The former Scotland international has initially turned down the offer to succeed Steve Cooper at the Liberty Stadium, the PA news agency understands.
The Sky Bet Championship club have refused to end their pursuit and conversations remain ongoing but it remains unlikely Martin will move to south Wales.
He was frontrunner for the role but former Derby and Chelsea assistant Jody Morris could now get the job.
Martin has impressed at the Dons since taking charge at the end of 2019, and last season only Manchester City and Barcelona had a higher possession percentage in Europe.
They finished 13th in Sky Bet League One last term, with Martin having guided them out of relegation danger in 2019-20 after taking over from Paul Tisdale.
Swansea are looking for a new manager after Cooper left by mutual consent earlier this month.
The former England Under-17 manager guided the Swans to the Sky Bet Championship play-off final but they lost 2-0 to Brentford in June.
QPR assistant boss John Eustace had looked poised to take control but opted to stay at Rangers after holding talks with Swansea.