Its head, Alpha Conde, said the AU demands an immediate return to constitutional order.
The military denies staging a coup, saying that Mr Mugabe is safe and that it was acting against "criminals" surrounding him.
Their move follows a power struggle over who might replace Mr Mugabe.
His vice-president, Emmerson Mnangagwa was fired last week, making Mr Mugabe's wife Grace the president's likely successor - but leaving top military officials feeling sidelined.
Mr Mugabe, 93, has dominated the country's political scene since it gained independence from the UK in 1980.
- What we know so far
- How can you tell a coup is happening?
- Zimbabwe media slow to cover takeover
Responding to the developments, Mr Conde, who is also Guinea's president, said Zimbabwean soldiers "had obviously attempted to take power".
The AU had "serious concern" at the situation and "reiterates its full support to the country's legal institutions", the statement said.
The BBC's Anne Soy in Zimbabwe points out that Egypt was ejected from the AU after its 2013 coup, so it may be the Zimbabwean military is trying to avoid antagonising the bloc by not describing their actions as a coup.
How the drama unfolded
After days of tension and rumour, soldiers seized the state broadcaster ZBC late on Tuesday.
A Zimbabwean army officer, Major General Sibusiso Moyo, went on air to say the military was targeting "criminals" around President Mugabe.
"This is not a military takeover of government," he insisted.
Maj Gen Moyo also said Mr Mugabe and his family were "safe and sound and their security is guaranteed". It is not clear who is leading the military action.
Since then, military vehicles have been out on the streets of Harare, while gunfire has been heard from northern suburbs where Mr Mugabe and a number of government officials live.
In a statement, the office of South African President Jacob Zuma, said: "President Zuma spoke to President Robert Mugabe earlier today who indicated that he was confined to his home but said that he was fine."
There has been no direct comment from President Mugabe, nor his wife Grace, whose whereabouts are unclear.
How we got here
The rivalry between Mrs Mugabe and Mr Mnangagwa has split the governing Zanu-PF.
Following a call for his dismissal from Mrs Mugabe, Mr Mnangagwa was removed from the vice presidency
But on Monday, army chief Gen Constantino Chiwenga said the army was prepared to act to end purges within Zanu-PF.
Gen Chiwenga is a close ally of Mr Mnangagwa and both are veterans of the 1970s war which ended white minority rule.
A supporter of Mrs Mugabe, Zanu-PF youth wing leader, Kudzai Chipanga, had responded by saying the military should "stay in the barracks".
But he has now apologised to Gen Chiwenga and other top military officials, saying "we are still young and make mistakes", ZBC reports.
An extraordinary gamble
By Andrew Harding, BBC Southern Africa correspondent
A quick show of military force, a few arrests - and then what?
These are, of course, unpredictable times for Zimbabwe and yet there is a chance that the army's extraordinary overnight gamble will pay off, and that President Robert Mugabe, humiliated and powerless, will nonetheless be allowed to retire with at least the pretence of dignity.
It is important to remember that Mr Mugabe is not being challenged by the Western governments he has warned against for decades, or by Zimbabwe's political opposition, or by a mass uprising against economic hardship.
This is, fundamentally, an internal power struggle within Zanu-PF and whoever emerges victorious can expect a newly purged party to fall obediently into line.
Mr Mugabe's mistake, at 93, was to assume he was still powerful enough to build a dynasty to back his wife, Grace, to succeed him.
Instead, his once loyal deputy, Emmerson Mnangagwa, may be poised to take control. If so, many foreign governments are likely to give him the benefit of the doubt and hope he can rescue Zimbabwe from years of misrule.
[BBC SOURCES]
0 comments