* Ares withdraws buyout offer for whole firm

* Shares fall to A$1.37 vs Ares withdrawn offer of A$1.85

* Review of wealth management and banking unit ends

* 'Most prized' asset AMP Capital still under review, Ares engaged

* Investors will be concerned if only most valuable asset sold-UBS

SYDNEY, Feb 11 (Reuters) - AMP Ltd revealed on Thursday that U.S.-based suitor Ares Management had dropped its A$6.36 billion ($4.91 billion) takeover offer, sending the Australian wealth manager's shares plunging more than 10%.

The decision by Ares to walk away was another major setback for scandal-hit AMP, which also reported a 32% fall in annual profits on Thursday due to heavy cash outflow from its core wealth management business.

Ares was the sole public suitor for Sydney-based AMP in its entirety after the company announced a portfolio review of all its businesses last year.

AMP said that its review of AMP Capital, considered its most valuable unit, was ongoing and that Ares was "engaged" with AMP as part of that process.

Representatives from Ares declined to comment.

"The completion of the asset review (with) Ares not proceeding with an offer, on all assets except for AMP Capital, will concern investors who need a complete sale and not a sale of the most prized asset only," UBS analysts said in a note.

Shares in AMP, for which Ares had offered A$1.85 each, fell as much as 10.7% to A$1.37, the lowest since Oct. 23, shortly before the company revealed Ares' approach.

"The market seems disappointed by the lack of a deal, but we would have been much more disappointed if it had been a bad deal," said Tim Hillier a fund manager at Allan Gray, AMP's second largest shareholder.

"We want management to be focused on improving the underlying business."

AMP has struggled to repair its reputation since a public financial sector inquiry in 2018 exposed systemic wrongdoing at the company, including charging customers for services it did not provide and misleading regulators.

AMP Chief Executive Francesco De Ferrari told journalists the company wanted to finalise the review process for AMP Capital "as soon as possible" although he gave no timeframe. He added that AMP could decide to retain the asset manager.

The company said it had closed the reviews of its wealth management and banking businesses.

Reporting 2020 full-year financial results, AMP said underlying profit fell to A$295 million ($227.8 million) from A$439 million a year ago as net cash outflows from its wealth management business rose 32% to A$8.3 billion.

A government scheme allowing workers to draw on their retirement savings during the coronavirus pandemic had accelerated those outflows.

AMP said it was committed to resuming dividend payments this year, seeking to return cash to investors from the sale of its Life Insurance unit.

With "earnings not recovering, the turnaround story for AMP is harder to see," UBS analysts said. (Reporting by Paulina Duran in Sydney and Anushka Trivedi and Sameer Manekar in Bengaluru; Editing by Ramakrishnan M., Stephen Coates and Jane Wardell)