* 96% of votes back deal, but 20.7% of shareholders against

* All-cash offer is one of Australia's biggest-ever buyouts

* UniSuper the only investor rolling scrip into unlisted vehicle

SYDNEY, Feb 3 (Reuters) - Sydney Airport shareholders approved on Thursday a A$23.6 billion ($16.85 billion) cash takeover by infrastructure investors, though many small retail investors voted against delisting Australia's only listed airport.

The purchase is a long-term bet on the travel sector, which has been battered by the coronavirus pandemic. Record-low interest rates are spurring pension funds and their investment managers to chase higher yields.

The Sydney Airport takeover, one of Australia's biggest-ever buyouts, was backed by 96% of votes cast, but 20.7% of shareholders, representing a stake of 4%, were against the deal in a final voting tally.

Several retail shareholders said at a meeting they were disappointed the bid lacked a scrip component as they wanted to stay invested in the company over the long term. One long-term shareholder said he faced a big tax bill from accepting cash.

Chairman David Gonski said the bidders came to the board with an offer that did not include the opportunity for shareholders to roll their interests into the unlisted vehicle.

There was however, one exception. The consortium's bid was contingent on the company's largest investor pension fund, UniSuper, folding its 15% stake into the unlisted company rather than accepting the cash on offer to others.

After the deal is complete, UniSuper will join IFM Investors, QSuper, AustralianSuper and U.S.-based Global Infrastructure Partners as major shareholders.

Australia's other major airports are unlisted and owned by pension funds and infrastructure investors, though New Zealand's Auckland International Airport remains listed.

Last month, an independent expert said the Sydney Airport deal was "fair and reasonable" to shareholders at a time of significant revenue uncertainty due to COVID-19 and the looming end of its monopoly when Western Sydney Airport opens in 2026.

Sydney Airport's domestic traffic fell 74% in 2021 from 2019 level before the pandemic, while international traffic was down by 95.5%. ($1=1.4010 Australian dollars) (Reporting by Jamie Freed; Editing by Clarence Fernandez)