One banker at the meeting with Osborne told Reuters the banks wanted to see concrete moves to ensure firms based in London would retain access to EU markets, however, because "no one in their right mind" would currently invest in Britain.
Since the June 23 referendum there have been fears of an exodus from the City of London if access to the EU's single market becomes significantly harder. Banks like JPMorgan (>> JPMorgan Chase & Co.) have said they could move thousands of jobs.
On Thursday JPMorgan, Goldman Sachs (>> Goldman Sachs Group Inc), Bank of America Merrill Lynch (>> Bank of America Corp) and Morgan Stanley (>> Morgan Stanley), as well as Britain's Asia-focused Standard Chartered (>> Standard Chartered PLC) said they would try to support London's financial sector.
"Today we met and agreed that we would work together ... with a common aim to help London retain its position as the leading international financial center," the banks said in a joint statement with Osborne.
Citi (>> Citigroup Inc), which had been unable to attend the meeting, signed the statement later on Thursday.
The statement said no other city in Europe had capital markets as deep as those of London, but the banks made no commitment about keeping jobs in Britain.
Banks at the meeting sought immediate reassurance over the fate of EU staff currently working in London, and urged a pragmatic approach to quickly ensure Britain retained access to the EU single market.
But any permanent trade deal is likely to be years away, given the normal length such agreements take and the political vacuum as Britain's ruling Conservatives picks a new leader.
The French government pledged on Wednesday to make its tax regime for expatriates the most favorable in Europe in a land grab for London banking business.
Goldman Sachs and Morgan Stanley have denied speculation they were poised to shift London-based staff and operations to Frankfurt.
Jamie Dimon, the head of JPMorgan Chase, said before the referendum that the bank could have to move up to 4,000 staff from the UK in the event of a Brexit vote.
Osborne, who was senior figure in the failed campaign to keep Britain in the EU, has spent years trying to expand London's role as a global hub for trade in China's renminbi and India's rupee, as well as Islamic finance.
On Wednesday the Lord Mayor of the City of London, the district in the capital that is home to many large banks, said the government needed to ensure EU exit talks preserved financial services firms' "passporting" rights that allow them to use London as a base for EU-wide activities.
Osborne has said Britain needs to ensure a heavy emphasis on trade ties, including financial services, in EU exit talks.
On Tuesday, he met British domestic banks, whose share prices have tumbled since the Brexit vote, and encouraged them to make more funds available for UK lending after the Bank of England eased capital rules.
(Additional reporting by Andrew MacAskill, editing by Jeremy Gaunt and Toby Chopra)
By David Milliken and Lawrence White