McCarthy, who is vying to become speaker of the U.S. House of Representatives, earlier told Fox News he had won bipartisan agreement to lift the mandate at a White House meeting with Biden, Senate Majority Leader Chuck Schumer, House Speaker Nancy Pelosi and Senate Republican leader Mitch McConnell.
McCarthy said it would be repealed as part of the must-pass $817 billion National Defense Authorization Act, or NDAA, an annual bill setting policy for the Pentagon that is expected to pass the Senate and House of Representatives this month.
But the White House said Biden had agreed only to consider the idea.
"Leader McCarthy raised this with the president and the president told him he would consider it," said White House spokesperson Olivia Dalton. "The secretary of defense has recommended retaining the mandate, and the president supports his position. Discussions about the NDAA are ongoing."
The mandate, which was imposed in August 2021, requires all U.S. service members to be vaccinated against COVID-19.
"You know what I was able to achieve in that meeting? To be able (to) - we're going to see in the NDAA - lift the vaccination mandate on our military men and women," McCarthy, the top House Republican, said in the interview, which aired late on Friday.
"I know I'm going to get that," McCarthy said. "We're working it out right now. I believe we're ... going to get that."
There was no immediate comment from the other three congressional leaders at the meeting.
The Pentagon's vaccine mandate has been the object of intense opposition from Republican conservatives, including several House lawmakers who are threatening to block McCarthy from becoming speaker when Republicans take control of the chamber on Jan. 3.
According to Defense Department data, 3,717 Marines, 1,816 soldiers and 2,064 sailors have been discharged for refusing to get vaccinated. But federal courts this year have blocked military services from punishing personnel who have refused the vaccines on religious grounds.
McCarthy presented the vaccine mandate deal as a sign of how he would lead the House as speaker. He also rebutted conservative criticism over his attendance at a White House state dinner for French President Emmanuel Macron.
"That's the things that we're going to have with the new Republican majority," McCarthy told Fox News.
"If somebody wants to argue about whether I'll represent this country right and respect the very first ally that helped us create this nation, I don't think they have their hearts in the right place."
(Editing by Deepa Babington)
By David Morgan and Jeff Mason