"I have asked Defence Minister Christine Lambrecht to relieve me from my duties with immediate effect," Vice Admiral Kay-Achim Schoenbach said in a statement. "The minister has accepted my request."
Schoenbach made the remarks to a think-tank discussion in India on Friday, and video was published on social media. The comments came at a sensitive time as Russia has amassed tens of thousands of troops on Ukraine's borders.
Diplomatic efforts are focused on preventing an escalation. Russia denies it is planning to invade Ukraine.
In New Delhi, Schoenbach, speaking in English, said Putin seeks to be treated as an equal by the West.
"What he (Putin) really wants is respect," Schoenbach said.
"And my God, giving someone respect is low cost, even no cost... It is easy to give him the respect he really demands - and probably also deserves," Schoenbach said, calling Russia an old and important country.
Schoenbach conceded Russia's actions in Ukraine needed to be addressed. But he added that "the Crimea peninsula is gone, it will never come back, this is a fact," contradicting the joint Western position that Moscow's annexation of the peninsula from Ukraine in 2014 cannot be accepted and must be reversed.
Prior to Schoenbach's resignation, the defence ministry publicly criticised his remarks, saying they did not reflect Germany's position in either content or wording.
Schoenbach apologized for his comments.
"My rash remarks in India ... are increasingly putting a strain on my office," he said. "I consider this step (the resignation) necessary to avert further damage to the German navy, the German forces, and, in particular, the Federal Republic of Germany."
The Ukrainian Foreign Ministry had called on Germany to publicly reject the navy chief's comments. Schoenbach's comments could impair Western efforts to de-escalate the situation, Ukraine said in a statement.
"Ukraine is grateful to Germany for the support it has already provided since 2014, as well as for the diplomatic efforts to resolve the Russian-Ukrainian armed conflict. But Germany's current statements are disappointing and run counter to that support and effort," Ukraine's Foreign Minister Dmytro Kuleba said separately in tweet.
(Reporting by Sabine Siebold; Editing by Catherine Evans and Cynthia Osterman)