The yen and Swiss franc gained earlier in the session, extending a flight to safety that began on Friday after Iranian Major-General Qassem Soleimani was killed in a U.S. drone strike on his convoy at Baghdad airport. That rally, however, has lost some steam.
"On the technical side, the yen has been strengthening even before the Iran thing so that play may be overdone below 108 yen and we're seeing the dollar coming back toward 109 yen," said John Doyle, vice president of dealing and trading at Tempus Inc in Washington.
"With something like this big, the major knee-jerk risk-off move is often short-lived and that's basically what we're seeing now. Equities basically are back flat and that correlates well with dollar/yen," he added.
U.S. President Donald Trump warned of a "major retaliation" if Iran hit back, while Iran's replacement commander vowed to expel the United States from the region.
On Sunday, Iran further distanced itself from the 2015 nuclear agreement with world powers, which the United States withdrew from in 2018, saying it would continue to cooperate with the United Nations nuclear watchdog but would respect no limits to its uranium enrichment work.
The yen surged on Monday to a three-month high around 107.75 versus the U.S. dollar, but was last down on the day as the greenback strengthened to 108.44 yen, up 0.2%.
The Swiss franc, another safe-haven currency, rose against the dollar, which fell 0.3% to 0.9690 franc.
The dollar index was down 0.2% at 96.674.
The greenback is sometimes seen as a safe-haven asset given that most central banks hold it as their main reserve currency and a big chunk of global companies trade using dollars, but the yen and the franc represent a more traditional safe-haven bet.
Implied volatility gauges in euro/dollar, the most traded currency pair, on the other hand, were relatively calm, suggesting investors are not yet fleeing to add protection to their portfolios by buying currency options.
A currency volatility index developed by Deutsche Bank was only marginally up and still close to its record lows.
Currencies sensitive to global risk appetite were weaker, including the Australian dollar, New Zealand dollar and Swedish crown.
The British pound was trading up 0.6% at $1.3158 <GBP=D3> ahead of a crucial week when British lawmakers are due to reconvene to debate the Brexit deal Prime Minister Boris Johnson has agreed with Brussels.
Graphic - Currency volatility index close to record lows: https://fingfx.thomsonreuters.com/gfx/mkt/13/619/619/Currency%20volatility%20index%20close%20to%20record%20lows.png
(Reporting by Gertrude Chavez-Dreyfuss; Editing by Paul Simao and Richard Chang)
By Gertrude Chavez-Dreyfuss