Spot gold slipped 0.4% to $1,804.79 per ounce by 12:22 p.m. EDT (1622 GMT), and U.S. gold futures also dropped 0.4%, to $1,806.90.
Gold prices jumped more than 1% on Wednesday on weaker-than-expected U.S. ADP jobs data, but gave up those gains after Fed Vice Chair Richard Clarida suggested the central bank could begin tapering its asset purchases later this year.
"Gold is trading weak relative to real yields, suggesting the metal's micro-structure is vulnerable to a downside catalyst and we're expecting a strong beat to Friday's jobs data, so there could be some further momentum to the downside," said TD Securities commodity strategist Daniel Ghali.
Friday's U.S. July non-farm payrolls report could shape the Fed's policy outlook, with economists in a Reuters poll predicting a gain of 870,000 jobs.
Higher interest rates raise the opportunity cost of holding non-interest bearing gold.
OANDA analyst Craig Erlam also said that recent comments from various Fed officials give the impression that the balance of power is falling in favor of the hawks, which a potentially strong jobs report would only accelerate, and that does not bode well for gold.
Elsewhere, platinum fell 1.5% to $1,010.77 while palladium rose 0.7% to $2,665.70.
Top automakers have recently said a chip shortage which has hindered auto production could persist for longer, potentially affecting demand for platinum and palladium which are used to curb emissions in cars.
"Certainly the fact that auto production might be constrained for longer than was expected is a downside risk for the platinum-palladium group metals," Ghali said.
Silver dropped 0.5% to $25.24 per ounce.
(Reporting by Nakul Iyer in Bengaluru; Editing by Paul Simao and Susan Fenton)
By Nakul Iyer
