On Wall Street, CSX Corp was one of the biggest drags on the benchmark S&P 500 index. The railroad tumbled 10.27% after it reported quarterly earnings that missed expectations and cut its full-year revenue forecast on weakness in its trade-related intermodal business. Fellow railroad Union Pacific lost 6.34% while Berkshire Hathaway, which owns railroad Burlington Northern Santa Fe, saw its Class B shares fall 2.55%.
The results come after U.S. President Donald Trump renewed his threat to tax another $325 billion of Chinese goods on Tuesday, which weighed on stocks. In addition, the U.S. could also face Chinese sanctions, following a World Trade Organization ruling on Tuesday, further complicating trade talks between the two countries.
"None of the underlying issues have really been put to bed," said Ed Campbell, portfolio manager and managing director at QMA in Newark, New Jersey, regarding trade concerns. "It's a risk factor that could come back at any time."
U.S. stocks have also eased over the past two sessions in part due to a sluggish start to the quarterly earnings season. Those declines also follow a rally that sent key stock averages to record peaks on expectations for lower U.S. rates.
Big banks such as Citi, JPMorgan and Wells Fargo have recorded drops in net interest margins, a sign low interest rates are hurting their bottomlines.
Bank of America shares were up 0.7% after it reported results on Wednesday but lowered its annual net interest income guidance.
While it is still early in what is expected to be a lacklustre reporting season, the earnings growth rate for the second quarter now stands at 0.4%, according to Refinitiv data. Expectations were recently calling for a quarterly decline in S&P 500 results.
The Dow Jones Industrial Average fell 115.64 points, or 0.42%, to 27,219.99, the S&P 500 lost 19.63 points, or 0.65%, to 2,984.41 and the Nasdaq Composite dropped 37.59 points, or 0.46%, to 8,185.21.
European shares closed lower as weakness in Swedish shares on some disappointing quarterly results and a decline in shares of oil majors helped snap a three-day winning streak.
The pan-European STOXX 600 index <.STOXX> lost 0.37% and MSCI's gauge of stocks across the globe <.MIWD00000PUS> shed 0.45%.
Along with the trade concerns, U.S. Treasury yields moved lower after data showed weakness in the housing market for a second straight month, even as mortgage rates have declined.
"The housing starts were a little weaker but the building permits were definitely significantly weaker," said Justin Lederer, an interest rates strategist at Cantor Fitzgerald in New York.
GRAPHIC: Housing starts, building permits Image -
Benchmark 10-year notes <US10YT=RR> last rose 20/32 in price to yield 2.052%, from 2.12% late on Tuesday.
The dollar retreated after notching strong gains on Tuesday following better-than-expected monthly retail sales data, while pound bounced after touching a 27-month low versus the greenback as no-deal Brexit concerns mounted.
The dollar index <.DXY> fell 0.18%, with the euro up 0.12% to $1.1223. Sterling was last trading at $1.2434, up 0.25% on the day.
(Additional reporting by Karen Brettell and April Joyner in New York; Editing by Bernadette Baum and Susan Thomas)
By Chuck Mikolajczak