The export decline highlights the immense task Yoshihide Suga, who is assured to be elected prime minister later on Wednesday, faces in driving an economic recovery.
Total exports fell 14.8% year-on-year in August, a smaller decline than the 16.1% expected by economists in a Reuters poll, official data showed on Wednesday.
That meant exports fell for their 21st straight month, marking the longest run of declines since a 23-month run through July 1987. That followed a 19.2% drop in the previous month.
The decline in August was driven by fewer shipments of cars and mineral fuels, though the pace of contraction eased somewhat from July as economic activity showed signs of picking up.
"Strong demand for ICT technology linked to working from home resulted in exports of electric machinery only falling 5.5% year-on-year," said Tom Learmouth, Japan economist at Capital Economics.
"However, export volumes may not reach pre-virus levels until early-2022," he said in a note.
Suga, who won a ruling Liberal Democratic Party (LDP) leadership election on Monday, will face a huge challenge to get the economy back on track after it posted its worst postwar contraction in the second quarter.
By region, shipments to the United States - Japan's key market - fell 21.3% in the year to August, weighed heavily by declines in engine parts and construction machinery.
Exports to China, Japan's largest trading partner, rose 5.1% year-on-year in August, helped by a sharp increase in shipments of semiconductors, the data showed.
That marked the second straight monthly rise in China-bound shipments, which showed signs of picking up, a finance ministry official said.
Exports to the rest of Asia declined 7.8%, weighed by shrinking exports of iron and steel products.
Overall imports fell 20.8% in the year to August, versus the median estimate for an 18.0% decrease.
As a result, the trade balance came to a surplus of 248.3 billion yen ($2.36 billion), versus the median estimate for a 37.5 billion yen shortfall.
By Daniel Leussink