But any such withdrawal of stimulus will begin from 2021 at the earliest, the survey showed, a sign that monetary policy in Japan could be in a holding pattern for the time being.
"There's a chance growth in overseas and Japanese economies could pick up next year," said Nobuyasu Atago, chief economist at Okasan Securities.
"The yen is stable and stock prices are firm," which could allow the BOJ to hold off on expanding stimulus, he added.
Twenty-five of 41 economists, or 61% of the total, expect the BOJ's next move to be a withdrawal of stimulus, the poll taken between Dec. 4-16 showed. That was up from 44% in a survey in November.
Most of them say it could happen in 2021 or later.
Sixteen of the economists, or 39%, think the BOJ will top up stimulus as its next step, down from 56% in last month's survey.
The BOJ is set to keep monetary policy steady this week as receding fears of a disorderly Brexit and signs of progress in U.S.-China trade talks take some pressure off the central bank to use its dwindling ammunition to underpin growth.
Japan's core consumer price index, which includes oil products but not fresh foods, is forecast to rise 0.6% this fiscal year and next year - well short of the BOJ's 2% inflation target.
Japanese policymakers have been under pressure to do more to underpin a fragile economic recovery, hit by the global trade war, typhoons and a sales tax hike that rolled out in October.
The world's third-largest economy is forecast to have shrunk by an annualised 3.2% in the fourth quarter, which would be the biggest contraction since April-June 2014, the poll found.
"Consumer spending is expected to worsen sharply in the current quarter as the tax increase puts a burden on households," said Yoshiki Shinke, chief economist at Dai-ichi Life Research Institute.
"Capital spending also likely weakened after having boosted before the tax hike. The economy will pick up next year but the pace of recovery is expected to be moderate."
Growth will rebound by 0.9% and 1.2% in the first and second quarters of 2020, according to median forecasts. The economy will expand 0.9% in the current fiscal year ending in March 2020 before slowing to 0.5% the following year, the poll predicted.
Some economists responded to the poll before the government announced its plan to compile a 13.2 trillion yen ($122 billion) fiscal package to support growth.
Based on information on media reports of the spending plan, most analysts expect the stimulus package to lift growth by less than one percentage point.
"Labor shortage will be a bottleneck" and dent the effect of the stimulus package as the government could face difficulty executing public works projects smoothly, said Shuji Tonouchi, senior market economist at Mitsubishi UFJ Morgan Stanley Securities.
(Reporting by Kaori Kaneko; Polling by Shaloo Shrivastava and Anisha Sheth in Bengaluru; Editing by Leika Kihara and Ross Finley)
By Kaori Kaneko
Stocks treated in this article : Euro / Japanese Yen (EUR/JPY)
, Euro / British Pound (EUR/GBP)
, London Brent Oil
, Morgan Stanley
, Mitsubishi UFJ Financial Group, Inc.
, Okasan Securities Group Inc.
, Mitsubishi Corporation
, Core Corporation
, Hewlett Packard Enterprise