The Nikkei share average <.N225> rose 0.14% to 21,565 while the broader Topix fell 0.22% to 1,574.89, with small cap shares hit the most.
Topix small <.TOPXS> which consists of main board shares excluding 500 of the biggest companies fell 0.7%, while Topix 500 <.TOPX500> fell 0.17%, with Topix core 30 <.TOPXC> of top 30 firms falling just 0.05%.
Nikkei's gains stemmed from rises in a few heavyweights in the index, including Fast Retailing and FamilyMart Uny, which rose 1.5% and 4.1% respectively.
Defensive shares, including FamilyMart, were in demand as investors continued to favour sectors seen as least susceptible to global risks.
Retailers <.IRETL.T> rose 0.3%, with drugstore chain operator Welcia Holdings gaining 4.3% and rival Tsuruha Holdings 3.2%. Food companies <.IFOOD.T> rose 0.5% with NH Foods up 1.9%.
Telecom firms also did well, with KDDI rising 2.3% and Softbank Corp 0.4%.
On the other hand, electronic part makers and other semi-conductor-related firms bucked the trend, after Apple Inc led U.S. technology shares lower on Monday following a brokerage downgrade.
Apple suppliers followed in its footsteps, with Murata Manufacturing falling 2.0% and Taiyo Yuden down 4.0% and TDK Corp off 1.5%.
Shin-etsu Chemical fell 2.1% while Keyence shed 1.4%.
The Nikkei has been stuck in a narrow range between 21,500 and 21,800 this month as investors seek more clarity on the U.S. Federal Reserve's policy and as the global economic outlook.
"There are some questions over whether the Fed can really cut rates this month after such a strong reading on payrolls data," said Hiroshi Masushima, market analyst at Monex Securities.
Masushima said the Nikkei was unlikely to move much until next month when investors have a clearer idea on the Fed's policy and Japanese earnings outlook. In fact, expectations of a range-bound market are wide spread.
The Nikkei volatility index <.JNIV>, a measure of investors' volatility expectations based on option pricing, hitting a 1-1/2-year low.
(Additional reporting by Tomo Uetake; Editing by Jacqueline Wong and Michael Perry)
By Hideyuki Sano