By Esther Fung

Giant mall owner Simon Property Group is pursuing an unorthodox business strategy, smashing the foundation of the traditional industry model on the one hand while trying to glue certain elements of it back together with the other.

Simon has been in talks with Amazon.com Inc. to convert department stores into warehouse distribution hubs, say people familiar with the matter, an innovation that could plug large holes filled by ailing tenants such as Sears Holdings Corp. and J.C. Penney Co. But the move would challenge a long-held industry belief that anchor tenants are crucial for attracting foot traffic to other stores.

At the same time, Simon has been trying to keep aspects of the old business model intact, despite heightened pressure from e-commerce and other forces that have pushed some once-prominent retailers into bankruptcy. It is using cash to help stabilize some companies whose demise would mean closing stores in Simon malls.

The mall operator and a partner recently agreed to buy apparel retailers Brooks Brothers Inc. and Lucky Brand Dungarees LLC out of chapter 11. With another partner, Simon is in advanced talks to buy J.C. Penney.

One thing the two strategies have in common: a focus on maintaining occupancy levels at Simon malls as high as possible. Analysts say there is still a glut of malls, and that as many as half of the roughly 1,100 U.S. malls could shut for good as tenants close their doors.

Some view Simon's moves as boosting the company's income while buying time to reconfigure malls with only one anchor department store.

"Nobody in the $5.5 trillion-a-year retail business stands to lose more from bankrupt tenants or declining tenants than Simon Property right now, " said Nick Egelanian, president of SiteWorks, a retail consulting firm. "They need to keep rents flowing. I see it as the natural conclusion of the 50-year run of the mall industry."

Some investors said they think Simon Property is being astute and making out-of-favor investments that might seem foolhardy now but could pay off in the long term. Shares of the largest U.S. mall owner by number of malls are flat since June 30, when news started to emerge that it was interested in buying more flagging retailers. The S&P 500 rose around 9.1% during the same period.

Bill Smead, chief investment officer of Smead Capital Management, recently increased its investment in Simon Property Group to around 2.3% of its portfolio. He used an analogy to suggest it makes sense for Simon to pursue out-of-favor retailers right now because they come at bargain prices but could offer an upside when the economy turns.

"You buy straw hats in the winter. And for retail, this is winter," Mr. Smead said.

During an earnings call last week, Simon Property Chief Executive David Simon addressed concerns about the higher frequency of retailer investments. He said the company's aggregate investment in the two retailers is very small, just less than $50 million, and that ratings firms wouldn't think twice about it.

"If we didn't believe in the brand and we didn't think we could make money, we wouldn't do it. And it's...probably the same people that told Amazon to stay just in the book business, OK?" said Mr. Simon. "There's just nothing out there that says you can't make smart investments outside of your core businesses."

As part of a consortium, Simon purchased apparel retailer Aéropostale Inc. out of bankruptcy in 2016. The mall operator said in February it invested $25 million in the retailer and has received $13 million in distributions, and that Aéropostale earnings have reached around $80 million from a loss of $100 million three years ago. Simon also purchased Forever 21 Inc. with some partners early this year.

Still, some analysts said that in one sense Simon Property has been restrained. The landlord has made attempts for only three out of the 26 retailers to have filed for bankruptcy so far this year. "They're not going after everyone," said Alexander Goldfarb, a senior research analyst at Piper Sandler Cos.

Mr. Simon declined to comment about any logistics partnership with Amazon on the call, but he noted that more retailers are distributing their e-commerce orders from their stores through curbside pickup. "That's a good trend long term for us," said Mr. Simon.

The discussions with Amazon over converting some department stores into fulfillment centers are still in preliminary stages and a variety of uses are being considered, including a hybrid of retail and warehouses, according to a person familiar with the matter.

Simon risks alienating other mall tenants that would prefer a traditional anchor, rather than their chief online competitor. But if Amazon could include a retail-store portion such as a bookstore, a cashierless convenience store and returns drop-off, it would be additive to mall traffic that other mall tenants would appreciate, analysts from Jefferies Research said.

Write to Esther Fung at esther.fung@wsj.com