By Deepa Seetharaman

More than eight years ago, an investor contacted Instagram co-founder Kevin Systrom to inform him that Mark Zuckerberg was interested in buying the fast-growing photo-sharing app. Mr. Systrom's first question was direct: would the Facebook Inc. chief executive go into "destroy mode" if told no?

That question of how Facebook used its considerable market power to grow through strategic acquisitions was a big one in Wednesday's antitrust hearings, with the Instagram deal as the starring episode thanks to new documents released by the House antitrust subcommittee.

The allegation from several lawmakers, antitrust critics and competitors broadly is that Facebook has long adopted a "copy, acquire, and kill" strategy -- as one lawmaker described it -- when rivals emerge. The giant threatens to mimic startups and possibly squash them if they don't submit to a deal. Facebook officials were taken aback that the House released the documents and posted them on Twitter, according to people familiar with the matter.

When asked about the tactic during the hearing, Mr. Zuckerberg said Facebook has adapted features popularized by other platforms if they are seen as useful to users and said its acquisitions are successful in hindsight in part because the company invested heavily in helping them grow. He added that the Federal Trade Commission reviewed the relevant documents in 2012 when they examined the Instagram purchase and voted not to challenge the deal.

Facebook is currently preparing to launch its Instagram Reels service to take on TikTok, the video-sharing app that is hugely popular among young people. Facebook is offering some TikTok influencers cash to switch to Reels, The Wall Street Journal reported this week.

The Instagram acquisition for $1 billion, considered one of the bargains of the century, has increasingly become controversial among those who believe Facebook has gained too much power, and the documents released Wednesday paint a picture of a company keenly aware of competitive threats.

U.S. antitrust law is designed in part to block any acquisitions that substantially lessen competition.

Just a few months from going public in May 2012, Mr. Zuckerberg was shaken by a new crop of mobile-oriented social-media upstarts that were nibbling away at Facebook's user base, emails among the documents released Wednesday show. At the time, consumers were shifting to mobile phones, an area where Facebook was weak.

In internal discussions with other executives, Mr. Zuckerberg repeatedly expressed concerns that Instagram, Snap Inc. and other mobile-first social-media companies could undermine Facebook.

"The businesses are nascent but the networks are established, the brands are already meaningful and if they grow to a large scale, they could be very disruptive to us," Mr. Zuckerberg wrote in a February 2012 email to then Chief Financial Officer David Ebersman. "I'm curious if we should consider going after one or two of them."

In an email exchange in February 2012, Mr. Ebersman asked Mr. Zuckerberg what he was trying to accomplish by potentially buying Instagram. Mr. Ebersman laid out several reasons for doing such a deal, including neutralizing a competitor, acquiring talent or integrating another product with Facebook.

Mr. Zuckerberg responded that it was a combination of the first and third options. "One way of looking at this is that what we're really buying is time," he said. Buying Instagram or another startup "will give us a year or more to integrate their dynamics before anyone can get close to their scale again." He added: "New products won't get much traction since we'll already have their mechanics deployed at scale."

Nearly an hour later, Mr. Zuckerberg sent another message. "I didn't mean to imply that we'd be buying them to prevent them from competing with us in any way."

By that point, Mr. Zuckerberg was already contacting friends and former colleagues about buying Instagram. That included former Facebook executive and venture capitalist Matt Cohler, according to the documents. Mr. Cohler, who was an investor in Instagram, then messaged Mr. Systrom about a potential deal. Messrs. Cohler and Systrom discussed how Mr. Zuckerberg might respond to being rebuffed.

"If I make the "leave instagram alone for facebook's sake" argument, he will conclude that it's best to crush instagram," Mr. Cohler said in a written message. "I guess there's not much that can be done about that though."

In private discussions with Mr. Systrom, Mr. Zuckerberg at times expressed impatience with the Instagram co-founder's reasons for staying independent. In March 2012, Mr. Zuckerberg informed Mr. Systrom that Facebook was developing its own strategy for photos. "So how we engage now will also determine how much we're partners vs competitors down the line," he said in a chat.

On Wednesday, Mr. Zuckerberg defended his comments to Rep. Pramila Jayapal (D., Wash.) "I don't view those conversations as a threat in any way."

Mr. Zuckerberg has always been sharp elbowed when it comes to protecting Facebook from competition, according to the documents and people who have worked with him. Over the years, he has told employees that they shouldn't be "too proud to copy," the Journal reported in 2017.

In 2013, Facebook purchased a security app called Onavo, which gave Facebook access to private data showing how apps were performing relative to others. Onavo data was used to help Facebook determine whether or not to buy WhatsApp in 2014, Mr. Zuckerberg said Wednesday in the hearing.

In an internal Q&A in early April 2012, Mr. Zuckerberg addressed the growing threat of Instagram. "The bad news is that they are growing really quickly, they have a lot of momentum, and it's going to be tough to dislodge them," he said in a video recording. "We have a hard battle ahead of ourselves there."

On April 9, 2012, Facebook announced a deal with Instagram, which had about a dozen employees and no revenue. It now has more than 1 billion monthly users and is considered a key driver of Facebook's growth, though the company doesn't break out the unit's performance.

In an email exchange the day the Instagram deal was announced, Mr. Zuckerberg told a colleague that they were right to worry about Instagram, rather than Google+, the fledgling social network from the tech giant.

Mr. Zuckerberg added: "One thing about startups though is you can often acquire them."

Jeff Horwitz contributed to this article.

Write to Deepa Seetharaman at Deepa.Seetharaman@wsj.com