By Sam Schechner

A system Google set up to promote competition on Android has left some smaller search engines having trouble gaining traction, fueling rivals' complaints about the tech giant's compliance with a European Union antitrust decision ahead of potential U.S. charges.

Since March, Alphabet Inc.-owned Google has been showing people in Europe who set up new mobile devices running the company's Android operating system what it calls a "choice screen," a list of rival search engines that they can select as the device's default. The system is part of Google's compliance with a 2018 decision that found the company used Android's dominance to strong-arm phone-makers into pre-installing its search engine.

But some small search engines that are relatively popular in Europe failed to win spots in large European countries in the latest round of auctions to appear on the choice screen, according to people familiar with the results. The results, which cover the fourth quarter of the year, are set to be announced on Monday.

DuckDuckGo Inc., maker of a U.S.-based search engine that doesn't collect data about its users, lost the auction in all but four small European countries, the people said. Berlin-based Ecosia, which donates most of its profit to planting trees, also didn't win a slot in any large European country, the people added.

The major winners of the auctions -- which offer three spots in each of 31 countries to outside search engines -- include Microsoft Corp.'s Bing, as well as a handful of other small search engines, the people said. Google doesn't participate in the auctions but is offered automatically as a choice in every country along with the auction winners.

"In developing the choice screen for Europe, we carefully balanced providing users with yet more choice while ensuring that we can continue to invest in developing and maintaining the open-source Android platform for the long-term," a Google spokeswoman said. Google has defended its use of auctions saying that "an auction is a fair and objective method to determine which search providers are included."

The elimination of some smaller search engines gives fodder to Google rivals who have complained that the company has crafted its compliance with the EU's antitrust decisions in ways that don't fundamentally change the competitive landscape. DuckDuckGo and Ecosia are the most popular small search engines in Europe, with 0.5% and 0.3% market share as of August, respectively according to Statcounter.

At issue is a hard-fought debate about whether competition enforcement can -- or should -- go beyond a narrow focus on consumer welfare, or should attempt more radical interventions to redraw the balance of market power in cases of abuse.

DuckDuckGo and Ecosia have both been vocal in the past to say auctions unfairly benefit Google. They also say the rivals most likely to succeed in an auction are those that are the most profitable, rather than those that are best for, or most popular among, consumers, and have called for Google to offer longer lists of alternatives without requiring companies to pay.

"The auction itself is flawed," said Gabriel Weinberg, founder and chief executive of DuckDuckGo, who said he bid as much as he estimated his company could sustainably afford. "Anyone with less ads is getting hurt. Anyone doing privacy is getting hurt. Anyone doing things where they give away profit [to support causes] is getting hurt."

Microsoft declined to comment. The European Commission, the EU's top antitrust authority, said it continues to monitor Google's implementation of its remedy and has been discussing "the selection mechanism of rival search providers" with Google "following relevant feedback from the market."

Wall Street Journal publisher News Corp is a longtime Google critic and has complained about some of the company's conduct to regulators.

The dispute over Google's compliance with EU antitrust decisions comes as the U.S. Dept. of Justice is moving toward bringing a lawsuit against the company, The Wall Street Journal has reported. The department's investigation into search is likely to be the key focus of a lawsuit that could come within days, the Journal said last week. DuckDuckGo has also provided evidence for U.S. Congressional investigators. [LINK:

In the EU, Google's choice screen auctions are part of its effort to comply with a 2018 decision from the European Commission. The commission found, among other alleged violations, that Google had abused its control of the Android operating system by forcing phone makers to pre-install Google Search and Google's Chrome browser if they also wanted to include Google's Play store for apps, by far the most common way to get Android apps.

Google appealed the decision, but still had to change its contracts with phone makers to eliminate that requirement. Under pressure from EU regulators, Google later offered an additional concession: it would create a choice screen for Android device users to choose alternative search engines to be their default.

Such measures have in the past appeared to cut into Google's market share, for example in Russia. In Europe, however, rivals have complained because Google said it would structure the choice screen in the EU as an auction. Google later changed how it structured the auctions to charge less, but DuckDuckGo and others still said that a "pay-to-play" model risked eliminating them.

Those complaints echo similar issues that rivals have raised about Google's compliance with a separate EU antitrust decision, which alleged that Google abused the dominance of its search engine to boost its own product ads in a box above search results for websites offering product comparisons. Google in that case also set up an auction for product-comparison sites to bid to appear in the box. Rivals initially won few of the auctions, and complained, though an EU official said earlier this month that Google's compliance has since improved.

Initially, DuckDuckGo won spots in the auctions for all European countries, including Germany, France and the U.K. But before the fourth quarter auction, DuckDuckGo said it lowered its bids somewhat to account for its aggregate estimate of how much it actually profits from each user. DuckDuckGo's Mr. Weinberg declined to say what he bid, but said that he believes his company can no longer profitably appear in the Google choice screen.

"Our data shows that the preference menu once it's fully implemented, will drive significant traffic," Mr. Weinberg said. "Our being out this time means we're going to be out forever."

Write to Sam Schechner at sam.schechner@wsj.com