Damien Abad, the minister for Solidarity and the Disabled, strongly denies the accusation published on Saturday by investigative website Mediapart and has rejected calls from the opposition to resign.
Elisabeth Borne, the newly appointed prime minister, said at the weekend that she would be ruthless with anyone guilty of sexual misconduct, but that she would only take action if a new criminal complaint was made or judicial investigations launched. The government reiterated that stance on Wednesday.
Rejecting a request by women's rights groups, the prosecutor's office said it could not open a probe "for now," for lack of a complaint by the alleged victim herself and lack of details on who she is.
But lawmakers in Macron's camp worried the allegations - which Abad strongly rejects - could still have a long lasting impact.
"This feeds a kind of disgust with politics and boosts extreme (parties)," one female lawmaker in Macron's camp said, speaking on condition of anonymity because of the sensitivity of the topic.
"It's at odds with Emmanuel Macron making the cause of women a priority of his second mandate," she said, adding that she feared it would have an impact on the legislative election.
The two women quoted by Mediapart said Abad had forced them to have unwanted sexual relations with him in late 2010 and early 2011.
One twice filed a complaint, which was closed without further action, and the other one hasn't, and it was her case that the women's rights NGO raised with prosecutors.
Abad - who was poached from the opposition conservative party to join the government following Macron's winning a second presidential mandate - says he has done nothing wrong.
Some 53% of voters consider a minister accused of rape or sexual assault should resign, while others say that should not be the case before there is a ruling, a survey by Elabe pollsters on Tuesday showed.
"It's really difficult," a heavyweight in Macron's camp said, while hoping the immediate political impact would not be too damaging adding that voters "talk to me about the cost of living, health and the environment, not about Abad."
Opinion polls have so far shown Macron and his allies are likely to win the June 12-19 parliamentary elections.
(Additional reporting by Matthieu Protard; Writing by Ingrid Melander; Editing by)
By Elizabeth Pineau