Replika, a San Francisco startup launched in 2017, offers customers bespoke avatars that talk and listen to them.
It has led the way among English speakers, and is free to use, though it brings in around $2 million in monthly revenue from selling bonus features such as voice chats.
The 'virtual friend' is marketed as being able to improve the emotional well-being of the user.
But the Italian watchdog said that by intervening in the user's mood, it "may increase the risks for individuals still in a developmental stage or in a state of emotional fragility".
The agency also pointed to the absence of an age-verification mechanism, such as filters for minors or a blocking device if users do not explicitly state their age.
Replika breaches European Privacy Regulations and processes personal data unlawfully as it cannot be based, even implicitly, on a contract that a minor is unable to sign, the watchdog said.
Replika's developer, the U.S. company Luka Inc., must notify the Italian authority of measures taken to implement its requirements in 20 days and could be fined up to 20 million euros ($21.80 million), or up to 4% of its global annual turnover, the statement said.
Replika did not immediately respond to a Reuters email seeking comment.
($1 = 0.9176 euros)
(Reporting by Elvira Pollina; Writing by Gianluca Semeraro; editing by Gavin Jones and Arun Koyyur)
