Dohle had resigned "at his own request and on the best of mutual terms", the company added in the statement.
Nihar Malaviya, who is currently president and chief operating officer of Penguin Random House U.S., will take over as interim CEO from Jan. 1, the company said.
Bertelsmann, the German media group that owns Penguin, initially said it would appeal a U.S. judge's decision but then decided to scrap the deal after discussions with Paramount Global, Simon & Schuster's owner.
Judge Florence Pan of the U.S. District Court for the District of Columbia ruled on Oct. 31 that the Justice Department had shown the deal could substantially lessen competition "in the market for the U.S. publishing rights to anticipated top-selling books."
With the deal's dissolution, Penguin will pay a $200 million termination fee to Paramount.
(Reporting by Klaus Lauer, Writing by Miranda Murray)