Shares in the company, which makes engines for large airliners, lost 4% in early trading, making the company one of the top losers in Britain's bluechip index.
Rolls-Royce said on Tuesday that Singer, chief operating officer of ValueAct, left on December 9 after nearly four years as a non-executive director.
ValueAct owns a 9.35% stake in Rolls-Royce according to Thomson Reuters data. When Singer joined the board, the company's chairman Ian Davis said he would remain on it as long as ValueAct remained a significant shareholder.
Analysts at Jefferies dismissed worries that Singer's departure meant ValueAct was set to change its shareholding.
"We think it inevitable some will interpret the board change as signalling ValueAct is imminently going to reduce its shareholding, but such was the Relationship/Confidentiality Agreement between Rolls-Royce, ValueAct, and Bradley Singer that we believe there to be no substantive change," they said in a note.
A spokesman for ValueAct declined to comment on its future trading plans.
ValueAct's shareholding in Rolls-Royce was first announced in 2015 and Singer joined the board in 2016, at a time of turmoil for the company. He said on Tuesday that the company was now on a "solid path forward".
ValueAct states on its website that it regularly owns stakes for three to five years but it can hold them for ten years or more.
(Writing by Sarah Young; editing by Kate Holton)