Many of those rules were conceived after the 2008 financial crisis and require the largest firms on Wall Street to hold more capital against future losses.
"We have reduced the odds that (a) systemically important firm could fail," Yellen said at a press conference to discuss monetary policy. "I would say it's very important not to roll back."
President-elect Donald Trump has said he wants to erase some banking rules that are too costly to satisfy and are hindering the economy.
Yellen said existing banking rules could be tweaked but that they are mostly a benefit.
Banking rules, Yellen said, "provide us another way of ensuring that the firms we count on to supply credit to households and businesses would be able to go on doing that even in the face of a severely adverse shock."
(Reporting By Lucia Mutikani and Patrick Rucker; Editing by Andrea Ricci)