Kwarteng fired the previous occupant of the job, Tom Scholar, in one of his first acts after being named chancellor of the exchequer last month.
Truss promised to sweep aside what she called "Treasury orthodoxy" as she campaigned for leadership of the Conservative Party but an announcement by Kwarteng of big and unfunded tax cuts on Sept. 23 caused turmoil in financial markets.
The Telegraph and the Financial Times said Antonia Romeo, currently the top civil servant at Ministry of Justice, was Kwarteng's top choice to lead the Treasury.
Romeo worked closely with Truss at the Department for International Trade, the FT said.
A spokesperson for the finance ministry said: "We will announce the new permanent secretary in due course once they've been appointed."
(Writing by William Schomberg; Editing by Nick Macfie)