The decision was made a day after Brazilians joined in mammoth nationwide street demonstrations criticizing her administration and the governing Worker's Party.
Rousseff's office yesterday confirmed that Eugenio Aragao took over the job following the resignation of Wellington Cesar.
Cesar stepped down after 11 days on the job because a Supreme Court ruling made him choose between the Cabinet position and his 25-year career as prosecutor in the state of Bahia.
Cesar got the job after long-time Rousseff ally Jose Eduardo Cardozo quit two weeks ago to become solicitor-general.
Cardozo had held the position for more than five years and was facing mounting pressure from politicians affected by the widespread corruption investigation at state-run oil giant Petrobras.
Brazil's justice minister is also the head of the country's federal police, which is a key part of the Petrobras probe alongside federal prosecutors and federal Judge Sergio Moro.
Rousseff faces impeachment proceedings in Congress for allegedly breaking fiscal rules. The government hopes that pro-government demonstrations scheduled for Friday will help shore up Rousseff's position.