The dismissals began Friday night, according to two people cited by The Associated Press. An email sent by one of the fired inspectors general said "roughly 17" inspectors general had been removed.
The watchdog's reporting comes in the wake of several high-profile deaths in federal lockups in recent years, most notably the murder James "Whitey" Bulger and the suicide of Jeffrey Epstein.
The inspector general's office found inmates served moldy bread, containers of food covered by what appear to be rodent droppings and insects in bags of cereal.
The Justice Department on Friday outlined a broad new push to support crime victims, including coordinating with state and local authorities in cases where federal charges won't be brought.
The report found the U.S. Park Police made the call to clear Lafayette Park before knowing of then-President Donald Trump's visit to a damaged church in the area.