The Federal Housing Finance Agency’s Inspector General wants Fannie Mae and Freddie Mac to penalize lenders for delays in repurchasing loans via an aggressive application of buyback late fees. The FHFA issued a “contract harmonization directive” in January 2012 calling for the two GSEs to develop “consistent timelines and collection standards for fees and penalties and additional types of penalties and remedies.”