Fannie, Freddie Helped by Special Servicers. While nonbank servicers are under scrutiny from a number of different parties, Fannie Mae and Freddie Mac said special servicers reduce credit losses compared with traditional bank servicers. In its latest earnings report, Freddie said it facilitated the transfer of servicing for $55.6 billion in unpaid principal balance to special servicers in 2013. “Some of these specialty servicers have grown rapidly in the last two years and now service a large share of our loans,” the GSE said.