Bankers fear enforcement action amid allegations that dealer rates vary by race.