External and internal auditors, as well as regulators, should have dug deeper when the first signs of Libor manipulation surfaced, say participants at American Banker's Risk Management roundtable.