Breaking Rules for Learning