The phrase "red rag" refers to something that causes a strong reaction or provokes anger or irritation in someone, just like a bull gets agitated by seeing the color red.