The phrase "perverse delight" refers to finding joy or satisfaction in something that is considered unconventional, strange, or even morally wrong. It means taking pleasure in something others might find unusual or inappropriate. Full definition
I confess I take perverse delight as a theologian in the controversies surrounding postmodernism. (firstthings.com)
As a state official responsible for criticizing local efforts, it's possible for me to take a certain perverse delight in the continuing evidence of unequal educational outcomes. (religion-online.org)