The phrase "bad apples" is used to refer to a few individuals in a group who behave in a negative or unethical way, spoiling the reputation of the entire group.