Even if you've been late on a credit card payment or two, it won't affect your score unless you're at least 30 days late - getting hit with an occasional late fee isn't a problem.
How long negative information stays on a credit report — Late fees will not affect a creditscore, while a late payment history will — at least, until it drops off a credit report in seven years... (See Negative credit information)