There is that suggests heavy alcohol use while breastfeeding can cause gross motor issues and poor infant growth, but the research is contradictory. (momtricks.com)
«Short, Medium, and Long Term Consequences of Poor Infant health: An Analysis Using Siblings and Twins». (mpra.ub.uni-muenchen.de)
Smoke exposure is also linked with poorer infant sleep, which could contribute to irritability (Mennella et al 2007). (parentingscience.com)