We may otherwise be giving back some of the benefits via: leaky chimney dampers; low R value masonry and metal chimneys which penetrate the building envelope (both heating and cooling penalties depending on the season); convective air currents arising from temperature differentials inside large chimneys; metal pipe air intakes which can be quite difficult to insulate, channeling cold into the envelope 24/7 during heating season;
backdrafting through leaky woodstove doors; and so forth.