Originally, the hats were random drops. This still remains the case for pre-60, pre-BC players.
The 'guaranteed' drops for level 60 players were off big bosses in OK, such as Dark Master Gandling in Scholomance.
Then, when TBC came out, they dropped off heroic instance bosses, such as Heroic Mech, for those with TBC and post 60.
Now that WotLK has come out, and if you are post 70, these are where they come from, confirmed:
Winter Veil bosses of Northrend:
Grand Magus Telestra (The Nexus) - Red Winter Hat
Jedoga Shadowseeker in (Ahn'Kahet: The Old Kingdom) - Red Winter Hat
Mage-Lord Urom in (The Oculus) - Green Winter Hat
Also: You do not need to be in heroic mode to obtain the hat, just need to hit the specific boss. Also confirmed: the model will be wearing the hat if it drops one.
WoW Official WV Info