Unless you're a Mod you shouldn't be wearing a button-down collar. I've never understood why you would need to button down a collar?!
It's originally a sporting thing... and from there it caught on in the North-Eastern USA and became a cornerstone of the 'Ivy' look, i.e. the way rich students at Ivy League colleages (Harvard, Princeton etc.) dressed, that became a kind of standard for wealthy WASPs. The mod use of button-downs was a bit later. But I really like button-downs, especially with a tweed jacket or suit. Not with anything more 'city' though...
[QUOTE 2791101, member: 259"]And for God's sake don't even
think about wearing jeans with a suit jacket.

[/quote]
No indeed! However, a good pair of jeans (and I mean some serious selvedge denim, not dad jeans) can work with a tweed jacket or other rougher-textured sport coat, though, even with a blazer - again, the Ivy influence. But you should never break a suit, unless it's specifically designed to be broken (particular types casual suits, sometimes called blazer suits these days).