[brudgers]

My advice, start from Linux. It exposes all the moving parts, has the best tooling, and pretty much anything you learn about it is applicable to Windows or OSX - though there are parts of both that are unique. A mainline distro with a long term support release makes sense.

Hardware, cheap laptop and a cheap desktop and a decent keyboard. I am partial to the Microsoft Natural Ergonomic 4000. With two computers running Linux you can learn about networking, hardware configuration, ssh, and using the shell.

Next, Emacs [or maybe Vim] there's a reason these are around after decades. Emacs is fully programmable which if your into programming makes logical sense. Plus it forces you to use the keyboard and the keyboard is the fastest most reliable interface.

Language? For learning programming nothing comes close to Racket/Scheme. Racket is huge and provides tools for just about any programming paradigm. The two best programming introductions are How to Design Programs and The Structure and Interpretation of Computer Programs. Both are free on the internet.

Racket: http://racket-lang.org

HtDP: http://www.ccs.neu.edu/home/matthias/HtDP2e/

SICP: http://mitpress.mit.edu/sicp/full-text/book/book.html