www.gokgs.com is the server that's probably biggest in the English speaking world. The desktop client is Java based, there is an Android client, and we're waiting for an HTML client (it's in closed Alpha). There are several other servers that cater more to Asian players, but you can certainly play on them.
There is also online-go.com, which is web-based. It has real-time games, but the majority of games on the server are correspondance.
Resources: senseis.xmp.net (an old wiki that has lots of content, but few active contributors), and lifein19x19.com/forum, a forum dedicated to Go.
I didn't know about the KGS HTML client. Either I should hurry up with my side-project or abandon it, as that's exactly what I'm building (but obviously without the already established KGS user base).
There's also http://www.dragongoserver.net/ for those who prefer "mail time" games, i.e. ones in the order of 1 day per move, obviously asynchronous. Pretty neat if you find it hard to carve a continuous hour for a synchronous game.
On another semi-unrelated note, if you ever want to play teaching games with friends you try out this Meteor-based webapp I made. It doesn't enforce any rules, but it does show all piece movements in realtime as you drag them around, so it's a bit more like playing in person. It should be easy to run locally or on your own free Meteor instance. http://github.com/muraiki/senseinogoban There's a demo server up but since anyone can join, don't expect your board to stay in shape. :)
Someday I'd like to make it more feature complete, such as adding accounts and multiple rooms, but since I just started a new job I won't have time for a while!
http://online-go.com is a new go server with a nice web interface. It runs acceptably well through the Android web browser if you're just looking to play games. KGS is still a better place to find games and players to talk to, though.
Since it hasn't been mentioned, Tygem is available for Windows or iPad (strange combo, I know). Not big with the English speaking crowd, but it's filled with top-notch Korean and Chinese players so it's great entertainment for dan level players.
Yellow Mountain Imports has a good looking and feeling bamboo set (bowls, board, yunzi stones) for under $100. GoGameGuru has some cheap sets too, but not with bowls. Both have a shin "kaya" set for under $200.
I went to the local go club a few times before picking up my own go equipment. I wanted to get some "hands on" feel for the board and stones, to pick the brain of people who have played longer than me on a greater variety of equipment, and to find out how genuinely terrible I am at go. I live in a town where there is a decent sized Asian market area. While some of the shops had go equipment, it was subpar and way too expensive for my blood.
The set MichaelGG is talking about is the first set that I picked up for myself. It was the centerpiece of my living room for several years, and has taken all of the abuse I can throw at it (over eager new players, drunk friends, young kids, pets, the go club) in stride.
As I was still learning and teaching the game I picked up a smaller, reversible 9x9 and 13x13 board with plastic stones, and that set never sees the light of day. It's far too light and chintzy to play on now. If you're sure you want go equipment, I would take at least a bamboo board and single convex yunzi stones. The upfront cost might seem like much, but the set from YMI will last at least a generation if not longer.
There is also online-go.com, which is web-based. It has real-time games, but the majority of games on the server are correspondance.
Resources: senseis.xmp.net (an old wiki that has lots of content, but few active contributors), and lifein19x19.com/forum, a forum dedicated to Go.