[thaumasiotes]

The port number is one field in a TCP or UDP packet. When you listen, you register with the operating system (or whatever controls the network card, I guess) saying that you want a particular port. When a packet comes in (on the network card), it will be routed to you (on your socket) if it's addressed to you (by including a port number you're registered for).

You can think of the port number as the second half of your IP address. As far as the networking goes, an IP address and a port number are basically the same thing. The port number is just the lower bits of the "combined IP address".