5 mins on amazon will find you what you are looking for.
There are a whole bunch of feature phone and basic carrier/prepaid phones out there.
Some of them will also do usb tethering but its a bit difficult to actually set up since it needs obscure drivers.
You can also go for the higher end blackberry which still keeps physical keys and runs full blown android.