The psychology of light is much more complicated than just number of lumens being thrown about. Because HPS lamps throw light in pretty narrowly defined bands, people feel less comfortable, and aren't able to notice things as well compared to light that is lower wattage, but more full spectrum ( http://www.lrc.rpi.edu/researchAreas/pdf/GrotonFinalReport.p... ). Like a lot of things, quality of light counts more than quantity.
LED lights have much longer lifespans: 50k to 100k hours, compared to ~30K hours for high pressure sodium lamps [1]. Reduced labour and parts costs could account for the decision.
Also, the referenced document shows LED lights as emitting 70 - 150 lm/W, so power consumption may be better than you think.
It's poor form to ask for a source of something that would take you 15 seconds to google.
> Allegedly, Cree has developed a white LED with up to 254 lm/W
Interesting considering that the theoretical maximum is 251 lm/W - so I wonder which law of physics they violated to do that. (Most likely it's not actually white, but reddish.)
The hps operates at higher power and comes in models that scatter the light in all directions (so you need a reflector). The spectrum is constrained (as is for led) but you'll find it difficult to change it. All those lumens are in an unfavorable part of the spectrum. LED is much better at fooling our eyes, their spectrum can be tuned to sit close to the receptiveness maxima for our eyes.
Under scotopic (night) viewing conditions, it is lower than that. Also when you take into account the losses due to the reflectors to get the light out of a lamp, the nice little lambertian LED emitters require much less optics to do a good job of providing light on a wide road area.