Hacker News new | ask | show | jobs
by aldoushuxley001 2649 days ago
I was just using urllib3 to post a form on another website and get the resulting html page, then parsed it with BeautifulSoup.

Since it was just a one off use case and ultimately very simple, I didn't see the need for any more functionality. Why bother with the extra packages? Or do you think it's still worthwhile to use Requests even still? Is it not just unnecessary bloat that might slow runtime?
1 comments
diminoten 2649 days ago
There's a lot to unpack in your comment, but I'll just work with the most easily verifiable thing for you; what was the response time of the resource you were querying with urllib3, and do you think using requests instead of urllib3 directly would be an order of magnitude (or two) more or less runtime?
link
aldoushuxley001 2649 days ago
I admittedly didn't test the response times between the two, but just felt adding additional dependencies was unnecessary. I don't realistically expect the speed to be too different between the two, but the less I have to rely on external libraries the better. If I can get the job done with urllib3 why use Requests?

Though admittedly, after reading OPs statement, I see that Requests might actually have some extra security that urllib3 alone might not have. But barring security improvements or the need for extra features that Requests has, seems like using Requests for my usecase would be adding unnecessary complexity.

link
diminoten 2649 days ago
> but just felt adding additional dependencies was unnecessary

This notion, especially in Python and HTTP client programming, is wrong and will cost you many many more hours than it will save you.

Requests is an entire order of magnitude easier to use than urllib3, and while we may be dealing in minutes for this specific scenario, you will make up for any time investment you pay to learn Requests the very next time you need to do HTTP related work in the language.

It's a matter of not overreacting to a cost, and you're paying way more than you should to get a much smaller gain than you could, if you paid that cost elsewhere (by learning/using Requests and how to manage dependencies in Python, which you have to do anyway with bs4).

link