[ricardo81]

I could be lost here (C/PHP/Node coder mainly in code I've used)

Why is it a special case to track HTTP/s requests, that otherwise couldn't be logged like any other process/function? I'd guess most people use libcurl and you can wrap something around that.

I guess I'm lost on why this is HTTP or Python specific, or if it is, fine.

[cle-b]

Unlike other tools such as proxies that allow you to trace HTTP requests, httpdbg makes it possible to link the HTTP request to the Python code that initiated it. This is why it is specific to Python and does not work with other languages.

[ricardo81]

I'm still not understanding.

If you're coding something up, why wouldn't you know that piece of code does a HTTP/s request? Based on what you said, it sounds like a scenario where a programmer doesn't know how a request was made. Are there examples of scenarios where that's the case?

Sounds like a bit of a security nightmare where there's code doing arbitrary requests.

[bityard]

Maybe you are working with an application or library that you didn't write, and want to see the raw requests and responses it generates without reading the entirety of the source code.

Maybe you are generating HTTP requests through an API and need to see which headers it sets by default, or which headers are or are not getting set due to a misconfiguration or bug.

There are probably loads more use cases, and if I actually did programming for a living, I could probably list a lot more.

[ricardo81]

The 3rd party library stuff makes sense, to an extent. But then you're debugging a 3rd party library.

[diegoallen]

If a 3rd party library you depend on has bugs, you have bugs. And you need to either submit a patch to the library or find a workaround.

[ricardo81]

Or just not use arbitrary 3rd party stuff hoping it works :)

libcurl is used on billions of devices across the world and has plenty of debugging capabilities.

MITM proxy works across all languages.

[whirlwin]

Here's a concrete scenario for you: Say you are in a team of 10 developers with a huge codebase that has accumulated over 5+ years. If you're new in the team, and you need to understand when a specific HTTP header is sent, or just snoop the value in the payload you otherwise wouldn't be able to see.

[ricardo81]

Snooping traffic isn't new though, so what's specific about this tool and Python.

[whirlwin]

How would you snoop outgoing HTTPS traffic otherwise easily anyway? mitmproxy requires some work to set up

[golergka]

> If you're coding something up, why wouldn't you know that piece of code does a HTTP/s request?

Because tracing all side-effects in a huge codebase with a lot of libraries and layers can be a daunting task.

Update: if you haven't worked with 20 year old >1m LOC codebase which went through many different teams and doesn't have any documentation whatsoever, you may lack necessary perspective to see value tools like this.

[ricardo81]

Sounds like people dealing with code they have no idea what it does. No amount of tools are going to help with that.

[actionfromafar]

I think you attract downvotes because tools are helpful. If you have a huge unknown codebase, it can be nice to attack it from different angles. Reading code is useful, but observing what it does in runtime can be useful, too. Also, with hairier code, it can be more useful to first observe and prod it like a black box.

[ricardo81]

Just the 1 downvote.

Yes, "tools are helpful", but whether there's a python/http specific tool that doesn't do what more generic tools do remains to be seen.

[seanc]

In the old days we'd use tcpdump and wireshark for this, but nowadays everything is encrypted up in the application layer so you need this kind of thing. Or tricky key dumping hacks.

[toomuchtodo]

https://www.charlesproxy.com/ ?

[robertlagrant]

I think the nice thing about HTTP for this is different parts of the stack can introduce default headers etc and it's helpful to be able to see the actual request after all that processing's been done.

[ricardo81]

With curl there's always CURLOPT_VERBOSE as per the library.