Host name in URL not shown for https requests about mitmproxy

[clouedoc]

Note to future readers: the cli option enabling host-header-as-url-authority is now --showhost

It can also be activated under the option menu (press uppercase O while in terminal mode) next to "showhost" (options are alphabetically sorted).

[yinguohang]

If you are using python script and try to get the url, you might try request.pretty_host instead of request.host.

[root-intruder]

If you are using python script and try to get the url, you might try request.pretty_host instead of request.host.

for keeping the whole URL request.pretty_url is also possible

[leoluk]

Workaround: returning headers['Host'][0] instead of host in read_request.

[cortesi]

Hi Leopold,

This is actually a complex issue that we have to think about carefully. The hostname we display can come from a number of sources:

- The host header
- The host specified in the CONNECT string on proxy connection
- The upstream host specified in a reverse proxy configuration
- The upstream IP address actually connected to (or deduced in transparent mode)

You can plausibly have a request in which all of these values is different. You can also have perfectly ordinary requests in which any of these are absent! What you're seeing in the current master is an interim solution while I think about the best way to resolve this.

So, I've looked at your patch, but I'm afraid it won't do. We can't assume that a host header exists, and we need something better than using just the host header even if it does exist. My current thinking is that we need an option in mitmproxy to toggle between showing the upstream IP, and the host header.

• In host header mode, we use the host header if it exists, otherwise we fall back to the IP. This is the default mode.
• In IP mode, we just use the IP.
• We expose the CONNECT strings and upstream reverse proxy configuration through the interface in other ways (probably the detailed flow view currently accessible through the X shortcut).

If you feel like tackling this as a project, let me know. Otherwise, this is definitely on my todo list, and I'll work on it before the next release.

Cheers,

Aldo

[leoluk]

Yeah… I realized that as well. Using the host header might have security implications as well (CVE-2009-0801), depending on the implementation details (I've only looked at the code for a few minutes). Despite the limitations of this approach, it seems to me that simply using the Host header is more useful than displaying the IP until there's a correct implementation.

I agree with you. I suggest to use the host header (or the IP if it doesn't exist) for the displayed URL because it's usually the most useful information. The values from the other sources could be displayed in the detailed view and there should definitely be a setting (for misbehaving or malicious clients, for example).

[cortesi]

I've just added an option that uses the Host header for display URL construction. You can access it using the --host option, or with the "o" then "h" keyboard shortcut in mitmproxy.

[leoluk]

Works perfectly! Thanks a lot.

[ogion]

hi, how can we do this in Java?