Skip to content

gh-102327: Extend docs for "url" and "headers" parameters to HTTPConnection.request() #102328

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
May 9, 2023
Merged

gh-102327: Extend docs for "url" and "headers" parameters to HTTPConnection.request() #102328

Changes from 6 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
86275f8
Document "url" parameter to HTTPConnection.request() (#102327)
davidfstr Feb 28, 2023
912df17
Alter example URL to be: https://docs.python.org/3/
davidfstr Apr 30, 2023
9013cd3
Alter RFC section link to appear with more-natural formatting
davidfstr Apr 30, 2023
c0960c6
Distribute new documentation to relevant parameter paragraphs
davidfstr May 2, 2023
231bd9e
Adjust formatting
davidfstr May 2, 2023
5a6293d
Fix trailing whitespace
davidfstr May 2, 2023
e6dc07c
Clarify phrasing for when absolute paths are required
davidfstr May 4, 2023
dcf2480
Fix whitespace
davidfstr May 4, 2023
f6f6e48
Merge branch 'main' into f/http_url_clarify
merwok May 9, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions Doc/library/http.client.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,9 @@ HTTPConnection Objects
encode_chunked=False)

This will send a request to the server using the HTTP request
method *method* and the selector *url*.
method *method* and the request URI *url*. The provided *url* must be
an absolute path to conform with :rfc:`RFC 2616 §5.1.2 <2616#section-5.1.2>`
when using most HTTP methods (like ``GET`` or ``POST``).
Copy link
Member

@merwok merwok May 2, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This line makes me wonder if PUT and PATCH are part of «most HTTP methods», so I have to follow the RFC link to see. It seems that yes, only OPTIONS * is given as a counter-example. I wonder if there is a way to rephrase that so that casual readers take away that url should nearly always be an absolute path, see link for exact rules.

What do other people think?

Copy link
Contributor Author

@davidfstr davidfstr May 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A more specific phrasing would be:

The provided *url* must be an absolute path
to conform with :rfc:`RFC 2616 §5.1.2 <2616#section-5.1.2>`,
unless connecting to an HTTP proxy server or
using the ``OPTIONS`` or ``CONNECT`` methods.

And further down:

A :rfc:`Host header <2616#section-14.23>` must be provided
to conform with :rfc:`RFC 2616 §5.1.2 <2616#section-5.1.2>`,
unless connecting to an HTTP proxy server or
using the ``OPTIONS`` or ``CONNECT`` methods.

A reader would still have to follow the link to determine the actual rules if they were talking to an HTTP proxy server (somewhat common?) or using OPTIONS/CONNECT (rare?).

Comments?

Copy link
Member

@merwok merwok May 3, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That’s very clear! The unless clause could even be in parentheses.

I never use proxies but I think there are two cases:

  • explicit proxy, when you send GET https://authority/path on a connection opened to http://proxy
  • transparent proxy, when you use an HTTP library that respects the http_proxy or https_proxy environment variables

I suppose the note here applies to the first kind only?

Copy link
Contributor Author

@davidfstr davidfstr May 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feedback applied. I think this change is ready to merge!

Copy link
Member

@merwok merwok May 4, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you reply to the question about proxy? 🙂

Copy link
Contributor Author

@davidfstr davidfstr May 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RFC 2616 §1.3 defines a "proxy" as:

An intermediary program which acts as both a server and a client
for the purpose of making requests on behalf of other clients.
Requests are serviced internally or by passing them on, with
possible translation, to other servers. A proxy MUST implement
both the client and server requirements of this specification. A
"transparent proxy" is a proxy that does not modify the request or
response beyond what is required for proxy authentication and
identification. A "non-transparent proxy" is a proxy that modifies
the request or response in order to provide some added service to
the user agent, such as group annotation services, media type
transformation, protocol reduction, or anonymity filtering. Except
where either transparent or non-transparent behavior is explicitly
stated, the HTTP proxy requirements apply to both types of
proxies.

Therefore I speculate that the following requirement from §5.1.2 applies when a Python program attempts to connect to any kind of proxy:

The absoluteURI form is REQUIRED when the request is being made to a
proxy.

I never use proxies myself so I have no empirical experience one way or the other.

Copy link
Member

@merwok merwok May 5, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, let’s hope the people using proxies know how to handle them!


If *body* is specified, the specified data is sent after the headers are
finished. It may be a :class:`str`, a :term:`bytes-like object`, an
Expand All @@ -279,7 +281,9 @@ HTTPConnection Objects
iterable are sent as is until the iterable is exhausted.

The *headers* argument should be a mapping of extra HTTP headers to send
with the request.
with the request. A :rfc:`Host header <2616#section-14.23>`
must be provided to conform with :rfc:`RFC 2616 §5.1.2 <2616#section-5.1.2>`
when using most HTTP methods (like ``GET`` or ``POST``).

If *headers* contains neither Content-Length nor Transfer-Encoding,
but there is a request body, one of those
Expand All @@ -298,6 +302,16 @@ HTTPConnection Objects
HTTPConnection object assumes that all encoding is handled by the
calling code. If it is ``True``, the body will be chunk-encoded.

For example, to perform a ``GET`` request to ``https://docs.python.org/3/``::

>>> import http.client
>>> host = "docs.python.org"
>>> conn = http.client.HTTPSConnection(host)
>>> conn.request("GET", "/3/", headers={"Host": host})
>>> response = conn.getresponse()
>>> print(response.status, response.reason)
200 OK

.. note::
Chunked transfer encoding has been added to the HTTP protocol
version 1.1. Unless the HTTP server is known to handle HTTP 1.1,
Expand Down