requests

Requests module.

wmflib.requests.DEFAULT_TIMEOUT: float = 5.0

the default timeout to use if none is passed, in seconds.

Type

float

class wmflib.requests.TimeoutHTTPAdapter(**kwargs: Any)[source]

Bases: requests.adapters.HTTPAdapter

Requests HTTP Adapter with default timeout for all requests.

Initialize the adapter with a default timeout, that can be overriden.

To override the default timeout of wmflib.requests.DEFAULT_REQUESTS_TIMEOUT` pass the timeout parameter when initializing an instance of this class.

Params:

As required by requests’s HTTPAdapter: https://2.python-requests.org/en/master/api/#requests.adapters.HTTPAdapter

send(request: requests.models.PreparedRequest, **kwargs: Any) requests.models.Response[source]

Override the send method to pass the default timeout if not set.

Params:

As required by requests’s HTTPAdapter: https://2.python-requests.org/en/master/api/#requests.adapters.HTTPAdapter.send The noqa is needed unless the exact signature is replicated.

wmflib.requests.http_session(name: str, *, timeout: float = 5.0, tries: int = 3, backoff: float = 1.0) requests.sessions.Session[source]

Return a new requests Session with User-Agent, default timeout and retry logic on failure already setup.

The returned session will retry any DELETE, GET, HEAD, OPTIONS, PUT, TRACE request that returns one of the following HTTP status code:

  • 429 Too Many Requests

  • 500 Internal Server Error

  • 502 Bad Gateway

  • 503 Service Unavailable

  • 504 Gateway Timeout

It will also retry any request that times out before the specified timeout. For non-idempotent HTTP methods the request will not be retried if the data has reached the server.

The retry interval between requests is determined by the backoff parameter, see below.

The timeout functionality is provided via the wmflib.requests.TimeoutHTTPAdapter and individual request can override the session timeout by specifying a timeout parameter.

Examples

With default parameters:

from wmflib.requests import http_session
session = http_session('AppName')  # The given name will be used in the User-Agent header, see below
# At this point the session can be used as a normal requests session

With customized parameters:

session = http_session('AppName', timeout=10.0, tries=5, backoff=2.0)
Parameters
  • name (str) –

    the name to use for the User-Agent header. It can be specified in the name/version format, if applicable. The resulting header will be set to:

    pywmflib/{version} {name} +https://wikitech.wikimedia.org/wiki/Python/Wmflib root@wikimedia.org
    

  • timeout (float) – the default timeout to use in all requests within this session, in seconds. Any request can override it passing the timeout parameter explicitely.

  • tries (int) – the total number of requests to perform before bailing out.

  • backoff (float) –

    the backoff factor to use, will generate a sleep between retries, in seconds, of:

    {backoff factor} * (2 ** ({number of total retries} - 1))
    

Returns

the pre-configured session.

Return type

requests.Session