throttle — Slow Down Wiki I/O#

Mechanics to slow down wiki read and/or write rate.

class throttle.ProcEntry(module_id, pid, time, site)[source]#

Bases: NamedTuple

ProcEntry namedtuple.

Create new instance of ProcEntry(module_id, pid, time, site)

Parameters:

  • module_id (str)

  • pid (int)

  • time (int)

  • site (str)

module_id: str#

Alias for field number 0

pid: int#

Alias for field number 1

site: str#

Alias for field number 3

time: int#

Alias for field number 2

class throttle.Throttle(site, *, mindelay=None, maxdelay=None, writedelay=None)[source]#

Bases: object

Control rate of access to wiki server.

Calling this object blocks the calling thread until at least 'delay' seconds have passed since the previous call.

Each Site initiates one Throttle object (site.throttle) to control the rate of access.

Parameters:

  • site (pywikibot.site.BaseSite | str) – site or sitename for this Throttle. If site is an empty string, it will not be written to the throttle.ctrl file.

  • mindelay (int | None) – The minimal delay, also used for read access

  • maxdelay (int | None) – The maximal delay

  • writedelay (int | float | None) – The write delay

checkMultiplicity()[source]#

Count running processes for site and set process_multiplicity.

Changed in version 7.0: process is not written to throttle.ctrl file if site is empty.

Return type:

None

checkdelay: int = 300#
drop()[source]#

Remove me from the list of running bot processes.

Return type:

None

property dropdelay#

Ignore processes that have not made a check in this many seconds.

Deprecated since version 8.4: use expiry instead.

expiry: int = 600#
getDelay(write=False)[source]#

Return the actual delay, accounting for multiple processes.

This value is the maximum wait between reads/writes, not taking into account of how much time has elapsed since the last access.

Parameters:

write (bool)

get_pid(module)[source]#

Get the global pid if the module is running multiple times.

Parameters:

module (str)

Return type:

int

lag(lagtime=None)[source]#

Seize the throttle lock due to server lag.

Usually the self.retry-after value from response_header of the last request if available which will be used for wait time. Otherwise lagtime from api maxlag is used. If neither self.retry_after nor lagtime is set, fallback to config.retry_wait.

If the lagtime is disproportionately high compared to self.retry_after value, the wait time will be increased.

This method is used by api.request. It will prevent any thread from accessing this site.

Parameters:

lagtime (float | None) – The time to wait for the next request which is the last maxlag time from api warning. This is only used as a fallback if self.retry_after isn’t set.

Return type:

None

property releasepid#

Free the process id after this many seconds.

Deprecated since version 8.4: use expiry instead.

setDelays(delay=None, writedelay=None, absolute=False)[source]#

Set the nominal delays in seconds. Defaults to config values.

Parameters:

absolute (bool)

Return type:

None

static wait(seconds)[source]#

Wait for seconds seconds.

Announce the delay if it exceeds a preset limit.

Parameters:

seconds (int | float)

Return type:

None

waittime(write=False)[source]#

Return waiting time in seconds.

The result is for a query that would be made right now.

Parameters:

write (bool)

throttle.pid: bool | int = False#

global process identifier

When the first Throttle is instantiated, it will set this variable to a positive integer, which will apply to all throttle objects created by this process.