Readahead helper for making large MediaWiki data dumps; reads in a previous XML dump to sequentially prefetch text records already normalized and decompressed. More...
Public Member Functions | |
| __construct ( $infile) | |
| close () | |
| debug ( $str) | |
| nextPage () | |
| nextRev () | |
| nextText () | |
| nodeContents () | |
| Shouldn't something like this be built-in to XMLReader? Fetches text contents of the current element, assuming no sub-elements or such scary things. | |
| prefetch ( $page, $rev) | |
| Attempts to fetch the text of a particular page revision from the dump stream. | |
| skipTo ( $name, $parent='page') | |
Protected Attributes | |
| $atEnd = false | |
| $atPageEnd = false | |
| $infiles = null | |
| $lastPage = 0 | |
| $lastRev = 0 | |
| XMLReader | $reader = null |
Detailed Description
Readahead helper for making large MediaWiki data dumps; reads in a previous XML dump to sequentially prefetch text records already normalized and decompressed.
This can save load on the external database servers, hopefully.
Assumes that dumps will be recorded in the canonical order:
- ascending by page_id
- ascending by rev_id within each page
- text contents are immutable and should not change once recorded, so the previous dump is a reliable source
Definition at line 42 of file BaseDump.php.
Constructor & Destructor Documentation
◆ __construct()
| BaseDump::__construct | ( | $infile | ) |
Definition at line 51 of file BaseDump.php.
References and().
Member Function Documentation
◆ close()
| BaseDump::close | ( | ) |
- Access:\n private
- Returns
- null
Definition at line 209 of file BaseDump.php.
Referenced by nextPage(), nodeContents(), and skipTo().
◆ debug()
| BaseDump::debug | ( | $str | ) |
Definition at line 97 of file BaseDump.php.
References wfDebug().
Referenced by prefetch(), and skipTo().
◆ nextPage()
| BaseDump::nextPage | ( | ) |
- Access:\n private
Definition at line 106 of file BaseDump.php.
References and(), close(), nodeContents(), and skipTo().
Referenced by prefetch().
◆ nextRev()
| BaseDump::nextRev | ( | ) |
- Access:\n private
Definition at line 126 of file BaseDump.php.
References and(), nodeContents(), and skipTo().
Referenced by prefetch().
◆ nextText()
| BaseDump::nextText | ( | ) |
- Access:\n private
- Returns
- string
Definition at line 140 of file BaseDump.php.
References and(), nodeContents(), and skipTo().
Referenced by prefetch().
◆ nodeContents()
| BaseDump::nodeContents | ( | ) |
Shouldn't something like this be built-in to XMLReader? Fetches text contents of the current element, assuming no sub-elements or such scary things.
- Returns
- string
- Access:\n private
Definition at line 182 of file BaseDump.php.
References $buffer, and close().
Referenced by nextPage(), nextRev(), and nextText().
◆ prefetch()
| BaseDump::prefetch | ( | $page, | |
| $rev ) |
Attempts to fetch the text of a particular page revision from the dump stream.
May return null if the page is unavailable.
- Parameters
-
int $page ID number of page to read int $rev ID number of revision to read
- Returns
- string|null
Definition at line 67 of file BaseDump.php.
References $rev, and(), debug(), nextPage(), nextRev(), and nextText().
◆ skipTo()
| BaseDump::skipTo | ( | $name, | |
| $parent = 'page' ) |
- Access:\n private
- Parameters
-
string $name string $parent
- Returns
- bool|null
Definition at line 152 of file BaseDump.php.
References $parent, close(), and debug().
Referenced by nextPage(), nextRev(), and nextText().
Member Data Documentation
◆ $atEnd
|
protected |
Definition at line 45 of file BaseDump.php.
◆ $atPageEnd
|
protected |
Definition at line 46 of file BaseDump.php.
◆ $infiles
|
protected |
Definition at line 49 of file BaseDump.php.
◆ $lastPage
|
protected |
Definition at line 47 of file BaseDump.php.
◆ $lastRev
|
protected |
Definition at line 48 of file BaseDump.php.
◆ $reader
|
protected |
Definition at line 44 of file BaseDump.php.
The documentation for this class was generated from the following file:
- includes/export/BaseDump.php
