MediaWiki  master
BaseDump Class Reference

Readahead helper for making large MediaWiki data dumps; reads in a previous XML dump to sequentially prefetch text records already normalized and decompressed. More...

Public Member Functions

 __construct ( $infile)
 
 close ()
 
 prefetch ( $page, $rev, $slot=SlotRecord::MAIN)
 Attempts to fetch the text of a particular page revision from the dump stream. More...
 

Protected Member Functions

 debug ( $str)
 

Protected Attributes

 $atEnd = false
 
 $atPageEnd = false
 
 $infiles = null
 
 $lastPage = 0
 
 $lastRev = 0
 
XMLReader $reader = null
 

Private Member Functions

 nextPage ()
 
 nextRev ()
 
 nextText ()
 
 nodeContents ()
 Shouldn't something like this be built-in to XMLReader? Fetches text contents of the current element, assuming no sub-elements or such scary things. More...
 
 skipTo ( $name, $parent='page')
 

Detailed Description

Readahead helper for making large MediaWiki data dumps; reads in a previous XML dump to sequentially prefetch text records already normalized and decompressed.

This can save load on the external database servers, hopefully.

Assumes that dumps will be recorded in the canonical order:

  • ascending by page_id
  • ascending by rev_id within each page
  • text contents are immutable and should not change once recorded, so the previous dump is a reliable source

Definition at line 44 of file BaseDump.php.

Constructor & Destructor Documentation

◆ __construct()

BaseDump::__construct (   $infile)

Definition at line 53 of file BaseDump.php.

Member Function Documentation

◆ close()

BaseDump::close ( )
Returns
null

Definition at line 217 of file BaseDump.php.

Referenced by nextPage(), nodeContents(), and skipTo().

◆ debug()

BaseDump::debug (   $str)
protected

Definition at line 113 of file BaseDump.php.

References wfDebug().

Referenced by prefetch(), and skipTo().

◆ nextPage()

BaseDump::nextPage ( )
private

Definition at line 119 of file BaseDump.php.

References close(), nodeContents(), and skipTo().

Referenced by prefetch().

◆ nextRev()

BaseDump::nextRev ( )
private

Definition at line 136 of file BaseDump.php.

References nodeContents(), and skipTo().

Referenced by prefetch().

◆ nextText()

BaseDump::nextText ( )
private
Returns
string

Definition at line 149 of file BaseDump.php.

References nodeContents(), and skipTo().

Referenced by prefetch().

◆ nodeContents()

BaseDump::nodeContents ( )
private

Shouldn't something like this be built-in to XMLReader? Fetches text contents of the current element, assuming no sub-elements or such scary things.

Returns
string

Definition at line 191 of file BaseDump.php.

References close().

Referenced by nextPage(), nextRev(), nextText(), and prefetch().

◆ prefetch()

BaseDump::prefetch (   $page,
  $rev,
  $slot = SlotRecord::MAIN 
)

Attempts to fetch the text of a particular page revision from the dump stream.

May return null if the page is unavailable.

Parameters
int$pageID number of page to read
int$revID number of revision to read
string$slotRole name of the slot to read
Returns
string|null

Definition at line 70 of file BaseDump.php.

References debug(), nextPage(), nextRev(), nextText(), nodeContents(), and skipTo().

◆ skipTo()

BaseDump::skipTo (   $name,
  $parent = 'page' 
)
private
Parameters
string$name
string$parent
Returns
bool|null

Definition at line 162 of file BaseDump.php.

References close(), and debug().

Referenced by nextPage(), nextRev(), nextText(), and prefetch().

Member Data Documentation

◆ $atEnd

BaseDump::$atEnd = false
protected

Definition at line 47 of file BaseDump.php.

◆ $atPageEnd

BaseDump::$atPageEnd = false
protected

Definition at line 48 of file BaseDump.php.

◆ $infiles

BaseDump::$infiles = null
protected

Definition at line 51 of file BaseDump.php.

◆ $lastPage

BaseDump::$lastPage = 0
protected

Definition at line 49 of file BaseDump.php.

◆ $lastRev

BaseDump::$lastRev = 0
protected

Definition at line 50 of file BaseDump.php.

◆ $reader

XMLReader BaseDump::$reader = null
protected

Definition at line 46 of file BaseDump.php.


The documentation for this class was generated from the following file: