Wikibase
MediaWiki Wikibase extension
Loading...
Searching...
No Matches
Wikibase\Repo\Dumpers\DumpGenerator Class Reference

DumpGenerator generates a dump of a given set of entities, excluding redirects. More...

+ Inheritance diagram for Wikibase\Repo\Dumpers\DumpGenerator:

Public Member Functions

 __construct ( $out, EntityPrefetcher $entityPrefetcher)
 
 setLimit ( $limit)
 Set maximum number of entities produced.
 
 setBatchSize ( $batchSize)
 Sets the batch size for processing.
 
 setProgressReporter (MessageReporter $progressReporter)
 
 setExceptionHandler (ExceptionHandler $exceptionHandler)
 
 setShardingFilter ( $shardingFactor, $shard)
 Set the sharding factor and desired shard.
 
 setEntityTypesFilter ( $types)
 Set the entity types to be included in the output.
 
 generateDump (EntityIdPager $idPager)
 Generates a dump, writing to the file handle provided to the constructor.
 

Protected Member Functions

 writeToDump ( $data)
 Writers the given string to the output provided to the constructor.
 
 preDump ()
 Do something before dumping data.
 
 postDump ()
 Do something after dumping data.
 
 preBatchDump ( $entities)
 Do something before dumping a batch of entities.
 
 preEntityDump ( $dumpCount)
 Do something before dumping entity.
 
 postEntityDump ( $dumpCount)
 Do something after dumping entity.
 
 generateDumpForEntityId (EntityId $entityId)
 Produce dump data for specific entity.
 

Protected Attributes

 $batchSize = 100
 
 $out
 
 $shardingFactor = 1
 
 $shard = 0
 
 $progressReporter
 
 $exceptionHandler
 
 $entityPrefetcher
 
 $entityTypes
 
 $limit = 0
 

Private Member Functions

 idMatchesFilters (EntityId $entityId)
 
 idMatchesShard (EntityId $entityId)
 
 idMatchesType (EntityId $entityId)
 
 dumpEntities (array $entityIds, &$dumpCount)
 Dump list of entities.
 

Detailed Description

DumpGenerator generates a dump of a given set of entities, excluding redirects.

Author
Daniel Kinzler

Constructor & Destructor Documentation

◆ __construct()

Wikibase\Repo\Dumpers\DumpGenerator::__construct (   $out,
EntityPrefetcher  $entityPrefetcher 
)
Parameters
resource$out
EntityPrefetcher$entityPrefetcher
Exceptions
InvalidArgumentException

Member Function Documentation

◆ dumpEntities()

Wikibase\Repo\Dumpers\DumpGenerator::dumpEntities ( array  $entityIds,
$dumpCount 
)
private

Dump list of entities.

Parameters
EntityId[]$entityIds
int&$dumpCountThe number of entities already dumped (will be updated).

◆ generateDump()

Wikibase\Repo\Dumpers\DumpGenerator::generateDump ( EntityIdPager  $idPager)

Generates a dump, writing to the file handle provided to the constructor.

Parameters
EntityIdPager$idPager

◆ generateDumpForEntityId()

Wikibase\Repo\Dumpers\DumpGenerator::generateDumpForEntityId ( EntityId  $entityId)
abstractprotected

Produce dump data for specific entity.

Parameters
EntityId$entityId
Exceptions
EntityLookupException
StorageException
Returns
string|null

Reimplemented in Wikibase\Repo\Dumpers\JsonDumpGenerator, and Wikibase\Repo\Dumpers\RdfDumpGenerator.

◆ idMatchesFilters()

Wikibase\Repo\Dumpers\DumpGenerator::idMatchesFilters ( EntityId  $entityId)
private

◆ idMatchesShard()

Wikibase\Repo\Dumpers\DumpGenerator::idMatchesShard ( EntityId  $entityId)
private

◆ idMatchesType()

Wikibase\Repo\Dumpers\DumpGenerator::idMatchesType ( EntityId  $entityId)
private

◆ postDump()

Wikibase\Repo\Dumpers\DumpGenerator::postDump ( )
protected

Do something after dumping data.

Reimplemented in Wikibase\Repo\Dumpers\JsonDumpGenerator, and Wikibase\Repo\Dumpers\RdfDumpGenerator.

◆ postEntityDump()

Wikibase\Repo\Dumpers\DumpGenerator::postEntityDump (   $dumpCount)
protected

Do something after dumping entity.

Parameters
int$dumpCount

◆ preBatchDump()

Wikibase\Repo\Dumpers\DumpGenerator::preBatchDump (   $entities)
protected

Do something before dumping a batch of entities.

Parameters
EntityId[]$entities

Reimplemented in Wikibase\Repo\Dumpers\RdfDumpGenerator.

◆ preDump()

Wikibase\Repo\Dumpers\DumpGenerator::preDump ( )
protected

Do something before dumping data.

Reimplemented in Wikibase\Repo\Dumpers\JsonDumpGenerator, and Wikibase\Repo\Dumpers\RdfDumpGenerator.

◆ preEntityDump()

Wikibase\Repo\Dumpers\DumpGenerator::preEntityDump (   $dumpCount)
protected

Do something before dumping entity.

Parameters
int$dumpCount

Reimplemented in Wikibase\Repo\Dumpers\JsonDumpGenerator.

◆ setBatchSize()

Wikibase\Repo\Dumpers\DumpGenerator::setBatchSize (   $batchSize)

Sets the batch size for processing.

The batch size is used as the limit when listing IDs via the EntityIdPager::getNextBatchOfIds() method, and also controls the interval of progress reports.

Parameters
int$batchSize
Exceptions
InvalidArgumentException

◆ setEntityTypesFilter()

Wikibase\Repo\Dumpers\DumpGenerator::setEntityTypesFilter (   $types)

Set the entity types to be included in the output.

Parameters
string[] | null$typesThe desired types (use null for any type).

◆ setExceptionHandler()

Wikibase\Repo\Dumpers\DumpGenerator::setExceptionHandler ( ExceptionHandler  $exceptionHandler)

◆ setLimit()

Wikibase\Repo\Dumpers\DumpGenerator::setLimit (   $limit)

Set maximum number of entities produced.

Parameters
int$limit

◆ setProgressReporter()

Wikibase\Repo\Dumpers\DumpGenerator::setProgressReporter ( MessageReporter  $progressReporter)

◆ setShardingFilter()

Wikibase\Repo\Dumpers\DumpGenerator::setShardingFilter (   $shardingFactor,
  $shard 
)

Set the sharding factor and desired shard.

For instance, to generate four dumps in parallel, use setShardingFilter( 4, 0 ) for the first dump, setShardingFilter( 4, 1 ) for the second dump, etc.

Parameters
int$shardingFactor
int$shard
Exceptions
InvalidArgumentException

◆ writeToDump()

Wikibase\Repo\Dumpers\DumpGenerator::writeToDump (   $data)
protected

Writers the given string to the output provided to the constructor.

Parameters
string$data

Member Data Documentation

◆ $batchSize

Wikibase\Repo\Dumpers\DumpGenerator::$batchSize = 100
protected

◆ $entityPrefetcher

Wikibase\Repo\Dumpers\DumpGenerator::$entityPrefetcher
protected

◆ $entityTypes

Wikibase\Repo\Dumpers\DumpGenerator::$entityTypes
protected

◆ $exceptionHandler

Wikibase\Repo\Dumpers\DumpGenerator::$exceptionHandler
protected

◆ $limit

Wikibase\Repo\Dumpers\DumpGenerator::$limit = 0
protected

◆ $out

Wikibase\Repo\Dumpers\DumpGenerator::$out
protected

◆ $progressReporter

Wikibase\Repo\Dumpers\DumpGenerator::$progressReporter
protected

◆ $shard

Wikibase\Repo\Dumpers\DumpGenerator::$shard = 0
protected

◆ $shardingFactor

Wikibase\Repo\Dumpers\DumpGenerator::$shardingFactor = 1
protected

The documentation for this class was generated from the following file: