MediaWiki  1.23.2
JobQueueAggregatorRedis.php
Go to the documentation of this file.
1 <?php
33  protected $redisPool;
34 
36  protected $servers;
37 
48  protected function __construct( array $params ) {
49  parent::__construct( $params );
50  $this->servers = isset( $params['redisServers'] )
51  ? $params['redisServers']
52  : array( $params['redisServer'] ); // b/c
53  $this->redisPool = RedisConnectionPool::singleton( $params['redisConfig'] );
54  }
55 
56  protected function doNotifyQueueEmpty( $wiki, $type ) {
57  $conn = $this->getConnection();
58  if ( !$conn ) {
59  return false;
60  }
61  try {
62  $conn->hDel( $this->getReadyQueueKey(), $this->encQueueName( $type, $wiki ) );
63 
64  return true;
65  } catch ( RedisException $e ) {
66  $this->handleException( $conn, $e );
67 
68  return false;
69  }
70  }
71 
72  protected function doNotifyQueueNonEmpty( $wiki, $type ) {
73  $conn = $this->getConnection();
74  if ( !$conn ) {
75  return false;
76  }
77  try {
78  $conn->hSet( $this->getReadyQueueKey(), $this->encQueueName( $type, $wiki ), time() );
79 
80  return true;
81  } catch ( RedisException $e ) {
82  $this->handleException( $conn, $e );
83 
84  return false;
85  }
86  }
87 
88  protected function doGetAllReadyWikiQueues() {
89  $conn = $this->getConnection();
90  if ( !$conn ) {
91  return array();
92  }
93  try {
94  $conn->multi( Redis::PIPELINE );
95  $conn->exists( $this->getReadyQueueKey() );
96  $conn->hGetAll( $this->getReadyQueueKey() );
97  list( $exists, $map ) = $conn->exec();
98 
99  if ( $exists ) { // cache hit
100  $pendingDBs = array(); // (type => list of wikis)
101  foreach ( $map as $key => $time ) {
102  list( $type, $wiki ) = $this->dencQueueName( $key );
103  $pendingDBs[$type][] = $wiki;
104  }
105  } else { // cache miss
106  // Avoid duplicated effort
107  $rand = wfRandomString( 32 );
108  $conn->multi( Redis::MULTI );
109  $conn->setex( "{$rand}:lock", 3600, 1 );
110  $conn->renamenx( "{$rand}:lock", $this->getReadyQueueKey() . ":lock" );
111  if ( $conn->exec() !== array( true, true ) ) { // lock
112  $conn->delete( "{$rand}:lock" );
113  return array(); // already in progress
114  }
115 
116  $pendingDBs = $this->findPendingWikiQueues(); // (type => list of wikis)
117 
118  $conn->delete( $this->getReadyQueueKey() . ":lock" ); // unlock
119 
120  $now = time();
121  $map = array();
122  foreach ( $pendingDBs as $type => $wikis ) {
123  foreach ( $wikis as $wiki ) {
124  $map[$this->encQueueName( $type, $wiki )] = $now;
125  }
126  }
127  $conn->hMSet( $this->getReadyQueueKey(), $map );
128  }
129 
130  return $pendingDBs;
131  } catch ( RedisException $e ) {
132  $this->handleException( $conn, $e );
133 
134  return array();
135  }
136  }
137 
138  protected function doPurge() {
139  $conn = $this->getConnection();
140  if ( !$conn ) {
141  return false;
142  }
143  try {
144  $conn->delete( $this->getReadyQueueKey() );
145  } catch ( RedisException $e ) {
146  $this->handleException( $conn, $e );
147 
148  return false;
149  }
150 
151  return true;
152  }
153 
160  protected function getConnection() {
161  $conn = false;
162  foreach ( $this->servers as $server ) {
163  $conn = $this->redisPool->getConnection( $server );
164  if ( $conn ) {
165  break;
166  }
167  }
168 
169  return $conn;
170  }
171 
177  protected function handleException( RedisConnRef $conn, $e ) {
178  $this->redisPool->handleError( $conn, $e );
179  }
180 
184  private function getReadyQueueKey() {
185  return "jobqueue:aggregator:h-ready-queues:v1"; // global
186  }
187 
193  private function encQueueName( $type, $wiki ) {
194  return rawurlencode( $type ) . '/' . rawurlencode( $wiki );
195  }
196 
201  private function dencQueueName( $name ) {
202  list( $type, $wiki ) = explode( '/', $name, 2 );
203 
204  return array( rawurldecode( $type ), rawurldecode( $wiki ) );
205  }
206 }
$time
see documentation in includes Linker php for Linker::makeImageLink & $time
Definition: hooks.txt:1358
RedisConnectionPool\singleton
static singleton(array $options)
Definition: RedisConnectionPool.php:115
JobQueueAggregatorRedis\doNotifyQueueEmpty
doNotifyQueueEmpty( $wiki, $type)
Definition: JobQueueAggregatorRedis.php:54
php
skin txt MediaWiki includes four core it has been set as the default in MediaWiki since the replacing Monobook it had been been the default skin since before being replaced by Vector largely rewritten in while keeping its appearance Several legacy skins were removed in the as the burden of supporting them became too heavy to bear Those in etc for skin dependent CSS etc for skin dependent JavaScript These can also be customised on a per user by etc This feature has led to a wide variety of user styles becoming that gallery is a good place to ending in php
Definition: skin.txt:62
JobQueueAggregatorRedis\doGetAllReadyWikiQueues
doGetAllReadyWikiQueues()
Definition: JobQueueAggregatorRedis.php:86
servers
storage can be distributed across multiple servers
Definition: memcached.txt:33
$params
$params
Definition: styleTest.css.php:40
JobQueueAggregatorRedis\doPurge
doPurge()
Definition: JobQueueAggregatorRedis.php:136
JobQueueAggregator
Class to handle tracking information about all queues.
Definition: JobQueueAggregator.php:30
JobQueueAggregatorRedis\dencQueueName
dencQueueName( $name)
Definition: JobQueueAggregatorRedis.php:199
JobQueueAggregatorRedis\encQueueName
encQueueName( $type, $wiki)
Definition: JobQueueAggregatorRedis.php:191
JobQueueAggregatorRedis\$servers
array $servers
List of Redis server addresses *.
Definition: JobQueueAggregatorRedis.php:34
array
the array() calling protocol came about after MediaWiki 1.4rc1.
List of Api Query prop modules.
list
deferred txt A few of the database updates required by various functions here can be deferred until after the result page is displayed to the user For updating the view updating the linked to tables after a etc PHP does not yet have any way to tell the server to actually return and disconnect while still running these but it might have such a feature in the future We handle these by creating a deferred update object and putting those objects on a global list
Definition: deferred.txt:11
JobQueueAggregatorRedis
Class to handle tracking information about all queues using PhpRedis.
Definition: JobQueueAggregatorRedis.php:31
RedisConnectionPool
Helper class to manage Redis connections.
Definition: RedisConnectionPool.php:38
JobQueueAggregatorRedis\$redisPool
RedisConnectionPool $redisPool
Definition: JobQueueAggregatorRedis.php:32
JobQueueAggregatorRedis\__construct
__construct(array $params)
@params include:
Definition: JobQueueAggregatorRedis.php:46
$name
Allows to change the fields on the form that will be generated $name
Definition: hooks.txt:336
JobQueueAggregatorRedis\getConnection
getConnection()
Get a connection to the server that handles all sub-queues for this queue.
Definition: JobQueueAggregatorRedis.php:158
JobQueueAggregatorRedis\handleException
handleException(RedisConnRef $conn, $e)
Definition: JobQueueAggregatorRedis.php:175
JobQueueAggregator\findPendingWikiQueues
findPendingWikiQueues()
Get all databases that have a pending job.
Definition: JobQueueAggregator.php:150
as
This document is intended to provide useful advice for parties seeking to redistribute MediaWiki to end users It s targeted particularly at maintainers for Linux since it s been observed that distribution packages of MediaWiki often break We ve consistently had to recommend that users seeking support use official tarballs instead of their distribution s and this often solves whatever problem the user is having It would be nice if this could such as
Definition: distributors.txt:9
RedisConnRef
Helper class to handle automatically marking connectons as reusable (via RAII pattern)
Definition: RedisConnectionPool.php:348
JobQueueAggregatorRedis\doNotifyQueueNonEmpty
doNotifyQueueNonEmpty( $wiki, $type)
Definition: JobQueueAggregatorRedis.php:70
$e
if( $useReadline) $e
Definition: eval.php:66
JobQueueAggregatorRedis\getReadyQueueKey
getReadyQueueKey()
Definition: JobQueueAggregatorRedis.php:182
wfRandomString
wfRandomString( $length=32)
Get a random string containing a number of pseudo-random hex characters.
Definition: GlobalFunctions.php:300
$type
$type
Definition: testCompression.php:46