MediaWiki  1.23.15
moveToExternal.php
Go to the documentation of this file.
1 <?php
24 define( 'REPORTING_INTERVAL', 1 );
25 
26 if ( !defined( 'MEDIAWIKI' ) ) {
27  require_once __DIR__ . '/../commandLine.inc';
28  require_once __DIR__ . '/../../includes/externalstore/ExternalStoreDB.php';
29  require_once 'resolveStubs.php';
30 
31  $fname = 'moveToExternal';
32 
33  if ( !isset( $args[0] ) ) {
34  print "Usage: php moveToExternal.php [-s <startid>] [-e <endid>] <cluster>\n";
35  exit;
36  }
37 
38  $cluster = $args[0];
39  $dbw = wfGetDB( DB_MASTER );
40 
41  if ( isset( $options['e'] ) ) {
42  $maxID = $options['e'];
43  } else {
44  $maxID = $dbw->selectField( 'text', 'MAX(old_id)', false, $fname );
45  }
46  $minID = isset( $options['s'] ) ? $options['s'] : 1;
47 
48  moveToExternal( $cluster, $maxID, $minID );
49 }
50 
51 function moveToExternal( $cluster, $maxID, $minID = 1 ) {
52  $fname = 'moveToExternal';
53  $dbw = wfGetDB( DB_MASTER );
54  $dbr = wfGetDB( DB_SLAVE );
55 
56  $count = $maxID - $minID + 1;
57  $blockSize = 1000;
58  $numBlocks = ceil( $count / $blockSize );
59  print "Moving text rows from $minID to $maxID to external storage\n";
60  $ext = new ExternalStoreDB;
61  $numMoved = 0;
62 
63  for ( $block = 0; $block < $numBlocks; $block++ ) {
64  $blockStart = $block * $blockSize + $minID;
65  $blockEnd = $blockStart + $blockSize - 1;
66 
67  if ( !( $block % REPORTING_INTERVAL ) ) {
68  print "oldid=$blockStart, moved=$numMoved\n";
70  }
71 
72  $res = $dbr->select( 'text', array( 'old_id', 'old_flags', 'old_text' ),
73  array(
74  "old_id BETWEEN $blockStart AND $blockEnd",
75  'old_flags NOT ' . $dbr->buildLike( $dbr->anyString(), 'external', $dbr->anyString() ),
76  ), $fname );
77  foreach ( $res as $row ) {
78  # Resolve stubs
79  $text = $row->old_text;
80  $id = $row->old_id;
81  if ( $row->old_flags === '' ) {
82  $flags = 'external';
83  } else {
84  $flags = "{$row->old_flags},external";
85  }
86 
87  if ( strpos( $flags, 'object' ) !== false ) {
88  $obj = unserialize( $text );
89  $className = strtolower( get_class( $obj ) );
90  if ( $className == 'historyblobstub' ) {
91  # resolveStub( $id, $row->old_text, $row->old_flags );
92  # $numStubs++;
93  continue;
94  } elseif ( $className == 'historyblobcurstub' ) {
95  $text = gzdeflate( $obj->getText() );
96  $flags = 'utf-8,gzip,external';
97  } elseif ( $className == 'concatenatedgziphistoryblob' ) {
98  // Do nothing
99  } else {
100  print "Warning: unrecognised object class \"$className\"\n";
101  continue;
102  }
103  } else {
104  $className = false;
105  }
106 
107  if ( strlen( $text ) < 100 && $className === false ) {
108  // Don't move tiny revisions
109  continue;
110  }
111 
112  # print "Storing " . strlen( $text ) . " bytes to $url\n";
113  # print "old_id=$id\n";
114 
115  $url = $ext->store( $cluster, $text );
116  if ( !$url ) {
117  print "Error writing to external storage\n";
118  exit;
119  }
120  $dbw->update( 'text',
121  array( 'old_flags' => $flags, 'old_text' => $url ),
122  array( 'old_id' => $id ), $fname );
123  $numMoved++;
124  }
125  }
126 }
DB_MASTER
const DB_MASTER
Definition: Defines.php:56
php
skin txt MediaWiki includes four core it has been set as the default in MediaWiki since the replacing Monobook it had been been the default skin since before being replaced by Vector largely rewritten in while keeping its appearance Several legacy skins were removed in the as the burden of supporting them became too heavy to bear Those in etc for skin dependent CSS etc for skin dependent JavaScript These can also be customised on a per user by etc This feature has led to a wide variety of user styles becoming that gallery is a good place to ending in php
Definition: skin.txt:62
ExternalStoreDB
DB accessable external objects.
Definition: ExternalStoreDB.php:31
wfGetDB
& wfGetDB( $db, $groups=array(), $wiki=false)
Get a Database object.
Definition: GlobalFunctions.php:3714
moveToExternal
if(!defined( 'MEDIAWIKI')) moveToExternal( $cluster, $maxID, $minID=1)
Definition: moveToExternal.php:51
$fname
if(!defined( 'MEDIAWIKI')) $fname
This file is not a valid entry point, perform no further processing unless MEDIAWIKI is defined.
Definition: Setup.php:35
$flags
it s the revision text itself In either if gzip is the revision text is gzipped $flags
Definition: hooks.txt:2124
$dbr
$dbr
Definition: testCompression.php:48
REPORTING_INTERVAL
const REPORTING_INTERVAL
Definition: moveToExternal.php:24
array
the array() calling protocol came about after MediaWiki 1.4rc1.
List of Api Query prop modules.
wfWaitForSlaves
wfWaitForSlaves( $maxLag=false, $wiki=false, $cluster=false)
Modern version of wfWaitForSlaves().
Definition: GlobalFunctions.php:3859
false
processing should stop and the error should be shown to the user * false
Definition: hooks.txt:188
$options
null means default in associative array with keys and values unescaped Should be merged with default with a value of false meaning to suppress the attribute in associative array with keys and values unescaped & $options
Definition: hooks.txt:1530
$count
$count
Definition: UtfNormalTest2.php:96
$args
if( $line===false) $args
Definition: cdb.php:62
DB_SLAVE
const DB_SLAVE
Definition: Defines.php:55
$ext
$ext
Definition: NoLocalSettings.php:34
as
This document is intended to provide useful advice for parties seeking to redistribute MediaWiki to end users It s targeted particularly at maintainers for Linux since it s been observed that distribution packages of MediaWiki often break We ve consistently had to recommend that users seeking support use official tarballs instead of their distribution s and this often solves whatever problem the user is having It would be nice if this could such as
Definition: distributors.txt:9
$res
$res
Definition: database.txt:21