MediaWiki  1.23.2
cleanupUploadStash.php
Go to the documentation of this file.
1 <?php
28 require_once __DIR__ . '/Maintenance.php';
29 
37 
38  public function __construct() {
39  parent::__construct();
40  $this->mDescription = "Clean up abandoned files in temporary uploaded file stash";
41  $this->setBatchSize( 50 );
42  }
43 
44  public function execute() {
45  global $wgUploadStashMaxAge;
46 
47  $repo = RepoGroup::singleton()->getLocalRepo();
48  $tempRepo = $repo->getTempRepo();
49 
50  $dbr = $repo->getSlaveDb();
51 
52  // how far back should this look for files to delete?
53  $cutoff = time() - $wgUploadStashMaxAge;
54 
55  $this->output( "Getting list of files to clean up...\n" );
56  $res = $dbr->select(
57  'uploadstash',
58  'us_key',
59  'us_timestamp < ' . $dbr->addQuotes( $dbr->timestamp( $cutoff ) ),
60  __METHOD__
61  );
62 
63  // Delete all registered stash files...
64  if ( $res->numRows() == 0 ) {
65  $this->output( "No stashed files to cleanup according to the DB.\n" );
66  } else {
67  // finish the read before starting writes.
68  $keys = array();
69  foreach ( $res as $row ) {
70  array_push( $keys, $row->us_key );
71  }
72 
73  $this->output( 'Removing ' . count( $keys ) . " file(s)...\n" );
74  // this could be done some other, more direct/efficient way, but using
75  // UploadStash's own methods means it's less likely to fall accidentally
76  // out-of-date someday
77  $stash = new UploadStash( $repo );
78 
79  $i = 0;
80  foreach ( $keys as $key ) {
81  $i++;
82  try {
83  $stash->getFile( $key, true );
84  $stash->removeFileNoAuth( $key );
85  } catch ( UploadStashException $ex ) {
86  $type = get_class( $ex );
87  $this->output( "Failed removing stashed upload with key: $key ($type)\n" );
88  }
89  if ( $i % 100 == 0 ) {
90  $this->output( "$i\n" );
91  }
92  }
93  $this->output( "$i done\n" );
94  }
95 
96  // Delete all the corresponding thumbnails...
97  $dir = $tempRepo->getZonePath( 'thumb' );
98  $iterator = $tempRepo->getBackend()->getFileList( array( 'dir' => $dir, 'adviseStat' => 1 ) );
99  $this->output( "Deleting old thumbnails...\n" );
100  $i = 0;
101  $batch = array(); // operation batch
102  foreach ( $iterator as $file ) {
103  if ( wfTimestamp( TS_UNIX, $tempRepo->getFileTimestamp( "$dir/$file" ) ) < $cutoff ) {
104  $batch[] = array( 'op' => 'delete', 'src' => "$dir/$file" );
105  if ( count( $batch ) >= $this->mBatchSize ) {
106  $this->doOperations( $tempRepo, $batch );
107  $i += count( $batch );
108  $batch = array();
109  $this->output( "$i\n" );
110  }
111  }
112  }
113  if ( count( $batch ) ) {
114  $this->doOperations( $tempRepo, $batch );
115  $i += count( $batch );
116  }
117  $this->output( "$i done\n" );
118 
119  // Apparently lots of stash files are not registered in the DB...
120  $dir = $tempRepo->getZonePath( 'public' );
121  $iterator = $tempRepo->getBackend()->getFileList( array( 'dir' => $dir, 'adviseStat' => 1 ) );
122  $this->output( "Deleting orphaned temp files...\n" );
123  if ( strpos( $dir, '/local-temp' ) === false ) { // sanity check
124  $this->error( "Temp repo is not using the temp container.", 1 ); // die
125  }
126  $i = 0;
127  $batch = array(); // operation batch
128  foreach ( $iterator as $file ) {
129  if ( wfTimestamp( TS_UNIX, $tempRepo->getFileTimestamp( "$dir/$file" ) ) < $cutoff ) {
130  $batch[] = array( 'op' => 'delete', 'src' => "$dir/$file" );
131  if ( count( $batch ) >= $this->mBatchSize ) {
132  $this->doOperations( $tempRepo, $batch );
133  $i += count( $batch );
134  $batch = array();
135  $this->output( "$i\n" );
136  }
137  }
138  }
139  if ( count( $batch ) ) {
140  $this->doOperations( $tempRepo, $batch );
141  $i += count( $batch );
142  }
143  $this->output( "$i done\n" );
144  }
145 
146  protected function doOperations( FileRepo $tempRepo, array $ops ) {
147  $status = $tempRepo->getBackend()->doQuickOperations( $ops );
148  if ( !$status->isOK() ) {
149  $this->error( print_r( $status->getErrorsArray(), true ) );
150  }
151  }
152 }
153 
154 $maintClass = "UploadStashCleanup";
155 require_once RUN_MAINTENANCE_IF_MAIN;
RepoGroup\singleton
static singleton()
Get a RepoGroup instance.
Definition: RepoGroup.php:53
php
skin txt MediaWiki includes four core it has been set as the default in MediaWiki since the replacing Monobook it had been been the default skin since before being replaced by Vector largely rewritten in while keeping its appearance Several legacy skins were removed in the as the burden of supporting them became too heavy to bear Those in etc for skin dependent CSS etc for skin dependent JavaScript These can also be customised on a per user by etc This feature has led to a wide variety of user styles becoming that gallery is a good place to ending in php
Definition: skin.txt:62
UploadStashCleanup\execute
execute()
Do the actual work.
Definition: cleanupUploadStash.php:44
$maintClass
$maintClass
Definition: cleanupUploadStash.php:154
wfTimestamp
wfTimestamp( $outputtype=TS_UNIX, $ts=0)
Get a timestamp string in one of various formats.
Definition: GlobalFunctions.php:2483
UploadStash
UploadStash is intended to accomplish a few things:
Definition: UploadStash.php:44
RUN_MAINTENANCE_IF_MAIN
require_once RUN_MAINTENANCE_IF_MAIN
Definition: maintenance.txt:50
Maintenance
Abstract maintenance class for quickly writing and churning out maintenance scripts with minimal effo...
Definition: maintenance.txt:39
UploadStashCleanup\doOperations
doOperations(FileRepo $tempRepo, array $ops)
Definition: cleanupUploadStash.php:146
true
null means default in associative array with keys and values unescaped Should be merged with default with a value of false meaning to suppress the attribute in associative array with keys and values unescaped noclasses just before the function returns a value If you return true
Definition: hooks.txt:1530
$dbr
$dbr
Definition: testCompression.php:48
FileRepo
Base class for file repositories.
Definition: FileRepo.php:37
UploadStashCleanup
Maintenance script to remove old or broken uploads from temporary uploaded file storage and clean up ...
Definition: cleanupUploadStash.php:36
array
the array() calling protocol came about after MediaWiki 1.4rc1.
List of Api Query prop modules.
global
when a variable name is used in a it is silently declared as a new masking the global
Definition: design.txt:93
FileRepo\getBackend
getBackend()
Get the file backend instance.
Definition: FileRepo.php:190
$file
if(PHP_SAPI !='cli') $file
Definition: UtfNormalTest2.php:30
UploadStashCleanup\__construct
__construct()
Default constructor.
Definition: cleanupUploadStash.php:38
$dir
if(count( $args)==0) $dir
Definition: importImages.php:49
TS_UNIX
const TS_UNIX
Unix time - the number of seconds since 1970-01-01 00:00:00 UTC.
Definition: GlobalFunctions.php:2426
as
This document is intended to provide useful advice for parties seeking to redistribute MediaWiki to end users It s targeted particularly at maintainers for Linux since it s been observed that distribution packages of MediaWiki often break We ve consistently had to recommend that users seeking support use official tarballs instead of their distribution s and this often solves whatever problem the user is having It would be nice if this could such as
Definition: distributors.txt:9
$keys
$keys
Definition: testCompression.php:63
$batch
$batch
Definition: linkcache.txt:23
Maintenance\error
error( $err, $die=0)
Throw an error to the user.
Definition: Maintenance.php:333
Maintenance\output
output( $out, $channel=null)
Throw some output to the user.
Definition: Maintenance.php:314
$res
$res
Definition: database.txt:21
Maintenance\setBatchSize
setBatchSize( $s=0)
Set the batch size.
Definition: Maintenance.php:254
UploadStashException
Definition: UploadStash.php:679
$type
$type
Definition: testCompression.php:46