MediaWiki master
|
Proxy backend that manages file layout rewriting for FileRepo. More...
Inherits Wikimedia\FileBackend\FileBackend.
Public Member Functions | |
__construct (array $config) | |
Create a new backend instance from configuration. | |
clearCache (array $paths=null) | |
Invalidate any in-process file stat and property cache. | |
concatenate (array $params) | |
Concatenate a list of storage files into a single file system file. | |
directoryExists (array $params) | |
Check if a directory exists at a given storage path. | |
fileExists (array $params) | |
Check if a file exists at a storage path in the backend. | |
getBackendPath ( $path, $latest=true) | |
Translate a legacy "title" path to its "sha1" counterpart. | |
getBackendPaths (array $paths, $latest=true) | |
Translate legacy "title" paths to their "sha1" counterparts. | |
getDirectoryList (array $params) | |
Get an iterator to list all directories under a storage directory. | |
getFeatures () | |
Get the a bitfield of extra features supported by the backend medium. | |
getFileContentsMulti (array $params) | |
Like getFileContents() except it takes an array of storage paths and returns an order preserved map of storage paths to their content. | |
getFileHttpUrl (array $params) | |
Return an HTTP URL to a given file that requires no authentication to use. | |
getFileList (array $params) | |
Get an iterator to list all stored files under a storage directory. | |
getFileProps (array $params) | |
Get the properties of the content of the file at a storage path in the backend. | |
getFileSha1Base36 (array $params) | |
Get a SHA-1 hash of the content of the file at a storage path in the backend. | |
getFileSize (array $params) | |
Get the size (bytes) of a file at a storage path in the backend. | |
getFileStat (array $params) | |
Get quick information about a file at a storage path in the backend. | |
getFileTimestamp (array $params) | |
Get the last-modified timestamp of the file at a storage path. | |
getFileXAttributes (array $params) | |
Get metadata about a file at a storage path in the backend. | |
getInternalBackend () | |
Get the underlying FileBackend that is being wrapped. | |
getLocalCopyMulti (array $params) | |
Like getLocalCopy() except it takes an array of storage paths and yields an order preserved-map of storage paths to temporary local file copies. | |
getLocalReferenceMulti (array $params) | |
Like getLocalReference() except it takes an array of storage paths and yields an order-preserved map of storage paths to temporary local file copies. | |
getPathForSHA1 ( $sha1) | |
Get the ultimate original storage path for a file. | |
getScopedLocksForOps (array $ops, StatusValue $status) | |
Get an array of scoped locks needed for a batch of file operations. | |
preloadCache (array $paths) | |
Preload persistent file stat cache and property cache into in-process cache. | |
preloadFileStat (array $params) | |
Preload file stat information (concurrently if possible) into in-process cache. | |
streamFile (array $params) | |
Stream the content of the file at a storage path in the backend. | |
Public Member Functions inherited from Wikimedia\FileBackend\FileBackend | |
clean (array $params) | |
Delete a storage directory if it is empty. | |
copy (array $params, array $opts=[]) | |
Performs a single copy operation. | |
create (array $params, array $opts=[]) | |
Performs a single create operation. | |
delete (array $params, array $opts=[]) | |
Performs a single delete operation. | |
describe (array $params, array $opts=[]) | |
Performs a single describe operation. | |
doOperation (array $op, array $opts=[]) | |
Same as doOperations() except it takes a single operation. | |
doOperations (array $ops, array $opts=[]) | |
This is the main entry point into the backend for write operations. | |
doQuickOperation (array $op, array $opts=[]) | |
Same as doQuickOperations() except it takes a single operation. | |
doQuickOperations (array $ops, array $opts=[]) | |
Perform a set of independent file operations on some files. | |
getContainerStoragePath ( $container) | |
Get the storage path for the given container for this backend. | |
getDomainId () | |
Get the domain identifier used for this backend (possibly empty). | |
getFileContents (array $params) | |
Get the contents of a file at a storage path in the backend. | |
getLocalCopy (array $params) | |
Get a local copy on disk of the file at a storage path in the backend. | |
getLocalReference (array $params) | |
Returns a file system file, identical in content to the file at a storage path. | |
getName () | |
Get the unique backend name. | |
getReadOnlyReason () | |
Get an explanatory message if this backend is read-only. | |
getRootStoragePath () | |
Get the root storage path of this backend. | |
getScopedFileLocks (array $paths, $type, StatusValue $status, $timeout=0) | |
Lock the files at the given storage paths in the backend. | |
getStreamerOptions () | |
Return options for use with HTTPFileStreamer. | |
getTopDirectoryList (array $params) | |
Same as FileBackend::getDirectoryList() except only lists directories that are immediately under the given directory. | |
getTopFileList (array $params) | |
Same as FileBackend::getFileList() except only lists files that are immediately under the given directory. | |
getWikiId () | |
Alias to getDomainId() | |
hasFeatures ( $bitfield) | |
Check if the backend medium supports a field of extra features. | |
isReadOnly () | |
Check if this backend is read-only. | |
lockFiles (array $paths, $type, $timeout=0) | |
Lock the files at the given storage paths in the backend. | |
move (array $params, array $opts=[]) | |
Performs a single move operation. | |
prepare (array $params) | |
Prepare a storage directory for usage. | |
publish (array $params) | |
Remove measures to block web access to a storage directory and the container it belongs to. | |
quickCopy (array $params, array $opts=[]) | |
Performs a single quick copy operation. | |
quickCreate (array $params, array $opts=[]) | |
Performs a single quick create operation. | |
quickDelete (array $params, array $opts=[]) | |
Performs a single quick delete operation. | |
quickDescribe (array $params, array $opts=[]) | |
Performs a single quick describe operation. | |
quickMove (array $params, array $opts=[]) | |
Performs a single quick move operation. | |
quickStore (array $params, array $opts=[]) | |
Performs a single quick store operation. | |
secure (array $params) | |
Take measures to block web access to a storage directory and the container it belongs to. | |
setLogger (LoggerInterface $logger) | |
store (array $params, array $opts=[]) | |
Performs a single store operation. | |
unlockFiles (array $paths, $type) | |
Unlock the files at the given storage paths in the backend. | |
Protected Member Functions | |
doClean (array $params) | |
doOperationsInternal (array $ops, array $opts) | |
doPrepare (array $params) | |
doPublish (array $params) | |
doQuickOperationsInternal (array $ops, array $opts) | |
doSecure (array $params) | |
getDB ( $index) | |
Get a connection to the repo file registry DB. | |
mungeOpPaths (array $ops) | |
Translate legacy "title" source paths to their "sha1" counterparts. | |
translateArrayResults ( $function, array $params) | |
Translates paths when the backend function returns results keyed by paths. | |
translateSrcParams ( $function, array $params) | |
Translates paths found in the "src" or "srcs" keys of a params array. | |
Protected Member Functions inherited from Wikimedia\FileBackend\FileBackend | |
header ( $header) | |
newStatus (... $args) | |
Yields the result of the status wrapper callback on either: | |
resetOutputBuffer () | |
resolveFSFileObjects (array $ops) | |
Convert FSFile 'src' paths to string paths (with an 'srcRef' field set to the FSFile) | |
scopedProfileSection ( $section) | |
wrapStatus (StatusValue $sv) | |
Protected Attributes | |
FileBackend | $backend |
Closure | $dbHandleFunc |
IDatabase[] | $dbs |
string | $repoName |
MapCacheLRU | $resolvedPathCache |
Protected Attributes inherited from Wikimedia\FileBackend\FileBackend | |
int | $concurrency |
How many operations can be done in parallel. | |
string | $domainId |
Unique domain name. | |
LockManager | $lockManager |
LoggerInterface | $logger |
string | $name |
Unique backend name. | |
string | $parallelize |
When to do operations in parallel. | |
callable null | $profiler |
string | $readOnly |
Read-only explanation message. | |
callable null | $statusWrapper |
array | $streamerOptions |
Option map for use with HTTPFileStreamer. | |
TempFSFileFactory | $tmpFileFactory |
Additional Inherited Members | |
Static Public Member Functions inherited from Wikimedia\FileBackend\FileBackend | |
static | extensionFromPath ( $path, $case='lowercase') |
Get the final extension from a storage or FS path. | |
static | isPathTraversalFree ( $path) |
Check if a relative path has no directory traversals. | |
static | isStoragePath ( $path) |
Check if a given path is a "mwstore://" path. | |
static | makeContentDisposition ( $type, $filename='') |
Build a Content-Disposition header value per RFC 6266. | |
static | normalizeStoragePath ( $storagePath) |
Normalize a storage path by cleaning up directory separators. | |
static | parentStoragePath ( $storagePath) |
Get the parent storage directory of a storage path. | |
static | resetOutputBufferTheDefaultWay () |
Default behavior of resetOutputBuffer(). | |
static | splitStoragePath ( $storagePath) |
Split a storage path into a backend name, a container name, and a relative file path. | |
Public Attributes inherited from Wikimedia\FileBackend\FileBackend | |
const | ATTR_HEADERS = 1 |
Bitfield flags for supported features. | |
const | ATTR_METADATA = 2 |
const | ATTR_UNICODE_PATHS = 4 |
Static Protected Member Functions inherited from Wikimedia\FileBackend\FileBackend | |
static | normalizeContainerPath ( $path) |
Validate and normalize a relative storage path. | |
Proxy backend that manages file layout rewriting for FileRepo.
LocalRepo may be configured to store files under their title names or by SHA-1. This acts as a shim in the latter case, providing backwards compatability for most callers. All "public"/"deleted" zone files actually go in an "original" container and are never changed.
This requires something like thumb_handler.php and img_auth.php for client viewing of files.
Definition at line 39 of file FileBackendDBRepoWrapper.php.
FileBackendDBRepoWrapper::__construct | ( | array | $config | ) |
Create a new backend instance from configuration.
This should only be called from within FileBackendGroup.
array | $config | Parameters include:
|
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 51 of file FileBackendDBRepoWrapper.php.
References $backend, Wikimedia\FileBackend\FileBackend\getDomainId(), and Wikimedia\FileBackend\FileBackend\getName().
FileBackendDBRepoWrapper::clearCache | ( | array | $paths = null | ) |
Invalidate any in-process file stat and property cache.
If $paths is given, then only the cache for those files will be cleared.
array | null | $paths | Storage paths (optional) |
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 249 of file FileBackendDBRepoWrapper.php.
FileBackendDBRepoWrapper::concatenate | ( | array | $params | ) |
Concatenate a list of storage files into a single file system file.
The target path should refer to a file that is already locked or otherwise safe from modification from other processes. Normally, the file will be a new temp file, which should be adequate.
array | $params | Operation parameters, include:
|
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 173 of file FileBackendDBRepoWrapper.php.
References $params, and translateSrcParams().
FileBackendDBRepoWrapper::directoryExists | ( | array | $params | ) |
Check if a directory exists at a given storage path.
For backends using key/value stores, a directory is said to exist whenever there exist any files with paths using the given directory path as a prefix followed by a forward slash. For example, if there is a file called "mwstore://backend/container/dir/path.svg" then directories are said to exist at "mwstore://backend/container" and "mwstore://backend/container/dir". These can be thought of as "virtual" directories.
Backends that directly use a filesystem layer might enumerate empty directories. The clean() method should always be used when files are deleted or moved if this is a concern. This is a trade-off to avoid write amplication/contention on file changes or read amplification when calling this method.
Storage backends with eventual consistency might return stale data.
array | $params | Parameters include:
|
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 233 of file FileBackendDBRepoWrapper.php.
References $params.
|
protected |
array | $params |
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 169 of file FileBackendDBRepoWrapper.php.
References $params.
|
protected |
array | $ops | |
array | $opts |
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 149 of file FileBackendDBRepoWrapper.php.
References mungeOpPaths().
|
protected |
array | $params |
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 157 of file FileBackendDBRepoWrapper.php.
References $params.
|
protected |
array | $params |
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 165 of file FileBackendDBRepoWrapper.php.
References $params.
|
protected |
array | $ops | |
array | $opts |
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 153 of file FileBackendDBRepoWrapper.php.
References mungeOpPaths().
|
protected |
array | $params |
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 161 of file FileBackendDBRepoWrapper.php.
References $params.
FileBackendDBRepoWrapper::fileExists | ( | array | $params | ) |
Check if a file exists at a storage path in the backend.
This returns false if only a directory exists at the path.
Callers that only care if a file is readily accessible can use non-strict comparisons on the result. If "does not exist" and "existence is unknown" must be distinguished, then strict comparisons to true/null should be used.
array | $params | Parameters include:
|
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 177 of file FileBackendDBRepoWrapper.php.
References $params, and translateSrcParams().
FileBackendDBRepoWrapper::getBackendPath | ( | $path, | |
$latest = true ) |
Translate a legacy "title" path to its "sha1" counterpart.
E.g. mwstore://local-backend/local-public/a/ab/<name>.jpg => mwstore://local-backend/local-original/x/y/z/<sha1>.jpg
string | $path | |
bool | $latest |
Definition at line 82 of file FileBackendDBRepoWrapper.php.
References $path, and getBackendPaths().
Referenced by mungeOpPaths(), and translateSrcParams().
FileBackendDBRepoWrapper::getBackendPaths | ( | array | $paths, |
$latest = true ) |
Translate legacy "title" paths to their "sha1" counterparts.
E.g. mwstore://local-backend/local-public/a/ab/<name>.jpg => mwstore://local-backend/local-original/x/y/z/<sha1>.jpg
string[] | $paths | |
bool | $latest |
Definition at line 97 of file FileBackendDBRepoWrapper.php.
References Wikimedia\FileBackend\FileBackend\$name, $path, DB_PRIMARY, DB_REPLICA, getDB(), and getPathForSHA1().
Referenced by getBackendPath(), mungeOpPaths(), preloadCache(), translateArrayResults(), and translateSrcParams().
|
protected |
Get a connection to the repo file registry DB.
int | $index |
Definition at line 288 of file FileBackendDBRepoWrapper.php.
References $dbHandleFunc.
Referenced by getBackendPaths().
FileBackendDBRepoWrapper::getDirectoryList | ( | array | $params | ) |
Get an iterator to list all directories under a storage directory.
If the directory is of the form "mwstore://backend/container", then all directories in the container will be listed. If the directory is of form "mwstore://backend/container/dir", then all directories directly under that directory will be listed. Results will be storage directories relative to the given directory.
Storage backends with eventual consistency might return stale data.
Failures during iteration can result in FileBackendError exceptions (since 1.22).
array | $params | Parameters include:
|
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 237 of file FileBackendDBRepoWrapper.php.
References $params.
FileBackendDBRepoWrapper::getFeatures | ( | ) |
Get the a bitfield of extra features supported by the backend medium.
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 245 of file FileBackendDBRepoWrapper.php.
FileBackendDBRepoWrapper::getFileContentsMulti | ( | array | $params | ) |
Like getFileContents() except it takes an array of storage paths and returns an order preserved map of storage paths to their content.
array | $params | Parameters include:
|
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 217 of file FileBackendDBRepoWrapper.php.
References $params, and translateArrayResults().
FileBackendDBRepoWrapper::getFileHttpUrl | ( | array | $params | ) |
Return an HTTP URL to a given file that requires no authentication to use.
The URL may be pre-authenticated (via some token in the URL) and temporary. This will return null if the backend cannot make an HTTP URL for the file.
This is useful for key/value stores when using scripts that seek around large files and those scripts (and the backend) support HTTP Range headers. Otherwise, one would need to use getLocalReference(), which involves loading the entire file on to local disk.
array | $params | Parameters include:
|
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 229 of file FileBackendDBRepoWrapper.php.
References $params, and translateSrcParams().
FileBackendDBRepoWrapper::getFileList | ( | array | $params | ) |
Get an iterator to list all stored files under a storage directory.
If the directory is of the form "mwstore://backend/container", then all files in the container will be listed. If the directory is of form "mwstore://backend/container/dir", then all files under that directory will be listed. Results will be storage paths relative to the given directory.
Storage backends with eventual consistency might return stale data.
Failures during iteration can result in FileBackendError exceptions (since 1.22).
array | $params | Parameters include:
|
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 241 of file FileBackendDBRepoWrapper.php.
References $params.
FileBackendDBRepoWrapper::getFileProps | ( | array | $params | ) |
Get the properties of the content of the file at a storage path in the backend.
This gives the result of FSFile::getProps() on a local copy of the file.
array | $params | Parameters include:
|
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 201 of file FileBackendDBRepoWrapper.php.
References $params, and translateSrcParams().
FileBackendDBRepoWrapper::getFileSha1Base36 | ( | array | $params | ) |
Get a SHA-1 hash of the content of the file at a storage path in the backend.
array | $params | Parameters include:
|
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 197 of file FileBackendDBRepoWrapper.php.
References $params, and translateSrcParams().
FileBackendDBRepoWrapper::getFileSize | ( | array | $params | ) |
Get the size (bytes) of a file at a storage path in the backend.
array | $params | Parameters include:
|
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 185 of file FileBackendDBRepoWrapper.php.
References $params, and translateSrcParams().
FileBackendDBRepoWrapper::getFileStat | ( | array | $params | ) |
Get quick information about a file at a storage path in the backend.
If the file does not exist, then this returns false. Otherwise, the result is an associative array that includes:
array | $params | Parameters include:
|
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 189 of file FileBackendDBRepoWrapper.php.
References $params, and translateSrcParams().
FileBackendDBRepoWrapper::getFileTimestamp | ( | array | $params | ) |
Get the last-modified timestamp of the file at a storage path.
array | $params | Parameters include:
|
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 181 of file FileBackendDBRepoWrapper.php.
References $params, and translateSrcParams().
FileBackendDBRepoWrapper::getFileXAttributes | ( | array | $params | ) |
Get metadata about a file at a storage path in the backend.
If the file does not exist, then this returns false. Otherwise, the result is an associative array that includes:
Use FileBackend::hasFeatures() to check how well this is supported.
array | $params | $params include:
|
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 193 of file FileBackendDBRepoWrapper.php.
References $params, and translateSrcParams().
FileBackendDBRepoWrapper::getInternalBackend | ( | ) |
Get the underlying FileBackend that is being wrapped.
Definition at line 68 of file FileBackendDBRepoWrapper.php.
References $backend.
FileBackendDBRepoWrapper::getLocalCopyMulti | ( | array | $params | ) |
Like getLocalCopy() except it takes an array of storage paths and yields an order preserved-map of storage paths to temporary local file copies.
Multiple calls to this method for the same path will create new copies.
array | $params | Parameters include:
|
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 225 of file FileBackendDBRepoWrapper.php.
References $params, and translateArrayResults().
FileBackendDBRepoWrapper::getLocalReferenceMulti | ( | array | $params | ) |
Like getLocalReference() except it takes an array of storage paths and yields an order-preserved map of storage paths to temporary local file copies.
Never modify the returned files since they might be originals, they might be shared among multiple callers of this method, or the backend might internally keep FSFile references for deferred operations.
array | $params | Parameters include:
|
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 221 of file FileBackendDBRepoWrapper.php.
References $params, and translateArrayResults().
FileBackendDBRepoWrapper::getPathForSHA1 | ( | $sha1 | ) |
Get the ultimate original storage path for a file.
Use this when putting a new file into the system
string | $sha1 | File SHA-1 base36 |
Definition at line 274 of file FileBackendDBRepoWrapper.php.
Referenced by getBackendPaths().
FileBackendDBRepoWrapper::getScopedLocksForOps | ( | array | $ops, |
StatusValue | $status ) |
Get an array of scoped locks needed for a batch of file operations.
Normally, FileBackend::doOperations() handles locking, unless the 'nonLocking' param is passed in. This function is useful if you want the files to be locked for a broader scope than just when the files are changing. For example, if you need to update DB metadata, you may want to keep the files locked until finished.
array | $ops | List of file operations to FileBackend::doOperations() |
StatusValue | $status | StatusValue to update on lock/unlock |
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 262 of file FileBackendDBRepoWrapper.php.
|
protected |
Translate legacy "title" source paths to their "sha1" counterparts.
This leaves destination paths alone since we don't want those to mutate
array[] | $ops |
Definition at line 347 of file FileBackendDBRepoWrapper.php.
References getBackendPath(), and getBackendPaths().
Referenced by doOperationsInternal(), and doQuickOperationsInternal().
FileBackendDBRepoWrapper::preloadCache | ( | array | $paths | ) |
Preload persistent file stat cache and property cache into in-process cache.
This should be used when stat calls will be made on a known list of a many files.
array | $paths | Storage paths |
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 253 of file FileBackendDBRepoWrapper.php.
References getBackendPaths().
FileBackendDBRepoWrapper::preloadFileStat | ( | array | $params | ) |
Preload file stat information (concurrently if possible) into in-process cache.
This should be used when stat calls will be made on a known list of a many files. This does not make use of the persistent file stat cache.
array | $params | Parameters include:
|
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 258 of file FileBackendDBRepoWrapper.php.
References $params, and translateSrcParams().
FileBackendDBRepoWrapper::streamFile | ( | array | $params | ) |
Stream the content of the file at a storage path in the backend.
If the file does not exists, an HTTP 404 error will be given. Appropriate HTTP headers (Status, Content-Type, Content-Length) will be sent if streaming began, while none will be sent otherwise. Implementations should flush the output buffer before sending data.
array | $params | Parameters include:
|
Reimplemented from Wikimedia\FileBackend\FileBackend.
Definition at line 205 of file FileBackendDBRepoWrapper.php.
References $params, and translateSrcParams().
|
protected |
Translates paths when the backend function returns results keyed by paths.
string | $function | |
array | $params |
Definition at line 324 of file FileBackendDBRepoWrapper.php.
References $params, $path, and getBackendPaths().
Referenced by getFileContentsMulti(), getLocalCopyMulti(), and getLocalReferenceMulti().
|
protected |
Translates paths found in the "src" or "srcs" keys of a params array.
string | $function | |
array | $params |
Definition at line 303 of file FileBackendDBRepoWrapper.php.
References $params, getBackendPath(), and getBackendPaths().
Referenced by concatenate(), fileExists(), getFileHttpUrl(), getFileProps(), getFileSha1Base36(), getFileSize(), getFileStat(), getFileTimestamp(), getFileXAttributes(), preloadFileStat(), and streamFile().
|
protected |
Definition at line 41 of file FileBackendDBRepoWrapper.php.
Referenced by __construct(), and getInternalBackend().
|
protected |
Definition at line 45 of file FileBackendDBRepoWrapper.php.
Referenced by getDB().
|
protected |
Definition at line 49 of file FileBackendDBRepoWrapper.php.
|
protected |
Definition at line 43 of file FileBackendDBRepoWrapper.php.
|
protected |
Definition at line 47 of file FileBackendDBRepoWrapper.php.