Base class for all backends using particular storage medium. More...

Inherits Wikimedia\FileBackend\FileBackend.

Inherited by FSFileBackend, MemoryFileBackend, and SwiftFileBackend.

Collaboration diagram for FileBackendStore:

[legend]

Public Member Functions
	__construct (array $config)

	clearCache (array $paths=null)
	Invalidate any in-process file stat and property cache.

	concatenate (array $params)
	Concatenate a list of storage files into a single file system file.

	copyInternal (array $params)
	Copy a file from one storage path to another in the backend.

	createInternal (array $params)
	Create a file in the backend with the given contents.

	deleteInternal (array $params)
	Delete a file at the storage path.

	describeInternal (array $params)
	Alter metadata for a file at the storage path.

	directoryExists (array $params)
	Check if a directory exists at a given storage path.

	executeOpHandlesInternal (array $fileOpHandles)
	Execute a list of FileBackendStoreOpHandle handles in parallel.

	fileExists (array $params)
	Check if a file exists at a storage path in the backend.

	getDirectoryList (array $params)
	Get an iterator to list all directories under a storage directory.

	getDirectoryListInternal ( $container, $dir, array $params)
	Do not call this function from places outside FileBackend.

	getFileContentsMulti (array $params)
	Like getFileContents() except it takes an array of storage paths and returns an order preserved map of storage paths to their content.

	getFileHttpUrl (array $params)

	getFileList (array $params)
	Get an iterator to list all stored files under a storage directory.

	getFileListInternal ( $container, $dir, array $params)
	Do not call this function from places outside FileBackend.

	getFileProps (array $params)
	Get the properties of the content of the file at a storage path in the backend.

	getFileSha1Base36 (array $params)
	Get a SHA-1 hash of the content of the file at a storage path in the backend.

	getFileSize (array $params)
	Get the size (bytes) of a file at a storage path in the backend.

	getFileStat (array $params)
	Get quick information about a file at a storage path in the backend.

	getFileTimestamp (array $params)
	Get the last-modified timestamp of the file at a storage path.

	getFileXAttributes (array $params)
	Get metadata about a file at a storage path in the backend.

	getLocalCopyMulti (array $params)
	Like getLocalCopy() except it takes an array of storage paths and yields an order preserved-map of storage paths to temporary local file copies.

	getLocalReferenceMulti (array $params)
	Like getLocalReference() except it takes an array of storage paths and yields an order-preserved map of storage paths to temporary local file copies.

	getOperationsInternal (array $ops)
	Return a list of FileOp objects from a list of operations.

	getPathsToLockForOpsInternal (array $performOps)
	Get a list of storage paths to lock for a list of operations Returns an array with LockManager::LOCK_UW (shared locks) and LockManager::LOCK_EX (exclusive locks) keys, each corresponding to a list of storage paths to be locked.

	getScopedLocksForOps (array $ops, StatusValue $status)
	Get an array of scoped locks needed for a batch of file operations.

	isPathUsableInternal ( $storagePath)
	Check if a file can be created or changed at a given storage path in the backend.

	isSingleShardPathInternal ( $storagePath)
	Check if a storage path maps to a single shard.

	maxFileSizeInternal ()
	Get the maximum allowable file size given backend medium restrictions and basic performance constraints.

	moveInternal (array $params)
	Move a file from one storage path to another in the backend.

	nullInternal (array $params)
	No-op file operation that does nothing.

	preloadCache (array $paths)
	Preload persistent file stat cache and property cache into in-process cache.

	preloadFileStat (array $params)
	Preload file stat information (concurrently if possible) into in-process cache.

	storeInternal (array $params)
	Store a file into the backend from a file on disk.

	streamFile (array $params)
	Stream the content of the file at a storage path in the backend.

Public Member Functions inherited from Wikimedia\FileBackend\FileBackend
	clean (array $params)
	Delete a storage directory if it is empty.

	copy (array $params, array $opts=[])
	Performs a single copy operation.

	create (array $params, array $opts=[])
	Performs a single create operation.

	delete (array $params, array $opts=[])
	Performs a single delete operation.

	describe (array $params, array $opts=[])
	Performs a single describe operation.

	doOperation (array $op, array $opts=[])
	Same as doOperations() except it takes a single operation.

	doOperations (array $ops, array $opts=[])
	This is the main entry point into the backend for write operations.

	doQuickOperation (array $op, array $opts=[])
	Same as doQuickOperations() except it takes a single operation.

	doQuickOperations (array $ops, array $opts=[])
	Perform a set of independent file operations on some files.

	getContainerStoragePath ( $container)
	Get the storage path for the given container for this backend.

	getDomainId ()
	Get the domain identifier used for this backend (possibly empty).

	getFeatures ()
	Get the a bitfield of extra features supported by the backend medium.

	getFileContents (array $params)
	Get the contents of a file at a storage path in the backend.

	getLocalCopy (array $params)
	Get a local copy on disk of the file at a storage path in the backend.

	getLocalReference (array $params)
	Returns a file system file, identical in content to the file at a storage path.

	getName ()
	Get the unique backend name.

	getReadOnlyReason ()
	Get an explanatory message if this backend is read-only.

	getRootStoragePath ()
	Get the root storage path of this backend.

	getScopedFileLocks (array $paths, $type, StatusValue $status, $timeout=0)
	Lock the files at the given storage paths in the backend.

	getStreamerOptions ()
	Return options for use with HTTPFileStreamer.

	getTopDirectoryList (array $params)
	Same as FileBackend::getDirectoryList() except only lists directories that are immediately under the given directory.

	getTopFileList (array $params)
	Same as FileBackend::getFileList() except only lists files that are immediately under the given directory.

	getWikiId ()
	Alias to getDomainId()

	hasFeatures ( $bitfield)
	Check if the backend medium supports a field of extra features.

	isReadOnly ()
	Check if this backend is read-only.

	lockFiles (array $paths, $type, $timeout=0)
	Lock the files at the given storage paths in the backend.

	move (array $params, array $opts=[])
	Performs a single move operation.

	prepare (array $params)
	Prepare a storage directory for usage.

	publish (array $params)
	Remove measures to block web access to a storage directory and the container it belongs to.

	quickCopy (array $params, array $opts=[])
	Performs a single quick copy operation.

	quickCreate (array $params, array $opts=[])
	Performs a single quick create operation.

	quickDelete (array $params, array $opts=[])
	Performs a single quick delete operation.

	quickDescribe (array $params, array $opts=[])
	Performs a single quick describe operation.

	quickMove (array $params, array $opts=[])
	Performs a single quick move operation.

	quickStore (array $params, array $opts=[])
	Performs a single quick store operation.

	secure (array $params)
	Take measures to block web access to a storage directory and the container it belongs to.

	setLogger (LoggerInterface $logger)

	store (array $params, array $opts=[])
	Performs a single store operation.

	unlockFiles (array $paths, $type)
	Unlock the files at the given storage paths in the backend.

Protected Member Functions

deleteContainerCache ( $container)

Delete the cached info for a container.

deleteFileCache ( $path)

Delete the cached stat info for a file path.

directoriesAreVirtual ()

Is this a key/value store where directories are just virtual? Virtual directories exists in so much as files exists that are prefixed with the directory path followed by a forward slash.

doClean (array $params)

doCleanInternal ( $container, $dir, array $params)

doClearCache (array $paths=null)

Clears any additional stat caches for storage paths.

doConcatenate (array $params)

doCopyInternal (array $params)

doCreateInternal (array $params)

doDeleteInternal (array $params)

doDescribeInternal (array $params)

doDirectoryExists ( $container, $dir, array $params)

doExecuteOpHandlesInternal (array $fileOpHandles)

doGetFileContentsMulti (array $params)

doGetFileSha1Base36 (array $params)

doGetFileStat (array $params)

doGetFileStatMulti (array $params)

Get file stat information (concurrently if possible) for several files.

doGetFileXAttributes (array $params)

doGetLocalCopyMulti (array $params)

doGetLocalReferenceMulti (array $params)

doMoveInternal (array $params)

doOperationsInternal (array $ops, array $opts)

doPrepare (array $params)

See also: FileBackend::prepare()

Parameters

array $params

Returns: StatusValue Good status without value for success, fatal otherwise.

doPrepareInternal ( $container, $dir, array $params)

doPrimeContainerCache (array $containerInfo)

Fill the backend-specific process cache given an array of resolved container names and their corresponding cached info.

doPublish (array $params)

doPublishInternal ( $container, $dir, array $params)

doQuickOperationsInternal (array $ops, array $opts)

doSecure (array $params)

doSecureInternal ( $container, $dir, array $params)

doStoreInternal (array $params)

doStreamFile (array $params)

fullContainerName ( $container)

Get the full container name, including the domain ID prefix.

getContainerHashLevels ( $container)

Get the sharding config for a container.

getContainerShard ( $container, $relPath)

Get the container name shard suffix for a given path.

getContainerSuffixes ( $container)

Get a list of full container shard suffixes for a container.

getContentType ( $storagePath, $content, $fsPath)

Get the content type to use in HEAD/GET requests for a file.

ingestFreshFileStats (array $stats, $latest)

Ingest file stat entries that just came from querying the backend (not cache)

primeContainerCache (array $items)

Do a batch lookup from cache for container stats for all containers used in a list of container names or storage paths objects.

primeFileCache (array $items)

Do a batch lookup from cache for file stats for all paths used in a list of storage paths or FileOp objects.

resolveContainerName ( $container)

Resolve a container name, checking if it's allowed by the backend.

resolveContainerPath ( $container, $relStoragePath)

Resolve a relative storage path, checking if it's allowed by the backend.

resolveStoragePath ( $storagePath)

Splits a storage path into an internal container name, an internal relative file name, and a container shard suffix.

resolveStoragePathReal ( $storagePath)

Like resolveStoragePath() except null values are returned if the container is sharded and the shard could not be determined or if the path ends with '/'.

sanitizeOpHeaders (array $op)

Normalize and filter HTTP headers from a file operation.

setConcurrencyFlags (array $opts)

Set the 'concurrency' option from a list of operation options.

setContainerCache ( $container, array $val)

Set the cached info for a container.

setFileCache ( $path, array $val)

Set the cached stat info for a file path.

Protected Member Functions inherited from Wikimedia\FileBackend\FileBackend

header ( $header)

newStatus (... $args)

Yields the result of the status wrapper callback on either:

resetOutputBuffer ()

resolveFSFileObjects (array $ops)

Convert FSFile 'src' paths to string paths (with an 'srcRef' field set to the FSFile)

scopedProfileSection ( $section)

wrapStatus (StatusValue $sv)

Static Protected Member Functions
static	isValidContainerName ( $container)
	Check if a full container name is valid.

static	isValidShortContainerName ( $container)
	Check if a short container name is valid.

static	normalizeXAttributes (array $xattr)
	Normalize file headers/metadata to the FileBackend::getFileXAttributes() format.

Static Protected Member Functions inherited from Wikimedia\FileBackend\FileBackend
static	normalizeContainerPath ( $path)
	Validate and normalize a relative storage path.

Protected Attributes
MapCacheLRU	$cheapCache
	Map of paths to small (RAM/disk) cache items.

MapCacheLRU	$expensiveCache
	Map of paths to large (RAM/disk) cache items.

	$maxFileSize = 32 * 1024 * 1024 * 1024

WANObjectCache	$memCache

callable null	$mimeCallback
	Method to get the MIME type of files.

array< string, array >	$shardViaHashLevels = []
	Map of container names to sharding config.

BagOStuff	$srvCache

const	CACHE_CHEAP_SIZE = 500

const	CACHE_EXPENSIVE_SIZE = 5

const	CACHE_TTL = 10

Protected Attributes inherited from Wikimedia\FileBackend\FileBackend
int	$concurrency
	How many operations can be done in parallel.

string	$domainId
	Unique domain name.

LockManager	$lockManager

LoggerInterface	$logger

string	$name
	Unique backend name.

string	$parallelize
	When to do operations in parallel.

callable null	$profiler

string	$readOnly
	Read-only explanation message.

callable null	$statusWrapper

array	$streamerOptions
	Option map for use with HTTPFileStreamer.

TempFSFileFactory	$tmpFileFactory

Additional Inherited Members
Static Public Member Functions inherited from Wikimedia\FileBackend\FileBackend
static	extensionFromPath ( $path, $case='lowercase')
	Get the final extension from a storage or FS path.

static	isPathTraversalFree ( $path)
	Check if a relative path has no directory traversals.

static	isStoragePath ( $path)
	Check if a given path is a "mwstore://" path.

static	makeContentDisposition ( $type, $filename='')
	Build a Content-Disposition header value per RFC 6266.

static	normalizeStoragePath ( $storagePath)
	Normalize a storage path by cleaning up directory separators.

static	parentStoragePath ( $storagePath)
	Get the parent storage directory of a storage path.

static	resetOutputBufferTheDefaultWay ()
	Default behavior of resetOutputBuffer().

static	splitStoragePath ( $storagePath)
	Split a storage path into a backend name, a container name, and a relative file path.

Public Attributes inherited from Wikimedia\FileBackend\FileBackend
const	ATTR_HEADERS = 1
	Bitfield flags for supported features.

const	ATTR_METADATA = 2

const	ATTR_UNICODE_PATHS = 4

Detailed Description

Base class for all backends using particular storage medium.

This class defines the methods as abstract that subclasses must implement. Outside callers should not use functions with "Internal" in the name.

The FileBackend operations are implemented using basic functions such as storeInternal(), copyInternal(), deleteInternal() and the like. This class is also responsible for path resolution and sanitization.

Stability: stable: to extend

Since: 1.19

Definition at line 43 of file FileBackendStore.php.

Constructor & Destructor Documentation

◆ __construct()

FileBackendStore::__construct ( array $config )

See also

FileBackend::__construct() Additional $config params include:

srvCache : BagOStuff cache to APC or the like.
wanCache : WANObjectCache object to use for persistent caching.
mimeCallback : Callback that takes (storage path, content, file system path) and returns the MIME type of the file or 'unknown/unknown'. The file system path parameter should be used if the content one is null.

Stability: stable: to call

Parameters

array $config

Reimplemented from Wikimedia\FileBackend\FileBackend.

Reimplemented in FSFileBackend, and SwiftFileBackend.

Definition at line 88 of file FileBackendStore.php.

Member Function Documentation

◆ clearCache()

FileBackendStore::clearCache ( array $paths = null )

final

Invalidate any in-process file stat and property cache.

If $paths is given, then only the cache for those files will be cleared.

See also: FileBackend::getFileStat()

Parameters

array | null $paths Storage paths (optional)

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 1467 of file FileBackendStore.php.

References $path, and doClearCache().

Referenced by copyInternal(), createInternal(), deleteInternal(), describeInternal(), SwiftFileBackend\doGetFileXAttributes(), doOperationsInternal(), doQuickOperationsInternal(), SwiftFileBackend\doStreamFile(), moveInternal(), and storeInternal().

◆ concatenate()

FileBackendStore::concatenate ( array $params )

final

Concatenate a list of storage files into a single file system file.

The target path should refer to a file that is already locked or otherwise safe from modification from other processes. Normally, the file will be a new temp file, which should be adequate.

Parameters

array

$params

Operation parameters, include:

srcs : ordered source storage paths (e.g. chunk1, chunk2, ...)
dst : file system path to 0-byte temp file
parallelize : try to do operations in parallel when possible

Returns: StatusValue

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 366 of file FileBackendStore.php.

References $params, doConcatenate(), Wikimedia\FileBackend\FileBackend\getScopedFileLocks(), Wikimedia\FileBackend\FileBackend\newStatus(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

◆ copyInternal()

FileBackendStore::copyInternal ( array $params )

final

Copy a file from one storage path to another in the backend.

This will overwrite any file that exists at the destination. Do not call this function from places outside FileBackend and FileOp.

$params include:

src : source storage path
dst : destination storage path
ignoreMissingSource : do nothing if the source file does not exist
headers : HTTP header name/value map
async : StatusValue will be returned immediately if supported. If the StatusValue is OK, then its value field will be set to a FileBackendStoreOpHandle object.
dstExists : Whether a file exists at the destination (optimization). Callers can use "false" if no existing file is being changed.

Parameters

array $params

Returns: StatusValue

Definition at line 225 of file FileBackendStore.php.

References $params, clearCache(), deleteFileCache(), doCopyInternal(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

◆ createInternal()

FileBackendStore::createInternal ( array $params )

final

Create a file in the backend with the given contents.

This will overwrite any file that exists at the destination. Do not call this function from places outside FileBackend and FileOp.

$params include:

content : the raw file contents
dst : destination storage path
headers : HTTP header name/value map
async : StatusValue will be returned immediately if supported. If the StatusValue is OK, then its value field will be set to a FileBackendStoreOpHandle object.
dstExists : Whether a file exists at the destination (optimization). Callers can use "false" if no existing file is being changed.

Parameters

array $params

Returns: StatusValue

Definition at line 138 of file FileBackendStore.php.

References $params, clearCache(), deleteFileCache(), doCreateInternal(), maxFileSizeInternal(), Wikimedia\FileBackend\FileBackend\newStatus(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

◆ deleteContainerCache()

FileBackendStore::deleteContainerCache ( $container )

finalprotected

Delete the cached info for a container.

The cache key is salted for a while to prevent race conditions.

Parameters

string $container Resolved container name

Definition at line 1801 of file FileBackendStore.php.

Referenced by doClean().

◆ deleteFileCache()

FileBackendStore::deleteFileCache ( $path )

finalprotected

Delete the cached stat info for a file path.

The cache key is salted for a while to prevent race conditions. Since negatives (404s) are not cached, this does not need to be called when a file is created at a path were there was none before.

Parameters

string $path Storage path

Definition at line 1902 of file FileBackendStore.php.

References $path.

Referenced by SwiftFileBackend\addMissingHashMetadata(), copyInternal(), createInternal(), deleteInternal(), describeInternal(), SwiftFileBackend\doStreamFile(), moveInternal(), and storeInternal().

◆ deleteInternal()

FileBackendStore::deleteInternal ( array $params )

final

Delete a file at the storage path.

Do not call this function from places outside FileBackend and FileOp.

$params include:

src : source storage path
ignoreMissingSource : do nothing if the source file does not exist
async : StatusValue will be returned immediately if supported. If the StatusValue is OK, then its value field will be set to a FileBackendStoreOpHandle object.

Parameters

array $params

Returns: StatusValue

Definition at line 259 of file FileBackendStore.php.

References clearCache(), deleteFileCache(), doDeleteInternal(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

◆ describeInternal()

FileBackendStore::describeInternal ( array $params )

final

Alter metadata for a file at the storage path.

Do not call this function from places outside FileBackend and FileOp.

$params include:

src : source storage path
headers : HTTP header name/value map
async : StatusValue will be returned immediately if supported. If the StatusValue is OK, then its value field will be set to a FileBackendStoreOpHandle object.

Parameters

array $params

Returns: StatusValue

Definition at line 330 of file FileBackendStore.php.

References $params, clearCache(), deleteFileCache(), doDescribeInternal(), Wikimedia\FileBackend\FileBackend\newStatus(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

◆ directoriesAreVirtual()

FileBackendStore::directoriesAreVirtual ( )

abstractprotected

Is this a key/value store where directories are just virtual? Virtual directories exists in so much as files exists that are prefixed with the directory path followed by a forward slash.

Returns: bool

Reimplemented in FSFileBackend, MemoryFileBackend, and SwiftFileBackend.

Referenced by doClean().

◆ directoryExists()

FileBackendStore::directoryExists ( array $params )

final

Check if a directory exists at a given storage path.

For backends using key/value stores, a directory is said to exist whenever there exist any files with paths using the given directory path as a prefix followed by a forward slash. For example, if there is a file called "mwstore://backend/container/dir/path.svg" then directories are said to exist at "mwstore://backend/container" and "mwstore://backend/container/dir". These can be thought of as "virtual" directories.

Backends that directly use a filesystem layer might enumerate empty directories. The clean() method should always be used when files are deleted or moved if this is a concern. This is a trade-off to avoid write amplication/contention on file changes or read amplification when calling this method.

Storage backends with eventual consistency might return stale data.

See also: FileBackend::EXISTENCE_ERROR; FileBackend::clean()

Parameters

array

$params

Parameters include:

dir : storage directory

Returns: bool|null Whether a directory exists or null (I/O error)

Since: 1.20

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 1085 of file FileBackendStore.php.

References $params, doDirectoryExists(), getContainerSuffixes(), resolveStoragePath(), and Wikimedia\FileBackend\FileBackend\splitStoragePath().

◆ doClean()

FileBackendStore::doClean ( array $params )

finalprotected

See also: FileBackend::clean()

Parameters

array $params

Returns: StatusValue

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 579 of file FileBackendStore.php.

References $params, deleteContainerCache(), directoriesAreVirtual(), doClean(), doCleanInternal(), getContainerSuffixes(), Wikimedia\FileBackend\FileBackend\getScopedFileLocks(), Wikimedia\FileBackend\FileBackend\getTopDirectoryList(), Wikimedia\FileBackend\FileBackend\newStatus(), resolveStoragePath(), Wikimedia\FileBackend\FileBackend\scopedProfileSection(), and Wikimedia\FileBackend\FileBackend\splitStoragePath().

Referenced by doClean().

◆ doCleanInternal()

FileBackendStore::doCleanInternal	(		$container,
			$dir,
		array	$params )

protected

See also: FileBackendStore::doClean()

Stability: stable: to override

Parameters

string	$container
string	$dir
array	$params

Returns: StatusValue

Reimplemented in SwiftFileBackend, and FSFileBackend.

Definition at line 634 of file FileBackendStore.php.

References Wikimedia\FileBackend\FileBackend\newStatus().

Referenced by doClean().

◆ doClearCache()

FileBackendStore::doClearCache ( array $paths = null )

protected

Clears any additional stat caches for storage paths.

Stability: stable: to override

See also: FileBackend::clearCache()

Parameters

string[] | null $paths Storage paths (optional)

Reimplemented in FSFileBackend.

Definition at line 1492 of file FileBackendStore.php.

Referenced by clearCache().

◆ doConcatenate()

FileBackendStore::doConcatenate ( array $params )

protected

See also: FileBackendStore::concatenate()

Stability: stable: to override

Parameters

array $params

Returns: StatusValue

Definition at line 394 of file FileBackendStore.php.

References $params, $path, Wikimedia\FileBackend\FileBackend\getLocalReference(), getLocalReferenceMulti(), and Wikimedia\FileBackend\FileBackend\newStatus().

Referenced by concatenate().

◆ doCopyInternal()

FileBackendStore::doCopyInternal ( array $params )

abstractprotected

See also: FileBackendStore::copyInternal()

Parameters

array $params

Returns: StatusValue

Reimplemented in FSFileBackend, MemoryFileBackend, and SwiftFileBackend.

Referenced by copyInternal().

◆ doCreateInternal()

FileBackendStore::doCreateInternal ( array $params )

abstractprotected

See also: FileBackendStore::createInternal()

Parameters

array $params

Returns: StatusValue

Reimplemented in FSFileBackend, MemoryFileBackend, and SwiftFileBackend.

Referenced by createInternal().

◆ doDeleteInternal()

FileBackendStore::doDeleteInternal ( array $params )

abstractprotected

See also: FileBackendStore::deleteInternal()

Parameters

array $params

Returns: StatusValue

Reimplemented in FSFileBackend, MemoryFileBackend, and SwiftFileBackend.

Referenced by deleteInternal().

◆ doDescribeInternal()

FileBackendStore::doDescribeInternal ( array $params )

protected

See also: FileBackendStore::describeInternal()

Stability: stable: to override

Parameters

array $params

Returns: StatusValue

Reimplemented in SwiftFileBackend.

Definition at line 351 of file FileBackendStore.php.

References Wikimedia\FileBackend\FileBackend\newStatus().

Referenced by describeInternal().

◆ doDirectoryExists()

FileBackendStore::doDirectoryExists	(		$container,
			$dir,
		array	$params )

abstractprotected

See also: FileBackendStore::directoryExists()

Parameters

string	$container	Resolved container name
string	$dir	Resolved path relative to container
array	$params

Returns: bool|null

Reimplemented in MemoryFileBackend, SwiftFileBackend, and FSFileBackend.

Referenced by directoryExists().

◆ doExecuteOpHandlesInternal()

FileBackendStore::doExecuteOpHandlesInternal ( array $fileOpHandles )

protected

See also: FileBackendStore::executeOpHandlesInternal()

Stability: stable: to override

Parameters

FileBackendStoreOpHandle[] $fileOpHandles

Exceptions

FileBackendError

Returns: StatusValue[] List of corresponding StatusValue objects

Reimplemented in FSFileBackend, and SwiftFileBackend.

Definition at line 1414 of file FileBackendStore.php.

Referenced by executeOpHandlesInternal().

◆ doGetFileContentsMulti()

FileBackendStore::doGetFileContentsMulti ( array $params )

protected

See also: FileBackendStore::getFileContentsMulti()

Stability: stable: to override

Parameters

array $params

Returns: string[]|bool[]|null[] Map of (path => string, false (missing), or null (error))

Reimplemented in SwiftFileBackend.

Definition at line 827 of file FileBackendStore.php.

References $path, and doGetLocalReferenceMulti().

Referenced by getFileContentsMulti().

◆ doGetFileSha1Base36()

FileBackendStore::doGetFileSha1Base36 ( array $params )

protected

See also: FileBackendStore::getFileSha1Base36()

Stability: stable: to override

Parameters

array $params

Returns: bool|string|null SHA1, false (missing file), or null (error)

Definition at line 935 of file FileBackendStore.php.

References Wikimedia\FileBackend\FileBackend\getLocalReference().

Referenced by getFileSha1Base36().

◆ doGetFileStat()

FileBackendStore::doGetFileStat ( array $params )

abstractprotected

See also: FileBackendStore::getFileStat()

Parameters

array $params

Returns: array|false|null

Reimplemented in FSFileBackend, MemoryFileBackend, and SwiftFileBackend.

Referenced by getFileStat().

◆ doGetFileStatMulti()

FileBackendStore::doGetFileStatMulti ( array $params )

protected

Get file stat information (concurrently if possible) for several files.

Stability: stable: to override

See also: FileBackend::getFileStat()

Parameters

array

$params

Parameters include:

srcs : list of source storage paths
latest : use the latest available data

Returns: array<string,array|false|null>|null Null if not supported. Otherwise a map of storage path to attribute map, false (missing file), or null (I/O error).

Since: 1.23

Reimplemented in SwiftFileBackend.

Definition at line 1524 of file FileBackendStore.php.

Referenced by preloadFileStat().

◆ doGetFileXAttributes()

FileBackendStore::doGetFileXAttributes ( array $params )

protected

See also: FileBackendStore::getFileXAttributes()

Stability: stable: to override

Parameters

array $params

Returns: array[][]|false|null Attributes, false (missing file), or null (error)

Reimplemented in SwiftFileBackend.

Definition at line 888 of file FileBackendStore.php.

Referenced by getFileXAttributes().

◆ doGetLocalCopyMulti()

FileBackendStore::doGetLocalCopyMulti ( array $params )

abstractprotected

See also: FileBackendStore::getLocalCopyMulti()

Parameters

array $params

Returns: string[]|bool[]|null[] Map of (path => TempFSFile, false (missing), or null (error))

Reimplemented in FSFileBackend, MemoryFileBackend, and SwiftFileBackend.

Referenced by doGetLocalReferenceMulti(), and getLocalCopyMulti().

◆ doGetLocalReferenceMulti()

FileBackendStore::doGetLocalReferenceMulti ( array $params )

protected

See also: FileBackendStore::getLocalReferenceMulti()

Stability: stable: to override

Parameters

array $params

Returns: string[]|bool[]|null[] Map of (path => FSFile, false (missing), or null (error))

Reimplemented in FSFileBackend.

Definition at line 1002 of file FileBackendStore.php.

References doGetLocalCopyMulti().

Referenced by doGetFileContentsMulti(), and getLocalReferenceMulti().

◆ doMoveInternal()

FileBackendStore::doMoveInternal ( array $params )

abstractprotected

See also: FileBackendStore::moveInternal()

Parameters

array $params

Returns: StatusValue

Reimplemented in FSFileBackend, MemoryFileBackend, and SwiftFileBackend.

Referenced by moveInternal().

◆ doOperationsInternal()

FileBackendStore::doOperationsInternal	(	array	$ops,
		array	$opts )

finalprotected

See also: FileBackend::doOperations()

Parameters

array	$ops
array	$opts

Returns: StatusValue

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 1254 of file FileBackendStore.php.

References FileOpBatch\attempt(), clearCache(), getOperationsInternal(), getPathsToLockForOpsInternal(), Wikimedia\FileBackend\FileBackend\getScopedFileLocks(), Wikimedia\FileBackend\FileBackend\newStatus(), preloadFileStat(), primeContainerCache(), Wikimedia\FileBackend\FileBackend\scopedProfileSection(), and setConcurrencyFlags().

◆ doPrepare()

FileBackendStore::doPrepare ( array $params )

finalprotected

See also: FileBackend::prepare()

Parameters

array $params

Returns: StatusValue Good status without value for success, fatal otherwise.

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 468 of file FileBackendStore.php.

References $params, doPrepareInternal(), getContainerSuffixes(), Wikimedia\FileBackend\FileBackend\newStatus(), resolveStoragePath(), Wikimedia\FileBackend\FileBackend\scopedProfileSection(), and Wikimedia\FileBackend\FileBackend\splitStoragePath().

◆ doPrepareInternal()

FileBackendStore::doPrepareInternal	(		$container,
			$dir,
		array	$params )

protected

See also: FileBackendStore::doPrepare()

Stability: stable: to override

Parameters

string	$container
string	$dir
array	$params

Returns: StatusValue Good status without value for success, fatal otherwise.

Reimplemented in SwiftFileBackend, and FSFileBackend.

Definition at line 501 of file FileBackendStore.php.

References Wikimedia\FileBackend\FileBackend\newStatus().

Referenced by doPrepare().

◆ doPrimeContainerCache()

FileBackendStore::doPrimeContainerCache ( array $containerInfo )

protected

Fill the backend-specific process cache given an array of resolved container names and their corresponding cached info.

Only containers that actually exist should appear in the map.

Stability: stable: to override

Parameters

array $containerInfo Map of resolved container names to cached info

Reimplemented in SwiftFileBackend.

Definition at line 1857 of file FileBackendStore.php.

Referenced by primeContainerCache().

◆ doPublish()

FileBackendStore::doPublish ( array $params )

finalprotected

See also: FileBackend::publish()

Parameters

array $params

Returns: StatusValue

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 542 of file FileBackendStore.php.

References $params, doPublishInternal(), getContainerSuffixes(), Wikimedia\FileBackend\FileBackend\newStatus(), resolveStoragePath(), Wikimedia\FileBackend\FileBackend\scopedProfileSection(), and Wikimedia\FileBackend\FileBackend\splitStoragePath().

◆ doPublishInternal()

FileBackendStore::doPublishInternal	(		$container,
			$dir,
		array	$params )

protected

See also: FileBackendStore::doPublish()

Stability: stable: to override

Parameters

string	$container
string	$dir
array	$params

Returns: StatusValue

Reimplemented in SwiftFileBackend, and FSFileBackend.

Definition at line 575 of file FileBackendStore.php.

References Wikimedia\FileBackend\FileBackend\newStatus().

Referenced by doPublish().

◆ doQuickOperationsInternal()

FileBackendStore::doQuickOperationsInternal	(	array	$ops,
		array	$opts )

finalprotected

See also: FileBackend::doQuickOperations()

Parameters

array	$ops
array	$opts

Returns: StatusValue

Since: 1.20

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 1316 of file FileBackendStore.php.

References Wikimedia\FileBackend\FileBackend\$concurrency, clearCache(), executeOpHandlesInternal(), getOperationsInternal(), Wikimedia\FileBackend\FileBackend\newStatus(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

◆ doSecure()

FileBackendStore::doSecure ( array $params )

finalprotected

See also: FileBackend::secure()

Parameters

array $params

Returns: StatusValue

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 505 of file FileBackendStore.php.

References $params, doSecureInternal(), getContainerSuffixes(), Wikimedia\FileBackend\FileBackend\newStatus(), resolveStoragePath(), Wikimedia\FileBackend\FileBackend\scopedProfileSection(), and Wikimedia\FileBackend\FileBackend\splitStoragePath().

◆ doSecureInternal()

FileBackendStore::doSecureInternal	(		$container,
			$dir,
		array	$params )

protected

See also: FileBackendStore::doSecure()

Stability: stable: to override

Parameters

string	$container
string	$dir
array	$params

Returns: StatusValue Good status without value for success, fatal otherwise.

Reimplemented in SwiftFileBackend, and FSFileBackend.

Definition at line 538 of file FileBackendStore.php.

References Wikimedia\FileBackend\FileBackend\newStatus().

Referenced by doSecure().

◆ doStoreInternal()

FileBackendStore::doStoreInternal ( array $params )

abstractprotected

See also: FileBackendStore::storeInternal()

Parameters

array $params

Returns: StatusValue

Reimplemented in FSFileBackend, MemoryFileBackend, and SwiftFileBackend.

Referenced by storeInternal().

◆ doStreamFile()

FileBackendStore::doStreamFile ( array $params )

protected

See also: FileBackendStore::streamFile()

Stability: stable: to override

Parameters

array $params

Returns: StatusValue

Reimplemented in SwiftFileBackend.

Definition at line 1059 of file FileBackendStore.php.

References $params, Wikimedia\FileBackend\FileBackend\getLocalReference(), and Wikimedia\FileBackend\FileBackend\newStatus().

Referenced by streamFile().

◆ executeOpHandlesInternal()

FileBackendStore::executeOpHandlesInternal ( array $fileOpHandles )

final

Execute a list of FileBackendStoreOpHandle handles in parallel.

The resulting StatusValue object fields will correspond to the order in which the handles where given.

Parameters

FileBackendStoreOpHandle[] $fileOpHandles

Returns: StatusValue[] Map of StatusValue objects

Exceptions

FileBackendError

Definition at line 1385 of file FileBackendStore.php.

References doExecuteOpHandlesInternal(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

Referenced by SwiftFileBackend\doCopyInternal(), SwiftFileBackend\doCreateInternal(), SwiftFileBackend\doDeleteInternal(), SwiftFileBackend\doDescribeInternal(), SwiftFileBackend\doMoveInternal(), doQuickOperationsInternal(), and SwiftFileBackend\doStoreInternal().

◆ fileExists()

FileBackendStore::fileExists ( array $params )

final

Check if a file exists at a storage path in the backend.

This returns false if only a directory exists at the path.

Callers that only care if a file is readily accessible can use non-strict comparisons on the result. If "does not exist" and "existence is unknown" must be distinguished, then strict comparisons to true/null should be used.

See also: FileBackend::EXISTENCE_ERROR; FileBackend::directoryExists()

Parameters

array

$params

Parameters include:

src : source storage path
latest : use the latest available data

Returns: bool|null Whether the file exists or null (I/O error)

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 638 of file FileBackendStore.php.

References getFileStat(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

Referenced by SwiftFileBackend\doStreamFile().

◆ fullContainerName()

FileBackendStore::fullContainerName ( $container )

finalprotected

Get the full container name, including the domain ID prefix.

Parameters

string $container

Returns: string

Definition at line 1735 of file FileBackendStore.php.

Referenced by resolveStoragePath().

◆ getContainerHashLevels()

FileBackendStore::getContainerHashLevels ( $container )

finalprotected

Get the sharding config for a container.

If greater than 0, then all file storage paths within the container are required to be hashed accordingly.

Parameters

string $container

Returns: array (integer levels, integer base, repeat flag) or (0, 0, false)

Definition at line 1695 of file FileBackendStore.php.

Referenced by getContainerShard(), and getContainerSuffixes().

◆ getContainerShard()

FileBackendStore::getContainerShard	(		$container,
			$relPath )

finalprotected

Get the container name shard suffix for a given path.

Any empty suffix means the container is not sharded.

Parameters

string	$container	Container name
string	$relPath	Storage path relative to the container

Returns: string|null Returns null if shard could not be determined

Definition at line 1643 of file FileBackendStore.php.

References getContainerHashLevels().

Referenced by resolveStoragePath().

◆ getContainerSuffixes()

FileBackendStore::getContainerSuffixes ( $container )

finalprotected

Get a list of full container shard suffixes for a container.

Parameters

string $container

Returns: array

Definition at line 1716 of file FileBackendStore.php.

References getContainerHashLevels().

Referenced by directoryExists(), doClean(), doPrepare(), doPublish(), doSecure(), getDirectoryList(), and getFileList().

◆ getContentType()

FileBackendStore::getContentType	(	$storagePath,
		$content,
		$fsPath )

protected

Get the content type to use in HEAD/GET requests for a file.

Stability: stable: to override

Parameters

string	$storagePath
string \| null	$content	File data
string \| null	$fsPath	File system path

Returns: string MIME type

Definition at line 2025 of file FileBackendStore.php.

Referenced by SwiftFileBackend\doCreateInternal(), and SwiftFileBackend\doStoreInternal().

◆ getDirectoryList()

FileBackendStore::getDirectoryList ( array $params )

final

Get an iterator to list all directories under a storage directory.

If the directory is of the form "mwstore://backend/container", then all directories in the container will be listed. If the directory is of form "mwstore://backend/container/dir", then all directories directly under that directory will be listed. Results will be storage directories relative to the given directory.

Storage backends with eventual consistency might return stale data.

Failures during iteration can result in FileBackendError exceptions (since 1.22).

See also: FileBackend::LIST_ERROR; FileBackend::directoryExists()

Parameters

array

$params

Parameters include:

dir : storage directory
topOnly : only return direct child dirs of the directory

Returns: \Traversable|array|null Directory list enumerator or null (initial I/O error)

Since: 1.20

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 1120 of file FileBackendStore.php.

References $params, getContainerSuffixes(), getDirectoryListInternal(), resolveStoragePath(), and Wikimedia\FileBackend\FileBackend\splitStoragePath().

◆ getDirectoryListInternal()

FileBackendStore::getDirectoryListInternal	(		$container,
			$dir,
		array	$params )

abstract

Do not call this function from places outside FileBackend.

See also: FileBackendStore::getDirectoryList()

Parameters

string	$container	Resolved container name
string	$dir	Resolved path relative to container
array	$params

Returns: Traversable|array|null Iterable list or null (error)

Reimplemented in MemoryFileBackend, SwiftFileBackend, and FSFileBackend.

Referenced by getDirectoryList().

◆ getFileContentsMulti()

FileBackendStore::getFileContentsMulti ( array $params )

Like getFileContents() except it takes an array of storage paths and returns an order preserved map of storage paths to their content.

See also: FileBackend::getFileContents()

Parameters

array

$params

Parameters include:

srcs : list of source storage paths
latest : use the latest available data
parallelize : try to do operations in parallel when possible

Returns: string[]|false[] Map of (path name => file content or false on failure)

Since: 1.20

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 806 of file FileBackendStore.php.

References $params, $path, doGetFileContentsMulti(), Wikimedia\FileBackend\FileBackend\scopedProfileSection(), and setConcurrencyFlags().

◆ getFileHttpUrl()

FileBackendStore::getFileHttpUrl ( array $params )

See also: FileBackend::getFileHttpUrl()

Stability: stable: to override

Parameters

array $params

Returns: string|null

Reimplemented from Wikimedia\FileBackend\FileBackend.

Reimplemented in SwiftFileBackend.

Definition at line 1028 of file FileBackendStore.php.

◆ getFileList()

FileBackendStore::getFileList ( array $params )

final

Get an iterator to list all stored files under a storage directory.

If the directory is of the form "mwstore://backend/container", then all files in the container will be listed. If the directory is of form "mwstore://backend/container/dir", then all files under that directory will be listed. Results will be storage paths relative to the given directory.

Storage backends with eventual consistency might return stale data.

Failures during iteration can result in FileBackendError exceptions (since 1.22).

See also: FileBackend::LIST_ERROR

Parameters

array

$params

Parameters include:

dir : storage directory
topOnly : only return direct child files of the directory (since 1.20)
adviseStat : set to true if stat requests will be made on the files (since 1.22)
forWrite : true if the list will inform a write operations (since 1.41)

Returns: \Traversable|array|null File list enumerator or null (initial I/O error)

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 1150 of file FileBackendStore.php.

References $params, getContainerSuffixes(), getFileListInternal(), resolveStoragePath(), and Wikimedia\FileBackend\FileBackend\splitStoragePath().

◆ getFileListInternal()

FileBackendStore::getFileListInternal	(		$container,
			$dir,
		array	$params )

abstract

Do not call this function from places outside FileBackend.

See also: FileBackendStore::getFileList()

Parameters

string	$container	Resolved container name
string	$dir	Resolved path relative to container
array	$params

Returns: Traversable|string[]|null Iterable list or null (error)

Reimplemented in MemoryFileBackend, SwiftFileBackend, and FSFileBackend.

Referenced by getFileList().

◆ getFileProps()

FileBackendStore::getFileProps ( array $params )

final

Get the properties of the content of the file at a storage path in the backend.

This gives the result of FSFile::getProps() on a local copy of the file.

Parameters

array

$params

Parameters include:

src : source storage path
latest : use the latest available data

Returns: array Properties map; FSFile::placeholderProps() if file missing or on I/O error

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 946 of file FileBackendStore.php.

References Wikimedia\FileBackend\FileBackend\getLocalReference(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

◆ getFileSha1Base36()

FileBackendStore::getFileSha1Base36 ( array $params )

final

Get a SHA-1 hash of the content of the file at a storage path in the backend.

See also: FileBackend::SHA1_FAIL

Parameters

array

$params

Parameters include:

src : source storage path
latest : use the latest available data

Returns: string|false Hash string or false (missing file or I/O error)

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 892 of file FileBackendStore.php.

References $params, $path, doGetFileSha1Base36(), Wikimedia\FileBackend\FileBackend\normalizeStoragePath(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

◆ getFileSize()

FileBackendStore::getFileSize ( array $params )

final

Get the size (bytes) of a file at a storage path in the backend.

See also: FileBackend::SIZE_FAIL

Parameters

array

$params

Parameters include:

src : source storage path
latest : use the latest available data

Returns: int|false File size in bytes or false (missing file or I/O error)

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 662 of file FileBackendStore.php.

References getFileStat(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

◆ getFileStat()

FileBackendStore::getFileStat ( array $params )

final

Get quick information about a file at a storage path in the backend.

If the file does not exist, then this returns false. Otherwise, the result is an associative array that includes:

mtime : the last-modified timestamp (TS_MW)
size : the file size (bytes) Additional values may be included for internal use only.

See also: FileBackend::STAT_ABSENT; FileBackend::STAT_ERROR

Parameters

array

$params

Parameters include:

src : source storage path
latest : use the latest available data

Returns: array|false|null Attribute map, false (missing file), or null (I/O error)

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 674 of file FileBackendStore.php.

References $params, $path, doGetFileStat(), ingestFreshFileStats(), Wikimedia\FileBackend\FileBackend\normalizeStoragePath(), primeFileCache(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

Referenced by SwiftFileBackend\doDescribeInternal(), SwiftFileBackend\doGetFileSha1base36(), SwiftFileBackend\doGetFileXAttributes(), fileExists(), getFileSize(), and getFileTimestamp().

◆ getFileTimestamp()

FileBackendStore::getFileTimestamp ( array $params )

final

Get the last-modified timestamp of the file at a storage path.

See also: FileBackend::TIMESTAMP_FAIL

Parameters

array

$params

Parameters include:

src : source storage path
latest : use the latest available data

Returns: string|false TS_MW timestamp or false (missing file or I/O error)

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 650 of file FileBackendStore.php.

References getFileStat(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

◆ getFileXAttributes()

FileBackendStore::getFileXAttributes ( array $params )

final

Get metadata about a file at a storage path in the backend.

If the file does not exist, then this returns false. Otherwise, the result is an associative array that includes:

headers : map of HTTP headers used for GET/HEAD requests (name => value)
metadata : map of file metadata (name => value) Metadata keys and headers names will be returned in all lower-case. Additional values may be included for internal use only.

Use FileBackend::hasFeatures() to check how well this is supported.

See also: FileBackend::XATTRS_FAIL

Parameters

array

$params

$params include:

src : source storage path
latest : use the latest available data

Returns: array|false File metadata array or false (missing file or I/O error)

Since: 1.23

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 844 of file FileBackendStore.php.

References $params, $path, doGetFileXAttributes(), Wikimedia\FileBackend\FileBackend\normalizeStoragePath(), normalizeXAttributes(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

◆ getLocalCopyMulti()

FileBackendStore::getLocalCopyMulti ( array $params )

final

Like getLocalCopy() except it takes an array of storage paths and yields an order preserved-map of storage paths to temporary local file copies.

Multiple calls to this method for the same path will create new copies.

See also: FileBackend::getLocalCopy()

Parameters

array

$params

Parameters include:

srcs : list of source storage paths
latest : use the latest available data
parallelize : try to do operations in parallel when possible

Returns: array Map of (path name => TempFSFile or false (missing) or null (error))

Since: 1.20

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 1006 of file FileBackendStore.php.

References $params, doGetLocalCopyMulti(), Wikimedia\FileBackend\FileBackend\scopedProfileSection(), and setConcurrencyFlags().

◆ getLocalReferenceMulti()

FileBackendStore::getLocalReferenceMulti ( array $params )

final

Like getLocalReference() except it takes an array of storage paths and yields an order-preserved map of storage paths to temporary local file copies.

Never modify the returned files since they might be originals, they might be shared among multiple callers of this method, or the backend might internally keep FSFile references for deferred operations.

See also: FileBackend::getLocalReference()

Parameters

array

$params

Parameters include:

srcs : list of source storage paths
latest : use the latest available data
parallelize : try to do operations in parallel when possible

Returns: array Map of (path name => FSFile or false (missing) or null (error))

Since: 1.20

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 955 of file FileBackendStore.php.

References $params, $path, doGetLocalReferenceMulti(), Wikimedia\FileBackend\FileBackend\normalizeStoragePath(), Wikimedia\FileBackend\FileBackend\scopedProfileSection(), and setConcurrencyFlags().

Referenced by doConcatenate().

◆ getOperationsInternal()

FileBackendStore::getOperationsInternal ( array $ops )

final

Return a list of FileOp objects from a list of operations.

Do not call this function from places outside FileBackend.

The result must have the same number of items as the input. An exception is thrown if an unsupported operation is requested.

Parameters

array[] $ops Same format as doOperations()

Returns: FileOp[]

Exceptions

FileBackendError

Definition at line 1191 of file FileBackendStore.php.

References $params.

Referenced by doOperationsInternal(), doQuickOperationsInternal(), and getScopedLocksForOps().

◆ getPathsToLockForOpsInternal()

FileBackendStore::getPathsToLockForOpsInternal ( array $performOps )

final

Get a list of storage paths to lock for a list of operations Returns an array with LockManager::LOCK_UW (shared locks) and LockManager::LOCK_EX (exclusive locks) keys, each corresponding to a list of storage paths to be locked.

All returned paths are normalized.

Parameters

FileOp[] $performOps List of FileOp objects

Returns: string[][] (LockManager::LOCK_UW => path list, LockManager::LOCK_EX => path list)

Definition at line 1230 of file FileBackendStore.php.

Referenced by doOperationsInternal(), and getScopedLocksForOps().

◆ getScopedLocksForOps()

FileBackendStore::getScopedLocksForOps	(	array	$ops,
		StatusValue	$status )

Get an array of scoped locks needed for a batch of file operations.

Normally, FileBackend::doOperations() handles locking, unless the 'nonLocking' param is passed in. This function is useful if you want the files to be locked for a broader scope than just when the files are changing. For example, if you need to update DB metadata, you may want to keep the files locked until finished.

See also: FileBackend::doOperations()

Parameters

array	$ops	List of file operations to FileBackend::doOperations()
StatusValue	$status	StatusValue to update on lock/unlock

Returns: ScopedLock|null RAII-style self-unlocking lock or null on failure

Since: 1.20

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 1248 of file FileBackendStore.php.

References getOperationsInternal(), getPathsToLockForOpsInternal(), and Wikimedia\FileBackend\FileBackend\getScopedFileLocks().

◆ ingestFreshFileStats()

FileBackendStore::ingestFreshFileStats	(	array	$stats,
			$latest )

finalprotected

Ingest file stat entries that just came from querying the backend (not cache)

Parameters

array<string,array|false|null> $stats Map of storage path => {

See also: doGetFileStat} result

Parameters

bool $latest Whether doGetFileStat()/doGetFileStatMulti() had the 'latest' flag

Returns: bool Whether all files have non-error stat replies

Definition at line 739 of file FileBackendStore.php.

References $path, $success, normalizeXAttributes(), and setFileCache().

Referenced by getFileStat(), and preloadFileStat().

◆ isPathUsableInternal()

FileBackendStore::isPathUsableInternal ( $storagePath )

abstract

Check if a file can be created or changed at a given storage path in the backend.

FS backends should check that the parent directory exists, files can be written under it, and that any file already there is both readable and writable. Backends using key/value stores should check if the container exists.

Parameters

string $storagePath

Returns: bool

Reimplemented in FSFileBackend, MemoryFileBackend, and SwiftFileBackend.

◆ isSingleShardPathInternal()

FileBackendStore::isSingleShardPathInternal ( $storagePath )

final

Check if a storage path maps to a single shard.

Container dirs like "a", where the container shards on "x/xy", can reside on several shards. Such paths are tricky to handle.

Parameters

string $storagePath

Returns: bool

Definition at line 1681 of file FileBackendStore.php.

References resolveStoragePath().

◆ isValidContainerName()

static FileBackendStore::isValidContainerName ( $container )

staticfinalprotected

Check if a full container name is valid.

This checks for length and illegal characters. Limiting the characters makes migrations to other stores easier.

Parameters

string $container

Returns: bool

Definition at line 1564 of file FileBackendStore.php.

Referenced by isValidShortContainerName().

◆ isValidShortContainerName()

static FileBackendStore::isValidShortContainerName ( $container )

staticfinalprotected

Check if a short container name is valid.

This checks for length and illegal characters. This may disallow certain characters that can appear in the prefix used to make the full container name.

Parameters

string $container

Returns: bool

Definition at line 1547 of file FileBackendStore.php.

References isValidContainerName().

◆ maxFileSizeInternal()

FileBackendStore::maxFileSizeInternal ( )

final

Get the maximum allowable file size given backend medium restrictions and basic performance constraints.

Do not call this function from places outside FileBackend and FileOp.

Returns: int Bytes

Definition at line 104 of file FileBackendStore.php.

Referenced by createInternal(), and storeInternal().

◆ moveInternal()

FileBackendStore::moveInternal ( array $params )

final

Move a file from one storage path to another in the backend.

This will overwrite any file that exists at the destination. Do not call this function from places outside FileBackend and FileOp.

$params include:

src : source storage path
dst : destination storage path
ignoreMissingSource : do nothing if the source file does not exist
headers : HTTP header name/value map
async : StatusValue will be returned immediately if supported. If the StatusValue is OK, then its value field will be set to a FileBackendStoreOpHandle object.
dstExists : Whether a file exists at the destination (optimization). Callers can use "false" if no existing file is being changed.

Parameters

array $params

Returns: StatusValue

Definition at line 295 of file FileBackendStore.php.

References $params, clearCache(), deleteFileCache(), doMoveInternal(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

◆ normalizeXAttributes()

static FileBackendStore::normalizeXAttributes ( array $xattr )

staticfinalprotected

Normalize file headers/metadata to the FileBackend::getFileXAttributes() format.

Parameters

array $xattr

Returns: array

Since: 1.22

Definition at line 1981 of file FileBackendStore.php.

References Wikimedia\FileBackend\FileBackend\$name.

Referenced by getFileXAttributes(), ingestFreshFileStats(), and primeFileCache().

◆ nullInternal()

FileBackendStore::nullInternal ( array $params )

final

No-op file operation that does nothing.

Do not call this function from places outside FileBackend and FileOp.

Parameters

array $params

Returns: StatusValue

Definition at line 362 of file FileBackendStore.php.

References Wikimedia\FileBackend\FileBackend\newStatus().

◆ preloadCache()

FileBackendStore::preloadCache ( array $paths )

final

Preload persistent file stat cache and property cache into in-process cache.

This should be used when stat calls will be made on a known list of a many files.

See also: FileBackend::getFileStat()

Parameters

array $paths Storage paths

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 1456 of file FileBackendStore.php.

References $path, primeContainerCache(), primeFileCache(), and resolveStoragePath().

◆ preloadFileStat()

FileBackendStore::preloadFileStat ( array $params )

final

Preload file stat information (concurrently if possible) into in-process cache.

This should be used when stat calls will be made on a known list of a many files. This does not make use of the persistent file stat cache.

See also: FileBackend::getFileStat()

Parameters

array

$params

Parameters include:

srcs : list of source storage paths
latest : use the latest available data

Returns: bool Whether all requests proceeded without I/O errors (since 1.24)

Since: 1.23

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 1495 of file FileBackendStore.php.

References $params, doGetFileStatMulti(), ingestFreshFileStats(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

Referenced by doOperationsInternal().

◆ primeContainerCache()

FileBackendStore::primeContainerCache ( array $items )

finalprotected

Do a batch lookup from cache for container stats for all containers used in a list of container names or storage paths objects.

This loads the persistent cache values into the process cache.

Parameters

array $items

Definition at line 1816 of file FileBackendStore.php.

References $path, doPrimeContainerCache(), resolveStoragePath(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

Referenced by doOperationsInternal(), SwiftFileBackend\getContainerStat(), and preloadCache().

◆ primeFileCache()

FileBackendStore::primeFileCache ( array $items )

finalprotected

Do a batch lookup from cache for file stats for all paths used in a list of storage paths or FileOp objects.

This loads the persistent cache values into the process cache.

Parameters

array $items List of storage paths

Definition at line 1921 of file FileBackendStore.php.

References $path, normalizeXAttributes(), resolveStoragePath(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

Referenced by getFileStat(), and preloadCache().

◆ resolveContainerName()

FileBackendStore::resolveContainerName ( $container )

protected

Resolve a container name, checking if it's allowed by the backend.

This is intended for internal use, such as encoding illegal chars. Subclasses can override this to be more restrictive.

Stability: stable: to override

Parameters

string $container

Returns: string|null

Definition at line 1752 of file FileBackendStore.php.

Referenced by resolveStoragePath().

◆ resolveContainerPath()

FileBackendStore::resolveContainerPath	(		$container,
			$relStoragePath )

protected

Resolve a relative storage path, checking if it's allowed by the backend.

This is intended for internal use, such as encoding illegal chars or perhaps getting absolute paths (e.g. FS based backends). Note that the relative path may be the empty string (e.g. the path is simply to the container).

Stability: stable: to override

Parameters

string	$container	Container name
string	$relStoragePath	Storage path relative to the container

Returns: string|null Path or null if not valid

Reimplemented in FSFileBackend, and SwiftFileBackend.

Definition at line 1767 of file FileBackendStore.php.

Referenced by resolveStoragePath().

◆ resolveStoragePath()

FileBackendStore::resolveStoragePath ( $storagePath )

finalprotected

Splits a storage path into an internal container name, an internal relative file name, and a container shard suffix.

Any shard suffix is already appended to the internal container name. This also checks that the storage path is valid and within this backend.

If the container is sharded but a suffix could not be determined, this means that the path can only refer to a directory and can only be scanned by looking in all the container shards.

Parameters

string $storagePath

Returns: array (container, path, container suffix) or (null, null, null) if invalid

Definition at line 1585 of file FileBackendStore.php.

References fullContainerName(), getContainerShard(), Wikimedia\FileBackend\FileBackend\normalizeContainerPath(), resolveContainerName(), resolveContainerPath(), and Wikimedia\FileBackend\FileBackend\splitStoragePath().

Referenced by directoryExists(), doClean(), doPrepare(), doPublish(), doSecure(), getDirectoryList(), getFileList(), isSingleShardPathInternal(), preloadCache(), primeContainerCache(), primeFileCache(), and resolveStoragePathReal().

◆ resolveStoragePathReal()

FileBackendStore::resolveStoragePathReal ( $storagePath )

finalprotected

Like resolveStoragePath() except null values are returned if the container is sharded and the shard could not be determined or if the path ends with '/'.

The latter case is illegal for FS backends and can confuse listings for object store backends.

This function is used when resolving paths that must be valid locations for files. Directory and listing functions should generally just use resolveStoragePath() instead.

See also: FileBackendStore::resolveStoragePath()

Parameters

string $storagePath

Returns: array (container, path) or (null, null) if invalid

Definition at line 1626 of file FileBackendStore.php.

References resolveStoragePath().

◆ sanitizeOpHeaders()

FileBackendStore::sanitizeOpHeaders ( array $op )

protected

Normalize and filter HTTP headers from a file operation.

This normalizes and strips long HTTP headers from a file operation. Most headers are just numbers, but some are allowed to be long. This function is useful for cleaning up headers and avoiding backend specific errors, especially in the middle of batch file operations.

Parameters

array $op Same format as doOperation()

Returns: array

Definition at line 1433 of file FileBackendStore.php.

References Wikimedia\FileBackend\FileBackend\$name.

◆ setConcurrencyFlags()

FileBackendStore::setConcurrencyFlags ( array $opts )

finalprotected

Set the 'concurrency' option from a list of operation options.

Parameters

array $opts Map of operation options

Returns: array

Definition at line 2001 of file FileBackendStore.php.

References Wikimedia\FileBackend\FileBackend\$concurrency.

Referenced by doOperationsInternal(), getFileContentsMulti(), getLocalCopyMulti(), and getLocalReferenceMulti().

◆ setContainerCache()

FileBackendStore::setContainerCache	(		$container,
		array	$val )

finalprotected

Set the cached info for a container.

Parameters

string	$container	Resolved container name
array	$val	Information to cache

Definition at line 1787 of file FileBackendStore.php.

Referenced by SwiftFileBackend\getContainerStat().

◆ setFileCache()

FileBackendStore::setFileCache	(		$path,
		array	$val )

finalprotected

Set the cached stat info for a file path.

Negatives (404s) are not cached. By not caching negatives, we can skip cache salting for the case when a file is created at a path were there was none before.

Parameters

string	$path	Storage path
array	$val	Stat information to cache

Definition at line 1878 of file FileBackendStore.php.

References $path.

Referenced by ingestFreshFileStats().

◆ storeInternal()

FileBackendStore::storeInternal ( array $params )

final

Store a file into the backend from a file on disk.

This will overwrite any file that exists at the destination. Do not call this function from places outside FileBackend and FileOp.

$params include:

src : source path on disk
dst : destination storage path
headers : HTTP header name/value map
async : StatusValue will be returned immediately if supported. If the StatusValue is OK, then its value field will be set to a FileBackendStoreOpHandle object.
dstExists : Whether a file exists at the destination (optimization). Callers can use "false" if no existing file is being changed.

Parameters

array $params

Returns: StatusValue

Definition at line 181 of file FileBackendStore.php.

References $params, clearCache(), deleteFileCache(), doStoreInternal(), maxFileSizeInternal(), Wikimedia\FileBackend\FileBackend\newStatus(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

◆ streamFile()

FileBackendStore::streamFile ( array $params )

final

Stream the content of the file at a storage path in the backend.

If the file does not exists, an HTTP 404 error will be given. Appropriate HTTP headers (Status, Content-Type, Content-Length) will be sent if streaming began, while none will be sent otherwise. Implementations should flush the output buffer before sending data.

Parameters

array

$params

Parameters include:

src : source storage path
headers : list of additional HTTP headers to send if the file exists
options : HTTP request header map with lower case keys (since 1.28). Supports: range : format is "bytes=(\\d*-\\d*)" if-modified-since : format is an HTTP date
headless : do not send HTTP headers (including those of "headers") (since 1.28)
latest : use the latest available data
allowOB : preserve any output buffers (since 1.28)

Returns: StatusValue

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 1032 of file FileBackendStore.php.

References $params, doStreamFile(), Wikimedia\FileBackend\FileBackend\newStatus(), and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

Member Data Documentation

◆ $cheapCache

MapCacheLRU FileBackendStore::$cheapCache

protected

Map of paths to small (RAM/disk) cache items.

Definition at line 49 of file FileBackendStore.php.

◆ $expensiveCache

MapCacheLRU FileBackendStore::$expensiveCache

protected

Map of paths to large (RAM/disk) cache items.

Definition at line 51 of file FileBackendStore.php.

◆ $maxFileSize

FileBackendStore::$maxFileSize = 32 * 1024 * 1024 * 1024

protected

Definition at line 59 of file FileBackendStore.php.

◆ $memCache

WANObjectCache FileBackendStore::$memCache

protected

Definition at line 45 of file FileBackendStore.php.

◆ $mimeCallback

callable null FileBackendStore::$mimeCallback

protected

Method to get the MIME type of files.

Definition at line 57 of file FileBackendStore.php.

◆ $shardViaHashLevels

array<string,array> FileBackendStore::$shardViaHashLevels = []

protected

Map of container names to sharding config.

Definition at line 54 of file FileBackendStore.php.

◆ $srvCache

BagOStuff FileBackendStore::$srvCache

protected

Definition at line 47 of file FileBackendStore.php.

◆ CACHE_CHEAP_SIZE

const FileBackendStore::CACHE_CHEAP_SIZE = 500

protected

Definition at line 62 of file FileBackendStore.php.

◆ CACHE_EXPENSIVE_SIZE

const FileBackendStore::CACHE_EXPENSIVE_SIZE = 5

protected

Definition at line 63 of file FileBackendStore.php.

◆ CACHE_TTL

const FileBackendStore::CACHE_TTL = 10

protected

Definition at line 61 of file FileBackendStore.php.

The documentation for this class was generated from the following file:

includes/libs/filebackend/FileBackendStore.php

Public Member Functions

Protected Member Functions

Static Protected Member Functions

Protected Attributes

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ __construct()

Member Function Documentation

◆ clearCache()

◆ concatenate()

◆ copyInternal()

◆ createInternal()

◆ deleteContainerCache()

◆ deleteFileCache()

◆ deleteInternal()

◆ describeInternal()

◆ directoriesAreVirtual()

◆ directoryExists()

◆ doClean()

◆ doCleanInternal()

◆ doClearCache()

◆ doConcatenate()

◆ doCopyInternal()

◆ doCreateInternal()

◆ doDeleteInternal()

◆ doDescribeInternal()

◆ doDirectoryExists()

◆ doExecuteOpHandlesInternal()

◆ doGetFileContentsMulti()

◆ doGetFileSha1Base36()

◆ doGetFileStat()

◆ doGetFileStatMulti()

◆ doGetFileXAttributes()

◆ doGetLocalCopyMulti()

◆ doGetLocalReferenceMulti()

◆ doMoveInternal()

◆ doOperationsInternal()

◆ doPrepare()

◆ doPrepareInternal()

◆ doPrimeContainerCache()

◆ doPublish()

◆ doPublishInternal()

◆ doQuickOperationsInternal()

◆ doSecure()

◆ doSecureInternal()

◆ doStoreInternal()

◆ doStreamFile()

◆ executeOpHandlesInternal()

◆ fileExists()

◆ fullContainerName()

◆ getContainerHashLevels()

◆ getContainerShard()

◆ getContainerSuffixes()

◆ getContentType()

◆ getDirectoryList()

◆ getDirectoryListInternal()

◆ getFileContentsMulti()

◆ getFileHttpUrl()

◆ getFileList()

◆ getFileListInternal()

◆ getFileProps()

◆ getFileSha1Base36()

◆ getFileSize()

◆ getFileStat()

◆ getFileTimestamp()

◆ getFileXAttributes()

◆ getLocalCopyMulti()

◆ getLocalReferenceMulti()

◆ getOperationsInternal()

◆ getPathsToLockForOpsInternal()

◆ getScopedLocksForOps()

◆ ingestFreshFileStats()

◆ isPathUsableInternal()

◆ isSingleShardPathInternal()

◆ isValidContainerName()

◆ isValidShortContainerName()

◆ maxFileSizeInternal()

◆ moveInternal()

◆ normalizeXAttributes()