Proxy backend that mirrors writes to several internal backends. More...

Inheritance diagram for FileBackendMultiWrite:

[legend]

Collaboration diagram for FileBackendMultiWrite:

[legend]

Public Member Functions
	__construct (array $config)
	Construct a proxy backend that consists of several internal backends.

	accessibilityCheck (array $paths)
	Check that a set of file paths are usable across all internal backends.

	clearCache (array $paths=null)
	Invalidate any in-process file stat and property cache.

	concatenate (array $params)
	Concatenate a list of storage files into a single file system file.

	consistencyCheck (array $paths)
	Check that a set of files are consistent across all internal backends.

	directoryExists (array $params)
	Check if a directory exists at a given storage path.

	fileExists (array $params)
	Check if a file exists at a storage path in the backend.

	getDirectoryList (array $params)
	Get an iterator to list all directories under a storage directory.

	getFeatures ()
	Get the a bitfield of extra features supported by the backend medium.

	getFileContentsMulti (array $params)
	Like getFileContents() except it takes an array of storage paths and returns an order preserved map of storage paths to their content.

	getFileHttpUrl (array $params)
	Return an HTTP URL to a given file that requires no authentication to use.

	getFileList (array $params)
	Get an iterator to list all stored files under a storage directory.

	getFileProps (array $params)
	Get the properties of the content of the file at a storage path in the backend.

	getFileSha1Base36 (array $params)
	Get a SHA-1 hash of the content of the file at a storage path in the backend.

	getFileSize (array $params)
	Get the size (bytes) of a file at a storage path in the backend.

	getFileStat (array $params)
	Get quick information about a file at a storage path in the backend.

	getFileTimestamp (array $params)
	Get the last-modified timestamp of the file at a storage path.

	getFileXAttributes (array $params)
	Get metadata about a file at a storage path in the backend.

	getLocalCopyMulti (array $params)
	Like getLocalCopy() except it takes an array of storage paths and yields an order preserved-map of storage paths to temporary local file copies.

	getLocalReferenceMulti (array $params)
	Like getLocalReference() except it takes an array of storage paths and yields an order-preserved map of storage paths to temporary local file copies.

	getScopedLocksForOps (array $ops, StatusValue $status)
	Get an array of scoped locks needed for a batch of file operations.

	preloadCache (array $paths)
	Preload persistent file stat cache and property cache into in-process cache.

	preloadFileStat (array $params)
	Preload file stat information (concurrently if possible) into in-process cache.

	resyncFiles (array $paths, $resyncMode=true)
	Check that a set of files are consistent across all internal backends and re-synchronize those files against the "multi master" if needed.

	streamFile (array $params)
	Stream the content of the file at a storage path in the backend.

Public Member Functions inherited from FileBackend
	clean (array $params)
	Delete a storage directory if it is empty.

	copy (array $params, array $opts=[])
	Performs a single copy operation.

	create (array $params, array $opts=[])
	Performs a single create operation.

	delete (array $params, array $opts=[])
	Performs a single delete operation.

	describe (array $params, array $opts=[])
	Performs a single describe operation.

	doOperation (array $op, array $opts=[])
	Same as doOperations() except it takes a single operation.

	doOperations (array $ops, array $opts=[])
	This is the main entry point into the backend for write operations.

	doQuickOperation (array $op)
	Same as doQuickOperations() except it takes a single operation.

	doQuickOperations (array $ops, array $opts=[])
	Perform a set of independent file operations on some files.

	getContainerStoragePath ( $container)
	Get the storage path for the given container for this backend.

	getDomainId ()
	Get the domain identifier used for this backend (possibly empty).

	getFileContents (array $params)
	Get the contents of a file at a storage path in the backend.

	getJournal ()
	Get the file journal object for this backend.

	getLocalCopy (array $params)
	Get a local copy on disk of the file at a storage path in the backend.

	getLocalReference (array $params)
	Returns a file system file, identical in content to the file at a storage path.

	getName ()
	Get the unique backend name.

	getReadOnlyReason ()
	Get an explanatory message if this backend is read-only.

	getRootStoragePath ()
	Get the root storage path of this backend.

	getScopedFileLocks (array $paths, $type, StatusValue $status, $timeout=0)
	Lock the files at the given storage paths in the backend.

	getTopDirectoryList (array $params)
	Same as FileBackend::getDirectoryList() except only lists directories that are immediately under the given directory.

	getTopFileList (array $params)
	Same as FileBackend::getFileList() except only lists files that are immediately under the given directory.

	getWikiId ()
	Alias to getDomainId()

	hasFeatures ( $bitfield)
	Check if the backend medium supports a field of extra features.

	isReadOnly ()
	Check if this backend is read-only.

	lockFiles (array $paths, $type, $timeout=0)
	Lock the files at the given storage paths in the backend.

	move (array $params, array $opts=[])
	Performs a single move operation.

	prepare (array $params)
	Prepare a storage directory for usage.

	publish (array $params)
	Remove measures to block web access to a storage directory and the container it belongs to.

	quickCopy (array $params)
	Performs a single quick copy operation.

	quickCreate (array $params)
	Performs a single quick create operation.

	quickDelete (array $params)
	Performs a single quick delete operation.

	quickDescribe (array $params)
	Performs a single quick describe operation.

	quickMove (array $params)
	Performs a single quick move operation.

	quickStore (array $params)
	Performs a single quick store operation.

	secure (array $params)
	Take measures to block web access to a storage directory and the container it belongs to.

	setLogger (LoggerInterface $logger)

	store (array $params, array $opts=[])
	Performs a single store operation.

	unlockFiles (array $paths, $type)
	Unlock the files at the given storage paths in the backend.

Protected Member Functions
	doClean (array $params)

	doDirectoryOp ( $method, array $params)

	doOperationsInternal (array $ops, array $opts)

	doPrepare (array $params)

	doPublish (array $params)

	doQuickOperationsInternal (array $ops)

	doSecure (array $params)

	fileStoragePathsForOps (array $ops)
	Get a list of file storage paths to read or write for a list of operations.

	getReadIndexFromParams (array $params)

	hasVolatileSources (array $ops)

	substOpBatchPaths (array $ops, FileBackendStore $backend)
	Substitute the backend name in storage path parameters for a set of operations with that of a given internal backend.

	substOpPaths (array $ops, FileBackendStore $backend)
	Same as substOpBatchPaths() but for a single operation.

	substPaths ( $paths, FileBackendStore $backend)
	Substitute the backend of storage paths with an internal backend's name.

	unsubstPaths ( $paths, FileBackendStore $backend)
	Substitute the backend of internal storage paths with the proxy backend's name.

Protected Member Functions inherited from FileBackend
	newStatus (... $args)
	Yields the result of the status wrapper callback on either:

	resetOutputBuffer ()

	resolveFSFileObjects (array $ops)
	Convert FSFile 'src' paths to string paths (with an 'srcRef' field set to the FSFile)

	scopedProfileSection ( $section)

	wrapStatus (StatusValue $sv)

Protected Attributes
bool	$asyncWrites = false

string bool	$autoResync = false

FileBackendStore[]	$backends = []
	Prioritized list of FileBackendStore objects.

int	$masterIndex = -1
	Index of master backend.

int	$readIndex = -1
	Index of read affinity backend.

int	$syncChecks = 0
	Bitfield.

Protected Attributes inherited from FileBackend
int	$concurrency
	How many operations can be done in parallel.

string	$domainId
	Unique domain name.

FileJournal	$fileJournal

LockManager	$lockManager

LoggerInterface	$logger

string	$name
	Unique backend name.

callable	$obResetFunc

string	$parallelize
	When to do operations in parallel.

callable null	$profiler

string	$readOnly
	Read-only explanation message.

callable	$statusWrapper

callable	$streamMimeFunc

TempFSFileFactory	$tmpFileFactory

Additional Inherited Members
Static Public Member Functions inherited from FileBackend
static	extensionFromPath ( $path, $case='lowercase')
	Get the final extension from a storage or FS path.

static	isPathTraversalFree ( $path)
	Check if a relative path has no directory traversals.

static	isStoragePath ( $path)
	Check if a given path is a "mwstore://" path.

static	makeContentDisposition ( $type, $filename='')
	Build a Content-Disposition header value per RFC 6266.

static	normalizeStoragePath ( $storagePath)
	Normalize a storage path by cleaning up directory separators.

static	parentStoragePath ( $storagePath)
	Get the parent storage directory of a storage path.

static	splitStoragePath ( $storagePath)
	Split a storage path into a backend name, a container name, and a relative file path.

Public Attributes inherited from FileBackend
const	ATTR_HEADERS = 1
	Bitfield flags for supported features.

const	ATTR_METADATA = 2

const	ATTR_UNICODE_PATHS = 4

Static Protected Member Functions inherited from FileBackend
static	normalizeContainerPath ( $path)
	Validate and normalize a relative storage path.

Detailed Description

Proxy backend that mirrors writes to several internal backends.

This class defines a multi-write backend. Multiple backends can be registered to this proxy backend and it will act as a single backend. Use this when all access to those backends is through this proxy backend. At least one of the backends must be declared the "master" backend.

Only use this class when transitioning from one storage system to another.

Read operations are only done on the 'master' backend for consistency. Write operations are performed on all backends, starting with the master. This makes a best-effort to have transactional semantics, but since requests may sometimes fail, the use of "autoResync" or background scripts to fix inconsistencies is important.

Since: 1.19

Definition at line 45 of file FileBackendMultiWrite.php.

Constructor & Destructor Documentation

◆ __construct()

FileBackendMultiWrite::__construct ( array $config )

Construct a proxy backend that consists of several internal backends.

Locking, journaling, and read-only checks are handled by the proxy backend.

Additional $config params include:

backends : Array of backend config and multi-backend settings. Each value is the config used in the constructor of a FileBackendStore class, but with these additional settings:
- class : The name of the backend class
- isMultiMaster : This must be set for one backend.
- readAffinity : Use this for reads without 'latest' set.
syncChecks : Integer bitfield of internal backend sync checks to perform. Possible bits include the FileBackendMultiWrite::CHECK_* constants. There are constants for SIZE, TIME, and SHA1. The checks are done before allowing any file operations.
autoResync : Automatically resync the clone backends to the master backend when pre-operation sync checks fail. This should only be used if the master backend is stable and not missing any files. Use "conservative" to limit resyncing to copying newer master backend files over older (or non-existing) clone backend files. Cases that cannot be handled will result in operation abortion.
replication : Set to 'async' to defer file operations on the non-master backends. This will apply such updates post-send for web requests. Note that any checks from "syncChecks" are still synchronous.

Parameters

array $config

Exceptions

LogicException

Reimplemented from FileBackend.

Definition at line 97 of file FileBackendMultiWrite.php.

References FileBackend\$domainId, FileBackend\$fileJournal, $masterIndex, and FileBackend\$name.

Member Function Documentation

◆ accessibilityCheck()

FileBackendMultiWrite::accessibilityCheck ( array $paths )

Check that a set of file paths are usable across all internal backends.

Parameters

array $paths List of storage paths

Returns: StatusValue

Definition at line 321 of file FileBackendMultiWrite.php.

References $path, FileBackend\newStatus(), and substPaths().

Referenced by doOperationsInternal().

◆ clearCache()

FileBackendMultiWrite::clearCache ( array $paths = null )

Invalidate any in-process file stat and property cache.

If $paths is given, then only the cache for those files will be cleared.

See also: FileBackend::getFileStat()

Parameters

array | null $paths Storage paths (optional)

Reimplemented from FileBackend.

Definition at line 762 of file FileBackendMultiWrite.php.

References substPaths().

Referenced by doOperationsInternal().

◆ concatenate()

FileBackendMultiWrite::concatenate ( array $params )

Concatenate a list of storage files into a single file system file.

The target path should refer to a file that is already locked or otherwise safe from modification from other processes. Normally, the file will be a new temp file, which should be adequate.

Parameters

array

$params

Operation parameters, include:

srcs : ordered source storage paths (e.g. chunk1, chunk2, ...)
dst : file system path to 0-byte temp file
parallelize : try to do operations in parallel when possible

Returns: StatusValue

Reimplemented from FileBackend.

Definition at line 624 of file FileBackendMultiWrite.php.

References concatenate(), getReadIndexFromParams(), FileBackend\newStatus(), and substOpPaths().

Referenced by concatenate().

◆ consistencyCheck()

FileBackendMultiWrite::consistencyCheck ( array $paths )

Check that a set of files are consistent across all internal backends.

This method should only be called if the files are locked or the backend is in read-only mode

Parameters

array $paths List of storage paths

Returns: StatusValue

Definition at line 233 of file FileBackendMultiWrite.php.

References $masterIndex, $path, FileBackend\newStatus(), substOpPaths(), substPaths(), and true.

Referenced by doOperationsInternal().

◆ directoryExists()

FileBackendMultiWrite::directoryExists ( array $params )

Check if a directory exists at a given storage path.

For backends using key/value stores, a directory is said to exist whenever there exist any files with paths using the given directory path as a prefix followed by a forward slash. For example, if there is a file called "mwstore://backend/container/dir/path.svg" then directories are said to exist at "mwstore://backend/container" and "mwstore://backend/container/dir". These can be thought of as "virtual" directories.

Backends that directly use a filesystem layer might enumerate empty directories. The clean() method should always be used when files are deleted or moved if this is a concern. This is a trade-off to avoid write amplication/contention on file changes or read amplification when calling this method.

Storage backends with eventual consistency might return stale data.

See also: FileBackend::EXISTENCE_ERROR; FileBackend::clean()

Parameters

array

$params

Parameters include:

dir : storage directory

Returns: bool|null Whether a directory exists or null (I/O error)

Since: 1.20

Reimplemented from FileBackend.

Definition at line 740 of file FileBackendMultiWrite.php.

References $masterIndex, and substOpPaths().

◆ doClean()

FileBackendMultiWrite::doClean ( array $params )

protected

See also: FileBackend::clean()

Parameters

array $params

Returns: StatusValue

Reimplemented from FileBackend.

Definition at line 588 of file FileBackendMultiWrite.php.

References doDirectoryOp().

◆ doDirectoryOp()

FileBackendMultiWrite::doDirectoryOp	(		$method,
		array	$params
	)

protected

Parameters

string	$method	One of (doPrepare,doSecure,doPublish,doClean)
array	$params	Method arguments

Returns: StatusValue

Definition at line 597 of file FileBackendMultiWrite.php.

References $masterIndex, FileBackend\newStatus(), and substOpPaths().

Referenced by doClean(), doPrepare(), doPublish(), and doSecure().

◆ doOperationsInternal()

FileBackendMultiWrite::doOperationsInternal	(	array	$ops,
		array	$opts
	)

finalprotected

See also: FileBackend::doOperations()

Parameters

array	$ops
array	$opts

Returns: StatusValue

Reimplemented from FileBackend.

Definition at line 141 of file FileBackendMultiWrite.php.

References $masterIndex, accessibilityCheck(), clearCache(), consistencyCheck(), fileStoragePathsForOps(), getScopedLocksForOps(), hasVolatileSources(), FileBackend\newStatus(), resyncFiles(), and substOpBatchPaths().

◆ doPrepare()

FileBackendMultiWrite::doPrepare ( array $params )

protected

See also: FileBackend::prepare()

Parameters

array $params

Returns: StatusValue

Reimplemented from FileBackend.

Definition at line 576 of file FileBackendMultiWrite.php.

References doDirectoryOp().

◆ doPublish()

FileBackendMultiWrite::doPublish ( array $params )

protected

See also: FileBackend::publish()

Parameters

array $params

Returns: StatusValue

Reimplemented from FileBackend.

Definition at line 584 of file FileBackendMultiWrite.php.

References doDirectoryOp().

◆ doQuickOperationsInternal()

FileBackendMultiWrite::doQuickOperationsInternal ( array $ops )

protected

See also: FileBackend::doQuickOperations()

Parameters

array $ops

Returns: StatusValue

Since: 1.20

Reimplemented from FileBackend.

Definition at line 543 of file FileBackendMultiWrite.php.

References $masterIndex, hasVolatileSources(), FileBackend\newStatus(), and substOpBatchPaths().

◆ doSecure()

FileBackendMultiWrite::doSecure ( array $params )

protected

See also: FileBackend::secure()

Parameters

array $params

Returns: StatusValue

Reimplemented from FileBackend.

Definition at line 580 of file FileBackendMultiWrite.php.

References doDirectoryOp().

◆ fileExists()

FileBackendMultiWrite::fileExists ( array $params )

Check if a file exists at a storage path in the backend.

This returns false if only a directory exists at the path.

Callers that only care if a file is readily accessible can use non-strict comparisons on the result. If "does not exist" and "existence is unknown" must be distinguished, then strict comparisons to true/null should be used.

See also: FileBackend::EXISTENCE_ERROR; FileBackend::directoryExists()

Parameters

array

$params

Parameters include:

src : source storage path
latest : use the latest available data

Returns: bool|null Whether the file exists or null (I/O error)

Reimplemented from FileBackend.

Definition at line 635 of file FileBackendMultiWrite.php.

References getReadIndexFromParams(), and substOpPaths().

Referenced by fileStoragePathsForOps().

◆ fileStoragePathsForOps()

FileBackendMultiWrite::fileStoragePathsForOps ( array $ops )

protected

Get a list of file storage paths to read or write for a list of operations.

Parameters

array $ops Same format as doOperations()

Returns: array List of storage paths to files (does not include directories)

Definition at line 440 of file FileBackendMultiWrite.php.

References fileExists().

Referenced by doOperationsInternal().

◆ getDirectoryList()

FileBackendMultiWrite::getDirectoryList ( array $params )

Get an iterator to list all directories under a storage directory.

If the directory is of the form "mwstore://backend/container", then all directories in the container will be listed. If the directory is of form "mwstore://backend/container/dir", then all directories directly under that directory will be listed. Results will be storage directories relative to the given directory.

Storage backends with eventual consistency might return stale data.

Failures during iteration can result in FileBackendError exceptions (since 1.22).

See also: FileBackend::LIST_ERROR; FileBackend::directoryExists()

Parameters

array

$params

Parameters include:

dir : storage directory
topOnly : only return direct child dirs of the directory

Returns: Traversable|array|null Directory list enumerator or null (initial I/O error)

Since: 1.20

Reimplemented from FileBackend.

Definition at line 746 of file FileBackendMultiWrite.php.

References $masterIndex, and substOpPaths().

◆ getFeatures()

FileBackendMultiWrite::getFeatures ( )

Get the a bitfield of extra features supported by the backend medium.

Returns: int Bitfield of FileBackend::ATTR_* flags

Since: 1.23

Reimplemented from FileBackend.

Definition at line 758 of file FileBackendMultiWrite.php.

References $masterIndex.

◆ getFileContentsMulti()

FileBackendMultiWrite::getFileContentsMulti ( array $params )

Like getFileContents() except it takes an array of storage paths and returns an order preserved map of storage paths to their content.

See also: FileBackend::getFileContents()

Parameters

array

$params

Parameters include:

srcs : list of source storage paths
latest : use the latest available data
parallelize : try to do operations in parallel when possible

Returns: string[]|false[] Map of (path name => file content or false on failure)

Since: 1.20

Reimplemented from FileBackend.

Definition at line 670 of file FileBackendMultiWrite.php.

References $path, getReadIndexFromParams(), substOpPaths(), and unsubstPaths().

◆ getFileHttpUrl()

FileBackendMultiWrite::getFileHttpUrl ( array $params )

Return an HTTP URL to a given file that requires no authentication to use.

The URL may be pre-authenticated (via some token in the URL) and temporary. This will return null if the backend cannot make an HTTP URL for the file.

This is useful for key/value stores when using scripts that seek around large files and those scripts (and the backend) support HTTP Range headers. Otherwise, one would need to use getLocalReference(), which involves loading the entire file on to local disk.

See also: FileBackend::TEMPURL_ERROR

Parameters

array

$params

Parameters include:

src : source storage path
ttl : lifetime (seconds) if pre-authenticated; default is 1 day

Returns: string|null URL or null (not supported or I/O error)

Since: 1.21

Reimplemented from FileBackend.

Definition at line 733 of file FileBackendMultiWrite.php.

References getReadIndexFromParams(), and substOpPaths().

◆ getFileList()

FileBackendMultiWrite::getFileList ( array $params )

Get an iterator to list all stored files under a storage directory.

If the directory is of the form "mwstore://backend/container", then all files in the container will be listed. If the directory is of form "mwstore://backend/container/dir", then all files under that directory will be listed. Results will be storage paths relative to the given directory.

Storage backends with eventual consistency might return stale data.

Failures during iteration can result in FileBackendError exceptions (since 1.22).

See also: FileBackend::LIST_ERROR

Parameters

array

$params

Parameters include:

dir : storage directory
topOnly : only return direct child files of the directory (since 1.20)
adviseStat : set to true if stat requests will be made on the files (since 1.22)

Returns: Traversable|array|null File list enumerator or null (initial I/O error)

Reimplemented from FileBackend.

Definition at line 752 of file FileBackendMultiWrite.php.

References $masterIndex, and substOpPaths().

◆ getFileProps()

FileBackendMultiWrite::getFileProps ( array $params )

Get the properties of the content of the file at a storage path in the backend.

This gives the result of FSFile::getProps() on a local copy of the file.

Parameters

array

$params

Parameters include:

src : source storage path
latest : use the latest available data

Returns: array Properties map; FSFile::placeholderProps() if file missing or on I/O error

Reimplemented from FileBackend.

Definition at line 691 of file FileBackendMultiWrite.php.

References getReadIndexFromParams(), and substOpPaths().

◆ getFileSha1Base36()

FileBackendMultiWrite::getFileSha1Base36 ( array $params )

Get a SHA-1 hash of the content of the file at a storage path in the backend.

See also: FileBackend::SHA1_FAIL

Parameters

array

$params

Parameters include:

src : source storage path
latest : use the latest available data

Returns: string|false Hash string or false (missing file or I/O error)

Reimplemented from FileBackend.

Definition at line 684 of file FileBackendMultiWrite.php.

References getReadIndexFromParams(), and substOpPaths().

◆ getFileSize()

FileBackendMultiWrite::getFileSize ( array $params )

Get the size (bytes) of a file at a storage path in the backend.

See also: FileBackend::SIZE_FAIL

Parameters

array

$params

Parameters include:

src : source storage path
latest : use the latest available data

Returns: int|false File size in bytes or false (missing file or I/O error)

Reimplemented from FileBackend.

Definition at line 649 of file FileBackendMultiWrite.php.

References getReadIndexFromParams(), and substOpPaths().

◆ getFileStat()

FileBackendMultiWrite::getFileStat ( array $params )

Get quick information about a file at a storage path in the backend.

If the file does not exist, then this returns false. Otherwise, the result is an associative array that includes:

mtime : the last-modified timestamp (TS_MW)
size : the file size (bytes) Additional values may be included for internal use only.

See also: FileBackend::STAT_ABSENT; FileBackend::STAT_ERROR

Parameters

array

$params

Parameters include:

src : source storage path
latest : use the latest available data

Returns: array|false|null Attribute map, false (missing file), or null (I/O error)

Reimplemented from FileBackend.

Definition at line 656 of file FileBackendMultiWrite.php.

References getReadIndexFromParams(), and substOpPaths().

◆ getFileTimestamp()

FileBackendMultiWrite::getFileTimestamp ( array $params )

Get the last-modified timestamp of the file at a storage path.

See also: FileBackend::TIMESTAMP_FAIL

Parameters

array

$params

Parameters include:

src : source storage path
latest : use the latest available data

Returns: string|false TS_MW timestamp or false (missing file or I/O error)

Reimplemented from FileBackend.

Definition at line 642 of file FileBackendMultiWrite.php.

References getReadIndexFromParams(), and substOpPaths().

◆ getFileXAttributes()

FileBackendMultiWrite::getFileXAttributes ( array $params )

Get metadata about a file at a storage path in the backend.

If the file does not exist, then this returns false. Otherwise, the result is an associative array that includes:

headers : map of HTTP headers used for GET/HEAD requests (name => value)
metadata : map of file metadata (name => value) Metadata keys and headers names will be returned in all lower-case. Additional values may be included for internal use only.

Use FileBackend::hasFeatures() to check how well this is supported.

See also: FileBackend::XATTRS_FAIL

Parameters

array

$params

$params include:

src : source storage path
latest : use the latest available data

Returns: array|false File metadata array or false (missing file or I/O error)

Since: 1.23

Reimplemented from FileBackend.

Definition at line 663 of file FileBackendMultiWrite.php.

References getReadIndexFromParams(), and substOpPaths().

◆ getLocalCopyMulti()

FileBackendMultiWrite::getLocalCopyMulti ( array $params )

Like getLocalCopy() except it takes an array of storage paths and yields an order preserved-map of storage paths to temporary local file copies.

Multiple calls to this method for the same path will create new copies.

See also: FileBackend::getLocalCopy()

Parameters

array

$params

Parameters include:

srcs : list of source storage paths
latest : use the latest available data
parallelize : try to do operations in parallel when possible

Returns: array Map of (path name => TempFSFile or null on failure)

Since: 1.20

Reimplemented from FileBackend.

Definition at line 719 of file FileBackendMultiWrite.php.

References $path, getReadIndexFromParams(), substOpPaths(), and unsubstPaths().

◆ getLocalReferenceMulti()

FileBackendMultiWrite::getLocalReferenceMulti ( array $params )

Like getLocalReference() except it takes an array of storage paths and yields an order-preserved map of storage paths to temporary local file copies.

Never modify the returned files since they might be originals, they might be shared among multiple callers of this method, or the backend might internally keep FSFile references for deferred operations.

See also: FileBackend::getLocalReference()

Parameters

array

$params

Parameters include:

srcs : list of source storage paths
latest : use the latest available data
parallelize : try to do operations in parallel when possible

Returns: array Map of (path name => FSFile or null on failure)

Since: 1.20

Reimplemented from FileBackend.

Definition at line 705 of file FileBackendMultiWrite.php.

References $path, getReadIndexFromParams(), substOpPaths(), and unsubstPaths().

◆ getReadIndexFromParams()

FileBackendMultiWrite::getReadIndexFromParams ( array $params )

protected

Parameters

array $params

Returns: int The master or read affinity backend index, based on $params['latest']

Definition at line 806 of file FileBackendMultiWrite.php.

References $readIndex.

Referenced by concatenate(), fileExists(), getFileContentsMulti(), getFileHttpUrl(), getFileProps(), getFileSha1Base36(), getFileSize(), getFileStat(), getFileTimestamp(), getFileXAttributes(), getLocalCopyMulti(), getLocalReferenceMulti(), preloadFileStat(), and streamFile().

◆ getScopedLocksForOps()

FileBackendMultiWrite::getScopedLocksForOps	(	array	$ops,
		StatusValue	$status
	)

Get an array of scoped locks needed for a batch of file operations.

Normally, FileBackend::doOperations() handles locking, unless the 'nonLocking' param is passed in. This function is useful if you want the files to be locked for a broader scope than just when the files are changing. For example, if you need to update DB metadata, you may want to keep the files locked until finished.

See also: FileBackend::doOperations()

Parameters

array	$ops	List of file operations to FileBackend::doOperations()
StatusValue	$status	StatusValue to update on lock/unlock

Returns: ScopedLock|null RAII-style self-unlocking lock or null on failure

Since: 1.20

Reimplemented from FileBackend.

Definition at line 781 of file FileBackendMultiWrite.php.

References $masterIndex, FileBackend\getScopedFileLocks(), substOpBatchPaths(), and unsubstPaths().

Referenced by doOperationsInternal().

◆ hasVolatileSources()

FileBackendMultiWrite::hasVolatileSources ( array $ops )

protected

Parameters

array $ops File operations for FileBackend::doOperations()

Returns: bool Whether there are file path sources with outside lifetime/ownership

Definition at line 533 of file FileBackendMultiWrite.php.

Referenced by doOperationsInternal(), and doQuickOperationsInternal().

◆ preloadCache()

FileBackendMultiWrite::preloadCache ( array $paths )

Preload persistent file stat cache and property cache into in-process cache.

This should be used when stat calls will be made on a known list of a many files.

See also: FileBackend::getFileStat()

Parameters

array $paths Storage paths

Reimplemented from FileBackend.

Definition at line 769 of file FileBackendMultiWrite.php.

References $readIndex, and substPaths().

◆ preloadFileStat()

FileBackendMultiWrite::preloadFileStat ( array $params )

Preload file stat information (concurrently if possible) into in-process cache.

This should be used when stat calls will be made on a known list of a many files. This does not make use of the persistent file stat cache.

See also: FileBackend::getFileStat()

Parameters

array

$params

Parameters include:

srcs : list of source storage paths
latest : use the latest available data

Returns: bool Whether all requests proceeded without I/O errors (since 1.24)

Since: 1.23

Reimplemented from FileBackend.

Definition at line 774 of file FileBackendMultiWrite.php.

References getReadIndexFromParams(), and substOpPaths().

◆ resyncFiles()

FileBackendMultiWrite::resyncFiles	(	array	$paths,
			$resyncMode = `true`
	)

Check that a set of files are consistent across all internal backends and re-synchronize those files against the "multi master" if needed.

This method should only be called if the files are locked

Parameters

array	$paths	List of storage paths
string \| bool	$resyncMode	False, True, or "conservative"; see __construct()

Returns: StatusValue

Definition at line 349 of file FileBackendMultiWrite.php.

References $masterIndex, $path, FileBackend\newStatus(), substOpPaths(), and true.

Referenced by doOperationsInternal().

◆ streamFile()

FileBackendMultiWrite::streamFile ( array $params )

Stream the content of the file at a storage path in the backend.

If the file does not exists, an HTTP 404 error will be given. Appropriate HTTP headers (Status, Content-Type, Content-Length) will be sent if streaming began, while none will be sent otherwise. Implementations should flush the output buffer before sending data.

Parameters

array

$params

Parameters include:

src : source storage path
headers : list of additional HTTP headers to send if the file exists
options : HTTP request header map with lower case keys (since 1.28). Supports: range : format is "bytes=(\\d*-\\d*)" if-modified-since : format is an HTTP date
headless : only include the body (and headers from "headers") (since 1.28)
latest : use the latest available data
allowOB : preserve any output buffers (since 1.28)

Returns: StatusValue

Reimplemented from FileBackend.

Definition at line 698 of file FileBackendMultiWrite.php.

References getReadIndexFromParams(), and substOpPaths().

◆ substOpBatchPaths()

FileBackendMultiWrite::substOpBatchPaths	(	array	$ops,
		FileBackendStore	$backend
	)

protected

Substitute the backend name in storage path parameters for a set of operations with that of a given internal backend.

Parameters

array	$ops	List of file operation arrays
FileBackendStore	$backend

Returns: array

Definition at line 471 of file FileBackendMultiWrite.php.

References substPaths().

Referenced by doOperationsInternal(), doQuickOperationsInternal(), getScopedLocksForOps(), and substOpPaths().

◆ substOpPaths()

FileBackendMultiWrite::substOpPaths	(	array	$ops,
		FileBackendStore	$backend
	)

protected

Same as substOpBatchPaths() but for a single operation.

Parameters

array	$ops	File operation array
FileBackendStore	$backend

Returns: array

Definition at line 493 of file FileBackendMultiWrite.php.

References substOpBatchPaths().

Referenced by concatenate(), consistencyCheck(), directoryExists(), doDirectoryOp(), fileExists(), getDirectoryList(), getFileContentsMulti(), getFileHttpUrl(), getFileList(), getFileProps(), getFileSha1Base36(), getFileSize(), getFileStat(), getFileTimestamp(), getFileXAttributes(), getLocalCopyMulti(), getLocalReferenceMulti(), preloadFileStat(), resyncFiles(), and streamFile().

◆ substPaths()

FileBackendMultiWrite::substPaths	(		$paths,
		FileBackendStore	$backend
	)

protected

Substitute the backend of storage paths with an internal backend's name.

Parameters

array \| string	$paths	List of paths or single string path
FileBackendStore	$backend

Returns: string[]|string

Definition at line 506 of file FileBackendMultiWrite.php.

References StringUtils\escapeRegexReplacement().

Referenced by accessibilityCheck(), clearCache(), consistencyCheck(), preloadCache(), and substOpBatchPaths().

◆ unsubstPaths()

FileBackendMultiWrite::unsubstPaths	(		$paths,
		FileBackendStore	$backend
	)

protected

Substitute the backend of internal storage paths with the proxy backend's name.

Parameters

array \| string	$paths	List of paths or single string path
FileBackendStore	$backend	internal storage backend

Returns: string[]|string

Definition at line 521 of file FileBackendMultiWrite.php.

References StringUtils\escapeRegexReplacement(), and FileBackend\getName().

Referenced by getFileContentsMulti(), getLocalCopyMulti(), getLocalReferenceMulti(), and getScopedLocksForOps().

Member Data Documentation

◆ $asyncWrites

bool FileBackendMultiWrite::$asyncWrites = false

protected

Definition at line 60 of file FileBackendMultiWrite.php.

◆ $autoResync

string bool FileBackendMultiWrite::$autoResync = false

protected

Definition at line 57 of file FileBackendMultiWrite.php.

◆ $backends

FileBackendStore [] FileBackendMultiWrite::$backends = []

protected

Prioritized list of FileBackendStore objects.

Definition at line 47 of file FileBackendMultiWrite.php.

◆ $masterIndex

int FileBackendMultiWrite::$masterIndex = -1

protected

Index of master backend.

Definition at line 50 of file FileBackendMultiWrite.php.

Referenced by __construct(), consistencyCheck(), directoryExists(), doDirectoryOp(), doOperationsInternal(), doQuickOperationsInternal(), getDirectoryList(), getFeatures(), getFileList(), getScopedLocksForOps(), and resyncFiles().

◆ $readIndex

int FileBackendMultiWrite::$readIndex = -1

protected

Index of read affinity backend.

Definition at line 52 of file FileBackendMultiWrite.php.

Referenced by getReadIndexFromParams(), and preloadCache().

◆ $syncChecks

int FileBackendMultiWrite::$syncChecks = 0

protected

Bitfield.

Definition at line 55 of file FileBackendMultiWrite.php.

The documentation for this class was generated from the following file:

includes/libs/filebackend/FileBackendMultiWrite.php

Public Member Functions

Protected Member Functions

Protected Attributes

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ __construct()

Member Function Documentation

◆ accessibilityCheck()

◆ clearCache()

◆ concatenate()

◆ consistencyCheck()

◆ directoryExists()

◆ doClean()

◆ doDirectoryOp()

◆ doOperationsInternal()

◆ doPrepare()

◆ doPublish()

◆ doQuickOperationsInternal()

◆ doSecure()

◆ fileExists()

◆ fileStoragePathsForOps()

◆ getDirectoryList()

◆ getFeatures()

◆ getFileContentsMulti()

◆ getFileHttpUrl()

◆ getFileList()

◆ getFileProps()

◆ getFileSha1Base36()

◆ getFileSize()

◆ getFileStat()

◆ getFileTimestamp()

◆ getFileXAttributes()

◆ getLocalCopyMulti()

◆ getLocalReferenceMulti()

◆ getReadIndexFromParams()

◆ getScopedLocksForOps()

◆ hasVolatileSources()

◆ preloadCache()

◆ preloadFileStat()

◆ resyncFiles()

◆ streamFile()

◆ substOpBatchPaths()

◆ substOpPaths()

◆ substPaths()

◆ unsubstPaths()

Member Data Documentation

◆ $asyncWrites

◆ $autoResync

◆ $backends

◆ $masterIndex

◆ $readIndex

◆ $syncChecks