Class for an OpenStack Swift (or Ceph RGW) based file backend. More...

Inheritance diagram for SwiftFileBackend:

[legend]

Collaboration diagram for SwiftFileBackend:

[legend]

Public Member Functions
	__construct (array $config)

	getDirectoryListInternal ( $fullCont, $dir, array $params)

	getDirListPageInternal ( $fullCont, $dir, &$after, $limit, array $params)
	Do not call this function outside of SwiftFileBackendFileList.

	getFeatures ()
	Get the a bitfield of extra features supported by the backend medium.

	getFileHttpUrl (array $params)

	getFileListInternal ( $fullCont, $dir, array $params)

	getFileListPageInternal ( $fullCont, $dir, &$after, $limit, array $params)
	Do not call this function outside of SwiftFileBackendFileList.

	isPathUsableInternal ( $storagePath)
	Check if a file can be created or changed at a given storage path in the backend.

	loadListingStatInternal ( $path, array $val)
	Do not call this function outside of SwiftFileBackendFileList.

	onError ( $status, $func, array $params, $err='', $code=0, $desc='')
	Log an unexpected exception for this backend.

Public Member Functions inherited from FileBackendStore
	clearCache (array $paths=null)
	Invalidate any in-process file stat and property cache.

	concatenate (array $params)
	Concatenate a list of storage files into a single file system file.

	copyInternal (array $params)
	Copy a file from one storage path to another in the backend.

	createInternal (array $params)
	Create a file in the backend with the given contents.

	deleteInternal (array $params)
	Delete a file at the storage path.

	describeInternal (array $params)
	Alter metadata for a file at the storage path.

	directoryExists (array $params)
	Check if a directory exists at a given storage path.

	executeOpHandlesInternal (array $fileOpHandles)
	Execute a list of FileBackendStoreOpHandle handles in parallel.

	fileExists (array $params)
	Check if a file exists at a storage path in the backend.

	getDirectoryList (array $params)
	Get an iterator to list all directories under a storage directory.

	getFileContentsMulti (array $params)
	Like getFileContents() except it takes an array of storage paths and returns an order preserved map of storage paths to their content.

	getFileList (array $params)
	Get an iterator to list all stored files under a storage directory.

	getFileProps (array $params)
	Get the properties of the content of the file at a storage path in the backend.

	getFileSha1Base36 (array $params)
	Get a SHA-1 hash of the content of the file at a storage path in the backend.

	getFileSize (array $params)
	Get the size (bytes) of a file at a storage path in the backend.

	getFileStat (array $params)
	Get quick information about a file at a storage path in the backend.

	getFileTimestamp (array $params)
	Get the last-modified timestamp of the file at a storage path.

	getFileXAttributes (array $params)
	Get metadata about a file at a storage path in the backend.

	getLocalCopyMulti (array $params)
	Like getLocalCopy() except it takes an array of storage paths and yields an order preserved-map of storage paths to temporary local file copies.

	getLocalReferenceMulti (array $params)
	Like getLocalReference() except it takes an array of storage paths and yields an order-preserved map of storage paths to temporary local file copies.

	getOperationsInternal (array $ops)
	Return a list of FileOp objects from a list of operations.

	getPathsToLockForOpsInternal (array $performOps)
	Get a list of storage paths to lock for a list of operations Returns an array with LockManager::LOCK_UW (shared locks) and LockManager::LOCK_EX (exclusive locks) keys, each corresponding to a list of storage paths to be locked.

	getScopedLocksForOps (array $ops, StatusValue $status)
	Get an array of scoped locks needed for a batch of file operations.

	isSingleShardPathInternal ( $storagePath)
	Check if a storage path maps to a single shard.

	maxFileSizeInternal ()
	Get the maximum allowable file size given backend medium restrictions and basic performance constraints.

	moveInternal (array $params)
	Move a file from one storage path to another in the backend.

	nullInternal (array $params)
	No-op file operation that does nothing.

	preloadCache (array $paths)
	Preload persistent file stat cache and property cache into in-process cache.

	preloadFileStat (array $params)
	Preload file stat information (concurrently if possible) into in-process cache.

	storeInternal (array $params)
	Store a file into the backend from a file on disk.

	streamFile (array $params)
	Stream the content of the file at a storage path in the backend.

Public Member Functions inherited from FileBackend
	clean (array $params)
	Delete a storage directory if it is empty.

	copy (array $params, array $opts=[])
	Performs a single copy operation.

	create (array $params, array $opts=[])
	Performs a single create operation.

	delete (array $params, array $opts=[])
	Performs a single delete operation.

	describe (array $params, array $opts=[])
	Performs a single describe operation.

	doOperation (array $op, array $opts=[])
	Same as doOperations() except it takes a single operation.

	doOperations (array $ops, array $opts=[])
	This is the main entry point into the backend for write operations.

	doQuickOperation (array $op)
	Same as doQuickOperations() except it takes a single operation.

	doQuickOperations (array $ops, array $opts=[])
	Perform a set of independent file operations on some files.

	getContainerStoragePath ( $container)
	Get the storage path for the given container for this backend.

	getDomainId ()
	Get the domain identifier used for this backend (possibly empty).

	getFileContents (array $params)
	Get the contents of a file at a storage path in the backend.

	getJournal ()
	Get the file journal object for this backend.

	getLocalCopy (array $params)
	Get a local copy on disk of the file at a storage path in the backend.

	getLocalReference (array $params)
	Returns a file system file, identical in content to the file at a storage path.

	getName ()
	Get the unique backend name.

	getReadOnlyReason ()
	Get an explanatory message if this backend is read-only.

	getRootStoragePath ()
	Get the root storage path of this backend.

	getScopedFileLocks (array $paths, $type, StatusValue $status, $timeout=0)
	Lock the files at the given storage paths in the backend.

	getTopDirectoryList (array $params)
	Same as FileBackend::getDirectoryList() except only lists directories that are immediately under the given directory.

	getTopFileList (array $params)
	Same as FileBackend::getFileList() except only lists files that are immediately under the given directory.

	getWikiId ()
	Alias to getDomainId()

	hasFeatures ( $bitfield)
	Check if the backend medium supports a field of extra features.

	isReadOnly ()
	Check if this backend is read-only.

	lockFiles (array $paths, $type, $timeout=0)
	Lock the files at the given storage paths in the backend.

	move (array $params, array $opts=[])
	Performs a single move operation.

	prepare (array $params)
	Prepare a storage directory for usage.

	publish (array $params)
	Remove measures to block web access to a storage directory and the container it belongs to.

	quickCopy (array $params)
	Performs a single quick copy operation.

	quickCreate (array $params)
	Performs a single quick create operation.

	quickDelete (array $params)
	Performs a single quick delete operation.

	quickDescribe (array $params)
	Performs a single quick describe operation.

	quickMove (array $params)
	Performs a single quick move operation.

	quickStore (array $params)
	Performs a single quick store operation.

	secure (array $params)
	Take measures to block web access to a storage directory and the container it belongs to.

	setLogger (LoggerInterface $logger)

	store (array $params, array $opts=[])
	Performs a single store operation.

	unlockFiles (array $paths, $type)
	Unlock the files at the given storage paths in the backend.

Protected Member Functions
	addMissingHashMetadata (array $objHdrs, $path)
	Fill in any missing object metadata and save it to Swift.

	authTokenHeaders (array $creds)

	convertSwiftDate ( $ts, $format=TS_MW)
	Convert dates like "Tue, 03 Jan 2012 22:01:04 GMT"/"2013-05-11T07:37:27.678360Z".

	createContainer ( $container, array $params)
	Create a Swift container.

	deleteContainer ( $container, array $params)
	Delete a Swift container.

	directoriesAreVirtual ()
	Is this a key/value store where directories are just virtual? Virtual directories exists in so much as files exists that are prefixed with the directory path followed by a forward slash.

	doCleanInternal ( $fullCont, $dir, array $params)

	doCopyInternal (array $params)

	doCreateInternal (array $params)

	doDeleteInternal (array $params)

	doDescribeInternal (array $params)

	doDirectoryExists ( $fullCont, $dir, array $params)

	doExecuteOpHandlesInternal (array $fileOpHandles)

	doGetFileContentsMulti (array $params)

	doGetFileSha1base36 (array $params)

	doGetFileStat (array $params)

	doGetFileStatMulti (array $params)
	Get file stat information (concurrently if possible) for several files.

	doGetFileXAttributes (array $params)

	doGetLocalCopyMulti (array $params)

	doMoveInternal (array $params)

	doPrepareInternal ( $fullCont, $dir, array $params)

	doPrimeContainerCache (array $containerInfo)
	Fill the backend-specific process cache given an array of resolved container names and their corresponding cached info.

	doPublishInternal ( $fullCont, $dir, array $params)

	doSecureInternal ( $fullCont, $dir, array $params)

	doStoreInternal (array $params)

	doStreamFile (array $params)

	extractMetadataHeaders (array $headers)

	extractMutableContentHeaders (array $headers)
	Filter/normalize a header map to only include mutable "content-"/"x-content-" headers.

	getAuthentication ()

	getContainerStat ( $container, $bypassCache=false)
	Get a Swift container stat array, possibly from process cache.

	getMetadataFromHeaders (array $headers)

	getStatFromHeaders (array $rhdrs)

	headersFromParams (array $params)
	Get headers to send to Swift when reading a file based on a FileBackend params array, e.g.

	resolveContainerPath ( $container, $relStoragePath)
	Resolve a relative storage path, checking if it's allowed by the backend.

	setContainerAccess ( $container, array $readUsers, array $writeUsers)
	Set read/write permissions for a Swift container.

	storageUrl (array $creds, $container=null, $object=null)

Protected Member Functions inherited from FileBackendStore
	deleteContainerCache ( $container)
	Delete the cached info for a container.

	deleteFileCache ( $path)
	Delete the cached stat info for a file path.

	doClean (array $params)

	doClearCache (array $paths=null)
	Clears any additional stat caches for storage paths.

	doConcatenate (array $params)

	doGetFileSha1Base36 (array $params)

	doGetLocalReferenceMulti (array $params)

	doOperationsInternal (array $ops, array $opts)

	doPrepare (array $params)

	doPublish (array $params)

	doQuickOperationsInternal (array $ops)

	doSecure (array $params)

	fullContainerName ( $container)
	Get the full container name, including the domain ID prefix.

	getContainerHashLevels ( $container)
	Get the sharding config for a container.

	getContainerShard ( $container, $relPath)
	Get the container name shard suffix for a given path.

	getContainerSuffixes ( $container)
	Get a list of full container shard suffixes for a container.

	getContentType ( $storagePath, $content, $fsPath)
	Get the content type to use in HEAD/GET requests for a file.

	ingestFreshFileStats (array $stats, $latest)
	Ingest file stat entries that just came from querying the backend (not cache)

	primeContainerCache (array $items)
	Do a batch lookup from cache for container stats for all containers used in a list of container names or storage paths objects.

	primeFileCache (array $items)
	Do a batch lookup from cache for file stats for all paths used in a list of storage paths or FileOp objects.

	resolveContainerName ( $container)
	Resolve a container name, checking if it's allowed by the backend.

	resolveStoragePath ( $storagePath)
	Splits a storage path into an internal container name, an internal relative file name, and a container shard suffix.

	resolveStoragePathReal ( $storagePath)
	Like resolveStoragePath() except null values are returned if the container is sharded and the shard could not be determined or if the path ends with '/'.

	sanitizeOpHeaders (array $op)
	Normalize and filter HTTP headers from a file operation.

	setConcurrencyFlags (array $opts)
	Set the 'concurrency' option from a list of operation options.

	setContainerCache ( $container, array $val)
	Set the cached info for a container.

	setFileCache ( $path, array $val)
	Set the cached stat info for a file path.

Protected Member Functions inherited from FileBackend
	newStatus (... $args)
	Yields the result of the status wrapper callback on either:

	resetOutputBuffer ()

	resolveFSFileObjects (array $ops)
	Convert FSFile 'src' paths to string paths (with an 'srcRef' field set to the FSFile)

	scopedProfileSection ( $section)

	wrapStatus (StatusValue $sv)

Protected Attributes
array	$authCreds

int	$authErrorTimestamp = null
	UNIX timestamp.

int	$authSessionTimestamp = 0
	UNIX timestamp.

int	$authTTL
	TTL in seconds.

MapCacheLRU	$containerStatCache
	Container stat cache.

MultiHttpClient	$http

bool	$isRGW = false
	Whether the server is an Ceph RGW.

array	$readUsers
	Additional users (account:user) with read permissions on public containers.

string	$rgwS3AccessKey
	S3 access key (RADOS Gateway)

string	$rgwS3SecretKey
	S3 authentication key (RADOS Gateway)

array	$secureReadUsers
	Additional users (account:user) with read permissions on private containers.

array	$secureWriteUsers
	Additional users (account:user) with write permissions on private containers.

BagOStuff	$srvCache

string	$swiftAuthUrl
	Authentication base URL (without version)

string	$swiftKey
	Secret key for user.

string	$swiftStorageUrl
	Override of storage base URL.

string	$swiftTempUrlKey
	Shared secret value for making temp URLs.

string	$swiftUser
	Swift user (account:user) to authenticate as.

array	$writeUsers
	Additional users (account:user) with write permissions on public containers.

Protected Attributes inherited from FileBackendStore
MapCacheLRU	$cheapCache
	Map of paths to small (RAM/disk) cache items.

MapCacheLRU	$expensiveCache
	Map of paths to large (RAM/disk) cache items.

	$maxFileSize = 4294967296

WANObjectCache	$memCache

callable	$mimeCallback
	Method to get the MIME type of files.

array	$shardViaHashLevels = []
	Map of container names to sharding config.

BagOStuff	$srvCache

Protected Attributes inherited from FileBackend
int	$concurrency
	How many operations can be done in parallel.

string	$domainId
	Unique domain name.

FileJournal	$fileJournal

LockManager	$lockManager

LoggerInterface	$logger

string	$name
	Unique backend name.

callable	$obResetFunc

string	$parallelize
	When to do operations in parallel.

callable null	$profiler

string	$readOnly
	Read-only explanation message.

callable	$statusWrapper

callable	$streamMimeFunc

TempFSFileFactory	$tmpFileFactory

Private Member Functions
	buildFileObjectListing (array $objects)
	Build a list of file objects, filtering out any directories and extracting any stat info if provided in $objects.

	getCredsCacheKey ( $username)
	Get the cache key for a container.

	objectListing ( $fullCont, $type, $limit, $after=null, $prefix=null, $delim=null)
	Get a list of objects under a container.

Additional Inherited Members
Static Public Member Functions inherited from FileBackend
static	extensionFromPath ( $path, $case='lowercase')
	Get the final extension from a storage or FS path.

static	isPathTraversalFree ( $path)
	Check if a relative path has no directory traversals.

static	isStoragePath ( $path)
	Check if a given path is a "mwstore://" path.

static	makeContentDisposition ( $type, $filename='')
	Build a Content-Disposition header value per RFC 6266.

static	normalizeStoragePath ( $storagePath)
	Normalize a storage path by cleaning up directory separators.

static	parentStoragePath ( $storagePath)
	Get the parent storage directory of a storage path.

static	splitStoragePath ( $storagePath)
	Split a storage path into a backend name, a container name, and a relative file path.

Public Attributes inherited from FileBackendStore
const	CACHE_CHEAP_SIZE = 500

const	CACHE_EXPENSIVE_SIZE = 5

const	CACHE_TTL = 10

Public Attributes inherited from FileBackend
const	ATTR_HEADERS = 1
	Bitfield flags for supported features.

const	ATTR_METADATA = 2

const	ATTR_UNICODE_PATHS = 4

Static Protected Member Functions inherited from FileBackendStore
static	isValidContainerName ( $container)
	Check if a full container name is valid.

static	isValidShortContainerName ( $container)
	Check if a short container name is valid.

static	normalizeXAttributes (array $xattr)
	Normalize file headers/metadata to the FileBackend::getFileXAttributes() format.

Static Protected Member Functions inherited from FileBackend
static	normalizeContainerPath ( $path)
	Validate and normalize a relative storage path.

Static Protected Attributes inherited from FileBackendStore
static string	$ABSENT_LATEST = 'FNE-L'
	File does not exist according to a "latest"-mode stat query.

static string	$ABSENT_NORMAL = 'FNE-N'
	File does not exist according to a normal stat query.

static false	$RES_ABSENT = false
	Idiom for "no result due to missing file" (since 1.34)

static null	$RES_ERROR = null
	Idiom for "no result due to I/O errors" (since 1.34)

Detailed Description

Class for an OpenStack Swift (or Ceph RGW) based file backend.

StatusValue messages should avoid mentioning the Swift account name. Likewise, error suppression should be used to avoid path disclosure.

Since: 1.19

Definition at line 36 of file SwiftFileBackend.php.

Constructor & Destructor Documentation

◆ __construct()

SwiftFileBackend::__construct ( array $config )

See also: FileBackendStore::__construct()

Parameters

array

$config

Params include:

swiftAuthUrl : Swift authentication server URL
swiftUser : Swift user used by MediaWiki (account:username)
swiftKey : Swift authentication key for the above user
swiftAuthTTL : Swift authentication TTL (seconds)
swiftTempUrlKey : Swift "X-Account-Meta-Temp-URL-Key" value on the account. Do not set this until it has been set in the backend.
swiftStorageUrl : Swift storage URL (overrides that of the authentication response). This is useful to set if a TLS proxy is in use.
shardViaHashLevels : Map of container names to sharding config with:
- base : base of hash characters, 16 or 36
- levels : the number of hash levels (and digits)
- repeat : hash subdirectories are prefixed with all the parent hash directory names (e.g. "a/ab/abc")
cacheAuthInfo : Whether to cache authentication tokens in APC, etc. If those are not available, then the main cache will be used. This is probably insecure in shared hosting environments.
rgwS3AccessKey : Rados Gateway S3 "access key" value on the account. Do not set this until it has been set in the backend. This is used for generating expiring pre-authenticated URLs. Only use this when using rgw and to work around http://tracker.newdream.net/issues/3454.
rgwS3SecretKey : Rados Gateway S3 "secret key" value on the account. Do not set this until it has been set in the backend. This is used for generating expiring pre-authenticated URLs. Only use this when using rgw and to work around http://tracker.newdream.net/issues/3454.
readUsers : Swift users with read access to public containers (account:username)
writeUsers : Swift users with write access to public containers (account:username)
secureReadUsers : Swift users with read access to private containers (account:username)
secureWriteUsers : Swift users with write access to private containers (account:username)

Reimplemented from FileBackendStore.

Definition at line 114 of file SwiftFileBackend.php.

Member Function Documentation

◆ addMissingHashMetadata()

SwiftFileBackend::addMissingHashMetadata	(	array	$objHdrs,
			$path )

protected

Fill in any missing object metadata and save it to Swift.

Parameters

array	$objHdrs	Object response headers
string	$path	Storage path to object

Returns: array New headers

Definition at line 752 of file SwiftFileBackend.php.

References $path, authTokenHeaders(), FileBackendStore\deleteFileCache(), extractMetadataHeaders(), extractMutableContentHeaders(), getAuthentication(), FileBackend\getLocalCopy(), FileBackend\getScopedFileLocks(), FileBackend\newStatus(), FileBackendStore\resolveStoragePathReal(), FileBackend\scopedProfileSection(), and storageUrl().

Referenced by doGetFileStatMulti().

◆ authTokenHeaders()

SwiftFileBackend::authTokenHeaders ( array $creds )

protected

Parameters

array $creds From getAuthentication()

Returns: array

Definition at line 1797 of file SwiftFileBackend.php.

Referenced by addMissingHashMetadata(), createContainer(), deleteContainer(), doExecuteOpHandlesInternal(), doGetFileContentsMulti(), doGetFileStatMulti(), doGetLocalCopyMulti(), doStreamFile(), getContainerStat(), objectListing(), and setContainerAccess().

◆ buildFileObjectListing()

SwiftFileBackend::buildFileObjectListing ( array $objects )

private

Build a list of file objects, filtering out any directories and extracting any stat info if provided in $objects.

Parameters

stdClass[] | string[] $objects List of stdClass items or object names

Returns: array List of (names,stat array or null) entries

Definition at line 1043 of file SwiftFileBackend.php.

References convertSwiftDate().

Referenced by getFileListPageInternal().

◆ convertSwiftDate()

SwiftFileBackend::convertSwiftDate	(		$ts,
			$format = TS_MW )

protected

Convert dates like "Tue, 03 Jan 2012 22:01:04 GMT"/"2013-05-11T07:37:27.678360Z".

Dates might also come in like "2013-05-11T07:37:27.678360" from Swift listings, missing the timezone suffix (though Ceph RGW does not appear to have this bug).

Parameters

string	$ts
int	$format	Output format (TS_* constant)

Returns: string

Exceptions

FileBackendError

Definition at line 735 of file SwiftFileBackend.php.

Referenced by buildFileObjectListing(), and getStatFromHeaders().

◆ createContainer()

SwiftFileBackend::createContainer	(		$container,
		array	$params )

protected

Create a Swift container.

Parameters

string	$container	Container name
array	$params

Returns: StatusValue

Definition at line 1488 of file SwiftFileBackend.php.

References $readUsers, $writeUsers, authTokenHeaders(), getAuthentication(), FileBackend\newStatus(), onError(), and storageUrl().

Referenced by doPrepareInternal().

◆ deleteContainer()

SwiftFileBackend::deleteContainer	(		$container,
		array	$params )

protected

Delete a Swift container.

Parameters

string	$container	Container name
array	$params

Returns: StatusValue

Definition at line 1536 of file SwiftFileBackend.php.

References authTokenHeaders(), getAuthentication(), FileBackend\newStatus(), onError(), and storageUrl().

Referenced by doCleanInternal().

◆ directoriesAreVirtual()

SwiftFileBackend::directoriesAreVirtual ( )

protected

Is this a key/value store where directories are just virtual? Virtual directories exists in so much as files exists that are prefixed with the directory path followed by a forward slash.

Returns: bool

Reimplemented from FileBackendStore.

Definition at line 1299 of file SwiftFileBackend.php.

◆ doCleanInternal()

SwiftFileBackend::doCleanInternal	(		$container,
			$dir,
		array	$params )

protected

See also: FileBackendStore::doClean()

Parameters

string	$container
string	$dir
array	$params

Returns: StatusValue

Reimplemented from FileBackendStore.

Definition at line 689 of file SwiftFileBackend.php.

References deleteContainer(), getContainerStat(), and FileBackend\newStatus().

◆ doCopyInternal()

SwiftFileBackend::doCopyInternal ( array $params )

protected

See also: FileBackendStore::copyInternal()

Parameters

array $params

Returns: StatusValue

Reimplemented from FileBackendStore.

Definition at line 393 of file SwiftFileBackend.php.

References FileBackendStore\executeOpHandlesInternal(), extractMutableContentHeaders(), FileBackend\newStatus(), onError(), and FileBackendStore\resolveStoragePathReal().

◆ doCreateInternal()

SwiftFileBackend::doCreateInternal ( array $params )

protected

See also: FileBackendStore::createInternal()

Parameters

array $params

Returns: StatusValue

Reimplemented from FileBackendStore.

Definition at line 249 of file SwiftFileBackend.php.

References FileBackendStore\executeOpHandlesInternal(), extractMutableContentHeaders(), FileBackendStore\getContentType(), FileBackend\newStatus(), onError(), and FileBackendStore\resolveStoragePathReal().

◆ doDeleteInternal()

SwiftFileBackend::doDeleteInternal ( array $params )

protected

See also: FileBackendStore::deleteInternal()

Parameters

array $params

Returns: StatusValue

Reimplemented from FileBackendStore.

Definition at line 515 of file SwiftFileBackend.php.

References FileBackendStore\executeOpHandlesInternal(), FileBackend\newStatus(), onError(), and FileBackendStore\resolveStoragePathReal().

◆ doDescribeInternal()

SwiftFileBackend::doDescribeInternal ( array $params )

protected

See also: FileBackendStore::describeInternal()

Parameters

array $params

Returns: StatusValue

Reimplemented from FileBackendStore.

Definition at line 557 of file SwiftFileBackend.php.

References FileBackend\$name, doGetFileStat(), FileBackendStore\executeOpHandlesInternal(), extractMutableContentHeaders(), FileBackendStore\getFileStat(), FileBackend\newStatus(), onError(), and FileBackendStore\resolveStoragePathReal().

◆ doDirectoryExists()

SwiftFileBackend::doDirectoryExists	(		$container,
			$dir,
		array	$params )

protected

See also: FileBackendStore::directoryExists()

Parameters

string	$container	Resolved container name
string	$dir	Resolved path relative to container
array	$params

Returns: bool|null

Reimplemented from FileBackendStore.

Definition at line 864 of file SwiftFileBackend.php.

References FileBackendStore\$RES_ERROR, and objectListing().

◆ doExecuteOpHandlesInternal()

SwiftFileBackend::doExecuteOpHandlesInternal ( array $fileOpHandles )

protected

See also: FileBackendStore::executeOpHandlesInternal()

Parameters

FileBackendStoreOpHandle[] $fileOpHandles

Exceptions

FileBackendError

Returns: StatusValue[] List of corresponding StatusValue objects

Reimplemented from FileBackendStore.

Definition at line 1320 of file SwiftFileBackend.php.

References $s, authTokenHeaders(), getAuthentication(), FileBackend\newStatus(), and storageUrl().

◆ doGetFileContentsMulti()

SwiftFileBackend::doGetFileContentsMulti ( array $params )

protected

See also: FileBackendStore::getFileContentsMulti()

Parameters

array $params

Returns: string[]|bool[]|null[] Map of (path => string, false (missing), or null (error))

Reimplemented from FileBackendStore.

Definition at line 806 of file SwiftFileBackend.php.

References $content, $path, FileBackendStore\$RES_ABSENT, FileBackendStore\$RES_ERROR, authTokenHeaders(), getAuthentication(), headersFromParams(), onError(), FileBackendStore\resolveStoragePathReal(), and storageUrl().

◆ doGetFileSha1base36()

SwiftFileBackend::doGetFileSha1base36 ( array $params )

protected

Definition at line 1094 of file SwiftFileBackend.php.

References FileBackendStore\$RES_ABSENT, FileBackendStore\$RES_ERROR, and FileBackendStore\getFileStat().

◆ doGetFileStat()

SwiftFileBackend::doGetFileStat ( array $params )

protected

See also: FileBackendStore::getFileStat()

Parameters

array $params

Reimplemented from FileBackendStore.

Definition at line 717 of file SwiftFileBackend.php.

References doGetFileStatMulti().

Referenced by doDescribeInternal().

◆ doGetFileStatMulti()

SwiftFileBackend::doGetFileStatMulti ( array $params )

protected

Get file stat information (concurrently if possible) for several files.

See also: FileBackend::getFileStat()

Parameters

array

$params

Parameters include:

srcs : list of source storage paths
latest : use the latest available data

Returns: array|null Map of storage paths to array|bool|null (returns null if not supported)

Since: 1.23

Reimplemented from FileBackendStore.

Definition at line 1634 of file SwiftFileBackend.php.

References $path, FileBackendStore\$RES_ABSENT, FileBackendStore\$RES_ERROR, addMissingHashMetadata(), authTokenHeaders(), getAuthentication(), getContainerStat(), getStatFromHeaders(), headersFromParams(), onError(), FileBackendStore\resolveStoragePathReal(), and storageUrl().

Referenced by doGetFileStat().

◆ doGetFileXAttributes()

SwiftFileBackend::doGetFileXAttributes ( array $params )

protected

See also: FileBackendStore::getFileXAttributes()

Parameters

array $params

Returns: array[][]|false|null Attributes, false (missing file), or null (error)

Reimplemented from FileBackendStore.

Definition at line 1079 of file SwiftFileBackend.php.

References FileBackendStore\$RES_ABSENT, FileBackendStore\$RES_ERROR, FileBackendStore\clearCache(), and FileBackendStore\getFileStat().

◆ doGetLocalCopyMulti()

SwiftFileBackend::doGetLocalCopyMulti ( array $params )

protected

See also: FileBackendStore::getLocalCopyMulti()

Parameters

array $params

Returns: string[]|bool[]|null[] Map of (path => TempFSFile, false (missing), or null (error))

Reimplemented from FileBackendStore.

Definition at line 1173 of file SwiftFileBackend.php.

References FileBackendStore\$ABSENT_NORMAL, $ext, $path, FileBackendStore\$RES_ABSENT, FileBackendStore\$RES_ERROR, authTokenHeaders(), FileBackend\extensionFromPath(), getAuthentication(), getStatFromHeaders(), headersFromParams(), onError(), FileBackendStore\resolveStoragePathReal(), and storageUrl().

◆ doMoveInternal()

SwiftFileBackend::doMoveInternal ( array $params )

protected

See also: FileBackendStore::moveInternal()

Parameters

array $params

Returns: StatusValue

Reimplemented from FileBackendStore.

Definition at line 448 of file SwiftFileBackend.php.

References FileBackendStore\executeOpHandlesInternal(), extractMutableContentHeaders(), FileBackend\newStatus(), onError(), and FileBackendStore\resolveStoragePathReal().

◆ doPrepareInternal()

SwiftFileBackend::doPrepareInternal	(		$container,
			$dir,
		array	$params )

protected

See also: FileBackendStore::doPrepare()

Parameters

string	$container
string	$dir
array	$params

Returns: StatusValue

Reimplemented from FileBackendStore.

Definition at line 616 of file SwiftFileBackend.php.

References createContainer(), getContainerStat(), and FileBackend\newStatus().

◆ doPrimeContainerCache()

SwiftFileBackend::doPrimeContainerCache ( array $containerInfo )

protected

Fill the backend-specific process cache given an array of resolved container names and their corresponding cached info.

Only containers that actually exist should appear in the map.

Parameters

array $containerInfo Map of resolved container names to cached info

Reimplemented from FileBackendStore.

Definition at line 1628 of file SwiftFileBackend.php.

◆ doPublishInternal()

SwiftFileBackend::doPublishInternal	(		$container,
			$dir,
		array	$params )

protected

See also: FileBackendStore::doPublish()

Parameters

string	$container
string	$dir
array	$params

Returns: StatusValue

Reimplemented from FileBackendStore.

Definition at line 665 of file SwiftFileBackend.php.

References $readUsers, $writeUsers, getContainerStat(), FileBackend\newStatus(), and setContainerAccess().

◆ doSecureInternal()

SwiftFileBackend::doSecureInternal	(		$container,
			$dir,
		array	$params )

protected

See also: FileBackendStore::doSecure()

Parameters

string	$container
string	$dir
array	$params

Returns: StatusValue

Reimplemented from FileBackendStore.

Definition at line 639 of file SwiftFileBackend.php.

References $readUsers, $writeUsers, getContainerStat(), FileBackend\newStatus(), and setContainerAccess().

◆ doStoreInternal()

SwiftFileBackend::doStoreInternal ( array $params )

protected

See also: FileBackendStore::storeInternal()

Parameters

array $params

Returns: StatusValue

Reimplemented from FileBackendStore.

Definition at line 304 of file SwiftFileBackend.php.

References FileBackendStore\executeOpHandlesInternal(), extractMutableContentHeaders(), FileBackendStore\getContentType(), FileBackend\newStatus(), onError(), and FileBackendStore\resolveStoragePathReal().

◆ doStreamFile()

SwiftFileBackend::doStreamFile ( array $params )

protected

See also: FileBackendStore::streamFile()

Parameters

array $params

Returns: StatusValue

Reimplemented from FileBackendStore.

Definition at line 1107 of file SwiftFileBackend.php.

References $header, FileBackend\$obResetFunc, authTokenHeaders(), FileBackendStore\clearCache(), FileBackendStore\deleteFileCache(), FileBackendStore\fileExists(), getAuthentication(), getContainerStat(), headersFromParams(), FileBackend\newStatus(), onError(), FileBackendStore\resolveStoragePathReal(), HTTPFileStreamer\send404Message(), storageUrl(), and HTTPFileStreamer\STREAM_HEADLESS.

◆ extractMetadataHeaders()

SwiftFileBackend::extractMetadataHeaders ( array $headers )

protected

See also: https://docs.openstack.org/api-ref/object-store

Parameters

string[] $headers Map of (header => value) for a swift object

Returns: string[] Map of (metadata header name => metadata value)

Definition at line 221 of file SwiftFileBackend.php.

References FileBackend\$name.

Referenced by addMissingHashMetadata(), and getMetadataFromHeaders().

◆ extractMutableContentHeaders()

SwiftFileBackend::extractMutableContentHeaders ( array $headers )

protected

Filter/normalize a header map to only include mutable "content-"/"x-content-" headers.

Mutable headers can be changed via HTTP POST even if the file content is the same

See also: https://docs.openstack.org/api-ref/object-store

Parameters

string[] $headers Map of (header => value) for a swift object

Returns: string[] Map of (header => value) for Content-* headers mutable via POST

Definition at line 183 of file SwiftFileBackend.php.

References FileBackend\$name.

Referenced by addMissingHashMetadata(), doCopyInternal(), doCreateInternal(), doDescribeInternal(), doMoveInternal(), doStoreInternal(), and getStatFromHeaders().

◆ getAuthentication()

SwiftFileBackend::getAuthentication ( )

protected

Returns: array|null Credential map

Definition at line 1716 of file SwiftFileBackend.php.

References $authCreds, $authSessionTimestamp, getCredsCacheKey(), and onError().

Referenced by addMissingHashMetadata(), createContainer(), deleteContainer(), doExecuteOpHandlesInternal(), doGetFileContentsMulti(), doGetFileStatMulti(), doGetLocalCopyMulti(), doStreamFile(), getContainerStat(), getFileHttpUrl(), objectListing(), and setContainerAccess().

◆ getContainerStat()

SwiftFileBackend::getContainerStat	(		$container,
			$bypassCache = false )

protected

Get a Swift container stat array, possibly from process cache.

Use $reCache if the file count or byte count is needed.

Parameters

string	$container	Container name
bool	$bypassCache	Bypass all caches and load from Swift

Returns: array|bool|null False on 404, null on failure

Definition at line 1436 of file SwiftFileBackend.php.

References FileBackendStore\$RES_ABSENT, FileBackendStore\$RES_ERROR, authTokenHeaders(), getAuthentication(), onError(), FileBackendStore\primeContainerCache(), FileBackend\scopedProfileSection(), FileBackendStore\setContainerCache(), and storageUrl().

Referenced by doCleanInternal(), doGetFileStatMulti(), doPrepareInternal(), doPublishInternal(), doSecureInternal(), doStreamFile(), and isPathUsableInternal().

◆ getCredsCacheKey()

SwiftFileBackend::getCredsCacheKey ( $username )

private

Get the cache key for a container.

Parameters

string $username

Returns: string

Definition at line 1807 of file SwiftFileBackend.php.

Referenced by getAuthentication(), and onError().

◆ getDirectoryListInternal()

SwiftFileBackend::getDirectoryListInternal	(		$fullCont,
			$dir,
		array	$params )

See also: FileBackendStore::getDirectoryListInternal()

Parameters

string	$fullCont
string	$dir
array	$params

Returns: SwiftFileBackendDirList

Reimplemented from FileBackendStore.

Definition at line 881 of file SwiftFileBackend.php.

◆ getDirListPageInternal()

SwiftFileBackend::getDirListPageInternal	(		$fullCont,
			$dir,
		&	$after,
			$limit,
		array	$params )

Do not call this function outside of SwiftFileBackendFileList.

Parameters

string	$fullCont	Resolved container name
string	$dir	Resolved storage directory with no trailing slash
string \| null	&$after	Resolved container relative path to list items after
int	$limit	Max number of items to list
array	$params	Parameters for getDirectoryList()

Returns: array List of container relative resolved paths of directories directly under $dir

Exceptions

FileBackendError

Definition at line 907 of file SwiftFileBackend.php.

References $path, objectListing(), and FileBackend\scopedProfileSection().

◆ getFeatures()

SwiftFileBackend::getFeatures ( )

Get the a bitfield of extra features supported by the backend medium.

Returns: int Bitfield of FileBackend::ATTR_* flags

Since: 1.23

Reimplemented from FileBackend.

Definition at line 147 of file SwiftFileBackend.php.

◆ getFileHttpUrl()

SwiftFileBackend::getFileHttpUrl ( array $params )

See also: FileBackend::getFileHttpUrl()

Parameters

array $params

Returns: string|null

Reimplemented from FileBackendStore.

Definition at line 1245 of file SwiftFileBackend.php.

References getAuthentication(), FileBackendStore\resolveStoragePathReal(), and storageUrl().

◆ getFileListInternal()

SwiftFileBackend::getFileListInternal	(		$fullCont,
			$dir,
		array	$params )

See also: FileBackendStore::getFileListInternal()

Parameters

string	$fullCont
string	$dir
array	$params

Returns: SwiftFileBackendFileList

Reimplemented from FileBackendStore.

Definition at line 892 of file SwiftFileBackend.php.

◆ getFileListPageInternal()

SwiftFileBackend::getFileListPageInternal	(		$fullCont,
			$dir,
		&	$after,
			$limit,
		array	$params )

Do not call this function outside of SwiftFileBackendFileList.

Parameters

string	$fullCont	Resolved container name
string	$dir	Resolved storage directory with no trailing slash
string \| null	&$after	Resolved container relative path of file to list items after
int	$limit	Max number of items to list
array	$params	Parameters for getDirectoryList()

Returns: array List of resolved container relative paths of files under $dir

Exceptions

FileBackendError

Definition at line 990 of file SwiftFileBackend.php.

References buildFileObjectListing(), objectListing(), and FileBackend\scopedProfileSection().

◆ getMetadataFromHeaders()

SwiftFileBackend::getMetadataFromHeaders ( array $headers )

protected

See also: https://docs.openstack.org/api-ref/object-store

Parameters

string[] $headers Map of (header => value) for a swift object

Returns: string[] Map of (metadata key name => metadata value)

Definition at line 238 of file SwiftFileBackend.php.

References FileBackend\$name, and extractMetadataHeaders().

Referenced by getStatFromHeaders().

◆ getStatFromHeaders()

SwiftFileBackend::getStatFromHeaders ( array $rhdrs )

protected

Parameters

array $rhdrs

Returns: array

Definition at line 1695 of file SwiftFileBackend.php.

References convertSwiftDate(), extractMutableContentHeaders(), and getMetadataFromHeaders().

Referenced by doGetFileStatMulti(), and doGetLocalCopyMulti().

◆ headersFromParams()

SwiftFileBackend::headersFromParams ( array $params )

protected

Get headers to send to Swift when reading a file based on a FileBackend params array, e.g.

that of getLocalCopy(). $params is currently only checked for a 'latest' flag.

Parameters

array $params

Returns: array

Definition at line 1311 of file SwiftFileBackend.php.

Referenced by doGetFileContentsMulti(), doGetFileStatMulti(), doGetLocalCopyMulti(), and doStreamFile().

◆ isPathUsableInternal()

SwiftFileBackend::isPathUsableInternal ( $storagePath )

Check if a file can be created or changed at a given storage path in the backend.

FS backends should check that the parent directory exists, files can be written under it, and that any file already there is both readable and writable. Backends using key/value stores should check if the container exists.

Parameters

string $storagePath

Returns: bool

Reimplemented from FileBackendStore.

Definition at line 165 of file SwiftFileBackend.php.

References getContainerStat(), and FileBackendStore\resolveStoragePathReal().

◆ loadListingStatInternal()

SwiftFileBackend::loadListingStatInternal	(		$path,
		array	$val )

Do not call this function outside of SwiftFileBackendFileList.

Parameters

string	$path	Storage path
array	$val	Stat value

Definition at line 1075 of file SwiftFileBackend.php.

References $path.

◆ objectListing()

SwiftFileBackend::objectListing	(	$fullCont,
		$type,
		$limit,
		$after = null,
		$prefix = null,
		$delim = null )

private

Get a list of objects under a container.

Either just the names or a list of stdClass objects with details can be returned.

Parameters

string	$fullCont
string	$type	('info' for a list of object detail maps, 'names' for names only)
int	$limit
string \| null	$after
string \| null	$prefix
string \| null	$delim

Returns: StatusValue With the list as value

Definition at line 1577 of file SwiftFileBackend.php.

References $type, authTokenHeaders(), getAuthentication(), FileBackend\newStatus(), onError(), and storageUrl().

Referenced by doDirectoryExists(), getDirListPageInternal(), and getFileListPageInternal().

◆ onError()

SwiftFileBackend::onError	(		$status,
			$func,
		array	$params,
			$err = '',
			$code = 0,
			$desc = '' )

Log an unexpected exception for this backend.

This also sets the StatusValue object to have a fatal error.

Parameters

StatusValue \| null	$status
string	$func
array	$params
string	$err	Error string
int	$code	HTTP status
string	$desc	HTTP StatusValue description

Definition at line 1822 of file SwiftFileBackend.php.

References getCredsCacheKey().

Referenced by createContainer(), deleteContainer(), doCopyInternal(), doCreateInternal(), doDeleteInternal(), doDescribeInternal(), doGetFileContentsMulti(), doGetFileStatMulti(), doGetLocalCopyMulti(), doMoveInternal(), doStoreInternal(), doStreamFile(), getAuthentication(), getContainerStat(), and objectListing().

◆ resolveContainerPath()

SwiftFileBackend::resolveContainerPath	(		$container,
			$relStoragePath )

protected

Resolve a relative storage path, checking if it's allowed by the backend.

This is intended for internal use, such as encoding illegal chars or perhaps getting absolute paths (e.g. FS based backends). Note that the relative path may be the empty string (e.g. the path is simply to the container).

Parameters

string	$container	Container name
string	$relStoragePath	Storage path relative to the container

Returns: string|null Path or null if not valid

Reimplemented from FileBackendStore.

Definition at line 155 of file SwiftFileBackend.php.

◆ setContainerAccess()

SwiftFileBackend::setContainerAccess	(		$container,
		array	$readUsers,
		array	$writeUsers )

protected

Set read/write permissions for a Swift container.

See also: http://docs.openstack.org/developer/swift/misc.html#acls

In general, we don't allow listings to end-users. It's not useful, isn't well-defined (lists are truncated to 10000 item with no way to page), and is just a performance risk.

Parameters

string	$container	Resolved Swift container
array	$readUsers	List of the possible criteria for a request to have access to read a container. Each item is one of the following formats: account:user : Grants access if the request is by the given user ".r:<regex>" : Grants access if the request is from a referrer host that matches the expression and the request is not for a listing. Setting this to '*' effectively makes a container public. -".rlistings:<regex>" : Grants access if the request is from a referrer host that matches the expression and the request is for a listing.
array	$writeUsers	A list of the possible criteria for a request to have access to write to a container. Each item is of the following format: account:user : Grants access if the request is by the given user

Returns: StatusValue

Definition at line 1400 of file SwiftFileBackend.php.

References $readUsers, $writeUsers, authTokenHeaders(), getAuthentication(), FileBackend\newStatus(), and storageUrl().

Referenced by doPublishInternal(), and doSecureInternal().

◆ storageUrl()

SwiftFileBackend::storageUrl	(	array	$creds,
			$container = null,
			$object = null )

protected

Parameters

array	$creds	From getAuthentication()
string \| null	$container
string \| null	$object

Returns: string

Definition at line 1781 of file SwiftFileBackend.php.

Referenced by addMissingHashMetadata(), createContainer(), deleteContainer(), doExecuteOpHandlesInternal(), doGetFileContentsMulti(), doGetFileStatMulti(), doGetLocalCopyMulti(), doStreamFile(), getContainerStat(), getFileHttpUrl(), objectListing(), and setContainerAccess().

Member Data Documentation

◆ $authCreds

array SwiftFileBackend::$authCreds

protected

Definition at line 71 of file SwiftFileBackend.php.

Referenced by getAuthentication().

◆ $authErrorTimestamp

int SwiftFileBackend::$authErrorTimestamp = null

protected

UNIX timestamp.

Definition at line 75 of file SwiftFileBackend.php.

◆ $authSessionTimestamp

int SwiftFileBackend::$authSessionTimestamp = 0

protected

UNIX timestamp.

Definition at line 73 of file SwiftFileBackend.php.

Referenced by getAuthentication().

◆ $authTTL

int SwiftFileBackend::$authTTL

protected

TTL in seconds.

Definition at line 40 of file SwiftFileBackend.php.

◆ $containerStatCache

MapCacheLRU SwiftFileBackend::$containerStatCache

protected

Container stat cache.

Definition at line 68 of file SwiftFileBackend.php.

◆ $http

MultiHttpClient SwiftFileBackend::$http

protected

Definition at line 38 of file SwiftFileBackend.php.

◆ $isRGW

bool SwiftFileBackend::$isRGW = false

protected

Whether the server is an Ceph RGW.

Definition at line 78 of file SwiftFileBackend.php.

◆ $readUsers

array SwiftFileBackend::$readUsers

protected

Additional users (account:user) with read permissions on public containers.

Definition at line 56 of file SwiftFileBackend.php.

Referenced by createContainer(), doPublishInternal(), doSecureInternal(), and setContainerAccess().

◆ $rgwS3AccessKey

string SwiftFileBackend::$rgwS3AccessKey

protected

S3 access key (RADOS Gateway)

Definition at line 52 of file SwiftFileBackend.php.

◆ $rgwS3SecretKey

string SwiftFileBackend::$rgwS3SecretKey

protected

S3 authentication key (RADOS Gateway)

Definition at line 54 of file SwiftFileBackend.php.

◆ $secureReadUsers

array SwiftFileBackend::$secureReadUsers

protected

Additional users (account:user) with read permissions on private containers.

Definition at line 60 of file SwiftFileBackend.php.

◆ $secureWriteUsers

array SwiftFileBackend::$secureWriteUsers

protected

Additional users (account:user) with write permissions on private containers.

Definition at line 62 of file SwiftFileBackend.php.

◆ $srvCache

BagOStuff SwiftFileBackend::$srvCache

protected

Definition at line 65 of file SwiftFileBackend.php.

◆ $swiftAuthUrl

string SwiftFileBackend::$swiftAuthUrl

protected

Authentication base URL (without version)

Definition at line 42 of file SwiftFileBackend.php.

◆ $swiftKey

string SwiftFileBackend::$swiftKey

protected

Secret key for user.

Definition at line 48 of file SwiftFileBackend.php.

◆ $swiftStorageUrl

string SwiftFileBackend::$swiftStorageUrl

protected

Override of storage base URL.

Definition at line 44 of file SwiftFileBackend.php.

◆ $swiftTempUrlKey

string SwiftFileBackend::$swiftTempUrlKey

protected

Shared secret value for making temp URLs.

Definition at line 50 of file SwiftFileBackend.php.

◆ $swiftUser

string SwiftFileBackend::$swiftUser

protected

Swift user (account:user) to authenticate as.

Definition at line 46 of file SwiftFileBackend.php.

◆ $writeUsers

array SwiftFileBackend::$writeUsers

protected

Additional users (account:user) with write permissions on public containers.

Definition at line 58 of file SwiftFileBackend.php.

Referenced by createContainer(), doPublishInternal(), doSecureInternal(), and setContainerAccess().

The documentation for this class was generated from the following file:

includes/libs/filebackend/SwiftFileBackend.php

Public Member Functions

Protected Member Functions

Protected Attributes

Private Member Functions

Additional Inherited Members

Detailed Description

Constructor & Destructor Documentation

◆ __construct()

Member Function Documentation

◆ addMissingHashMetadata()

◆ authTokenHeaders()

◆ buildFileObjectListing()

◆ convertSwiftDate()

◆ createContainer()

◆ deleteContainer()

◆ directoriesAreVirtual()

◆ doCleanInternal()

◆ doCopyInternal()

◆ doCreateInternal()

◆ doDeleteInternal()

◆ doDescribeInternal()

◆ doDirectoryExists()

◆ doExecuteOpHandlesInternal()

◆ doGetFileContentsMulti()

◆ doGetFileSha1base36()

◆ doGetFileStat()

◆ doGetFileStatMulti()

◆ doGetFileXAttributes()

◆ doGetLocalCopyMulti()

◆ doMoveInternal()

◆ doPrepareInternal()

◆ doPrimeContainerCache()

◆ doPublishInternal()

◆ doSecureInternal()

◆ doStoreInternal()

◆ doStreamFile()

◆ extractMetadataHeaders()

◆ extractMutableContentHeaders()

◆ getAuthentication()

◆ getContainerStat()

◆ getCredsCacheKey()

◆ getDirectoryListInternal()

◆ getDirListPageInternal()

◆ getFeatures()

◆ getFileHttpUrl()

◆ getFileListInternal()

◆ getFileListPageInternal()

◆ getMetadataFromHeaders()

◆ getStatFromHeaders()

◆ headersFromParams()

◆ isPathUsableInternal()

◆ loadListingStatInternal()

◆ objectListing()

◆ onError()

◆ resolveContainerPath()

◆ setContainerAccess()

◆ storageUrl()

Member Data Documentation

◆ $authCreds

◆ $authErrorTimestamp

◆ $authSessionTimestamp

◆ $authTTL

◆ $containerStatCache

◆ $http

◆ $isRGW

◆ $readUsers

◆ $rgwS3AccessKey

◆ $rgwS3SecretKey

◆ $secureReadUsers

◆ $secureWriteUsers

◆ $srvCache

◆ $swiftAuthUrl

◆ $swiftKey

◆ $swiftStorageUrl

◆ $swiftTempUrlKey

◆ $swiftUser

◆ $writeUsers