MediaWiki master
Wikimedia\FileBackend\SwiftFileBackend Class Reference

Class for an OpenStack Swift (or Ceph RGW) based file backend. More...

Inherits Wikimedia\FileBackend\FileBackendStore.

Collaboration diagram for Wikimedia\FileBackend\SwiftFileBackend:

Public Member Functions

 __construct (array $config)
 
 addShellboxInputFile (BoxedCommand $command, string $boxedName, array $params)
 Add a file to a Shellbox command as an input file.
 
 getDirectoryListInternal ( $fullCont, $dir, array $params)
 
 getDirListPageInternal ( $fullCont, $dir, &$after, $limit, array $params)
 Do not call this function outside of SwiftFileBackendFileList.
 
 getFeatures ()
 Get the a bitfield of extra features supported by the backend medium.
 
 getFileHttpUrl (array $params)
 
 getFileListInternal ( $fullCont, $dir, array $params)
 
 getFileListPageInternal ( $fullCont, $dir, &$after, $limit, array $params)
 Do not call this function outside of SwiftFileBackendFileList.
 
 isPathUsableInternal ( $storagePath)
 Check if a file can be created or changed at a given storage path in the backend.
 
 loadListingStatInternal ( $path, array $val)
 Do not call this function outside of SwiftFileBackendFileList.
 
 onError ( $status, $func, array $params, $err='', $code=0, $desc='', $body='')
 Log an unexpected exception for this backend.
 
 setLogger (LoggerInterface $logger)
 
- Public Member Functions inherited from Wikimedia\FileBackend\FileBackendStore
 clearCache (?array $paths=null)
 Invalidate any in-process file stat and property cache.
 
 concatenate (array $params)
 Concatenate a list of storage files into a single file system file.
 
 copyInternal (array $params)
 Copy a file from one storage path to another in the backend.
 
 createInternal (array $params)
 Create a file in the backend with the given contents.
 
 deleteInternal (array $params)
 Delete a file at the storage path.
 
 describeInternal (array $params)
 Alter metadata for a file at the storage path.
 
 directoryExists (array $params)
 Check if a directory exists at a given storage path.
 
 executeOpHandlesInternal (array $fileOpHandles)
 Execute a list of FileBackendStoreOpHandle handles in parallel.
 
 fileExists (array $params)
 Check if a file exists at a storage path in the backend.
 
 getDirectoryList (array $params)
 Get an iterator to list all directories under a storage directory.
 
 getFileContentsMulti (array $params)
 Like getFileContents() except it takes an array of storage paths and returns an order preserved map of storage paths to their content.
 
 getFileList (array $params)
 Get an iterator to list all stored files under a storage directory.
 
 getFileProps (array $params)
 Get the properties of the content of the file at a storage path in the backend.
 
 getFileSha1Base36 (array $params)
 Get a SHA-1 hash of the content of the file at a storage path in the backend.
 
 getFileSize (array $params)
 Get the size (bytes) of a file at a storage path in the backend.
 
 getFileStat (array $params)
 Get quick information about a file at a storage path in the backend.
 
 getFileTimestamp (array $params)
 Get the last-modified timestamp of the file at a storage path.
 
 getFileXAttributes (array $params)
 Get metadata about a file at a storage path in the backend.
 
 getLocalCopyMulti (array $params)
 Like getLocalCopy() except it takes an array of storage paths and yields an order preserved-map of storage paths to temporary local file copies.
 
 getLocalReferenceMulti (array $params)
 Like getLocalReference() except it takes an array of storage paths and yields an order-preserved map of storage paths to temporary local file copies.
 
 getOperationsInternal (array $ops)
 Return a list of FileOp objects from a list of operations.
 
 getPathsToLockForOpsInternal (array $performOps)
 Get a list of storage paths to lock for a list of operations Returns an array with LockManager::LOCK_UW (shared locks) and LockManager::LOCK_EX (exclusive locks) keys, each corresponding to a list of storage paths to be locked.
 
 getScopedLocksForOps (array $ops, StatusValue $status)
 Get an array of scoped locks needed for a batch of file operations.
 
 isSingleShardPathInternal ( $storagePath)
 Check if a storage path maps to a single shard.
 
 maxFileSizeInternal ()
 Get the maximum allowable file size given backend medium restrictions and basic performance constraints.
 
 moveInternal (array $params)
 Move a file from one storage path to another in the backend.
 
 nullInternal (array $params)
 No-op file operation that does nothing.
 
 preloadCache (array $paths)
 Preload persistent file stat cache and property cache into in-process cache.
 
 preloadFileStat (array $params)
 Preload file stat information (concurrently if possible) into in-process cache.
 
 storeInternal (array $params)
 Store a file into the backend from a file on disk.
 
 streamFile (array $params)
 Stream the content of the file at a storage path in the backend.
 
- Public Member Functions inherited from Wikimedia\FileBackend\FileBackend
 clean (array $params)
 Delete a storage directory if it is empty.
 
 copy (array $params, array $opts=[])
 Performs a single copy operation.
 
 create (array $params, array $opts=[])
 Performs a single create operation.
 
 delete (array $params, array $opts=[])
 Performs a single delete operation.
 
 describe (array $params, array $opts=[])
 Performs a single describe operation.
 
 doOperation (array $op, array $opts=[])
 Same as doOperations() except it takes a single operation.
 
 doOperations (array $ops, array $opts=[])
 This is the main entry point into the backend for write operations.
 
 doQuickOperation (array $op, array $opts=[])
 Same as doQuickOperations() except it takes a single operation.
 
 doQuickOperations (array $ops, array $opts=[])
 Perform a set of independent file operations on some files.
 
 getContainerStoragePath ( $container)
 Get the storage path for the given container for this backend.
 
 getDomainId ()
 Get the domain identifier used for this backend (possibly empty).
 
 getFileContents (array $params)
 Get the contents of a file at a storage path in the backend.
 
 getLocalCopy (array $params)
 Get a local copy on disk of the file at a storage path in the backend.
 
 getLocalReference (array $params)
 Returns a file system file, identical in content to the file at a storage path.
 
 getName ()
 Get the unique backend name.
 
 getReadOnlyReason ()
 Get an explanatory message if this backend is read-only.
 
 getRootStoragePath ()
 Get the root storage path of this backend.
 
 getScopedFileLocks (array $paths, $type, StatusValue $status, $timeout=0)
 Lock the files at the given storage paths in the backend.
 
 getStreamerOptions ()
 Return options for use with HTTPFileStreamer.
 
 getTopDirectoryList (array $params)
 Same as FileBackend::getDirectoryList() except only lists directories that are immediately under the given directory.
 
 getTopFileList (array $params)
 Same as FileBackend::getFileList() except only lists files that are immediately under the given directory.
 
 getWikiId ()
 Alias to getDomainId()
 
 hasFeatures ( $bitfield)
 Check if the backend medium supports a field of extra features.
 
 isReadOnly ()
 Check if this backend is read-only.
 
 lockFiles (array $paths, $type, $timeout=0)
 Lock the files at the given storage paths in the backend.
 
 move (array $params, array $opts=[])
 Performs a single move operation.
 
 prepare (array $params)
 Prepare a storage directory for usage.
 
 publish (array $params)
 Remove measures to block web access to a storage directory and the container it belongs to.
 
 quickCopy (array $params, array $opts=[])
 Performs a single quick copy operation.
 
 quickCreate (array $params, array $opts=[])
 Performs a single quick create operation.
 
 quickDelete (array $params, array $opts=[])
 Performs a single quick delete operation.
 
 quickDescribe (array $params, array $opts=[])
 Performs a single quick describe operation.
 
 quickMove (array $params, array $opts=[])
 Performs a single quick move operation.
 
 quickStore (array $params, array $opts=[])
 Performs a single quick store operation.
 
 secure (array $params)
 Take measures to block web access to a storage directory and the container it belongs to.
 
 store (array $params, array $opts=[])
 Performs a single store operation.
 
 unlockFiles (array $paths, $type)
 Unlock the files at the given storage paths in the backend.
 

Protected Member Functions

 addMissingHashMetadata (array $objHdrs, $path)
 Fill in any missing object metadata and save it to Swift.
 
 authTokenHeaders (array $creds)
 
 convertSwiftDate ( $ts, $format=TS_MW)
 Convert dates like "Tue, 03 Jan 2012 22:01:04 GMT"/"2013-05-11T07:37:27.678360Z".
 
 createContainer ( $container, array $params)
 Create a Swift container.
 
 deleteContainer ( $container, array $params)
 Delete a Swift container.
 
 directoriesAreVirtual ()
 Is this a key/value store where directories are just virtual? Virtual directories exists in so much as files exists that are prefixed with the directory path followed by a forward slash.
 
 doCleanInternal ( $fullCont, $dir, array $params)
 
 doCopyInternal (array $params)
 
 doCreateInternal (array $params)
 
 doDeleteInternal (array $params)
 
 doDescribeInternal (array $params)
 
 doDirectoryExists ( $fullCont, $dir, array $params)
 
 doExecuteOpHandlesInternal (array $fileOpHandles)
 
 doGetFileContentsMulti (array $params)
 
 doGetFileSha1base36 (array $params)
 
 doGetFileStat (array $params)
 
 doGetFileStatMulti (array $params)
 Get file stat information (concurrently if possible) for several files.
 
 doGetFileXAttributes (array $params)
 
 doGetLocalCopyMulti (array $params)
 
 doMoveInternal (array $params)
 
 doPrepareInternal ( $fullCont, $dir, array $params)
 
See also
FileBackendStore::doPrepare()
Stability: stable
to override
Parameters
string$container
string$dir
array$params
Returns
StatusValue Good status without value for success, fatal otherwise.

 
 doPrimeContainerCache (array $containerInfo)
 Fill the backend-specific process cache given an array of resolved container names and their corresponding cached info.
 
 doPublishInternal ( $fullCont, $dir, array $params)
 
 doSecureInternal ( $fullCont, $dir, array $params)
 
 doStoreInternal (array $params)
 
 doStreamFile (array $params)
 
 extractMetadataHeaders (array $headers)
 
 extractMutableContentHeaders (array $headers)
 Filter/normalize a header map to only include mutable "content-"/"x-content-" headers.
 
 getAuthentication ()
 Get the cached auth token.
 
 getContainerStat ( $container, $bypassCache=false)
 Get a Swift container stat map, possibly from process cache.
 
 getMetadataFromHeaders (array $headers)
 
 getStatFromHeaders (array $rhdrs)
 
 headersFromParams (array $params)
 Get headers to send to Swift when reading a file based on a FileBackend params array, e.g.
 
 resolveContainerPath ( $container, $relStoragePath)
 Resolve a relative storage path, checking if it's allowed by the backend.
 
 setContainerAccess ( $container, array $readUsers, array $writeUsers)
 Set read/write permissions for a Swift container.
 
 storageUrl (array $creds, $container=null, $object=null)
 
- Protected Member Functions inherited from Wikimedia\FileBackend\FileBackendStore
 deleteContainerCache ( $container)
 Delete the cached info for a container.
 
 deleteFileCache ( $path)
 Delete the cached stat info for a file path.
 
 doClean (array $params)
 
 doClearCache (?array $paths=null)
 Clears any additional stat caches for storage paths.
 
 doConcatenate (array $params)
 
 doGetFileSha1Base36 (array $params)
 
 doGetLocalReferenceMulti (array $params)
 
 doOperationsInternal (array $ops, array $opts)
 
 doPrepare (array $params)
 
See also
FileBackend::prepare()
Parameters
array$params
Returns
StatusValue Good status without value for success, fatal otherwise.

 
 doPublish (array $params)
 
 doQuickOperationsInternal (array $ops, array $opts)
 
 doSecure (array $params)
 
 fullContainerName ( $container)
 Get the full container name, including the domain ID prefix.
 
 getContainerHashLevels ( $container)
 Get the sharding config for a container.
 
 getContainerShard ( $container, $relPath)
 Get the container name shard suffix for a given path.
 
 getContainerSuffixes ( $container)
 Get a list of full container shard suffixes for a container.
 
 getContentType ( $storagePath, $content, $fsPath)
 Get the content type to use in HEAD/GET requests for a file.
 
 ingestFreshFileStats (array $stats, $latest)
 Ingest file stat entries that just came from querying the backend (not cache)
 
 primeContainerCache (array $items)
 Do a batch lookup from cache for container stats for all containers used in a list of container names or storage paths objects.
 
 primeFileCache (array $items)
 Do a batch lookup from cache for file stats for all paths used in a list of storage paths or FileOp objects.
 
 resolveContainerName ( $container)
 Resolve a container name, checking if it's allowed by the backend.
 
 resolveStoragePath ( $storagePath)
 Splits a storage path into an internal container name, an internal relative file name, and a container shard suffix.
 
 resolveStoragePathReal ( $storagePath)
 Like resolveStoragePath() except null values are returned if the container is sharded and the shard could not be determined or if the path ends with '/'.
 
 sanitizeOpHeaders (array $op)
 Normalize and filter HTTP headers from a file operation.
 
 setConcurrencyFlags (array $opts)
 Set the 'concurrency' option from a list of operation options.
 
 setContainerCache ( $container, array $val)
 Set the cached info for a container.
 
 setFileCache ( $path, array $val)
 Set the cached stat info for a file path.
 
- Protected Member Functions inherited from Wikimedia\FileBackend\FileBackend
 header ( $header)
 
 newStatus (... $args)
 Yields the result of the status wrapper callback on either:
 
 resetOutputBuffer ()
 
 resolveFSFileObjects (array $ops)
 Convert FSFile 'src' paths to string paths (with an 'srcRef' field set to the FSFile)
 
 scopedProfileSection ( $section)
 
 wrapStatus (StatusValue $sv)
 

Protected Attributes

array null $authCreds
 
int null $authErrorTimestamp = null
 UNIX timestamp.
 
int $authTTL
 TTL in seconds.
 
bool $canShellboxGetTempUrl
 
MapCacheLRU $containerStatCache
 Container stat cache.
 
MultiHttpClient $http
 
bool $isRGW = false
 Whether the server is an Ceph RGW.
 
array $readUsers
 Additional users (account:user) with read permissions on public containers.
 
string $rgwS3AccessKey
 S3 access key (RADOS Gateway)
 
string $rgwS3SecretKey
 S3 authentication key (RADOS Gateway)
 
array $secureReadUsers
 Additional users (account:user) with read permissions on private containers.
 
array $secureWriteUsers
 Additional users (account:user) with write permissions on private containers.
 
string null $shellboxIpRange
 
BagOStuff $srvCache
 
string $swiftAuthUrl
 Authentication base URL (without version)
 
string $swiftKey
 Secret key for user.
 
string $swiftStorageUrl
 Override of storage base URL.
 
string $swiftTempUrlKey
 Shared secret value for making temp URLs.
 
string $swiftUser
 Swift user (account:user) to authenticate as.
 
array $writeUsers
 Additional users (account:user) with write permissions on public containers.
 
- Protected Attributes inherited from Wikimedia\FileBackend\FileBackendStore
MapCacheLRU $cheapCache
 Map of paths to small (RAM/disk) cache items.
 
MapCacheLRU $expensiveCache
 Map of paths to large (RAM/disk) cache items.
 
int $maxFileSize = 32 * 1024 * 1024 * 1024
 Size in bytes, defaults to 32 GiB.
 
WANObjectCache $memCache
 
callable null $mimeCallback
 Method to get the MIME type of files.
 
array< string, array > $shardViaHashLevels = []
 Map of container names to sharding config.
 
BagOStuff $srvCache
 
const CACHE_CHEAP_SIZE = 500
 
const CACHE_EXPENSIVE_SIZE = 5
 
const CACHE_TTL = 10
 
- Protected Attributes inherited from Wikimedia\FileBackend\FileBackend
int $concurrency
 How many operations can be done in parallel.
 
string $domainId
 Unique domain name.
 
LockManager $lockManager
 
LoggerInterface $logger
 
string $name
 Unique backend name.
 
string $parallelize
 When to do operations in parallel.
 
callable null $profiler
 
string $readOnly
 Read-only explanation message.
 
callable null $statusWrapper
 
array $streamerOptions
 Option map for use with HTTPFileStreamer.
 
TempFSFileFactory $tmpFileFactory
 

Additional Inherited Members

- Static Public Member Functions inherited from Wikimedia\FileBackend\FileBackend
static extensionFromPath ( $path, $case='lowercase')
 Get the final extension from a storage or FS path.
 
static isPathTraversalFree ( $path)
 Check if a relative path has no directory traversals.
 
static isStoragePath ( $path)
 Check if a given path is a "mwstore://" path.
 
static makeContentDisposition ( $type, $filename='')
 Build a Content-Disposition header value per RFC 6266.
 
static normalizeStoragePath ( $storagePath)
 Normalize a storage path by cleaning up directory separators.
 
static parentStoragePath ( $storagePath)
 Get the parent storage directory of a storage path.
 
static resetOutputBufferTheDefaultWay ()
 Default behavior of resetOutputBuffer().
 
static splitStoragePath ( $storagePath)
 Split a storage path into a backend name, a container name, and a relative file path.
 
- Public Attributes inherited from Wikimedia\FileBackend\FileBackend
const ATTR_HEADERS = 1
 Bitfield flags for supported features.
 
const ATTR_METADATA = 2
 
const ATTR_UNICODE_PATHS = 4
 
- Static Protected Member Functions inherited from Wikimedia\FileBackend\FileBackendStore
static isValidContainerName ( $container)
 Check if a full container name is valid.
 
static isValidShortContainerName ( $container)
 Check if a short container name is valid.
 
static normalizeXAttributes (array $xattr)
 Normalize file headers/metadata to the FileBackend::getFileXAttributes() format.
 
- Static Protected Member Functions inherited from Wikimedia\FileBackend\FileBackend
static normalizeContainerPath ( $path)
 Validate and normalize a relative storage path.
 

Detailed Description

Class for an OpenStack Swift (or Ceph RGW) based file backend.

StatusValue messages should avoid mentioning the Swift account name. Likewise, error suppression should be used to avoid path disclosure.

Since
1.19

Definition at line 55 of file SwiftFileBackend.php.

Constructor & Destructor Documentation

◆ __construct()

Wikimedia\FileBackend\SwiftFileBackend::__construct ( array $config)
See also
FileBackendStore::__construct()
Parameters
array$configParams include:
  • swiftAuthUrl : Swift authentication server URL
  • swiftUser : Swift user used by MediaWiki (account:username)
  • swiftKey : Swift authentication key for the above user
  • swiftAuthTTL : Swift authentication TTL (seconds)
  • swiftTempUrlKey : Swift "X-Account-Meta-Temp-URL-Key" value on the account. Do not set this until it has been set in the backend.
  • canShellboxGetTempUrl : Set this to true to generate a TempURL allowing Shellbox to directly fetch files from Swift. swiftTempUrlKey should be set.
  • shellboxIpRange : An IP range string to use when generating TempURLs for Shellbox. Specifying this will improve security by preventing exfiltrated TempURLs from being usable outside the server.
  • swiftStorageUrl : Swift storage URL (overrides that of the authentication response). This is useful to set if a TLS proxy is in use.
  • shardViaHashLevels : Map of container names to sharding config with:
    • base : base of hash characters, 16 or 36
    • levels : the number of hash levels (and digits)
    • repeat : hash subdirectories are prefixed with all the parent hash directory names (e.g. "a/ab/abc")
  • cacheAuthInfo : Whether to cache authentication tokens in APC, etc. If those are not available, then the main cache will be used. This is probably insecure in shared hosting environments.
  • rgwS3AccessKey : Rados Gateway S3 "access key" value on the account. Do not set this until it has been set in the backend. This is used for generating expiring pre-authenticated URLs. Only use this when using rgw and to work around http://tracker.newdream.net/issues/3454.
  • rgwS3SecretKey : Rados Gateway S3 "secret key" value on the account. Do not set this until it has been set in the backend. This is used for generating expiring pre-authenticated URLs. Only use this when using rgw and to work around http://tracker.newdream.net/issues/3454.
  • readUsers : Swift users with read access to public containers (account:username)
  • writeUsers : Swift users with write access to public containers (account:username)
  • secureReadUsers : Swift users with read access to private containers (account:username)
  • secureWriteUsers : Swift users with write access to private containers (account:username)
  • connTimeout : The HTTP connect timeout to use when connecting to Swift, in seconds.
  • reqTimeout : The HTTP request timeout to use when communicating with Swift, in seconds.

Reimplemented from Wikimedia\FileBackend\FileBackendStore.

Definition at line 147 of file SwiftFileBackend.php.

Member Function Documentation

◆ addMissingHashMetadata()

◆ addShellboxInputFile()

Wikimedia\FileBackend\SwiftFileBackend::addShellboxInputFile ( BoxedCommand $command,
string $boxedName,
array $params )

Add a file to a Shellbox command as an input file.

Parameters
BoxedCommand$command
string$boxedName
array$paramsParameters include:
  • src : source storage path
  • latest : use the latest available data
Returns
StatusValue
Since
1.43

Reimplemented from Wikimedia\FileBackend\FileBackendStore.

Definition at line 1308 of file SwiftFileBackend.php.

References $params, Wikimedia\FileBackend\SwiftFileBackend\$shellboxIpRange, $url, Wikimedia\FileBackend\SwiftFileBackend\getFileHttpUrl(), and Wikimedia\FileBackend\FileBackend\newStatus().

◆ authTokenHeaders()

Wikimedia\FileBackend\SwiftFileBackend::authTokenHeaders ( array $creds)
protected
Parameters
array$credsFrom getAuthentication()
Returns
array

Definition at line 1885 of file SwiftFileBackend.php.

◆ convertSwiftDate()

Wikimedia\FileBackend\SwiftFileBackend::convertSwiftDate ( $ts,
$format = TS_MW )
protected

Convert dates like "Tue, 03 Jan 2012 22:01:04 GMT"/"2013-05-11T07:37:27.678360Z".

Dates might also come in like "2013-05-11T07:37:27.678360" from Swift listings, missing the timezone suffix (though Ceph RGW does not appear to have this bug).

Parameters
string$ts
int$formatOutput format (TS_* constant)
Returns
string
Exceptions
FileBackendError

Definition at line 799 of file SwiftFileBackend.php.

Referenced by Wikimedia\FileBackend\SwiftFileBackend\getStatFromHeaders().

◆ createContainer()

Wikimedia\FileBackend\SwiftFileBackend::createContainer ( $container,
array $params )
protected

Create a Swift container.

Parameters
string$containerContainer name
array$params
Returns
StatusValue Good status without value for success, fatal otherwise.

Definition at line 1559 of file SwiftFileBackend.php.

References $params, Wikimedia\FileBackend\SwiftFileBackend\$readUsers, Wikimedia\FileBackend\SwiftFileBackend\$writeUsers, Wikimedia\FileBackend\FileBackend\newStatus(), and Wikimedia\FileBackend\SwiftFileBackend\onError().

Referenced by Wikimedia\FileBackend\SwiftFileBackend\doPrepareInternal().

◆ deleteContainer()

Wikimedia\FileBackend\SwiftFileBackend::deleteContainer ( $container,
array $params )
protected

Delete a Swift container.

Parameters
string$containerContainer name
array$params
Returns
StatusValue

Definition at line 1603 of file SwiftFileBackend.php.

References $params, Wikimedia\FileBackend\FileBackend\newStatus(), and Wikimedia\FileBackend\SwiftFileBackend\onError().

Referenced by Wikimedia\FileBackend\SwiftFileBackend\doCleanInternal().

◆ directoriesAreVirtual()

Wikimedia\FileBackend\SwiftFileBackend::directoriesAreVirtual ( )
protected

Is this a key/value store where directories are just virtual? Virtual directories exists in so much as files exists that are prefixed with the directory path followed by a forward slash.

Returns
bool

Reimplemented from Wikimedia\FileBackend\FileBackendStore.

Definition at line 1397 of file SwiftFileBackend.php.

◆ doCleanInternal()

Wikimedia\FileBackend\SwiftFileBackend::doCleanInternal ( $container,
$dir,
array $params )
protected
See also
FileBackendStore::doClean()
Stability: stable
to override
Parameters
string$container
string$dir
array$params
Returns
StatusValue

Reimplemented from Wikimedia\FileBackend\FileBackendStore.

Definition at line 757 of file SwiftFileBackend.php.

References $params, Wikimedia\FileBackend\SwiftFileBackend\deleteContainer(), Wikimedia\FileBackend\SwiftFileBackend\getContainerStat(), and Wikimedia\FileBackend\FileBackend\newStatus().

◆ doCopyInternal()

◆ doCreateInternal()

◆ doDeleteInternal()

Wikimedia\FileBackend\SwiftFileBackend::doDeleteInternal ( array $params)
protected

◆ doDescribeInternal()

◆ doDirectoryExists()

Wikimedia\FileBackend\SwiftFileBackend::doDirectoryExists ( $container,
$dir,
array $params )
protected
See also
FileBackendStore::directoryExists()
Parameters
string$containerResolved container name
string$dirResolved path relative to container
array$params
Returns
bool|null

Reimplemented from Wikimedia\FileBackend\FileBackendStore.

Definition at line 926 of file SwiftFileBackend.php.

◆ doExecuteOpHandlesInternal()

Wikimedia\FileBackend\SwiftFileBackend::doExecuteOpHandlesInternal ( array $fileOpHandles)
protected
See also
FileBackendStore::executeOpHandlesInternal()
Stability: stable
to override
Parameters
FileBackendStoreOpHandle[]$fileOpHandles
Exceptions
FileBackendError
Returns
StatusValue[] List of corresponding StatusValue objects

Reimplemented from Wikimedia\FileBackend\FileBackendStore.

Definition at line 1418 of file SwiftFileBackend.php.

References Wikimedia\FileBackend\FileBackend\newStatus().

◆ doGetFileContentsMulti()

Wikimedia\FileBackend\SwiftFileBackend::doGetFileContentsMulti ( array $params)
protected
See also
FileBackendStore::getFileContentsMulti()
Stability: stable
to override
Parameters
array$params
Returns
string[]|bool[]|null[] Map of (path => string, false (missing), or null (error))

Reimplemented from Wikimedia\FileBackend\FileBackendStore.

Definition at line 868 of file SwiftFileBackend.php.

References $params, $path, Wikimedia\FileBackend\SwiftFileBackend\headersFromParams(), Wikimedia\FileBackend\SwiftFileBackend\onError(), and Wikimedia\FileBackend\FileBackendStore\resolveStoragePathReal().

◆ doGetFileSha1base36()

Wikimedia\FileBackend\SwiftFileBackend::doGetFileSha1base36 ( array $params)
protected

◆ doGetFileStat()

Wikimedia\FileBackend\SwiftFileBackend::doGetFileStat ( array $params)
protected
See also
FileBackendStore::getFileStat()
Parameters
array$params
Returns
array|false|null

Reimplemented from Wikimedia\FileBackend\FileBackendStore.

Definition at line 781 of file SwiftFileBackend.php.

References $params, and Wikimedia\FileBackend\SwiftFileBackend\doGetFileStatMulti().

Referenced by Wikimedia\FileBackend\SwiftFileBackend\doDescribeInternal().

◆ doGetFileStatMulti()

Wikimedia\FileBackend\SwiftFileBackend::doGetFileStatMulti ( array $params)
protected

Get file stat information (concurrently if possible) for several files.

Stability: stable
to override
See also
FileBackend::getFileStat()
Parameters
array$paramsParameters include:
  • srcs : list of source storage paths
  • latest : use the latest available data
Returns
array<string,array|false|null>|null Null if not supported. Otherwise a map of storage path to attribute map, false (missing file), or null (I/O error).
Since
1.23

Reimplemented from Wikimedia\FileBackend\FileBackendStore.

Definition at line 1685 of file SwiftFileBackend.php.

References $params, $path, Wikimedia\FileBackend\SwiftFileBackend\addMissingHashMetadata(), Wikimedia\FileBackend\SwiftFileBackend\getContainerStat(), Wikimedia\FileBackend\SwiftFileBackend\getStatFromHeaders(), Wikimedia\FileBackend\SwiftFileBackend\headersFromParams(), Wikimedia\FileBackend\SwiftFileBackend\onError(), and Wikimedia\FileBackend\FileBackendStore\resolveStoragePathReal().

Referenced by Wikimedia\FileBackend\SwiftFileBackend\doGetFileStat().

◆ doGetFileXAttributes()

Wikimedia\FileBackend\SwiftFileBackend::doGetFileXAttributes ( array $params)
protected
See also
FileBackendStore::getFileXAttributes()
Stability: stable
to override
Parameters
array$params
Returns
array[][]|false|null Attributes, false (missing file), or null (error)

Reimplemented from Wikimedia\FileBackend\FileBackendStore.

Definition at line 1141 of file SwiftFileBackend.php.

References Wikimedia\FileBackend\FileBackendStore\clearCache(), and Wikimedia\FileBackend\FileBackendStore\getFileStat().

◆ doGetLocalCopyMulti()

Wikimedia\FileBackend\SwiftFileBackend::doGetLocalCopyMulti ( array $params)
protected
See also
FileBackendStore::getLocalCopyMulti()
Parameters
array$params
Returns
string[]|bool[]|null[] Map of (path => TempFSFile, false (missing), or null (error))

Reimplemented from Wikimedia\FileBackend\FileBackendStore.

Definition at line 1236 of file SwiftFileBackend.php.

References $params, $path, Wikimedia\FileBackend\FileBackend\extensionFromPath(), Wikimedia\FileBackend\SwiftFileBackend\getStatFromHeaders(), Wikimedia\FileBackend\SwiftFileBackend\headersFromParams(), Wikimedia\FileBackend\SwiftFileBackend\onError(), and Wikimedia\FileBackend\FileBackendStore\resolveStoragePathReal().

◆ doMoveInternal()

◆ doPrepareInternal()

Wikimedia\FileBackend\SwiftFileBackend::doPrepareInternal ( $fullCont,
$dir,
array $params )
protected

See also
FileBackendStore::doPrepare()
Stability: stable
to override
Parameters
string$container
string$dir
array$params
Returns
StatusValue Good status without value for success, fatal otherwise.

Reimplemented from Wikimedia\FileBackend\FileBackendStore.

Definition at line 682 of file SwiftFileBackend.php.

References $params, Wikimedia\FileBackend\SwiftFileBackend\createContainer(), Wikimedia\FileBackend\SwiftFileBackend\getContainerStat(), and Wikimedia\FileBackend\FileBackend\newStatus().

◆ doPrimeContainerCache()

Wikimedia\FileBackend\SwiftFileBackend::doPrimeContainerCache ( array $containerInfo)
protected

Fill the backend-specific process cache given an array of resolved container names and their corresponding cached info.

Only containers that actually exist should appear in the map.

Stability: stable
to override
Parameters
array$containerInfoMap of resolved container names to cached info

Reimplemented from Wikimedia\FileBackend\FileBackendStore.

Definition at line 1679 of file SwiftFileBackend.php.

◆ doPublishInternal()

Wikimedia\FileBackend\SwiftFileBackend::doPublishInternal ( $container,
$dir,
array $params )
protected

◆ doSecureInternal()

Wikimedia\FileBackend\SwiftFileBackend::doSecureInternal ( $container,
$dir,
array $params )
protected
See also
FileBackendStore::doSecure()
Stability: stable
to override
Parameters
string$container
string$dir
array$params
Returns
StatusValue Good status without value for success, fatal otherwise.

Reimplemented from Wikimedia\FileBackend\FileBackendStore.

Definition at line 701 of file SwiftFileBackend.php.

References $params, Wikimedia\FileBackend\SwiftFileBackend\$readUsers, Wikimedia\FileBackend\SwiftFileBackend\$writeUsers, Wikimedia\FileBackend\SwiftFileBackend\getContainerStat(), Wikimedia\FileBackend\FileBackend\newStatus(), and Wikimedia\FileBackend\SwiftFileBackend\setContainerAccess().

◆ doStoreInternal()

◆ doStreamFile()

◆ extractMetadataHeaders()

Wikimedia\FileBackend\SwiftFileBackend::extractMetadataHeaders ( array $headers)
protected
See also
https://docs.openstack.org/api-ref/object-store
Parameters
string[]$headersMap of (header => value) for a swift object
Returns
string[] Map of (metadata header name => metadata value)

Definition at line 277 of file SwiftFileBackend.php.

References Wikimedia\FileBackend\FileBackend\$name.

Referenced by Wikimedia\FileBackend\SwiftFileBackend\addMissingHashMetadata(), and Wikimedia\FileBackend\SwiftFileBackend\getMetadataFromHeaders().

◆ extractMutableContentHeaders()

Wikimedia\FileBackend\SwiftFileBackend::extractMutableContentHeaders ( array $headers)
protected

Filter/normalize a header map to only include mutable "content-"/"x-content-" headers.

Mutable headers can be changed via HTTP POST even if the file content is the same

See also
https://docs.openstack.org/api-ref/object-store
Parameters
string[]$headersMap of (header => value) for a swift object
Returns
string[] Map of (header => value) for Content-* headers mutable via POST

Definition at line 237 of file SwiftFileBackend.php.

References Wikimedia\FileBackend\FileBackend\$name.

Referenced by Wikimedia\FileBackend\SwiftFileBackend\addMissingHashMetadata(), Wikimedia\FileBackend\SwiftFileBackend\doCopyInternal(), Wikimedia\FileBackend\SwiftFileBackend\doCreateInternal(), Wikimedia\FileBackend\SwiftFileBackend\doDescribeInternal(), Wikimedia\FileBackend\SwiftFileBackend\doMoveInternal(), Wikimedia\FileBackend\SwiftFileBackend\doStoreInternal(), and Wikimedia\FileBackend\SwiftFileBackend\getStatFromHeaders().

◆ getAuthentication()

Wikimedia\FileBackend\SwiftFileBackend::getAuthentication ( )
protected

Get the cached auth token.

Returns
array|null Credential map

Definition at line 1771 of file SwiftFileBackend.php.

References Wikimedia\FileBackend\SwiftFileBackend\$authCreds, and Wikimedia\FileBackend\SwiftFileBackend\$authErrorTimestamp.

Referenced by Wikimedia\FileBackend\SwiftFileBackend\getFileHttpUrl().

◆ getContainerStat()

Wikimedia\FileBackend\SwiftFileBackend::getContainerStat ( $container,
$bypassCache = false )
protected

◆ getDirectoryListInternal()

Wikimedia\FileBackend\SwiftFileBackend::getDirectoryListInternal ( $fullCont,
$dir,
array $params )
See also
FileBackendStore::getDirectoryListInternal()
Parameters
string$fullCont
string$dir
array$params
Returns
SwiftFileBackendDirList

Reimplemented from Wikimedia\FileBackend\FileBackendStore.

Definition at line 943 of file SwiftFileBackend.php.

References $params.

◆ getDirListPageInternal()

Wikimedia\FileBackend\SwiftFileBackend::getDirListPageInternal ( $fullCont,
$dir,
& $after,
$limit,
array $params )

Do not call this function outside of SwiftFileBackendFileList.

Parameters
string$fullContResolved container name
string$dirResolved storage directory with no trailing slash
string | null&$afterResolved container relative path used for continuation paging
int$limitMax number of items to list
array$paramsParameters for getDirectoryList()
Returns
string[] List of resolved container relative directories directly under $dir
Exceptions
FileBackendError

Definition at line 969 of file SwiftFileBackend.php.

References $params, $path, and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

◆ getFeatures()

Wikimedia\FileBackend\SwiftFileBackend::getFeatures ( )

Get the a bitfield of extra features supported by the backend medium.

Stability: stable
to override
Returns
int Bitfield of FileBackend::ATTR_* flags
Since
1.23

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 201 of file SwiftFileBackend.php.

◆ getFileHttpUrl()

Wikimedia\FileBackend\SwiftFileBackend::getFileHttpUrl ( array $params)

◆ getFileListInternal()

Wikimedia\FileBackend\SwiftFileBackend::getFileListInternal ( $fullCont,
$dir,
array $params )
See also
FileBackendStore::getFileListInternal()
Parameters
string$fullCont
string$dir
array$params
Returns
SwiftFileBackendFileList

Reimplemented from Wikimedia\FileBackend\FileBackendStore.

Definition at line 954 of file SwiftFileBackend.php.

References $params.

◆ getFileListPageInternal()

Wikimedia\FileBackend\SwiftFileBackend::getFileListPageInternal ( $fullCont,
$dir,
& $after,
$limit,
array $params )

Do not call this function outside of SwiftFileBackendFileList.

Parameters
string$fullContResolved container name
string$dirResolved storage directory with no trailing slash
string | null&$afterResolved container relative path of file to list items after
int$limitMax number of items to list
array$paramsParameters for getFileList()
Returns
array[] List of (name, stat map or null) tuples under $dir
Exceptions
FileBackendError

Definition at line 1052 of file SwiftFileBackend.php.

References $params, and Wikimedia\FileBackend\FileBackend\scopedProfileSection().

◆ getMetadataFromHeaders()

Wikimedia\FileBackend\SwiftFileBackend::getMetadataFromHeaders ( array $headers)
protected
See also
https://docs.openstack.org/api-ref/object-store
Parameters
string[]$headersMap of (header => value) for a swift object
Returns
string[] Map of (metadata key name => metadata value)

Definition at line 294 of file SwiftFileBackend.php.

References Wikimedia\FileBackend\FileBackend\$name, and Wikimedia\FileBackend\SwiftFileBackend\extractMetadataHeaders().

Referenced by Wikimedia\FileBackend\SwiftFileBackend\getStatFromHeaders().

◆ getStatFromHeaders()

◆ headersFromParams()

Wikimedia\FileBackend\SwiftFileBackend::headersFromParams ( array $params)
protected

Get headers to send to Swift when reading a file based on a FileBackend params array, e.g.

that of getLocalCopy(). $params is currently only checked for a 'latest' flag.

Parameters
array$params
Returns
array

Definition at line 1409 of file SwiftFileBackend.php.

References $params.

Referenced by Wikimedia\FileBackend\SwiftFileBackend\doGetFileContentsMulti(), Wikimedia\FileBackend\SwiftFileBackend\doGetFileStatMulti(), Wikimedia\FileBackend\SwiftFileBackend\doGetLocalCopyMulti(), and Wikimedia\FileBackend\SwiftFileBackend\doStreamFile().

◆ isPathUsableInternal()

Wikimedia\FileBackend\SwiftFileBackend::isPathUsableInternal ( $storagePath)

Check if a file can be created or changed at a given storage path in the backend.

FS backends should check that the parent directory exists, files can be written under it, and that any file already there is both readable and writable. Backends using key/value stores should check if the container exists.

Parameters
string$storagePath
Returns
bool

Reimplemented from Wikimedia\FileBackend\FileBackendStore.

Definition at line 219 of file SwiftFileBackend.php.

References Wikimedia\FileBackend\SwiftFileBackend\getContainerStat(), and Wikimedia\FileBackend\FileBackendStore\resolveStoragePathReal().

◆ loadListingStatInternal()

Wikimedia\FileBackend\SwiftFileBackend::loadListingStatInternal ( $path,
array $val )

Do not call this function outside of SwiftFileBackendFileList.

Parameters
string$pathStorage path
array$valStat value

Definition at line 1137 of file SwiftFileBackend.php.

References $path.

◆ onError()

◆ resolveContainerPath()

Wikimedia\FileBackend\SwiftFileBackend::resolveContainerPath ( $container,
$relStoragePath )
protected

Resolve a relative storage path, checking if it's allowed by the backend.

This is intended for internal use, such as encoding illegal chars or perhaps getting absolute paths (e.g. FS based backends). Note that the relative path may be the empty string (e.g. the path is simply to the container).

Stability: stable
to override
Parameters
string$containerContainer name
string$relStoragePathStorage path relative to the container
Returns
string|null Path or null if not valid

Reimplemented from Wikimedia\FileBackend\FileBackendStore.

Definition at line 209 of file SwiftFileBackend.php.

◆ setContainerAccess()

Wikimedia\FileBackend\SwiftFileBackend::setContainerAccess ( $container,
array $readUsers,
array $writeUsers )
protected

Set read/write permissions for a Swift container.

See also
http://docs.openstack.org/developer/swift/misc.html#acls

In general, we don't allow listings to end-users. It's not useful, isn't well-defined (lists are truncated to 10000 item with no way to page), and is just a performance risk.

Parameters
string$containerResolved Swift container
array$readUsersList of the possible criteria for a request to have access to read a container. Each item is one of the following formats:
  • account:user : Grants access if the request is by the given user
  • ".r:<regex>" : Grants access if the request is from a referrer host that matches the expression and the request is not for a listing. Setting this to '*' effectively makes a container public. -".rlistings:<regex>" : Grants access if the request is from a referrer host that matches the expression and the request is for a listing.
array$writeUsersA list of the possible criteria for a request to have access to write to a container. Each item is of the following format:
  • account:user : Grants access if the request is by the given user
Returns
StatusValue Good status without value for success, fatal otherwise.

Definition at line 1484 of file SwiftFileBackend.php.

References Wikimedia\FileBackend\SwiftFileBackend\$readUsers, Wikimedia\FileBackend\SwiftFileBackend\$writeUsers, and Wikimedia\FileBackend\FileBackend\newStatus().

Referenced by Wikimedia\FileBackend\SwiftFileBackend\doPublishInternal(), and Wikimedia\FileBackend\SwiftFileBackend\doSecureInternal().

◆ setLogger()

Wikimedia\FileBackend\SwiftFileBackend::setLogger ( LoggerInterface $logger)

Reimplemented from Wikimedia\FileBackend\FileBackend.

Definition at line 196 of file SwiftFileBackend.php.

References Wikimedia\FileBackend\FileBackend\$logger.

◆ storageUrl()

Wikimedia\FileBackend\SwiftFileBackend::storageUrl ( array $creds,
$container = null,
$object = null )
protected
Parameters
array$credsFrom getAuthentication()
string | null$container
string | null$object
Returns
string

Definition at line 1869 of file SwiftFileBackend.php.

Referenced by Wikimedia\FileBackend\SwiftFileBackend\getFileHttpUrl().

Member Data Documentation

◆ $authCreds

array null Wikimedia\FileBackend\SwiftFileBackend::$authCreds
protected

◆ $authErrorTimestamp

int null Wikimedia\FileBackend\SwiftFileBackend::$authErrorTimestamp = null
protected

UNIX timestamp.

Definition at line 99 of file SwiftFileBackend.php.

Referenced by Wikimedia\FileBackend\SwiftFileBackend\getAuthentication().

◆ $authTTL

int Wikimedia\FileBackend\SwiftFileBackend::$authTTL
protected

TTL in seconds.

Definition at line 62 of file SwiftFileBackend.php.

◆ $canShellboxGetTempUrl

bool Wikimedia\FileBackend\SwiftFileBackend::$canShellboxGetTempUrl
protected

Definition at line 74 of file SwiftFileBackend.php.

◆ $containerStatCache

MapCacheLRU Wikimedia\FileBackend\SwiftFileBackend::$containerStatCache
protected

Container stat cache.

Definition at line 94 of file SwiftFileBackend.php.

◆ $http

MultiHttpClient Wikimedia\FileBackend\SwiftFileBackend::$http
protected

Definition at line 60 of file SwiftFileBackend.php.

◆ $isRGW

bool Wikimedia\FileBackend\SwiftFileBackend::$isRGW = false
protected

Whether the server is an Ceph RGW.

Definition at line 102 of file SwiftFileBackend.php.

◆ $readUsers

array Wikimedia\FileBackend\SwiftFileBackend::$readUsers
protected

◆ $rgwS3AccessKey

string Wikimedia\FileBackend\SwiftFileBackend::$rgwS3AccessKey
protected

S3 access key (RADOS Gateway)

Definition at line 78 of file SwiftFileBackend.php.

◆ $rgwS3SecretKey

string Wikimedia\FileBackend\SwiftFileBackend::$rgwS3SecretKey
protected

S3 authentication key (RADOS Gateway)

Definition at line 80 of file SwiftFileBackend.php.

◆ $secureReadUsers

array Wikimedia\FileBackend\SwiftFileBackend::$secureReadUsers
protected

Additional users (account:user) with read permissions on private containers.

Definition at line 86 of file SwiftFileBackend.php.

◆ $secureWriteUsers

array Wikimedia\FileBackend\SwiftFileBackend::$secureWriteUsers
protected

Additional users (account:user) with write permissions on private containers.

Definition at line 88 of file SwiftFileBackend.php.

◆ $shellboxIpRange

string null Wikimedia\FileBackend\SwiftFileBackend::$shellboxIpRange
protected

◆ $srvCache

BagOStuff Wikimedia\FileBackend\SwiftFileBackend::$srvCache
protected

Definition at line 91 of file SwiftFileBackend.php.

◆ $swiftAuthUrl

string Wikimedia\FileBackend\SwiftFileBackend::$swiftAuthUrl
protected

Authentication base URL (without version)

Definition at line 64 of file SwiftFileBackend.php.

◆ $swiftKey

string Wikimedia\FileBackend\SwiftFileBackend::$swiftKey
protected

Secret key for user.

Definition at line 70 of file SwiftFileBackend.php.

◆ $swiftStorageUrl

string Wikimedia\FileBackend\SwiftFileBackend::$swiftStorageUrl
protected

Override of storage base URL.

Definition at line 66 of file SwiftFileBackend.php.

◆ $swiftTempUrlKey

string Wikimedia\FileBackend\SwiftFileBackend::$swiftTempUrlKey
protected

Shared secret value for making temp URLs.

Definition at line 72 of file SwiftFileBackend.php.

◆ $swiftUser

string Wikimedia\FileBackend\SwiftFileBackend::$swiftUser
protected

Swift user (account:user) to authenticate as.

Definition at line 68 of file SwiftFileBackend.php.

◆ $writeUsers

array Wikimedia\FileBackend\SwiftFileBackend::$writeUsers
protected

The documentation for this class was generated from the following file: