locking
Module to manage distributed locks.
Bases:
SpicerackCheckError
Exception raised when unable to acquire the etcd lock, the operation should be retried.
- exception spicerack.locking.InvalidLockError[source]
Bases:
LockError
Exception raised when the parameters for a lock are invalid.
- exception spicerack.locking.LockError[source]
Bases:
SpicerackError
Generic exception for errors of this module.
- exception spicerack.locking.LockExistingError[source]
Bases:
LockError
Exception raised if a lock with the same ID already exists for the given key.
Bases:
SpicerackCheckError
Exception raised when unable to acquire the Spicerack lock, the operation should be retried.
- exception spicerack.locking.LockUnreadableError[source]
Bases:
LockError
Exception raised when unable to properly parse an existing lock from the backend.
- exception spicerack.locking.LockUnwritableError[source]
Bases:
LockError
Exception raised when unable to properly serialize the lock in order to save it in the backend.
- class spicerack.locking.ConcurrentLock(concurrency: int, owner: str, ttl: int, uuid: str = <factory>, created: datetime.datetime = <factory>) None [source]
Bases:
object
A single concurrent lock object.
- Parameters:
concurrency (
int
) -- how many concurrent locks can be acquired for the same key. If set to zero it means that there is no concurrency limit and an infinite number of clients can run concurrently, but they are tracked.ttl (
int
) -- the time to live in seconds of the lock. If expired it will be automatically discarded.owner (
str
) -- a way to identify the owner of the lock, usually in the form{user}@{hostname} [{pid}]
.uuid (
str
, default:<factory>
) -- the unique identifier of the concurrent lock.created (
datetime.datetime
, default:<factory>
) -- when the lock has been created, used for checking if the TTL has expired.
- classmethod from_dict(lock_id: str, lock_obj: dict[str, str | int]) spicerack.locking.ConcurrentLock [source]
Get a ConcurrentLock instance from a dict, suitable to be used to JSON deserialize.
- Parameters:
- Return type:
- Returns:
a concurrent lock instance.
- Raises:
spicerack.locking.InvalidLockError -- if unable to parse the created datetime.
- to_dict() dict[str, str | int] [source]
Returns the dict representation of the object suitable for JSON serialization.
Intentionally not using
dataclasses.asdict()
to skip self.uuid and to convert the created datetime to string.- Return type:
dict
[str
,typing.Union
[str
,int
]]- Returns:
the object as dict with the created datetime converted to string.
- class spicerack.locking.KeyLocks(key: str, locks: dict[str, spicerack.locking.ConcurrentLock] = <factory>) None [source]
Bases:
object
Manage a lock object for a given key with concurrent locks.
- Parameters:
key (
str
) -- the lock full key path to be used in the backend.locks (
dict
[str
,spicerack.locking.ConcurrentLock
], default:<factory>
) -- the concurrent locks for the given key.
- add(lock: spicerack.locking.ConcurrentLock) None [source]
Add the concurrent lock to the lock object if all criteria are met.
- Parameters:
lock (
spicerack.locking.ConcurrentLock
) -- the concurrent lock to add.- Raises:
spicerack.locking.LockUnavailableError -- when the max concurrency has been reached and is not possible to acquire a lock.
spicerack.locking.LockExistingError -- if a lock with the same ID already exists.
- Return type:
- classmethod from_json(key: str, json_str: str) spicerack.locking.KeyLocks [source]
Create an instance of this class from a JSON string.
- Parameters:
- Return type:
- Returns:
the lock instance.
- Raises:
spicerack.locking.LockUnreadableError -- if unable to decode the JSON lock.
- remove(lock_id: str) spicerack.locking.ConcurrentLock | None [source]
Remove the concurrent lock identified by the lock ID.
- to_json() str [source]
Return the JSON serialization of the current instance.
- Return type:
- Returns:
a string with a JSON-serialized object.
- Raises:
spicerack.locking.LockUnwritableError -- if unable to convert the curent instance to JSON.
- class spicerack.locking.Lock(*, config: dict, prefix: str, owner: str, dry_run: bool = True) None [source]
Bases:
object
Manage a Spicerack lock.
Example object created in the backend (etcd):
'/spicerack/locks/cookbooks/sre.foo.bar' => { '4e2677c7-541a-4f9e-afbb-cfdb69c440a1': { 'created': '2023-07-16 16:46:52.161053', 'owner': 'user@host [12345]', 'concurrency': 5, 'ttl': 120, } }
Initialize the instance.
- Parameters:
config (
dict
) -- the etcd configuration dictionary to be passed to the etcd client.prefix (
str
) -- the name of the directory to use to prefix the lock. Must be one ofspicerack.locking.ALLOWED_PREFIXES
.owner (
str
) -- a way to identify the owner of the lock, usually in the form{user}@{hostname} [{pid}]
.dry_run (
bool
, default:True
) -- whether this is a DRY-RUN.
- Raises:
spicerack.locking.LockError -- if the provided prefix is not one of the allowed ones.
- acquire(name: str, *, concurrency: int, ttl: int) str [source]
Acquire a lock for the given name and parameters and return the lock identifier.
Examples:
# Get a lock for a given name allowing to have other 2 concurrent runs in parallel with a TTL of 30 minutes lock_id = lock.acquire("some.lock.identifier", concurrency=3, ttl=1800) # Do something lock.release("some.lock.identifier", lock_id) # Acquire an exclusive lock for the given name with a TTL of 1 hour lock_id = lock.acquire("some.lock.identifier", concurrency=1, ttl=1800) # Do something lock.release("some.lock.identifier", lock_id)
- Parameters:
name (
str
) -- the lock name, cannot contain/
as that's a directory separator in the data structure.concurrency (
int
) -- how many concurrent clients can hold the same lock. If set to zero it means that there is no concurrency limit and an infinite number of clients can run concurrently, but they are tracked.ttl (
int
) -- the amount of seconds this lock is valid for. All locks that have passed their TTL are considered expired and will be automatically removed.
- Return type:
- Returns:
the lock unique identifier.
- acquired(name: str, *, concurrency: int, ttl: int) Iterator[None] [source]
Context manager to perform actions while holding a lock for the given name and parameters.
Examples:
# Get a lock for a given name allowing to have other 2 concurrent runs in parallel with a TTL of 30 minutes with lock.acquired("some.lock.identifier", concurrency=3, ttl=1800): # Do something # Acquire an exclusive lock for the given name with a TTL of 1 hour with lock.acquired("some.lock.identifier", concurrency=1, ttl=1800): # Do something
- Parameters:
name (
str
) -- the lock name, cannot contain/
as that's a directory separator in the data structure.concurrency (
int
) -- how many concurrent clients can hold the same lock. If set to zero it means that there is no concurrency limit and an infinite number of clients can run concurrently, but they are tracked.ttl (
int
) -- the amount of seconds this lock is valid for. All locks that have passed their TTL are considered expired and will be automatically removed.
- Yields:
Nothing, it just gives back control to the client's code while holding the lock.
- Return type:
- get(name: str) spicerack.locking.KeyLocks [source]
Get the existing locks for the given name. If missing returns a new object with no locks.
- Parameters:
name (
str
) -- the lock name, cannot contain/
as that's a directory separator in the data structure.- Raises:
spicerack.locking.LockError -- for any etcd errors beside the key not found one.
- Return type:
- Returns:
the existing locks for the given name.
- class spicerack.locking.NoLock(*_args: Any, **_kwargs: Any) None [source]
Bases:
object
A noop locking class that does nothing.
Has the same APIs of the
spicerack.locking.Lock
, to be used when locking support is disabled.Initialize the instance.
- Parameters:
*_args (
typing.Any
) -- accept any positional arguments thatspicerack.locking.Lock.__init__()
accepts.**_kwargs (
typing.Any
) -- accept any keyword argument thatspicerack.locking.Lock.__init__()
accepts.
- acquire(*_args: Any, **_kwags: Any) str [source]
Dummy method that just returns an empty string.
- Parameters:
*_args (
typing.Any
) -- accept any positional arguments thatspicerack.locking.Lock.acquire()
accepts.**_kwargs -- accept any keyword argument that
spicerack.locking.Lock.acquire()
accepts.
- Return type:
- Returns:
an empty string.
- acquired(*_args: Any, **_kwargs: Any) Iterator[None] [source]
Context manager that just yields.
- Parameters:
*_args (
typing.Any
) -- accept any positional arguments thatspicerack.locking.Lock.acquired()
accepts.**_kwargs (
typing.Any
) -- accept any keyword argument thatspicerack.locking.Lock.acquired()
accepts.
- Yields:
Nothing, it just gives back control to the client's code.
- Return type:
- get(name: str, *_args: Any, **_kwags: Any) spicerack.locking.KeyLocks [source]
Dummy method that just returns an empty KeyLocks.
- Parameters:
name (
str
) -- the lock name, cannot contain/
as that's a directory separator in the data structure.*_args (
typing.Any
) -- accept any positional arguments thatspicerack.locking.Lock.acquire()
accepts.**_kwargs -- accept any keyword argument that
spicerack.locking.Lock.acquire()
accepts.
- Return type:
- Returns:
an empty instance.
- release(*_args: Any, **_kwags: Any) None [source]
Dummy method that does nothing.
- Parameters:
*_args (
typing.Any
) -- accept any positional arguments thatspicerack.locking.Lock.release()
accepts.**_kwargs -- accept any keyword argument that
spicerack.locking.Lock.release()
accepts.
- Return type:
- spicerack.locking.get_lock_instance(*, config_file: pathlib.Path | None, prefix: str, owner: str, dry_run: bool = True) spicerack.locking.Lock | spicerack.locking.NoLock [source]
Get a lock instance based on the configuration file and prefix.
- Parameters:
config_file (
typing.Optional
[pathlib.Path
]) -- the path to the configuration file for the locking backend orNone
to disable the locking support and return aspicerack.locking.NoLock
instance. When the configuration file is present aspicerack.locking.Lock
instance is returned instead. The configuration is also automatically merged with the~/.etcdrc
config file of the running user, if present.prefix (
str
) -- the name of the directory to use to prefix the lock. Must be one ofspicerack.locking.ALLOWED_PREFIXES
.owner (
str
) -- a way to identify the owner of the lock, usually in the form{user}@{hostname} [{pid}]
.dry_run (
bool
, default:True
) -- whether this is a DRY-RUN.
- Raises:
spicerack.locking.LockError -- if the provided prefix is not one of the allowed ones.
- Return type:
typing.Union
[spicerack.locking.Lock
,spicerack.locking.NoLock
]- Returns:
The locking instance or a dummy instance that has the same API of the locking one but does nothing if the config_file.
- spicerack.locking.ALLOWED_PREFIXES: tuple[str, ...] = ('cookbooks', 'custom', 'modules')
The allowed values for the prefix parameter to be used as path prefix for the locks keys.
- spicerack.locking.COOKBOOKS_CUSTOM_PREFIX: str = 'custom'
The path prefix to use for the key of locks acquired from inside a cookbook.
- spicerack.locking.COOKBOOKS_PREFIX: str = 'cookbooks'
The path prefix to use for the key of locks acquired by spicerack for each cookbook execution.