Transports

Abstract transport and state machine for hosts state.

class cumin.transports.Command(command, timeout=None, ok_codes=None)[source]

Bases: object

Class to represent a command.

__eq__(other)[source]

Equality operation. Allow to directly compare a Command object to another or a string.

Parameters:according to Python's Data model object.__eq__().
Returns:True if the other object is equal to this one, False otherwise.
Return type:bool
Raises:exceptions.ValueError -- if the comparing object is not an instance of Command or a str.
__init__(command, timeout=None, ok_codes=None)[source]

Command constructor.

Parameters:
  • command (str) -- the command to execute.
  • timeout (int, optional) -- the command's timeout in seconds.
  • ok_codes (list, optional) -- a list of exit codes to be considered successful for the command. The exit code zero is considered successful by default, if this option is set it override it. If set to an empty list [], it means that any code is considered successful.
__ne__(other)[source]

Inequality operation. Allow to directly compare a Command object to another or a string.

Parameters:according to Python's Data model object.__ne__().
Returns:True if the other object is different to this one, False otherwise.
Return type:bool
Raises:exceptions.ValueError -- if the comparing object is not an instance of Command or a str.
__repr__()[source]

Return the representation of the Command.

The representation allow to instantiate a new Command instance with the same properties.

Returns:the representation of the object.
Return type:str
__str__()[source]

Return the string representation of the command.

Returns:the string representation of the object.
Return type:str
ok_codes

List of exit codes to be considered successful for the execution of the Command.

Getter:Returns the current ok_codes or a list with the element 0 if not set.
Setter:list[int], None: list of exit codes to be considered successful for the execution of the command on each host. Must be a list of int in the range 0-255 included, or None to unset it. The exit code 0 is considered successful by default, but it can be overriden setting this property. Set it to an empty list to consider any exit code successful.
Raises:cumin.transports.WorkerError -- if trying to set it to an invalid value.
timeout

Timeout of the Command.

Getter:Returns the current timeout or None if not set.
Setter:float, int, None: the timeout in seconds for the execution of the command on each host. Both float and int are accepted and converted internally to float. If None the timeout is reset to its default value.
Raises:cumin.transports.WorkerError -- if trying to set it to an invalid value.
class cumin.transports.State(init=None)[source]

Bases: object

State machine for the state of a host.

current

int: the current state.

pending, scheduled, running, success, failed, timeout

int: the available valid states, according to valid_states.

is_pending, is_scheduled, is_running, is_success, is_failed, is_timeout

bool: True if this is the current state, False otherwise.

__getattr__(name)[source]

Attribute accessor.

Accessible properties:
 
  • current (int): retuns the current state.
  • is_{valid_state_name} (bool): for each valid state name, returns True if the current state matches the state in the variable name. False otherwise.
Parameters:

according to Python's Data model object.__getattr__().

Raises:

exceptions.AttributeError -- if the attribute name is not available.

__init__(init=None)[source]

State constructor. The initial state is set to pending it not provided.

Parameters:init (int, optional) -- the initial state from where to start. The pending state will be used if not set.
Raises:cumin.transports.InvalidStateError -- if init is an invalid state.
__repr__()[source]

Return the representation of the State.

The representation allow to instantiate a new State instance with the same properties.

Returns:the representation of the object.
Return type:str
__str__()[source]

Return the string representation of the state.

Returns:the string representation of the object.
Return type:str
_cmp(other)[source]

Comparison operation. Allow to directly compare a state object to another or to an integer.

Raises ValueError if the comparing object is not an instance of State or an integer.

Arguments: other -- the object to compare the current instance to

allowed_state_transitions = {0: (1,), 1: (2,), 2: (2, 3, 4, 5), 3: (0,), 4: (), 5: ()}

dictionary with {valid state: tuple of valid states} mapping of allowed transitions for any valid state.

Type:dict
failed = 4

Valid states.

pending = 0

Valid states.

running = 2

Valid states.

scheduled = 1

Valid states.

states_representation = ('pending', 'scheduled', 'running', 'success', 'failed', 'timeout')

tuple with the string representations of the valid states.

Type:tuple()
success = 3

Valid states.

timeout = 5

Valid states.

update(new)[source]

Transition the state from the current state to the new one, if the transition is allowed.

Parameters:new (int) -- the new state to set. Only specific state transitions are allowed.
Raises:cumin.transports.StateTransitionError -- if the transition is not allowed, see allowed_state_transitions.
valid_states = range(0, 6)

valid states indexes.

Type:list
class cumin.transports.BaseWorker(config, target)[source]

Bases: object

abstract Worker interface to be extended by concrete workers.

__init__(config, target)[source]

Worker constructor. Setup environment variables and initialize properties.

Parameters:
  • config (dict) -- a dictionary with the parsed configuration file.
  • target (Target) -- a Target instance.
commands

Commands for the current execution.

Getter:Returns the current command list or an empty list if not set.
Setter:list[Command], list[str]: a list of Command objects or str to be executed in the hosts. The elements are converted to Command automatically.
Raises:cumin.transports.WorkerError -- if trying to set it with invalid data.
execute()[source]

abstract Execute the task as configured.

Returns:0 on success, a positive integer on failure.
Return type:int
Raises:cumin.transports.WorkerError -- if misconfigured.
get_results()[source]

abstract Iterate over the results (generator).

Yields:tuple -- with (hosts, result) for each host(s) of the current execution.
handler

abstract Get and set the handler for the current execution.

Getter:Returns the current handler or None if not set.
Setter:str, EventHandler, None: an event handler to be notified of the progress during execution. Its interface depends on the actual transport chosen. Accepted values are: * None => don't use an event handler (default) * str => a string label to choose one of the available default EventHandler classes in that transport, * an event handler class object (not instance)
success_threshold

Success threshold for the current execution.

Getter:float: returns the current success_threshold or 1.0 (100%) if not set.
Setter:float, None: The success ratio threshold that must be reached to consider the run successful. A float between 0 and 1 or None to reset it. The specific meaning might change based on the chosen transport.
Raises:cumin.transports.WorkerError -- if trying to set it to an invalid value.
timeout

Global timeout for the current execution.

Getter:int: returns the current timeout or 0 (no timeout) if not set.
Setter:int, None: timeout for the current execution in seconds. Must be a positive integer or None to reset it.
Raises:cumin.transports.WorkerError -- if trying to set it to an invalid value.
exception cumin.transports.InvalidStateError[source]

Bases: cumin.CuminError

Exception raised when an invalid transition for a node's State was attempted.

with_traceback()

Exception.with_traceback(tb) -- set self.__traceback__ to tb and return self.

class cumin.transports.NoProgress[source]

Bases: object

Used as a null object to disable the display of progress bars.

__init__

inherited Initialize self. See help(type(self)) for accurate signature.

close()[source]

Does nothing

init(num_hosts)[source]

Does nothing

update_failed(num_hosts=1)[source]

Does nothing

update_success(num_hosts=1)[source]

Does nothing

class cumin.transports.ProgressBars[source]

Bases: object

Progress bars for the status of successful / failed hosts.

The ProgressBars needs to be notified of the total number of hosts when the operation starts, and then notified of successes and failures.

__init__()[source]

Create the progress bars.

Note that the progress bars themselves are not initalized at object creation. init() needs to be called before using the progress bars.

close()[source]

Closes the progress bars.

init(num_hosts)[source]

Initialize the progress bars.

Parameters:num_hosts (int) -- the total number of hosts
update_failed(num_hosts=1)[source]

Updates the number of failed hosts.

Parameters:num_hosts (int) -- increment to the number of hosts that have completed in error
update_success(num_hosts=1)[source]

Updates the number of successful hosts.

Parameters:num_hosts (int) -- increment to the number of hosts that have completed successfully
exception cumin.transports.StateTransitionError[source]

Bases: cumin.CuminError

Exception raised when an invalid transition for a node's State was attempted.

with_traceback()

Exception.with_traceback(tb) -- set self.__traceback__ to tb and return self.

class cumin.transports.Target(hosts, batch_size=None, batch_size_ratio=None, batch_sleep=None)[source]

Bases: object

Targets management class.

__init__(hosts, batch_size=None, batch_size_ratio=None, batch_sleep=None)[source]

Constructor, inizialize the Target with the list of hosts and additional parameters.

Parameters:
  • hosts (ClusterShell.NodeSet.NodeSet, list) -- hosts that will be targeted, both ClusterShell.NodeSet.NodeSet and list are accepted and converted automatically to ClusterShell.NodeSet.NodeSet internally.
  • batch_size (int, optional) -- set the batch size so that no more that this number of hosts are targeted at any given time. It must be a positive integer. If greater than the number of hosts it will be auto-resized to the number of hosts.
  • batch_size_ratio (float, optional) -- set the batch size with a ratio so that no more that this fraction of hosts are targeted at any given time. It must be a float between 0 and 1 and will raise exception if after rounding it there are 0 hosts selected.
  • batch_sleep (float, optional) -- sleep time in seconds between the end of execution of one host in the batch and the start in the next host. It must be a positive float.
Raises:

cumin.transports.WorkerError -- if the hosts parameter is empty or invalid, if both the batch_size and batch_size_ratio parameters are set or if the batch_size_ratio selects no hosts.

_compute_batch_size(batch_size, hosts)[source]

Compute the batch_size based on the hosts size and return the value to be used.

Parameters:
  • batch_size (int, None) -- a positive integer to indicate the batch_size to apply when executing the worker or None to get its default value of all the hosts. If greater than the number of hosts, the number of hosts will be used as value instead.
  • hosts (ClusterShell.NodeSet.NodeSet) -- the list of hosts to use to calculate the batch size.
Returns:

the effective batch_size to use.

Return type:

int

static _compute_batch_sleep(batch_sleep)[source]

Validate batch_sleep and return its value or a default value.

Parameters:batch_sleep (float, None) -- a positive float indicating the sleep in seconds to apply between one batched host and the next, or None to get its default value.
Returns:the effective batch_sleep to use.
Return type:float
first_batch

First batch of the hosts to target.

Getter:Returns a ClusterShell.NodeSet.NodeSet of the first batch of hosts, according to the batch_size.
exception cumin.transports.WorkerError[source]

Bases: cumin.CuminError

Custom exception class for worker errors.

with_traceback()

Exception.with_traceback(tb) -- set self.__traceback__ to tb and return self.

cumin.transports.raise_error(property_name, message, value)[source]

Raise a WorkerError exception.

Parameters:
  • property_name (str) -- the name of the property that raised the exception.
  • message (str) -- the message to use for the exception.
  • value (mixed) -- the value that raised the exception.
cumin.transports.validate_list(property_name, value, allow_empty=False)[source]

Validate a list.

Parameters:
  • property_name (str) -- the name of the property to validate.
  • value (list) -- the value to validate.
  • allow_empty (bool, optional) -- whether to consider an empty list valid.
Raises:

cumin.transports.WorkerError -- if trying to set it to an invalid value.

cumin.transports.validate_positive_float(property_name, value)[source]

Validate a positive float or None.

Parameters:
  • property_name (str) -- the name of the property to validate.
  • value (float, None) -- the value to validate.
Raises:

cumin.transports.WorkerError -- if trying to set it to an invalid value.

cumin.transports.validate_positive_integer(property_name, value)[source]

Validate a positive integer or None.

Parameters:
  • property_name (str) -- the name of the property to validate.
  • value (int, None) -- the value to validate.
Raises:

cumin.transports.WorkerError -- if trying to set it to an invalid value.

Available transports