Spicerack - Automation framework for the WMF production infrastructure¶
Spicerack provides an entry point to all the libraries needed to automate and orchestrate tasks inside the Wikimedia
Foundation's production infrastructure. It provides also an entry point script
cookbook to list and run the
available cookbooks, both one by one or via an interactive menu.
Collection of cookbooks to automate and orchestrate operations in the WMF infrastructure.
The cookbooks will be executed by the
cookbook entry point script of the
The cookbooks must be structured in a tree, as they can be run also from an interactive menu that shows the tree from an arbitrary entry point downwards.
Each cookbook filename must be a valid Python module name, hence all lowercase, with underscore if that improves readability and that doesn't start with a number.
Given that the cookbooks are imported dynamically, a broader set of characters like dashes and starting with a number are technically allowed.
Example of cookbooks tree:
cookbooks |-- __init__.py |-- top_level_cookbook.py |-- group1 | |-- __init__.py | `-- important_cookbook.py `-- group2 |-- __init__.py `-- subgroup1 |-- __init__.py `-- some_task.py
Each cookbook must define:
A title setting a string variable
__title__at the module-level with the desired static value.
argument_parser() -> argparse.ArgumentParserfunction that accepts no arguments and return the
argparse.ArgumentParserinstance to use to parse the arguments of the cookbook. This function is optional, if not defined it means that the cookbook doesn't accept any argument. If a cookbook without that doesn't define an
argument_parser()function is called with CLI arguments it's considered an error.
args (argparse.Namespace, None): the parsed CLI arguments according to the parser returned by the
Noneif no CLI arguments were passed and the cookbook doesn't define an
argument_parser()function. Cookbooks are encouraged to define an
argument_parser()function so that an help message is automatically available with
-h/--helpand it can be shown both when running a cookbook directly or in the interactive menu.
spicerack (spicerack.Spicerack): an instance of
spicerack.Spicerackinitialized based on the generic CLI arguments parsed to the
cookbookentry point script. It allows to access all the libraries available in the
The logging is already pre-setup by the
cookbook entry point script that initialize the root logger, so that each
cookbook can just initialize its own
logging instance and log. A special logger to send notification to the
#wikimedia-operations IRC channel is also available through the
spicerack argument passed to the cookbook's
Example of logging:
import logging logger = logging.getLogger(__name__) # pylint: disable=invalid-name logger.info('message')
All the available modules in the Spicerack package are exposed to the cookbooks through the
spicerack argument to
run() function, that offers helper methods to obtain initialized instances of all the available libraries.
This instance exposes also some of the global CLI arguments parsed by the
cookbook entry point script such as
verbose as getters. See
spicerack.Spicerack for more details.