Release Notes

Cumin Changelog

v5.1.1 (2025-03-14)

Bug fixes

  • puppetdb: fix inventory query with quoted part in dot-notation for the /inventory endpoint.

v5.1.0 (2025-03-13)

New features

  • puppetdb: add support for structured facts and more querying the inventory endpoint of PuppetDB, that allows to query structured facts and other resources exposed by this endpoint with the PuppetDB's dot notation.

Minor improvements

  • cli: log an eventual exception to stderr when --trace is set in addition to the log entry in the logs on file for better visibility (T384539).

Bug fixes

  • query: do not error on no match in first subquery. When running queries that have subqueries and the first subquery is empty, cumin is currently raising an InvalidQueryError while that's still a valid result for the first subquery and should not raise.

Miscellanea

  • docs: removed deprecated call to sphinx_rtd_theme.

v5.0.0 (2025-01-16)

Dependencies breaking changes

  • Drop Python support for 3.7, 3.8, add suppoert for 3.11, 3.12 and 3.13.

    • Drop support for Python 3.7 and 3.8 (up to Debian Buster).

    • Add support for Python 3.11, 3.12 and 3.13.

    • Drop testing on Python 3.10 to speed up CI as that's a version not present in any Debian release.

    • Update minimum dependencies to the version in Debian Bullseye (apart few exceptions).

    • Move the tests for minimum version from Python 3.7 to 3.9.

    • Fix .gitignore matching for .coverage* files.

    • Add temporary pinning of some sphinx-contrib packages to make it work both on bullseye and sid.

API breaking changes

  • puppetdb: drop support for deprecated API v3. As PuppetDB have since long time deprecated version 3 of the API in favor of version 4 and there is no plan of a version 5, cleaning also the multi-version support and stop reading the version from the configuration file.

Minor improvements

  • puppetdb backend: updated the query so that we only select the certname, which is all cumin needs, it also uses group by to only return unique results.

Bug fixes

  • setup.py: force a newer sphinx_rtd_theme.

  • tests: make it compatible with urllib3 v2.0+.

  • tox.ini: make it compatible with tox 4.0+.

  • tox.ini: add compatibility with newer Sphinx 7.1+ using sphinx-build instead of setup.py build_sphinx..

  • doc: update .readthedocs.yml configuration to be compatible with their version 2 of the configuration file schema.

  • doc: don't fail on warning on readthedocs, we currently have a warning to keep the compatibility between bullseye and sid when generating the documentation.

Miscellanea

  • doc: mention inclusion into Debian upstream.

  • Use importlib.metadata instead of pkg_resources now that Python 3.7 support has been dropped, remove the need for the pkg_resource library and use importlib.metadata instead.

  • integration tests: use linuxserver/openssh-server and adapt config to use a dedicated user and not the root user to the container.

v4.2.0 (2023-01-12)

Minor improvements

  • backends.puppetdb: add a timeout to all requests calls to PuppetDB APIs. Allow to set the timeout from the configuration file, with a default of 30 seconds if not set.

  • backends.puppetdb: support using TLS client certificates when communicating with the PuppetDB server.

  • setup.py: support Python 3.10 and Pyparsing 3

  • cumin: set __version__ during Debian build:

    • In order to better support building in a pure Debian environment where the tests and the man page generation are done without installing the package, add support for reading the setuptools_scm fake version when exported via SETUPTOOLS_SCM_PRETEND_VERSION.

Bug fixes

  • CLI: minor fix on confirmation message.

  • backends.puppetdb: fix urllib3 import. More recent versions of requests don't ship urllib3 embedded, import that directly.

  • setup.py: add max version limit for Jinja2 in the tests-min environment because it breaks the old Sphinx version with more recent versions.

  • grammars: remove usage of leaveWhitespace. The calls to pyparsing's leaveWhitespace in various grammars was not really necessary and breaks with pyparsing 3+. Removing them in order to add support for pyparsing 3.

Miscellanea

  • pylint: fix newly reported pylint issues and removed unnecessary disable comments.

  • tests: fix typo in test name

  • prospector: disable pylint's consider-using-f-string error as the code still uses format(). To be converted to f-string in the future.

  • doc: set the default language. Latest Sphinx 5.0 release requires language to not be None, it raises a warning otherwise, and we fail on warnings.

  • Add configuration for the release script. This adds a WMF-specific configuration file to be used by the release script to make new releases (git tag, debian package, release to apt.w.o, PyPI release).

  • mypy: remove unnecessary ignores now that the upstream bug has been fixed.

  • doc: remove broken badges from the README.

  • setup.py: temporary fix for test dependencies

  • setup.py: add python_requires metadata.

  • setup.py: set a temporary upper limit for pylint and prospector.

  • Refactor tox to simplify the testing matrix testing the minimum version of dependencies only with the oldest supported Python version (3.7).

v4.1.1 (2021-06-23)

New features

  • config: add support for Kerberos auth (T244840):

    • In an environment where when running Cumin the user authenticates the SSH connection to the remote hosts via Kerberos, in case the user doesn't have a valid Kerberos ticket, it would get a cryptic authentication failure message.

    • In order to present the user a more meaningful error, a new configuration stanza named kerberos is added, see the example configuration file for all the details.

    • When configured to do so Cumin will ensure that the running user has a valid Kerberos ticket before trying to SSH to the target hosts, and present the user a nicer error message otherwise.

    • When using Cumin as a library, the cumin.ensure_kerberos_ticket function can be used to achieve the same functionality.

Miscellanea

  • setup.py: add types dependencies for mypy for dependencies that don't have yet type hints.

  • doc: use add_css_file() instead of add_stylesheet() as Sphinx version 4 removed the old name.

v4.1.0 (2021-05-03)

CLI breaking changes

  • cli: change confirmation input check

    • When asking for confirmation to execute a command in interactive mode, instead of y/n ask the user to enter the exact number of affected hosts to make sure they are aware of the impact of the action to be performed and prevent muscle memory errors.

    • Inspired by: https://rachelbythebay.com/w/2020/10/26/num/

  • cli: in dry-run mode send the list of hosts to stdout (T212783).

    • To simplify the usage from other tools that want to consume the generated list of hosts when not executing commands (dry-run mode), using cumin just to query matching hosts, send the list to stdout while the rest of the output is sent to stderr.

    • In conjunction with the new -n/--no-color option it should allow for an easy piping into additional tools using cumin as a selector of hosts:

      cumin -n "QUERY" 2> /dev/null | some_other_tool

New features

  • cli: add a -n/--no-colors option to suppress any colored output (T212783).

  • cli/clustershell: allow to disable progress bars (T212783):

    • Expose in the clustershell module the possibility to disable the show of the progress bars during execution.

    • Allow to disable the progress bars also in the CLI with a --no-progress flag.

  • config: allow using tilde ~ to specify config paths. This allows abstracting away the paths and seamlessly allowing per-user configurations.

  • config: expand user's home directory for logging. Allow to specify a log path relative to the user's home directory in the configuration file for the log_file entry that will expand ~ when present.

  • clustershell: allow to choose different reporters (T212783):

    • Make the event handler reporter configuratble between:

      • A null reporter that doesn't print anything (NullReporter)

      • A Tqdm-compatible reporter that prints just the success/failure results to stderr but doesn't print the actual command outputs (TqdmQuietReporter)

      • A Tqdm-compatible reporter that prints the command outputs to stdout and the success/failure results to stderr, as it was until now (TqdmReporter, the default).

  • setup.py: support more recent PyParsing versions

    • In order to be able to build Cumin on Debian bullseye, add support for more recent versions of PyParsing that introduced backward incompatible changes.

Minor improvements

  • Add support for Python 3.8 and 3.9

Bug fixes

  • tests: fix dependencies for tests (T270795).

    • Remove the limitation on prospector as the upstream bug was fixed.

    • Exclude flake8 from the minimum requirements as we just run the unit tests with the minimum requirements.

    • This will require the removal of python3-flake8 from the Build-Depends on debian/control when doing the next release as flake8 is not needed when building the package.

  • tests: fix integration tests as the newer versions of the sshd docker container needs a specific environment variable to enable the root user SSH access.

  • setup.py: Add missing long_description_content_type parameter.

  • doc: fix sphinx warning in docstring.

Miscellanea

  • setup.py: revert tqdm upper limit constraint.

    • As the upstream issue has been fixed in tqdm v4.48.0, remove the upper limit constraint.

    • Note: cumin will have output issues if used with a tqdm between v4.24.0 and v4.48.0 excluded.

  • Use @abstractmethod instead of @abstractproperty. The latter it's actually deprecated in favor of usin @abstractmethod in conjunction with @property and @example.setter.

  • Extracting obvious reporting code to a Reporter class to be able to expose the reporting functionality via the library APIs (T212783).

  • Introduce an interface for progress bars.

  • tox: add mypy environment.

    • In order to start adding type hints to the project, add a mypy environment to tox to ensure those added are correct.

    • Keep the configuration very light for now until type hints are added to the whole project.

  • tests: remove unnecessary environmental variables config. As cumin can run as a normal user some configuration to make it think it was run as root is not needed anymore across unit and integration tests.

  • integration tests: add undeduplicated output test.

    • The case of undeduplicated output, like when there is only one target host, was not tested by the integration tests. Adding a test to cover that use case too.

  • tests: fix pip backtracking

    • With the current setup of minimizing the number of different virtualenvs used by tox we ended up hitting an issue of pip backtracking. As prospector seems to be the most likely culprit here because has a lot of dependencies, and in the past too we had issues between prospector and flake8 dependencies, move prospector to its own virtualenv.

    • Add also mypy as an explicit dependency.

  • tests: fix minimum dependency and pytest warning.

    • Change the behaviour of the -min environment in tox to test with the minimum supported version of only the real dependencies and not the ones used only for the tests, with the only exception of Sphinx-related dependencies that are needed to build the manpage during the Debian build process.

    • Update pytest's command line options to prevent deprecation warnings.

v4.0.0 (2020-09-10)

  • No changes from the RC1 release.

v4.0.0rc1 (2020-06-09)

Dependency breaking changes

  • tqdm: limit the compatible versions of tqdm allowed to a small range of versions between 4.19.4 and 4.24.0) due to an upstream bug, see tqdm issue #777. The 4.23.4-1~wmf1 version of tqdm is available as a Debian package for buster in the Wikimedia APT repository in the component/spicerack component.

New features

  • Replace colorama with custom module (T217038).

    • In Debian stretch there is a regression in colorama in conjunction with tqdm that leads to a slow down of the progress of the script proportional to the amount of data printed to stdout/err. Colorama starts having very huge stacktraces and the process is stuck at 100% CPU for an increasingly amount of time while more data is printed.

    • Given the very simple usage of colors that is made in Cumin as of now, it seems much more feasible to replace the colorama library (as all that cross-OS support is not needed) and add a simple module with ANSI escape sequence support.

    • Use a type (metaclass) to be able to override __getattr__ for the static methods of the classes that use it and to automatically define a method for each color in a DRY way without code duplication.

    • Define a Colored class that uses ColoredType as metaclass to inherit its type with the custom behaviour.

    • For each color defined in ColoredType.COLORS a method of Colored is defined, e.g. Colored.red().

    • The Colored class has a disabled property that can be set to True to globally disable coloring. This could for example be integrated later into the CLI as an option to disable colors or allow to add some code to the color.py module to autodetect when not in a TTY and automatically disable all colors.

  • Allow running cumin as a regular user (T218440).

  • backends.puppetdb: make the PuppetDB backend process primitive types for queries (T207037).

    • Modify the grammar to recognize primitive PuppetDB types, communicate quotedness to the final output as appropriate.

  • backends.puppetdb: allow to override the URL scheme in the configuration (T218441).

    • In some environments the PuppetDB hosts might listen only on HTTP on localhost and the Cumin host might connect to it via an SSH tunnel.

    • Allow to override the default HTTPS scheme of the PuppetDB URL in the configuration.

  • backends.puppetdb: fix regex matching.

    • Fix regex matching in PuppetDB queries that requires that all backslashes are escaped according to the PuppetDB API. See PuppetDB documentation on regexp-match.

  • backends.openstack: add custom parameters for the client (T201881).

    • The instantiation of the novaclient Client might require additional parameters based on the specific OpenStack installation, like for example a region_name.

    • Add a generic client_params section to the configuration to allow to set arbitrary additional parameters that will be passed to the novalicent's Client.

  • CLI: improve help message (T204680).

    • Specify that the --debug and --trace options affect the logs and not the output and where to find the logs.

Miscellanea

  • Add official support to Python 3.7, deprecate support for 3.4, 3.5 and 3.6.

  • setup.py: make it compatible with Debian buster.

    • Add support for Debian Buster, using its versions as minimum required version for dependencies except tqdm.

    • For tqdm restrict the possible versions to a specific range, that is the only one that works fine with multiple progress bars and colors.

    • Remove support for Debian Stretch

  • transports.clustershell: extract progress bars from clustershell event handling.

  • tests: fix any newly reported issue by the various linters and static checkers.

  • tests: refactor some tests taking advantage of pytest functionalities.

  • tests: refactor tox configuration.

  • Updated documentation according to external dependency changes.

  • flake8: enforce import order and adopt W504.

    • Add flake8-import-order to enforce the import order using the edited style that corresponds to our styleguide, see: Python imports.

    • Fix all out of order imports.

    • For line breaks around binary operators, adopt W504 (breaking before the operator) and ignore W503, following PEP8 suggestion, see: PEP8 binary operator.

    • Fix all line breaks around binary operators to follow W504.

  • test: improve integration tests

    • Don't hide the output of the setup commands, it's useful to both see that the output is visually correct and allow to debug any error in setting up the integration tests.

    • Allow to pass arguments to the integrations tests so that the deletion of the test instances and temporarily generated files can be accessed for debugging.

  • doc: fix and improve documentation.

    • Adapt Sphinx settings according to the newer version used.

    • Fix links to the documentation of external libraries.

    • Add and include the diagram image for the available transitions for the cumin.transports.State class.

    • Improve docstrings for a better generated documentation result.

    • Remove unnecessary Sphinx helper functions, now correctly handled by Sphinx natively.

  • doc: split HTML and manpage generation.

    • Add a man tox environment to build only the manpage.

    • Add a dedicated man-min environment to build the manpage with the minimum version of Sphinx, that is the one of Debian Buster and that will be used to generate the manpage when building the Debian package.

    • Let the sphinx tox environment just build the HTML documentation.

v3.0.2 (2018-07-30)

Bug Fixes

  • Fix the -o/--output option (bytes->str conversion) (T200622):

    • The migration to Python3 left the -o/--output option of the CLI with some decoding issue from bytes to string.

    • Uniforming all calls to bytes.decode() not specifying the encoding as utf-8 is the default in Python 3.

    • Add integration tests for the -o/--output option.

  • CLI: fix setup_logging() when called without path (T188627):

    • Fix the setup_logging() function when it's called with a filename without a path, in order to log directly into the current directory. Thanks goes to aggro for reporting it.

  • Fix debugging log message conversion. The Command.timeout can also be None in case is not set, converting it to string instead of integer for the debug messages.

Miscellanea

  • Updated PyPI URLs to the new PyPI website

v3.0.1 (2018-02-19)

Bug Fixes

  • CLI: fix help message

v3.0.0 (2018-02-19)

API breaking changes

  • Migration to Python 3, dropping support of Python 2. Besides the usual Py2 -> Py3 conversions, the main changes are:

    • Add nodeset() and nodeset_fromlist() functions in the cumin module to instantiate ClusterShell's NodeSet objects with the resolver set to RESOLVER_NOGROUP, due to ClusterShell issue #368.

    • Bump dependency on ClusterShell library to 1.8.

    • Adapt callbacks in ClusterShell backend to the new ClusterShell's API signatures of version 1.8.

    • Use threading.Lock() calls as context managers for the with statement.

    • Use Colorama autoreset feature, simplifying its related calls.

New features

  • Backends: add known hosts files backend:

    • The knownhosts backend allow to use Cumin taking advantage of existing SSH known hosts files that are not hashed. It allow to write arbitrarily complex queries with subgroups and boolean operators, but each item must be either the hostname itself, or using host expansion with the powerful ClusterShell's NodeSet syntax.

    • See the example configuration on how to configure this backend with the list of known hosts files to be parsed.

    • The typical use case for the knownhosts backend is when the known hosts file(s) are generated and kept updated by some external configuration manager or tool that is not yet supported as a backend for Cumin. It can also work as a fallback backend in case the primary backend is unavailable but the known hosts file(s) are still up to date.

  • Batch size: allow to specify it in percentage (T187185):

    • Transports: allow to specify a batch_size_ratio as a float number in the Target constructor to set the batch_size as a percentage of the hosts list.

    • CLI: make the --batch-size option to accept both integers and percentage (i.e. 50%) values.

v2.0.0 (2018-01-19)

API breaking changes

  • Logging: uniform loggers (T179002):

    • Remove optional parameter logger from all classes where it was accepted, the classes instantiate the proper logger based on the current module and class name.

  • ClusterShell backend: fix execute() return code:

    • The return code of the execute() method was not respecting the parent class contract for its return code when there are no commands set or no hosts to target.

    • Make the Target class raise a WorkerError exception on instantiation if there are no target hosts.

    • Make the execute() method raise a WorkerError exception if there are no commands to execute.

New features

  • Backends: add support to external backends plugins (T178342):

    • Custom external backends can be developed outside of Cumin and used by Cumin as any other backend.

    • The external backends must:

      • Be present in Python PATH.

      • Define a GRAMMAR_PREFIX attribute that doesn't conflict with built-in backends prefixes.

      • Define a query_class attribute pointing to a class that inherit from cumin.backends.BaseQuery.

    • The CLI is not anymore able to enforce that the --backend parameter is valid when parsing the command line arguments, but will fail later on with a clear message.

  • PuppetDB backend: add support for PuppetDB API v4 (T182575):

    • Allow to set the API version via configuration.

    • Default to API v4 as v3 is obsolete.

    • Use POST for API v4 to overcome GET limits on large queries, fixes T166397.

    • Bumped minimum version for requests-mock to 1.3.0.

Minor improvements

  • Logging: uniform loggers (T179002):

    • Use proper hierarchical loggers across the project.

    • For classes inherited from a base abstract class, the logger is defined only in the base abstract class, with the name of the concrete class that is calling it.

    • Changed CLI logging format to take advantage of the hirarchical logging.

  • Logging: use % syntax for parameters (T179002):

    • For optimization purposes and to adhere to Python best practices, use %s syntax in logging messages and pass the replacement parameters to the logging function. Some messages are still pre-formatted before the call to the logging function because used also for other purposes.

    • pylint: re-enable the check for logging-format-interpolation.

v1.3.0 (2017-11-03)

New features

  • PuppetDB backend: Class, Roles and Profiles shortcuts (T178279):

    • It is becoming common practice to use the role/profile paradigm in Puppet, where each host has only one role named Role::Module::Name that includes multiple profiles of the type Profile::Module::Name. If this practice is used, queries for those resources in Cumin will be very common and not user-friendly, requiring to write queries of the type R:Class = Role::Module::Name. Add support to Roles and Profiles so that they can be queried via shortcuts with O:Module::Name for roles and P:Module::Name for profiles.

    • Add also a generic class shortcut to quickly query a class resource with C:class_name or C:path::to::class.

    • The special syntax for fields @field and parameters %param are also supported. When querying for any of the above shortcuts, like P:Module::Name%param = value. The generated query will include two subqueries in AND between them, one for the class title and the other for the class parameter.

Minor improvements

  • Refactor documentation:

    • Moved most of the content from the README to the classes, function and documentation pages where it really belongs.

    • Add documentation files for an introduction to cumin, how to install it, how to develop it and with the release notes.

    • Add animated GIF to the README and documentation introduction.

Bug Fixes

  • Documentation: amend CHANGELOG and TODO for the addition of the manpage in v1.2.2 (T159308).

  • Documentation: add ReadTheDocs specific configuration.

  • Documentation: fix ReadTheDocs CSS override

v1.2.2 (2017-10-11)

Minor improvements

  • Dependencies: split the OpenStack dependencies into a separate extras_require in setup.py. This allows to install Cumin without all the dependencies needed for the OpenStack backend, if that is not needed.

  • Docstrings: use Google Style Python Docstrings to allow to automatically generate documentation with Sphinx.

  • Documentation: converted README, CHANGELOG and TODO from Markdown to reStructuredText. PyPI renders only reStructuredText while GitHub renders both. Moving to reStructuredText to be PyPI friendly and allow to write more powerful documentation.

  • CLI: extract the ArgumentParser definition from parse_args() into a get_parser() function for easier testability and documentation generation. Uniform help messages in ArgumentParser options.

  • setup.py: prepare for PyPi submission. Include the full README.rst as long description.

  • Documentation: setup Sphinx to generate the documentation and to auto-document the API and CLI.

  • Testing: refactored tox.ini to reduce the number of virtualenv while expanding the available environments for static analysis and tests performed, including running unit tests with the minimum supported versions of all the dependencies.

  • CLI: add manpage (T159308)

v1.2.1 (2017-09-27)

New features

  • OpenStack backend: allow to set default query params in the configuration (T176314): Allow to set arbitrary default query params in the configuration for the OpenStack backend. This is useful for example if Cumin is installed inside an OpenStack project to automatically search only within the instances of the current project. See the example in the provided doc/examples/config.yaml file.

Bug Fixes

  • Configuration: do not raise on empty configuration or aliases. Moved the check of required parameters where needed, in order to raise explicit exceptions with a more meaningful message for the user.

  • Exceptions: convert remaining spurious exceptions to CuminError or improve their error message.

v1.1.1 (2017-09-26)

Bug Fixes

  • OpenStack: limit grammar to not overlap with the global one.

v1.1.0 (2017-09-21)

New features

  • Backends: add OpenStack backend (T175711).

Bug Fixes

  • CLI: fix --version option.

  • Installation: fix data_files installation directory (T174008)

  • Transports: better handling of empty list (T174911):

    • BaseWorker: accept an empty list in the command setter. It's its default value, there is no point in forbidding a client to set it to the same value.

    • ClusterShellWorker: return immediately if there are no target hosts.

  • Clustershell: make call to tqdm.write() explicit where to send the output, not relying on its default.

v1.0.0 (2017-08-23)

CLI breaking changes

  • CLI: migrate to timeout per command (T164838):

    • the global timeout command line options changes from -t/--timeout to --global-timeout.

    • the -t/--timeout option is now used to set the timeout for each command in each host independently.

Configuration breaking changes

  • Query: add multi-query support (T170394):

    • Remove the backend configuration key as it is not anymore used.

    • Add a new optional default_backend configuration key. If set the query will be first executed with the default backend, and if failing the parsing it will be executed with the global multi-query grammar. This allow to keep backward compatibility with the query that were executed with previous versions of Cumin.

API breaking changes

  • PuppetDB backend: consistently use InvalidQueryError (T162151).

  • Transports: refactor command handling to support new features (T164838), (T164833) and (T171679):

    • Transports: move BaseWorker helper methods to module functions.

    • Transports: add Command class.

    • Transports: use the new Command class in BaseWorker, moving from a list of strings to a list of Command objects.

    • Transports: maintain backward compatibility and easy of usage automatically converting a list of strings to a list of Command objects when setting the commands property.

    • Allow to set the ok_codes property of the transports.Command class to an empty list to consider any return code as successful. The case in which no return code should be treated successful has no practical use.

    • ClusterShell: adapt the calls to commands for the new Command objects.

  • Configuration: move configuration loader from the cli module to the main cumin module (T169640):

    • add a cumin.Config class.

    • move the parse_config helper to cumin's main module from the cli one, to allow to easily load the configuration also when it's used as a Python library.

  • QueryBuilder: move query string to build() method. The constructor of the QueryBuilder was changed to not accept anymore a query string directly, but just the configuration and the optional logger. The query string is now a required parameter of the build() method. This properly split configuration and parameters, allowing to easily build() multiple queries with the same QueryBuilder instance.

  • Transports: convert hosts to ClusterShell's NodeSet (T170394):

  • in preparation for the multi-query support, start moving the transports to accept a ClusterShell's NodeSet instead of a list of nodes. With the new multi-query support the backends too will return only NodeSets.

  • Query: add multi-query support (T170394):

    • Aliases are now global and must use the global grammar syntax.

    • Query class: the public build() method has become private and now is sufficient to call the execute(query_string) method. Example usage:

      config = cumin.Config(args.config)
hosts = query.Query(config, logger=logger).execute(query_string)

    • Query class: the public methods open_subgroup() and close_subgroup() have become private, _open_subgroup() and _close_subgroup() respectively.

  • Transports: improve target management (T171684):

    • Add a Target class to handle all the target-related configuration.

    • Let the BaseWorker require an instance of the Target class and delegate to it for all the target-related configuration.

    • This changes the BaseWorker constructor signature and removes the hosts, batch_size and batch_sleep setters/getters.

New features

  • CLI: automatically set dry-run mode when no commands are specified (T161887).

  • ClusterShell transport: output directly when only a single host is targeted. When the commands are executed against only one host, print the output directly as it comes, to give the user an immediate feedback. There is no advantage to collect the output for de-duplication in this case (T164827).

  • Transports: allow to specify a timeout per Command (T164838).

  • Transports: allow to specify exit codes per Command (T164833). Allow to specify for each Command object a list of exit codes to be considered successful when executing its specific command.

  • ClusterShell backend: allow to specify exit codes per Command (T164833).

  • ClusterShell backend: allow to set a timeout per Command (T164838).

  • CLI: add -i/--interactive option (T165838). When set, this option drops into a Python shell (REPL) after the execution, allowing the user to manipulate the results with the full power of Python. In this first iteration it can be used only when one command is specified.

  • CLI: add -o/--output to get the output in different formats (T165842). Allow to have txt and json output when only one command is specified. In this first iteration the formatted output will be printed after the standard output with a separator, in a next iteration the standard output will be suppressed.

  • Query and grammar: add support for aliases (T169640):

    • Allow aliases of the form A:alias_name into the grammar.

    • Automatically replace recursively all the aliases directly in the QueryBuilder, to make it completely transparent for the backends.

  • Configuration: automatically load aliases from file (T169640). When loading the configuration, automatically load also any aliases present in the aliases.yaml file in the same directory of the configuration file, if present.

  • Query: add multi-query support (T170394):

    • Each backend has now its own grammar and parsing rules as they are completely independent from each other.

    • Add a new global grammar that allows to execute blocks of queries with different backends and aggregate the results.

  • CLI: add an option to ignore exit codes of commands (T171679). Add the -x/--ignore-exit-codes option to consider any executed command as successful, ignoring the returned exit codes. This can be useful for a cleaner output and the usage of batches when running troubleshooting commands for which the return code might be ignored (i.e. grep).

Minor improvements

  • CLI: improve configuration error handling (T158747).

  • Fix Pylint and other validation tools reported errors (T154588).

  • Package metadata and testing tools improvements (T154588):

    • Fill setup.py with all the parameters, suitable for a future submission to PyPI.

    • Autodetect the version from Git tags and expose it in the module using setuptools_scm.

    • CLI: add a --version option to print the current version and exit.

    • Tests: use pytest to run the tests.

    • Tests: convert tests from unittest to pytest.

    • Tests: make tox use the dependencies in setup.py, removing the now unnecessary requirements files.

    • Tests: add security analyzer Bandit to tox.

    • Tests: add Prospector to tox, that in turns runs multiple additional tools: dodgy, mccabe, pep257, pep8, profile-validator, pyflakes, pylint, pyroma, vulture.

  • Tests: simplify and improve parametrized tests. Take advantage of pytest.mark.parametrize to run the same test multiple times with different parameters instead of looping inside the same test. This not only simplifies the code but also will make each parametrized test fail independently allowing an easier debugging.

  • CLI: simplify imports and introspection.

  • Logging: add a custom trace() logging level:

    • Add an additional custom logging level after DEBUG called TRACE mainly for development debugging.

    • Fail in case the same log level is already set with a different name. This could happen when used as a library.

    • CLI: add the --trace option to enable said logging level.

  • Tests: improved tests fixture usage and removed usage of the example configuration present in the documentation from the tests.

  • Transports: improve command list validation of the transports.Command class to not allow an empty list for the commands property (T171679).

Bug Fixes

  • PuppetDB backend: do not auto upper case the first character when the query is a regex (T161730).

  • PuppetDB backend: forbid resource's parameters regex as PuppetDB API v3 do not support regex match for resource's parameters (T162151).

  • ClusterShell transport: fix set of list options (T164824).

  • Transports: fix success_threshold getter when set to 0 (T167392).

  • Transports: fix ok_codes getter for empty list (T167394).

  • QueryBuilder: fix subgroup close at the end of query. When a query was having subgroups that were closed at the end of the query, QueryBuilder was not calling the close_subgroup() method of the related backend as it should have. For example in a query like host1* and (R:Class = Foo or R:Class = Bar).

  • Fix test dependency issue. Due to a braking API change in the latest version of Vulture, Prospector is not working anymore with the installed version of Vulture due to missing constraint in their setup.py. See Prospector issue #230 for more details.

v0.0.2 (2017-03-15)

Configuration breaking changes

  • Add support for batch processing (T159968):

    • Moved the environment block in the configuration file to the top level from within a specific transport.

API breaking changes

  • Add support for batch processing (T159968):

    • Refactored the BaseWorker class (and the ClusterShellWorker accordingly) to avoid passing a lot of parameters to the execute() method, moving them to setters and getters with validation and default values, respectively.

    • Add state machine for a transport's node state.

    • Add CuminError exception and make all custom exceptions inherit from it to allow to easily catch only Cumin's exceptions.

  • ClusterShell transport: always require an event handler (T159968):

    • Since the addition of the batch capability running without an event handler doesn't really work because only the first batch will be scheduled.

    • Updated the CLI to work transparently and set the mode to sync when there is only one command.

    • Unify the reporting lines format and logic between sync and async modes for coherence.

New features

  • Add support for not in simple hosts selection queries (T158748).

  • Add support for batch processing (T159968):

    • It's now possible to specify a batch_size and a batch_sleep parameters to define the size of a sliding batch and an optional sleep between hosts executions.

    • ClusterShell transport: the batches behaves accordingly to the specified mode when multiple commands are specified:

      • sync: the first command is executed in a sliding batch until executed on all hosts or aborted due unmet success ratio. Then the execution of the second command will start if the success ratio is reached.

      • async: all the commands are executed in series in the first batch, and then will proceed with the next hosts with a sliding batch, if the success ratio is met.

    • Improves logging for backends and transport.

    • CLI: updated to use the batch functionality, use the transport return value as return code on exit.

    • Improves test coverage.

  • PuppetDB backend: automatically upper case the first character in resource names (T159970).

Minor improvements

  • Moved config.yaml to a doc/examples/ directory. It simplify the ship of the example file when packaging.

  • Allow to ignore selected urllib3 warnings (T158758).

  • Add codecov and codacy config and badges.

  • Fixing minor issues reported by codacy (T158967).

  • Add integration tests for ClusterShell transport using Docker (T159969).

Bug Fixes

  • Match the whole string for hosts regex matching (T158746).

v0.0.1 (2017-02-17)

  • First released version (T154588).