In order to build on Debian Stretch without backported packages, relax a bit the urllib3 dependency as the only goal for to specify it is to avoid conflicts with the latest version.
documentations: fix Sphinx configuration
In order to avoid issues while building the Debian package on Stretch where Sphinx
1.4.9is available, change configuration to:
Reduce minimum Sphinx version to
setup.cfgthat is applied to every Sphinx run, and move it directly into
tox.inias a command line
-Woption, that will be executed only by
toxand not during the Debian package build process.
prometheus: add timeout support to
ganeti: add timeout support
cookbook API: drop
No current cookbook is using the dynamic way to provide a title through
This abstraction has not proven to be useful and the fact to mangle dynamically the title of a cookbook based on the current parameter while you can then execute it with different ones doesn't seem very useful, dropping it completely from the Cookbook API
doc: mark Sphinx warnings as error
To make the documentation building process more robust make Sphinx fail on warnings too
Sphinx > 1.5and will require to use the backport version while building the package on Debian Stretch
doc: add checker to ensure modules are documented
It's common when adding a new module to forget to add the few bits required to auto-generated its documentation
Add a check to ensure that all Spicerack modules are listed in the documentation API index and that the linked files exists
ganeti: Fix RAPI port
prometheus: fix base URL template
doc: autodoc missing API modules
urllib3version due to
Add emacs ignores to gitignore
tests: temporarily force
bandit < 1.6.0
Due to a bug upstream bandit 1.6.0 doesn't honor the excluded directories, causing the failure of the bandit tox environments. Temporarily forcing its version
Add basic Ganeti RAPI support
Add basic Prometheus support
elasticsearch: logging during shard allocation was too verbose, some messages lowered to debug level
flake8: enforce import order and adopt
flake8-import-orderto enforce the import order using the
editedstyle that corresponds to our styleguide, see: mediawiki.org: Coding_conventions/Python
Mark spicerack as local and do not specify any organization-specific packages to avoid to keep a manually curated list of packages
Fix all out of order imports
For line breaks around binary operators, adopt
W504(breaking before the operator) and ignore
W503, following PEP8 suggestion, see: PEP0008#line_break_binary_operator
Fix all line breaks around binary operators to follow
elasticsearch: Retrieve hostname and fqdn from node attributes
elasticsearch: make unfreezing writes more robust (T219640)
elasticsearch: cleanup test by introducing a method to mock API calls
elasticsearch: rename elasticsearchclusters to elasticsearch_clusters
tox: fix typo in environment name
Add Python type hints and mypy check, not for variables and properties as we're still supporting Python 3.5
setup.py: revert commit 3d7ab9b that forced the
urllib3version installed as it's not needed anymore
tests/docs: unify usage of
ipmi: add password reset functionality
elasticsearch: upgrade rows one after the other
remote: suppress Cumin's output. As a workaround for a regression in colorama for stretch
Expose hostname from Reason.
elasticsearch: use the admin Reason to get current hostname
debmonitor: fix missing variable for logging line
elasticsearch: fix typo (xarg instead of xargs)
doc: fix reStructuredText formatting
Drop support for Python 3.4
Add support for Python 3.7
tests: refactor tox environments
elasticsearch: support cluster names which have
elasticsearch: systemctl iterates explicitly on elasticsearch instances.
remove_on_errorparameter to the
hosts_downtimed()context manager to decide wether to remove the downtime or not on error.
elasticsearch: raise logging level to ERROR for elasticsearch
elasticsearch: retry on all urllib3 exceptions
elasticsearch: retry on TransportError while waiting for node to be up
Change !log formatting to match Stashbot expectations.
icinga: add context manager for downtimed hosts:
Add a context manager to allow to execute other commands while the hosts are downtimed, removing the downtime at the end.
management: add management module:
Add a management module with a
Managementclass to interact with the management console names.
For now just add a
get_fqdn()method to automatically calculate the management FQDN for a given hostname.
When running in DRY-RUN mode no real changes are done and usually the
@retrydecorated methods are checking for some action to be propagated or completed. Hence when in DRY-RUN mode they tend to fail and retry until the tries attempts are exhausted, adding unnecessary time to the DRY-RUN.
With this patch the
retry()decorator is able to automagically detect if it's a DRY-RUN mode when called by any instance method that has a
self._dry_runproperty or, in the special case of
RemoteHostsAdapterderived instances, it has a
delete()method to remove a host from PuppetDB and clean up everything on the Puppet master.
spicerack: expose the
ownergetter to Reason class:
Add a public getter for the owner part of a reason, that retuns in a standard format the user running the code and the host where it's running.
decorators: improve tests.
documentation: fine-tune generated documentation.
dns: remove unused
Add missing timeout to requests calls.
dns: fix logging message.
elasticsearch_cluster: fix issues found during live tests.
ipmi: fix typos in docstrings.
ipmi: add support for DRY RUN mode
config: add load_ini_config() function to parse INI files.
debmonitor: use the existing configuration file
Instead of requiring a new configuration file, use the existing one already setup by Puppet for the debmonitor client.
Inject the path of the Debmonitor config into the ctor with a default value.
puppet: add default
batch_sizewhen running puppet
Allow to specify the
batch_sizewhen running puppet on a set of hosts.
Add a default
batch_sizeto avoid to overload the Puppet master hosts.
phabricator: remove unneded pylint ignore
mediawiki: update maintenance host Cumin query
remote: add workaround for Cumin bug
To avoid unnecessary waiting on the most common use case of
reboot(), that is with only one host, unset the default
batch_sleepas a workaround for T213296.
puppet: fix regenerate_certificate()
When re-generating the certificate, Puppet will exit with status code
1both if successful or on failure.
Restrict the accepted exit codes to
Detect errors in the output and raises if any.
debmonitor: add debmonitor module
phabricator: add phabricator module
admin_reason: fix default value for task
API breaking changes¶
cookbook: split main into
Add administrative module.
dns: add dns module.
Add elasticsearch_cluster module.
Add Icinga module.
Add ipmi module.
Add Puppet module.
puppet: add additional methods to
puppet: add PuppetMaster class.
remote: add more host functionalities.
doc: add documentation and its generation.
administrative: fix Reason's signature
elasticsearch_cluster: fix tests for Python 3.5.
icinga: fix typo in test docstring.
interactive: check TTY in
mediawiki: kill also HHVM on stop_cronjobs.
Fix typo in README.rst.
tests: fix randomly failing pylint check.
setup.py: update curator version to match our current elasticsearch version.
tests: fix lint ignore.
mediawiki: improve siteinfo checks.
dnsdisc: improve TTL checks.
tests: improve prospector tests.
dnsdisc: catch dnspython exceptions.
setup.py: add missing fields and fix missing comma.
mediawiki: ignore exit codes on stop_cronjobs.
logging: minor improvements and a fix.
log: remove relic from switchdc.
mysql: refactor sync check to avoid GTID.
Add redis_cluster module.
add methods for checking if a datacenter can be depooled.
refactor to use confctl's
set_readonly()and add checks.