Python API¶
Spicerack Python API autodoc.
Spicerack package.
- class spicerack.Spicerack(*, verbose: bool = False, dry_run: bool = True, cumin_config: str = '/etc/cumin/config.yaml', cumin_installer_config: str = '/etc/cumin/config-installer.yaml', conftool_config: str = '/etc/conftool/config.yaml', conftool_schema: str = '/etc/conftool/schema.yaml', debmonitor_config: str = '/etc/debmonitor.conf', etcd_config: str = '', spicerack_config_dir: str = '/etc/spicerack', http_proxy: str = '', get_cookbook_callback: collections.abc.Callable[[Spicerack, str, collections.abc.Sequence[str]], BaseItem | None] | None = None, extender_class: type[SpicerackExtenderBase] | None = None) None [source]¶
Bases:
object
Spicerack service locator.
Initialize the service locator for the Spicerack library.
- Parameters:
verbose (
bool
, default:False
) -- whether to set the verbose mode.dry_run (
bool
, default:True
) -- whether this is a DRY-RUN.cumin_config (
str
, default:'/etc/cumin/config.yaml'
) -- the path to Cumin's configuration file.cumin_installer_config (
str
, default:'/etc/cumin/config-installer.yaml'
) -- the path to Cumin's configuration file for the Debian installer.conftool_config (
str
, default:'/etc/conftool/config.yaml'
) -- the path to Conftool's configuration file.conftool_schema (
str
, default:'/etc/conftool/schema.yaml'
) -- the path to Conftool's schema file.debmonitor_config (
str
, default:'/etc/debmonitor.conf'
) --the path to Debmonitor's INI configuration file. It must have at least the following schema:
[DEFAULT] server=debmonitor.example.com cert=/etc/debmonitor/ssl/cert.pem key=/etc/debmonitor/ssl/server.key
etcd_config (
str
, default:''
) -- the path to the Etcd configuration file. Set to empty string to disable the locking support.spicerack_config_dir (
str
, default:'/etc/spicerack'
) -- the path for the root configuration directory for Spicerack. Module-specific configuration will be loaded fromconfig_dir/module_name/
.http_proxy (
str
, default:''
) -- thescheme://url:port
of the HTTP proxy to use for external calls.get_cookbook_callback (
typing.Optional
[collections.abc.Callable
[[Spicerack,str
,collections.abc.Sequence
[str
]],typing.Optional
[spicerack._menu.BaseItem
]]], default:None
) -- a callable to retrieve a CookbookItem to execute a cookbook from inside another cookbook.extender_class (
typing.Optional
[type
[SpicerackExtenderBase]], default:None
) -- an optional class object that inherits fromspicerack.SpicerackExtenderBase
to dynamically add accessors to Spicerack. If not set no extenders will be registered, even ifexternal_modules_dir
is specified in the configuration.
- admin_reason(reason: str, task_id: str | None = None) spicerack.administrative.Reason [source]¶
Get an administrative Reason instance.
- Parameters:
reason (
str
) -- the reason to use to justify an administrative action. Seespicerack.administrative.Reason
for all the details.task_id (
typing.Optional
[str
], default:None
) -- the task ID to mention in the reason.
- Return type:
- alerting_hosts(target_hosts: spicerack.typing.TypeHosts, *, verbatim_hosts: bool = False) spicerack.alerting.AlertingHosts [source]¶
Get an AlertingHosts instance.
- Parameters:
target_hosts (
typing.TypeVar
(TypeHosts
,collections.abc.Sequence
[str
],ClusterShell.NodeSet.NodeSet
)) -- the target hosts either as a NodeSet instance or a sequence of strings.verbatim_hosts (
bool
, default:False
) -- ifTrue
use the hosts passed verbatim as is, if insteadFalse
, the default, consider the given target hosts as FQDNs and extract their hostnames to be used in Icinga.
- Return type:
- alertmanager(instance_name: str = '') spicerack.alertmanager.Alertmanager [source]¶
Get an Alertmanager instance.
- Parameters:
instance_name (
str
, default:''
) -- the Alertmanager instance defined in the alertmanager/config.yaml config file to interact with- Return type:
Note
To interact with Alertmanager alerts attached to an
instance
usespicerack.Spicerack.alertmanager_hosts()
instead.
- alertmanager_hosts(target_hosts: spicerack.typing.TypeHosts, *, instance_name: str = '', verbatim_hosts: bool = False) spicerack.alertmanager.AlertmanagerHosts [source]¶
Get an AlertmanagerHosts instance.
Note
To interact with both Icinga and Alertmanager alerts, use
spicerack.Spicerack.alerting_hosts()
instead.- Parameters:
target_hosts (
typing.TypeVar
(TypeHosts
,collections.abc.Sequence
[str
],ClusterShell.NodeSet.NodeSet
)) -- the target hosts either as a NodeSet instance or a sequence of strings.instance_name (
str
, default:''
) -- the Alertmanager instance defined in the alertmanager/config.yaml config file to interact withverbatim_hosts (
bool
, default:False
) -- ifTrue
use the hosts passed verbatim as is, if insteadFalse
, the default, consider the given target hosts as FQDNs and extract their hostnames to be used in Icinga.
- Return type:
- apt_get(remote_hosts: spicerack.remote.RemoteHosts) spicerack.apt.AptGetHosts [source]¶
Get an APTGet instance for the given remote hosts.
Examples
>>> hosts = spicerack.remote().query('A:myalias') >>> apt_get = spicerack.apt_get(hosts)
- Parameters:
remote_hosts (
spicerack.remote.RemoteHosts
) -- the instance with the target hosts.- Return type:
- confctl(entity_name: str) spicerack.confctl.ConftoolEntity [source]¶
Get a Conftool specific entity instance.
- Parameters:
entity_name (
str
) -- the name of a Conftool entity. Possible values:node
,service
,discovery
,mwconfig
.- Return type:
- debmonitor() spicerack.debmonitor.Debmonitor [source]¶
Get a Debmonitor instance to interact with a Debmonitor website.
- Raises:
KeyError -- if any configuration option is missing.
- Return type:
- dhcp(datacenter: str) spicerack.dhcp.DHCP [source]¶
Return a DHCP configuration manager for the specified datacenter.
- Parameters:
datacenter (
str
) -- the datacenter for which the DHCP servers will be targeted.- Return type:
- discovery(*records: str) spicerack.dnsdisc.Discovery [source]¶
Get a Discovery instance for the given records.
- Parameters:
*records (
str
) -- arbitrary positional arguments, each one must be a Discovery DNS record name.- Return type:
- dns() wmflib.dns.Dns [source]¶
Get a Dns instance that will use the operating system default nameserver(s).
- Return type:
- elasticsearch_clusters(clustergroup: str, write_queue_datacenters: collections.abc.Sequence[str]) spicerack.elasticsearch_cluster.ElasticsearchClusters [source]¶
Get an ElasticsearchClusters instance.
- Parameters:
clustergroup (
str
) -- name of cluster group e.gsearch_eqiad
.write_queue_datacenters (
collections.abc.Sequence
[str
]) -- Sequence of which core DCs to query write queues for.
- Return type:
- etcdctl(*, remote_host: spicerack.remote.RemoteHosts) spicerack.toolforge.etcdctl.EtcdctlController [source]¶
Add etcdctl control capabilities to the given RemoteHost.
- Params:
- remote_host: Single remote host (that should map one single host)
to add capabilities to.
- Return type:
- Returns:
A wrapped RemoteHost with the etcdctl control related methods.
- ganeti() spicerack.ganeti.Ganeti [source]¶
Get an instance to interact with Ganeti.
- Raises:
KeyError -- If the configuration file does not contain the correct keys.
- Return type:
- icinga_hosts(target_hosts: spicerack.typing.TypeHosts, *, verbatim_hosts: bool = False) spicerack.icinga.IcingaHosts [source]¶
Get an IcingaHosts instance.
Note
To interact with both Icinga and Alertmanager alerts, use
spicerack.Spicerack.alerting_hosts()
instead.- Parameters:
target_hosts (
typing.TypeVar
(TypeHosts
,collections.abc.Sequence
[str
],ClusterShell.NodeSet.NodeSet
)) -- the target hosts either as a NodeSet instance or a sequence of strings.verbatim_hosts (
bool
, default:False
) -- ifTrue
use the hosts passed verbatim as is, if insteadFalse
, the default, consider the given target hosts as FQDNs and extract their hostnames to be used in Icinga.
- Return type:
- ipmi(target: str) spicerack.ipmi.Ipmi [source]¶
Get an Ipmi instance to send remote IPMI commands to management consoles.
- Parameters:
target (
str
) -- the management console FQDN or IP to target.- Return type:
- kafka() spicerack.kafka.Kafka [source]¶
Get an instance to interact with Kafka.
- Raises:
KeyError -- If the configuration file does not contain the correct keys.
- Return type:
- kubernetes(group: str, cluster: str) spicerack.k8s.Kubernetes [source]¶
Get a kubernetes client for the specified cluster.
- Parameters:
- Return type:
- lock() spicerack.locking.Lock | spicerack.locking.NoLock [source]¶
Get a Lock instance to acquire custom locks with concurrency and TTL around specific lines of code.
In case the locking support is disabled, a
spicerack.locking.NoLock
instance is returned, with the same API of thespicerack.locking.Lock
class but that does nothing.- Return type:
typing.Union
[spicerack.locking.Lock
,spicerack.locking.NoLock
]- Returns:
The lock instance already initialized with a dedicated prefix for the keys.
- mediawiki() spicerack.mediawiki.MediaWiki [source]¶
Get a MediaWiki instance.
- Return type:
- mysql() spicerack.mysql.Mysql [source]¶
Get a Mysql instance.
- Return type:
- mysql_legacy() spicerack.mysql_legacy.MysqlLegacy [source]¶
Get a MysqlLegacy instance.
- Return type:
- netbox(*, read_write: bool = False) spicerack.netbox.Netbox [source]¶
Get a Netbox instance to interact with Netbox's API.
- Parameters:
read_write (
bool
, default:False
) -- whether to use a read-write token.- Return type:
- netbox_server(hostname: str, *, read_write: bool = False) spicerack.netbox.NetboxServer [source]¶
Get a NetboxServer instance to interact with a server in Netbox, both physical and virtual.
- Parameters:
- Raises:
spicerack.netbox.NetboxError -- if unable to find or load the server.
- Return type:
- peeringdb(*, ttl: int = 86400) spicerack.peeringdb.PeeringDB [source]¶
Get a PeeringDB instance to interact with the PeeringDB API.
- Parameters:
ttl (
int
, default:86400
) -- the cache timeout in seconds. If cached items are older than the given TTL they will be ignored and fetched again.- Return type:
- phabricator(bot_config_file: str, section: str = 'phabricator_bot') wmflib.phabricator.Phabricator [source]¶
Get a Phabricator instance to interact with a Phabricator website.
- Parameters:
bot_config_file (
str
) --the path to the configuration file for the Phabricator bot, with the following structure:
[section_name] host = https://phabricator.example.com/api/ username = phab-bot token = api-12345
section (
str
, default:'phabricator_bot'
) -- the name of the section of the configuration file where to find the required parameters.
- Return type:
- prometheus() wmflib.prometheus.Prometheus [source]¶
Get a Prometheus instance.
- Return type:
- puppet(remote_hosts: spicerack.remote.RemoteHosts) spicerack.puppet.PuppetHosts [source]¶
Get a PuppetHosts instance for the given remote hosts.
- Parameters:
remote_hosts (
spicerack.remote.RemoteHosts
) -- the instance with the target hosts.- Return type:
- puppet_master() spicerack.puppet.PuppetMaster [source]¶
Get a PuppetMaster instance to manage hosts and certificates from a Puppet master.
- Return type:
- puppet_server() spicerack.puppet.PuppetServer [source]¶
Get a PuppetServer instance to manage hosts and certificates from a Puppet master.
- Return type:
- redfish(hostname: str, username: str = 'root', password: str = '') spicerack.redfish.Redfish [source]¶
Get an instance to talk to the Redfish API of a physical server.
Notes
At the moment only Dell hardware is supported.
- Parameters:
hostname (
str
) -- the hostname (not FQDN) of the physical server to manage.username (
str
, default:'root'
) -- the username for the management console.password (
str
, default:''
) -- the password for the management console for the given user. If empty or not provided would use the production management password and ask the user for it if not already in memory.
- Raises:
spicerack.exceptions.SpicerackError -- if not a physical server or unable to find the management IP.
- Return type:
- redis_cluster(cluster: str) spicerack.redis_cluster.RedisCluster [source]¶
Get a RedisCluster instance.
- Parameters:
cluster (
str
) -- the name of the cluster.- Return type:
- remote(installer: bool = False) spicerack.remote.Remote [source]¶
Get a Remote instance.
- Parameters:
installer (
bool
, default:False
) -- whether to use the special configuration to connect to a Debian installer or freshly re-imaged host prior to its first Puppet run.- Return type:
- reposync(name: str) spicerack.reposync.RepoSync [source]¶
Get a Reposync instance.
- Parameters:
name (
str
) -- the name of the repo to sync.- Return type:
- requests_session(name: str, **kwargs: Any) requests.sessions.Session [source]¶
Return a new requests Session with timeout and retry logic.
- Return type:
- Params:
according to
wmflib.requests.http_session()
.
- run_cookbook(cookbook: str, args: collections.abc.Sequence[str] = ()) int [source]¶
Run another Cookbook within the current run.
The other Cookbook will be executed with the current setup and will log in the same file of the current Cookbook that is running.
- Parameters:
cookbook (
str
) -- the path to the cookbook to execute, either in Spicerack's dot notation or the relative path to the Python file to execute.args (
collections.abc.Sequence
[str
], default:()
) -- an iterable sequence of strings with the Cookbook's argument. The Cookbook will be executed with the same global arguments used for the current run.
- Return type:
- Returns:
The exit code of the Cookbook, 0 if successful, non-zero if not.
- Raises:
spicerack.exceptions.SpicerackError -- if the
get_cookbook_callback
callback is not set or unable to find the cookbook with the given name.
- service_catalog() spicerack.service.Catalog [source]¶
Get a Catalog instance that reflects Puppet's service::catalog hieradata variable.
- Return type:
- thanos() wmflib.prometheus.Thanos [source]¶
Get a Thanos instance.
- Return type:
- property actions: ActionsDict¶
Returns a dictionary to log and record cookbook actions.
- property authdns_active_hosts: RemoteHosts¶
Get a RemoteHosts instance to target the active authoritative nameservers.
Examples
>>> authdns_hosts = spicerack.authdns_active_hosts >>> authdns_hosts.run_sync('some command')
- property authdns_servers: dict[str, str]¶
Getter for the authoritative DNS nameservers currently active in production.
Returns a dictionary where keys are the hostnames and values are the IPs of the active authoritative nameservers.
- property config_dir: Path¶
Getter for Spicerack's configuration file directory where the module-specific configs are available.
They can be accessed with:
self.config_dir / module_name / config_file
- property icinga_master_host: RemoteHosts¶
Returns the instance to execute commands on the Icinga master host.
- property irc_logger: Logger¶
Returns the logger instance to write to IRC in the #wikimedia-operations channel.
- property management_password: str¶
Returns the management password.
It will be asked to the user if not already cached by the current instance.
- property netbox_master_host: RemoteHosts¶
Returns the instance to execute commands on the Netbox master host.
- property requests_proxies: dict[str, str] | None¶
Returns the HTTP proxy configuration for the Requests module or
None
if no HTTP proxy is set.
- class spicerack.SpicerackExtenderBase(*, spicerack: spicerack.Spicerack)[source]¶
Bases:
object
Base class to create a Spicerack extender. Necessary when the
external_modules_dir
configuration is set.Initialize the instance.
- Parameters:
spicerack (
spicerack.Spicerack
) -- the Spicerack instance.
Subpackages and Submodules
- administrative
- alerting
- alertmanager
- apt
- confctl
- constants
- cookbook
- debmonitor
- decorators
- dhcp
- dnsdisc
- elasticsearch_cluster
ElasticsearchClusterCheckError
ElasticsearchClusterError
ElasticsearchCluster
ElasticsearchCluster.check_green()
ElasticsearchCluster.check_yellow_w_no_moving_shards()
ElasticsearchCluster.flush_markers()
ElasticsearchCluster.force_allocation_of_all_unassigned_shards()
ElasticsearchCluster.frozen_writes()
ElasticsearchCluster.get_nodes()
ElasticsearchCluster.is_node_in_cluster_nodes()
ElasticsearchCluster.reset_indices_to_read_write()
ElasticsearchCluster.stopped_replication()
ElasticsearchClusters
ElasticsearchClusters.flush_markers()
ElasticsearchClusters.force_allocation_of_all_unassigned_shards()
ElasticsearchClusters.frozen_writes()
ElasticsearchClusters.get_next_clusters_nodes()
ElasticsearchClusters.reset_indices_to_read_write()
ElasticsearchClusters.stopped_replication()
ElasticsearchClusters.wait_for_all_write_queues_empty()
ElasticsearchClusters.wait_for_green()
ElasticsearchClusters.wait_for_yellow_w_no_moving_shards()
ElasticsearchHosts
NodesGroup
create_elasticsearch_clusters()
- exceptions
- ganeti
- icinga
IcingaCheckError
IcingaError
IcingaStatusNotFoundError
IcingaStatusParseError
CommandFile
HostStatus
HostsStatus
IcingaHosts
IcingaHosts.downtime()
IcingaHosts.downtime_services()
IcingaHosts.downtimed()
IcingaHosts.get_status()
IcingaHosts.recheck_all_services()
IcingaHosts.recheck_failed_services()
IcingaHosts.remove_downtime()
IcingaHosts.remove_service_downtimes()
IcingaHosts.run_icinga_command()
IcingaHosts.services_downtimed()
IcingaHosts.wait_for_downtimed()
IcingaHosts.wait_for_optimal()
IcingaStatus
ServiceStatus
ICINGA_DOMAIN
MIN_DOWNTIME_SECONDS
- interactive
- ipmi
- k8s
- kafka
- locking
- mediawiki
MediaWikiCheckError
MediaWikiError
MediaWiki
MediaWiki.check_config_line()
MediaWiki.check_periodic_jobs_disabled()
MediaWiki.check_periodic_jobs_enabled()
MediaWiki.check_siteinfo()
MediaWiki.get_maintenance_host()
MediaWiki.get_master_datacenter()
MediaWiki.get_siteinfo()
MediaWiki.scap_sync_config_file()
MediaWiki.set_master_datacenter()
MediaWiki.set_readonly()
MediaWiki.set_readwrite()
MediaWiki.stop_periodic_jobs()
- mysql
- mysql_legacy
MysqlLegacyError
MysqlLegacyReplagError
Instance
Instance.clean_data_dir()
Instance.get_replication_info()
Instance.post_clone_reset_with_slave_stopped()
Instance.prep_src_for_cloning()
Instance.replication_lag()
Instance.restart_mysql()
Instance.resume_replication()
Instance.run_query()
Instance.set_replication_parameters()
Instance.show_slave_status()
Instance.start_mysql()
Instance.start_slave()
Instance.status_mysql()
Instance.stop_mysql()
Instance.stop_slave()
Instance.upgrade()
Instance.use_gtid()
Instance.wait_for_replication()
Instance.primary
MysqlLegacy
MysqlLegacy.check_core_masters_heartbeats()
MysqlLegacy.check_core_masters_in_sync()
MysqlLegacy.get_core_dbs()
MysqlLegacy.get_core_masters_heartbeats()
MysqlLegacy.get_dbs()
MysqlLegacy.set_core_masters_readonly()
MysqlLegacy.set_core_masters_readwrite()
MysqlLegacy.verify_core_masters_readonly()
MysqlLegacy.heartbeat_query
MysqlLegacyRemoteHosts
ReplicationInfo
CORE_SECTIONS
REPLICATION_ROLES
- netbox
- peeringdb
- puppet
- redfish
RedfishError
RedfishTaskNotCompletedError
ChassisResetPolicy
DellSCP
DellSCPPowerStatePolicy
DellSCPRebootPolicy
DellSCPTargetPolicy
Redfish
Redfish.change_user_password()
Redfish.chassis_reset()
Redfish.check_connection()
Redfish.find_account()
Redfish.get_power_state()
Redfish.last_reboot()
Redfish.most_recent_member()
Redfish.multipush_upload()
Redfish.poll_task()
Redfish.request()
Redfish.submit_files()
Redfish.submit_task()
Redfish.upload_file()
Redfish.wait_reboot_since()
Redfish.bios_version
Redfish.firmware_version
Redfish.hostname
Redfish.interface
Redfish.log_entries
Redfish.log_service
Redfish.manager
Redfish.manufacturer
Redfish.model
Redfish.multipushuri
Redfish.oob_info
Redfish.oob_manager
Redfish.oob_model
Redfish.pushuri
Redfish.reboot_message_id
Redfish.storage_manager
Redfish.system
Redfish.system_info
Redfish.system_manager
Redfish.update_service
Redfish.updateservice_info
RedfishDell
RedfishSupermicro
- redis_cluster
- remote
- reposync
- service
- toolforge
- toolforge.etcdctl
- typing