elasticsearch_cluster¶

ElasticsearchCluster module.

class spicerack.elasticsearch_cluster.ElasticsearchCluster(elasticsearch, remote, dry_run=True)[source]¶

Bases: object

Class to manage elasticsearch cluster.

Initialize ElasticsearchCluster

Parameters:	elasticsearch (elasticsearch.Elasticsearch) -- elasticsearch instance. remote (spicerack.remote.Remote) -- the Remote instance. dry_run (bool, optional) -- whether this is a DRY-RUN.

Todo

self._hostname class member will be replaced by the formatted message obtained via Reason, this can't be done right now as it needs to be inline with what the MW maint script and the Icinga check do at the moment.

check_green()[source]¶

Cluster health status.

Raises:	`spicerack.elasticsearch_cluster.ElasticsearchClusterCheckError` -- This is raised when request times and cluster is not green.

flush_markers(timeout=datetime.timedelta(0, 60))[source]¶

Flush markers unsynced.

Note

flush and flush_synced are called here because from experience, it results in fewer shards not syncing. This also makes the recovery faster.

Parameters:	timeout (datetime.timedelta) -- timedelta object for elasticsearch request timeout.

force_allocation_of_all_unassigned_shards()[source]¶: Manual allocation of unassigned shards.

frozen_writes(reason)[source]¶

Stop writes to all elasticsearch indices and enable them on exit.

Parameters:	reason (spicerack.administrative.Reason) -- Reason for freezing writes.

get_nodes()[source]¶

Get all Elasticsearch Nodes.

Returns:	dictionary of elasticsearch nodes in the cluster.
Return type:	dict

is_node_in_cluster_nodes(node)[source]¶

Checks if node is in a list of elasticsearch cluster nodes.

Parameters:	node (str) -- the elasticsearch host.
Returns:	`True` if node is present and `False` if not.
Return type:	bool

static split_node_name(node_name)[source]¶

Split node name into hostname and cluster group name.

Parameters:	node_name (str) -- node name containing hostname and cluster name separated by `-`.
Returns:	dictionary containing the node name and the cluster name.
Return type:	dict

stopped_replication()[source]¶: Context manager to perform actions while the cluster replication is stopped.

exception spicerack.elasticsearch_cluster.ElasticsearchClusterCheckError[source]¶

Bases: spicerack.exceptions.SpicerackCheckError

Custom Exception class for check errors of this module.

exception spicerack.elasticsearch_cluster.ElasticsearchClusterError[source]¶

Bases: spicerack.exceptions.SpicerackError

Custom Exception class for errors of this module.

class spicerack.elasticsearch_cluster.ElasticsearchClusters(clusters, remote, dry_run=True)[source]¶

Bases: object

Class to manage elasticsearch clusters.

Initialize ElasticsearchClusters.

Parameters:	clusters (list) -- list of `spicerack.elasticsearch_cluster.ElasticsearchCluster` instances. remote (spicerack.remote.Remote) -- the Remote instance. dry_run (bool, optional) -- whether this is a DRY-RUN.

flush_markers(timeout=datetime.timedelta(0, 60))[source]¶

Flush markers on all clusters.

Parameters:	timeout (datetime.timedelta, optional) -- timedelta object for elasticsearch request timeout.

force_allocation_of_all_unassigned_shards()[source]¶: Force allocation of unassigned shards on all clusters.

frozen_writes(reason)[source]¶

Freeze all writes to the clusters and then perform operations before unfreezing writes.

Parameters:	reason (spicerack.administrative.Reason) -- Reason for freezing writes.

get_next_clusters_nodes(started_before, size=1)[source]¶

Get next set of cluster nodes for cookbook operations like upgrade, rolling restart etc.

Parameters:	started_before (datetime.datetime) -- the time against after which we check if the node has been restarted. size (int, optional) -- size of nodes not restarted in a row.
Returns:	next eligible nodes for ElasticsearchHosts.
Return type:	spicerack.elasticsearch_cluster.ElasticsearchHosts

stopped_replication()[source]¶: Stops replication for all clusters.

wait_for_green(timeout=datetime.timedelta(0, 3600))[source]¶

Wait for green on all clusters.

Parameters:	timeout (datetime.timedelta, optional) -- timedelta object to represent how long to wait for green status on all clusters.

class spicerack.elasticsearch_cluster.ElasticsearchHosts(remote_host, next_nodes, dry_run=True)[source]¶

Bases: spicerack.remote.RemoteHostsAdapter

Remotehosts Adapter for managing elasticsearch nodes.

After calling the super's constructor, initialize other instance variables.

Parameters:	remote_host (spicerack.remote.RemoteHosts) -- the instance with the target hosts. next_nodes (list) -- list of dicts containing clusters hosts belong to. dry_run (bool, optional) -- whether this is a DRY-RUN.

depool_nodes()[source]¶: Depool the hosts.

get_remote_hosts()[source]¶

Returns elasticsearch remote hosts.

Returns:	RemoteHosts instance for this adapter.
Return type:	spicerack.remote.RemoteHosts

pool_nodes()[source]¶: Pool the hosts.

restart_elasticsearch()[source]¶: Restarts all elasticsearch instances.

start_elasticsearch()[source]¶: Starts all elasticsearch instances.

stop_elasticsearch()[source]¶: Stops all elasticsearch instances.

wait_for_elasticsearch_up(timeout=datetime.timedelta(0, 900))[source]¶

Check if elasticsearch instances on each node are up.

Parameters:	timeout (datetime.timedelta, optional) -- represent how long to wait for all instances to be up.

spicerack.elasticsearch_cluster.create_elasticsearch_clusters(clustergroup, remote, dry_run=True)[source]¶

Create ElasticsearchClusters instance.

Parameters:	clustergroup (str) -- name of cluster group. remote (spicerack.remote.Remote) -- the Remote instance. dry_run (bool, optional) -- whether this is a DRY-RUN.
Raises:	`spicerack.elasticsearch_cluster.ElasticsearchClusterError` -- Thrown when the requested cluster configuration is not found.
Returns:	ElasticsearchClusters instance.
Return type:	spicerack.elasticsearch_cluster.ElasticsearchClusters