elasticsearch_cluster¶

ElasticsearchCluster module.

class spicerack.elasticsearch_cluster.ElasticsearchCluster(elasticsearch, remote, dry_run=True)[source]¶

Bases: object

Class to manage elasticsearch cluster.

Initialize ElasticsearchCluster

Parameters:	elasticsearch (elasticsearch.Elasticsearch) -- elasticsearch instance. remote (spicerack.remote.Remote) -- the Remote instance. dry_run (bool, optional) -- whether this is a DRY-RUN.

check_green()[source]¶

Cluster health status.

Raises:	`spicerack.elasticsearch_cluster.ElasticsearchClusterCheckError` -- This is raised when request times and cluster is not green.

flush_markers(timeout=datetime.timedelta(0, 60))[source]¶

Flush markers unsynced.

Note

flush and flush_synced are called here because from experience, it results in fewer shards not syncing. This also makes the recovery faster.

Parameters:	timeout (datetime.timedelta) -- timedelta object for elasticsearch request timeout.

force_allocation_of_all_unassigned_shards()[source]¶: Manual allocation of unassigned shards.

frozen_writes(reason)[source]¶

Stop writes to all elasticsearch indices and enable them on exit.

Parameters:	reason (spicerack.administrative.Reason) -- Reason for freezing writes.

get_nodes()[source]¶

Get all Elasticsearch Nodes.

Returns:	dictionary of elasticsearch nodes in the cluster.
Return type:	dict

is_node_in_cluster_nodes(node)[source]¶

Checks if node is in a list of elasticsearch cluster nodes.

Parameters:	node (str) -- the elasticsearch host.
Returns:	`True` if node is present and `False` if not.
Return type:	bool

static split_node_name(node_name)[source]¶

Split node name into hostname and cluster group name.

Parameters:	node_name (str) -- node name containing hostname and cluster name separated by `-`.
Returns:	tuple containing the node name and the cluster name.
Return type:	(str, str)

stopped_replication()[source]¶: Context manager to perform actions while the cluster replication is stopped.

exception spicerack.elasticsearch_cluster.ElasticsearchClusterCheckError[source]¶

Bases: spicerack.exceptions.SpicerackCheckError

Custom Exception class for check errors of this module.

exception spicerack.elasticsearch_cluster.ElasticsearchClusterError[source]¶

Bases: spicerack.exceptions.SpicerackError

Custom Exception class for errors of this module.

class spicerack.elasticsearch_cluster.ElasticsearchClusters(clusters, remote, dry_run=True)[source]¶

Bases: object

Class to manage elasticsearch clusters.

Initialize ElasticsearchClusters.

Parameters:	clusters (list) -- list of `spicerack.elasticsearch_cluster.ElasticsearchCluster` instances. remote (spicerack.remote.Remote) -- the Remote instance. dry_run (bool, optional) -- whether this is a DRY-RUN.

flush_markers(timeout=datetime.timedelta(0, 60))[source]¶

Flush markers on all clusters.

Parameters:	timeout (datetime.timedelta, optional) -- timedelta object for elasticsearch request timeout.

force_allocation_of_all_unassigned_shards()[source]¶: Force allocation of unassigned shards on all clusters.

frozen_writes(reason)[source]¶

Freeze all writes to the clusters and then perform operations before unfreezing writes.

Parameters:	reason (spicerack.administrative.Reason) -- Reason for freezing writes.

get_next_clusters_nodes(started_before, size=1)[source]¶

Get next set of cluster nodes for cookbook operations like upgrade, rolling restart etc.

Nodes are selected from the row with the least restarted nodes. This ensure that a row is fully upgraded before moving to the next row. Since shards cannot move to a node with an older version of elasticsearch, this should help to keep all shards allocated at all time.

Parameters:	started_before (datetime.datetime) -- the time against after which we check if the node has been restarted. size (int, optional) -- size of nodes not restarted in a row.
Returns:	next eligible nodes for ElasticsearchHosts.
Return type:	spicerack.elasticsearch_cluster.ElasticsearchHosts

stopped_replication()[source]¶: Stops replication for all clusters.

wait_for_green(timeout=datetime.timedelta(0, 3600))[source]¶

Wait for green on all clusters.

Parameters:	timeout (datetime.timedelta, optional) -- timedelta object to represent how long to wait for green status on all clusters.

class spicerack.elasticsearch_cluster.ElasticsearchHosts(remote_hosts, nodes, dry_run=True)[source]¶

Bases: spicerack.remote.RemoteHostsAdapter

Remotehosts Adapter for managing elasticsearch nodes.

After calling the super's constructor, initialize other instance variables.

Parameters:	remote_hosts (spicerack.remote.RemoteHosts) -- the instance with the target hosts. nodes (list) -- list of dicts containing clusters hosts belong to. dry_run (bool, optional) -- whether this is a DRY-RUN.

depool_nodes()[source]¶: Depool the hosts.

get_remote_hosts()[source]¶

Returns elasticsearch remote hosts.

Returns:	RemoteHosts instance for this adapter.
Return type:	spicerack.remote.RemoteHosts

pool_nodes()[source]¶: Pool the hosts.

restart_elasticsearch()[source]¶: Restarts all elasticsearch instances.

start_elasticsearch()[source]¶: Starts all elasticsearch instances.

stop_elasticsearch()[source]¶: Stops all elasticsearch instances.

wait_for_elasticsearch_up(timeout=datetime.timedelta(0, 900))[source]¶

Check if elasticsearch instances on each node are up.

Parameters:	timeout (datetime.timedelta, optional) -- represent how long to wait for all instances to be up.

spicerack.elasticsearch_cluster.create_elasticsearch_clusters(clustergroup, remote, dry_run=True)[source]¶

Create ElasticsearchClusters instance.

Parameters:	clustergroup (str) -- name of cluster group. remote (spicerack.remote.Remote) -- the Remote instance. dry_run (bool, optional) -- whether this is a DRY-RUN.
Raises:	`spicerack.elasticsearch_cluster.ElasticsearchClusterError` -- Thrown when the requested cluster configuration is not found.
Returns:	ElasticsearchClusters instance.
Return type:	spicerack.elasticsearch_cluster.ElasticsearchClusters