mysql_legacy¶

MySQL shell module.

exception spicerack.mysql_legacy.MysqlLegacyError[source]¶

Bases: SpicerackError

Custom exception class for errors of this module.

exception spicerack.mysql_legacy.MysqlLegacyReplagError[source]¶

Bases: MysqlLegacyError

Custom exception class for errors related to replag in this module.

class spicerack.mysql_legacy.InstanceBase(host: spicerack.remote.RemoteHosts) → None[source]¶

Bases: object

Class to manage Single MariaDB Instances.

Initialize the instance.

Parameters:: host (spicerack.remote.RemoteHosts) -- the RemoteHosts instance that contains this MariaDB SingleInstance.

clean_data_dir(no_confirm: bool = False) → None[source]¶

Removes everything contained in the data directory.

Return type:: None

get_repl_info() → tuple[str, str][source]¶

Check replication status to send to the target node.

Return type:: tuple[str, str]

get_replication() → float[source]¶

Retrieves the current replication lag.

Return type:: float

post_asymmetrical_clone_fix(source_instance_data_dir: str, target_instance_data_dir: str) → None[source]¶

Fixes issues after an asymmetrical clone operation.

Return type:: None

post_clone_reset_with_slave_stopped() → None[source]¶

Prepares the MySQL instance for the first pooling operation.

Return type:: None

prep_src_for_cloning() → dict[str, str][source]¶

Helper that prepares source instance to be cloned.

Return type:: dict[str, str]

restart_mysql() → collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]][source]¶

Restarts mariadb service.

Return type:: collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]]

resume_replication() → None[source]¶

Resumes replication on the source MySQL instance.

Return type:: None

run_command(command: str) → collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]][source]¶

Execute the command via Remote.

Parameters:: command (str) -- the mysql query to be executed. Double quotes must be already escaped.
Raises:: spicerack.remote.RemoteExecutionError -- if the Cumin execution returns a non-zero exit code.
Return type:: collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]]

run_query(query: str) → Any[source]¶

Execute the query via Remote.

Parameters:: command -- the mysql query to be executed. Double quotes must be already escaped.
Raises:: spicerack.remote.RemoteExecutionError -- if the Cumin execution returns a non-zero exit code.
Return type:: typing.Any

set_replication_parameters(binlog_file: str, repl_position: str, primary_host: str, user: str, password: str) → None[source]¶

Sets the replication parameters for the MySQL instance.

Return type:: None

show_slave_status() → dict[source]¶

Returns the output of show slave status formatted as a dict.

Return type:: dict

start_mysql(skip_slave_start: bool = True) → collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]][source]¶

Starts mariadb service.

Return type:: collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]]

start_slave() → list[source]¶

Starts mariadb replication.

Return type:: list

status_mysql() → collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]][source]¶

Stops mariadb service.

Return type:: collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]]

stop_mysql() → collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]][source]¶

Stops mariadb service.

Return type:: collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]]

stop_slave() → list[source]¶

Stops mariadb replication.

Return type:: list

upgrade() → collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]][source]¶

Runs the relevant mysql_upgrade command to upgrade the instance content.

Return type:: collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]]

use_gtid() → collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]][source]¶

Runs SQL to use GTID.

Return type:: collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]]

wait_for_replication() → None[source]¶

Waits for replication to catch up.

Return type:: None

property primary: str¶: Retrieves the replication source of this cluster.

class spicerack.mysql_legacy.InstanceMulti(host: spicerack.remote.RemoteHosts, instance_name: str = '') → None[source]¶

Bases: InstanceBase

Class to manage Collocated (aka multiinstances) MariaDB Instances.

Initialize the instance.

Parameters:

host (spicerack.remote.RemoteHosts) -- the RemoteHosts instance that contains this MariaDB MultiInstance.
instance_name (str, default: '') -- The instance name you wish to identify this MultiInstance

class spicerack.mysql_legacy.MysqlLegacy(remote: spicerack.remote.Remote, dry_run: bool = True) → None[source]¶

Bases: object

Class to manage MySQL servers.

Initialize the instance.

Parameters:

remote (spicerack.remote.Remote) -- the Remote instance.
dry_run (bool, default: True) -- whether this is a DRY-RUN.

check_core_masters_heartbeats(datacenter: str, heartbeat_dc: str, heartbeats: dict[str, datetime.datetime]) → None[source]¶

Check the current heartbeat values in the core DB masters in DC are in sync with the provided heartbeats.

Parameters:

datacenter (str) -- the name of the datacenter from where to get the heartbeat values.
heartbeat_dc (str) -- the name of the datacenter for which to filter the heartbeat query.
heartbeats (dict[str, datetime.datetime]) -- a dictionary with the section name str as keys and heartbeat datetime.datetime for each core section as values.

Raises:

spicerack.mysql_legacy.MysqlLegacyError -- on failure to gather the heartbeat or convert it into a datetime.

Return type:

None

check_core_masters_in_sync(dc_from: str, dc_to: str) → None[source]¶

Check that all core masters in dc_to are in sync with the core masters in dc_from.

Parameters:

dc_from (str) -- the name of the datacenter from where to get the master positions.
dc_to (str) -- the name of the datacenter where to check that they are in sync.

Raises:

spicerack.remote.RemoteExecutionError -- on failure.

Return type:

None

get_core_dbs(*, datacenter: str | None = None, section: str | None = None, replication_role: str | None = None, excludes: tuple[str, ...] = ()) → spicerack.mysql_legacy.MysqlLegacyRemoteHosts[source]¶

Get an instance to operated on the core databases matching the parameters.

Parameters:

datacenter (typing.Optional[str], default: None) -- the name of the datacenter to filter for, accepted values are those specified in spicerack.constants.CORE_DATACENTERS.
replication_role (typing.Optional[str], default: None) -- the repication role to filter for, accepted values are those specified in spicerack.mysql_legacy.REPLICATION_ROLES.
section (typing.Optional[str], default: None) -- a specific section to filter for, accepted values are those specified in spicerack.mysql_legacy.CORE_SECTIONS.
excludes (tuple[str, ...], default: ()) -- sections to exclude from getting.

Raises:

spicerack.mysql_legacy.MysqlLegacyError -- on invalid data or unexpected matching hosts.

Return type:

spicerack.mysql_legacy.MysqlLegacyRemoteHosts

get_core_masters_heartbeats(datacenter: str, heartbeat_dc: str) → dict[str, datetime.datetime][source]¶

Get the current heartbeat values from core DB masters in DC for a given heartbeat DC.

Parameters:

datacenter (str) -- the name of the datacenter from where to get the heartbeat values.
heartbeat_dc (str) -- the name of the datacenter for which to filter the heartbeat query.

Return type:

dict[str, datetime.datetime]

Returns:

A dictionary with the section name str as keys and their heartbeat datetime.datetime as values. For example:

{'s1': datetime.datetime(2018, 1, 2, 11, 22, 33, 123456)}

Raises:

spicerack.mysql_legacy.MysqlLegacyError -- on failure to gather the heartbeat or convert it into a datetime.

get_dbs(query: str) → spicerack.mysql_legacy.MysqlLegacyRemoteHosts[source]¶

Get a MysqlLegacyRemoteHosts instance for the matching hosts.

Parameters:: query (str) -- the Remote query to use to fetch the DB hosts.
Return type:: spicerack.mysql_legacy.MysqlLegacyRemoteHosts

set_core_masters_readonly(datacenter: str) → None[source]¶

Set the core masters in read-only.

Parameters:

datacenter (str) -- the name of the datacenter to filter for.

Raises:

spicerack.remote.RemoteExecutionError -- on Remote failures.
spicerack.mysql_legacy.MysqlLegacyError -- on failing to verify the modified value.

Return type:

None

set_core_masters_readwrite(datacenter: str) → None[source]¶

Set the core masters in read-write.

Parameters:

datacenter (str) -- the name of the datacenter to filter for.

Raises:

spicerack.remote.RemoteExecutionError -- on Remote failures.
spicerack.mysql_legacy.MysqlLegacyError -- on failing to verify the modified value.

Return type:

None

verify_core_masters_readonly(datacenter: str, is_read_only: bool) → None[source]¶

Verify that the core masters are in read-only or read-write mode.

Parameters:

datacenter (str) -- the name of the datacenter to filter for.
is_read_only (bool) -- whether the read-only mode should be set or not.

Raises:

spicerack.mysql_legacy.MysqlLegacyError -- on failure.

Return type:

None

heartbeat_query: str = "SELECT ts FROM heartbeat.heartbeat WHERE datacenter = '{dc}' and shard = '{section}' ORDER BY ts DESC LIMIT 1"¶: Query pattern to check the heartbeat for a given datacenter and section.

class spicerack.mysql_legacy.MysqlLegacyRemoteHosts(remote_hosts: spicerack.remote.RemoteHosts) → None[source]¶

Bases: RemoteHostsAdapter

Custom RemoteHosts class for executing MySQL queries.

Initialize the MysqlLegacyRemoteHosts.

Parameters:: remote_hosts (spicerack.remote.RemoteHosts) -- a list of remote hosts on which to operate.
Raises:: spicerack.remote.RemoteError -- if no hosts were provided.

list_host_instance(grouped: bool = False) → list[spicerack.mysql_legacy.InstanceBase][source]¶

List MariaDB instances on the host.

Parameters:

grouped (bool, default: False) -- whether we want to return a "normal" NodeSet which groups everything

Raises:

spicerack.remote.RemoteExecutionError -- if the Cumin execution returns a non-zero exit code.
NotImplementedError -- if the replag is not fully caught on.

Return type:

list[spicerack.mysql_legacy.InstanceBase]

run_query(query: str, database: str = '', success_threshold: float = 1.0, batch_size: int | str | None = None, batch_sleep: float | None = None, is_safe: bool = False) → collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]][source]¶

Execute the query via Remote.

Parameters:

query (str) -- the mysql query to be executed. Double quotes must be already escaped.
database (str, default: '') -- an optional MySQL database to connect to before executing the query.
success_threshold (float, default: 1.0) -- to consider the execution successful, must be between 0.0 and 1.0.
batch_size (typing.Union[int, str, None], default: None) -- the batch size for cumin, either as percentage (e.g. 25%) or absolute number (e.g. 5).
batch_sleep (typing.Optional[float], default: None) -- the batch sleep in seconds to use in Cumin before scheduling the next host.
is_safe (bool, default: False) -- whether the command is safe to run also in dry-run mode because it's a read-only command that doesn't modify the state.

Raises:

spicerack.remote.RemoteExecutionError -- if the Cumin execution returns a non-zero exit code.

Return type:

collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]]

spicerack.mysql_legacy.convert_instancebase_to_instancemulti(instance: spicerack.mysql_legacy.InstanceBase, instance_name: str) → spicerack.mysql_legacy.InstanceMulti[source]¶

Converts InstanceBase to a named InstanceMulti.

Return type:: spicerack.mysql_legacy.InstanceMulti

spicerack.mysql_legacy.convert_instancemulti_to_instancebase(instance: spicerack.mysql_legacy.InstanceMulti) → spicerack.mysql_legacy.InstanceBase[source]¶

Converts InstanceMulti to InstanceBase.

Return type:: spicerack.mysql_legacy.InstanceBase

spicerack.mysql_legacy.CORE_SECTIONS: tuple[str, ...] = ('s1', 's2', 's3', 's4', 's5', 's6', 's7', 's8', 'x1', 'es4', 'es5')¶: Valid MySQL RW section names (external storage RO sections are not included here).

spicerack.mysql_legacy.REPLICATION_ROLES: tuple[str, ...] = ('master', 'slave', 'standalone')¶: Valid replication roles.