mysql_legacy

MySQL shell module.

exception spicerack.mysql_legacy.MysqlLegacyError[source]

Bases: SpicerackError

Custom exception class for errors of this module.

exception spicerack.mysql_legacy.MysqlLegacyReplagError[source]

Bases: MysqlLegacyError

Custom exception class for errors related to replag in this module.

class spicerack.mysql_legacy.Instance(host: spicerack.remote.RemoteHosts, *, name: str = '') None[source]

Bases: object

Class to manage MariaDB single intances and multiinstances.

Initialize the instance.

Parameters:
  • host (spicerack.remote.RemoteHosts) -- the RemoteHosts instance that contains this MariaDB SingleInstance.

  • name (str, default: '') -- the name of the instance in a multiinstance context. Leave it empty for single instances.

clean_data_dir(*, skip_confirmation: bool = False) None[source]

Removes everything contained in the data directory.

Parameters:

skip_confirmation (bool, default: False) -- execute the operation without any user confirmation.

Return type:

None

get_replication_info() spicerack.mysql_legacy.ReplicationInfo[source]

Get the replication information suitable to set a new node's replication.

Return type:

spicerack.mysql_legacy.ReplicationInfo

Returns:

The replication information object, useful to setup a new instance's replication to resume from the same position.

post_clone_reset_with_slave_stopped() None[source]

Prepares the MySQL instance for the first pooling operation.

Return type:

None

prep_src_for_cloning() spicerack.mysql_legacy.ReplicationInfo[source]

Helper that prepares source instance to be cloned.

Return type:

spicerack.mysql_legacy.ReplicationInfo

Returns:

The replication information object, useful to setup a new instance's replication to resume from the same position.

replication_lag() float[source]

Retrieves the current replication lag.

Return type:

float

Returns:

The replication lag in seconds.

Raises:

spicerack.mysql_legacy.MysqlLegacyError -- if no lag information is present or unable to parse the it.

restart_mysql() collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]][source]

Restarts mariadb service.

Return type:

collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]]

Returns:

The results of the remote restart command.

resume_replication() None[source]

Resumes replication on the source MySQL instance.

Return type:

None

run_query(query: str, database: str = '', **kwargs: Any) Any[source]

Execute the query via Remote.

Parameters:
  • query (str) -- the mysql query to be executed. Double quotes must be already escaped.

  • database (str, default: '') -- the optional database to use for the query execution.

  • **kwargs (typing.Any) -- any additional argument is passed to spicerack.remote.RemoteHosts.run_sync(). By default the print_progress_bars and print_output arguments are set to False.

Return type:

typing.Any

Returns:

The result of the remote command execution.

Raises:

spicerack.remote.RemoteExecutionError -- if the query execution via SSH returns a non-zero exit code.

set_replication_parameters(*, replication_info: spicerack.mysql_legacy.ReplicationInfo, user: str, password: str) None[source]

Sets the replication parameters for the MySQL instance.

Return type:

None

show_slave_status() dict[source]

Returns the output of show slave status formatted as a dict.

Return type:

dict

Returns:

the current slave status for the instance.

start_mysql() collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]][source]

Starts mariadb service.

Return type:

collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]]

Returns:

The results of the remote start command.

start_slave() None[source]

Starts mariadb replication.

Return type:

None

status_mysql() collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]][source]

Stops mariadb service.

Return type:

collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]]

Returns:

The results of the remote status command.

stop_mysql() collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]][source]

Stops mariadb service.

Return type:

collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]]

Returns:

The results of the remote status command.

stop_slave() None[source]

Stops mariadb replication.

Return type:

None

upgrade() collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]][source]

Runs the relevant mysql_upgrade command to upgrade the instance content.

Return type:

collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]]

Returns:

The results of the remote upgrade command.

use_gtid() None[source]

Runs SQL to use GTID.

Return type:

None

wait_for_replication(threshold: float = 1.0) None[source]

Waits for replication to catch up.

Parameters:

threshold (float, default: 1.0) -- the replication lag threshold in seconds under which the replication is considered in sync.

Raises:

spicerack.mysql_legacy.MysqlLegacyReplagError -- if the replication lag is still too high after all the retries.

Return type:

None

property primary: str

Retrieves the replication source of this cluster.

Raises:

spicerack.mysql_legacy.MysqlLegacyError -- if unable to find the master host of the current instance.

class spicerack.mysql_legacy.MysqlLegacy(remote: spicerack.remote.Remote, dry_run: bool = True) None[source]

Bases: object

Class to manage MySQL servers.

Initialize the instance.

Parameters:
check_core_masters_heartbeats(datacenter: str, heartbeat_dc: str, heartbeats: dict[str, datetime.datetime]) None[source]

Check the current heartbeat values in the core DB masters in DC are in sync with the provided heartbeats.

Parameters:
  • datacenter (str) -- the name of the datacenter from where to get the heartbeat values.

  • heartbeat_dc (str) -- the name of the datacenter for which to filter the heartbeat query.

  • heartbeats (dict[str, datetime.datetime]) -- a dictionary with the section name str as keys and heartbeat datetime.datetime for each core section as values.

Raises:

spicerack.mysql_legacy.MysqlLegacyError -- on failure to gather the heartbeat or convert it into a datetime.

Return type:

None

check_core_masters_in_sync(dc_from: str, dc_to: str) None[source]

Check that all core masters in dc_to are in sync with the core masters in dc_from.

Parameters:
  • dc_from (str) -- the name of the datacenter from where to get the master positions.

  • dc_to (str) -- the name of the datacenter where to check that they are in sync.

Raises:

spicerack.remote.RemoteExecutionError -- on failure.

Return type:

None

get_core_dbs(*, datacenter: str | None = None, section: str | None = None, replication_role: str | None = None, excludes: tuple[str, ...] = ()) spicerack.mysql_legacy.MysqlLegacyRemoteHosts[source]

Get an instance to operated on the core databases matching the parameters.

Parameters:
Raises:

spicerack.mysql_legacy.MysqlLegacyError -- on invalid data or unexpected matching hosts.

Return type:

spicerack.mysql_legacy.MysqlLegacyRemoteHosts

get_core_masters_heartbeats(datacenter: str, heartbeat_dc: str) dict[str, datetime.datetime][source]

Get the current heartbeat values from core DB masters in DC for a given heartbeat DC.

Parameters:
  • datacenter (str) -- the name of the datacenter from where to get the heartbeat values.

  • heartbeat_dc (str) -- the name of the datacenter for which to filter the heartbeat query.

Return type:

dict[str, datetime.datetime]

Returns:

A dictionary with the section name str as keys and their heartbeat datetime.datetime as values. For example:

{'s1': datetime.datetime(2018, 1, 2, 11, 22, 33, 123456)}

Raises:

spicerack.mysql_legacy.MysqlLegacyError -- on failure to gather the heartbeat or convert it into a datetime.

get_dbs(query: str) spicerack.mysql_legacy.MysqlLegacyRemoteHosts[source]

Get a MysqlLegacyRemoteHosts instance for the matching hosts.

Parameters:

query (str) -- the Remote query to use to fetch the DB hosts.

Return type:

spicerack.mysql_legacy.MysqlLegacyRemoteHosts

set_core_masters_readonly(datacenter: str) None[source]

Set the core masters in read-only.

Parameters:

datacenter (str) -- the name of the datacenter to filter for.

Raises:
Return type:

None

set_core_masters_readwrite(datacenter: str) None[source]

Set the core masters in read-write.

Parameters:

datacenter (str) -- the name of the datacenter to filter for.

Raises:
Return type:

None

verify_core_masters_readonly(datacenter: str, is_read_only: bool) None[source]

Verify that the core masters are in read-only or read-write mode.

Parameters:
  • datacenter (str) -- the name of the datacenter to filter for.

  • is_read_only (bool) -- whether the read-only mode should be set or not.

Raises:

spicerack.mysql_legacy.MysqlLegacyError -- on failure.

Return type:

None

heartbeat_query: str = "SELECT ts FROM heartbeat.heartbeat WHERE datacenter = '{dc}' and shard = '{section}' ORDER BY ts DESC LIMIT 1"

Query pattern to check the heartbeat for a given datacenter and section.

class spicerack.mysql_legacy.MysqlLegacyRemoteHosts(remote_hosts: spicerack.remote.RemoteHosts) None[source]

Bases: RemoteHostsAdapter

Custom RemoteHosts class for executing MySQL queries.

Initialize the instance.

Parameters:

remote_hosts (spicerack.remote.RemoteHosts) -- the instance to act on the remote hosts.

list_hosts_instances(*, group: bool = False) list[spicerack.mysql_legacy.Instance][source]

List MariaDB instances on the host.

Parameters:

group (bool, default: False) -- not yet implemented feature to allow parallelization.

Raises:
Return type:

list[spicerack.mysql_legacy.Instance]

run_query(query: str, database: str = '', **kwargs: Any) collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]][source]

Execute the query via Remote.

Parameters:
  • query (str) -- the mysql query to be executed. Double quotes must be already escaped.

  • database (str, default: '') -- an optional MySQL database to connect to before executing the query.

  • **kwargs (typing.Any) -- any additional argument is passed to spicerack.remote.RemoteHosts.run_sync(). By default the print_progress_bars and print_output arguments are set to False.

Raises:

spicerack.remote.RemoteExecutionError -- if the Cumin execution returns a non-zero exit code.

Return type:

collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]]

class spicerack.mysql_legacy.ReplicationInfo(primary: str, binlog: str, position: int, port: int) None[source]

Bases: object

Represent the minimum replication information needed to restore a replication from a given point.

Parameters:
  • primary (str) -- the FQDN of the primary host from where to replicate from.

  • binlog (str) -- the binlog file to replicate from.

  • position (int) -- the binlog position to replicate from.

  • port (int) -- the port of the master from where to replicate from.

spicerack.mysql_legacy.CORE_SECTIONS: tuple[str, ...] = ('s1', 's2', 's3', 's4', 's5', 's6', 's7', 's8', 'x1', 'es6', 'es7')

Valid MySQL RW core sections (external storage RO, parser cache, x2 and misc sections are not included here).

spicerack.mysql_legacy.REPLICATION_ROLES: tuple[str, ...] = ('master', 'slave', 'standalone')

Valid replication roles.