mysql_legacy

MySQL module (legacy).

Todo

replace with a proper MySQL module that uses a Python MySQL client, preferably in a parallel way.

exception spicerack.mysql_legacy.MysqlLegacyError[source]

Bases: SpicerackError

Custom exception class for errors of this module.

class spicerack.mysql_legacy.MysqlLegacy(remote: spicerack.remote.Remote, dry_run: bool = True) → None[source]

Bases: object

Class to manage MySQL servers.

Initialize the instance.

Parameters:
check_core_masters_heartbeats(datacenter: str, heartbeat_dc: str, heartbeats: dict[str, datetime.datetime]) → None[source]

Check the current heartbeat values in the core DB masters in DC are in sync with the provided heartbeats.

Parameters:

  • datacenter (str) -- the name of the datacenter from where to get the heartbeat values.

  • heartbeat_dc (str) -- the name of the datacenter for which to filter the heartbeat query.

  • heartbeats (dict[str, datetime.datetime]) -- a dictionary with the section name str as keys and heartbeat datetime.datetime for each core section as values.

Raises:

spicerack.mysql_legacy.MysqlLegacyError -- on failure to gather the heartbeat or convert it into a datetime.

Return type:

None

check_core_masters_in_sync(dc_from: str, dc_to: str) → None[source]

Check that all core masters in dc_to are in sync with the core masters in dc_from.

Parameters:

  • dc_from (str) -- the name of the datacenter from where to get the master positions.

  • dc_to (str) -- the name of the datacenter where to check that they are in sync.

Raises:

spicerack.remote.RemoteExecutionError -- on failure.

Return type:

None

get_core_dbs(*, datacenter: str | None = None, section: str | None = None, replication_role: str | None = None, excludes: tuple[str, ...] = ()) → spicerack.mysql_legacy.MysqlLegacyRemoteHosts[source]

Get an instance to operated on the core databases matching the parameters.

Parameters:
Raises:

spicerack.mysql_legacy.MysqlLegacyError -- on invalid data or unexpected matching hosts.

Return type:

spicerack.mysql_legacy.MysqlLegacyRemoteHosts

get_core_masters_heartbeats(datacenter: str, heartbeat_dc: str) → dict[str, datetime.datetime][source]

Get the current heartbeat values from core DB masters in DC for a given heartbeat DC.

Parameters:

  • datacenter (str) -- the name of the datacenter from where to get the heartbeat values.

  • heartbeat_dc (str) -- the name of the datacenter for which to filter the heartbeat query.

Return type:

dict[str, datetime.datetime]

Returns:

A dictionary with the section name str as keys and their heartbeat datetime.datetime as values. For example:

{'s1': datetime.datetime(2018, 1, 2, 11, 22, 33, 123456)}

Raises:

spicerack.mysql_legacy.MysqlLegacyError -- on failure to gather the heartbeat or convert it into a datetime.

get_dbs(query: str) → spicerack.mysql_legacy.MysqlLegacyRemoteHosts[source]

Get a MysqlLegacyRemoteHosts instance for the matching hosts.

Parameters:

query (str) -- the Remote query to use to fetch the DB hosts.

Return type:

spicerack.mysql_legacy.MysqlLegacyRemoteHosts

set_core_masters_readonly(datacenter: str) → None[source]

Set the core masters in read-only.

Parameters:

datacenter (str) -- the name of the datacenter to filter for.

Raises:
Return type:

None

set_core_masters_readwrite(datacenter: str) → None[source]

Set the core masters in read-write.

Parameters:

datacenter (str) -- the name of the datacenter to filter for.

Raises:
Return type:

None

verify_core_masters_readonly(datacenter: str, is_read_only: bool) → None[source]

Verify that the core masters are in read-only or read-write mode.

Parameters:

  • datacenter (str) -- the name of the datacenter to filter for.

  • is_read_only (bool) -- whether the read-only mode should be set or not.

Raises:

spicerack.mysql_legacy.MysqlLegacyError -- on failure.

Return type:

None

heartbeat_query: str = "SELECT ts FROM heartbeat.heartbeat WHERE datacenter = '{dc}' and shard = '{section}' ORDER BY ts DESC LIMIT 1"

Query pattern to check the heartbeat for a given datacenter and section.

class spicerack.mysql_legacy.MysqlLegacyRemoteHosts(remote_hosts: spicerack.remote.RemoteHosts) → None[source]

Bases: RemoteHostsAdapter

Custom RemoteHosts class to execute MySQL queries.

Initialize the instance.

Parameters:

remote_hosts (spicerack.remote.RemoteHosts) -- the instance to act on the remote hosts.

run_query(query: str, database: str = '', success_threshold: float = 1.0, batch_size: int | str | None = None, batch_sleep: float | None = None, is_safe: bool = False) → collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]][source]

Execute the query via Remote.

Parameters:

  • query (str) -- the mysql query to be executed. Double quotes must be already escaped.

  • database (str, default: '') -- an optional MySQL database to connect to before executing the query.

  • success_threshold (float, default: 1.0) -- to consider the execution successful, must be between 0.0 and 1.0.

  • batch_size (typing.Union[int, str, None], default: None) -- the batch size for cumin, either as percentage (e.g. 25%) or absolute number (e.g. 5).

  • batch_sleep (typing.Optional[float], default: None) -- the batch sleep in seconds to use in Cumin before scheduling the next host.

  • is_safe (bool, default: False) -- whether the command is safe to run also in dry-run mode because it's a read-only command that doesn't modify the state.

Raises:

spicerack.remote.RemoteExecutionError -- if the Cumin execution returns a non-zero exit code.

Return type:

collections.abc.Iterator[tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]]

spicerack.mysql_legacy.CORE_SECTIONS: tuple[str, ...] = ('s1', 's2', 's3', 's4', 's5', 's6', 's7', 's8', 'x1', 'es4', 'es5')

Valid MySQL RW section names (external storage RO sections are not included here).

spicerack.mysql_legacy.REPLICATION_ROLES: tuple[str, ...] = ('master', 'slave', 'standalone')

Valid replication roles.