mysql_legacy

MySQL module (legacy).

Todo

replace with a proper MySQL module that uses a Python MySQL client, preferably in a parallel way.

exception spicerack.mysql_legacy.MysqlLegacyError[source]

Bases: spicerack.exceptions.SpicerackError

Custom exception class for errors of this module.

class spicerack.mysql_legacy.MysqlLegacy(remote: spicerack.remote.Remote, dry_run: bool = True)[source]

Bases: object

Class to manage MySQL servers.

Initialize the instance.

Parameters
check_core_masters_heartbeats(datacenter: str, heartbeat_dc: str, heartbeats: Dict[str, datetime.datetime]) → None[source]

Check the current heartbeat values in the core DB masters in DC are in sync with the provided heartbeats.

Parameters

  • datacenter (str) -- the name of the datacenter from where to get the heartbeat values.

  • heartbeat_dc (str) -- the name of the datacenter for which to filter the heartbeat query.

  • heartbeats (dict) -- a dictionary with the section name str as keys and heartbeat datetime.datetime for each core section as values.

Raises

spicerack.mysql_legacy.MysqlLegacyError -- on failure to gather the heartbeat or convert it into a datetime.

check_core_masters_in_sync(dc_from: str, dc_to: str) → None[source]

Check that all core masters in dc_to are in sync with the core masters in dc_from.

Parameters

  • dc_from (str) -- the name of the datacenter from where to get the master positions.

  • dc_to (str) -- the name of the datacenter where to check that they are in sync.

Raises

spicerack.remote.RemoteExecutionError -- on failure.

get_core_dbs(*, datacenter: Optional[str] = None, section: Optional[str] = None, replication_role: Optional[str] = None, excludes: Tuple[str, ...] = ()) → spicerack.mysql_legacy.MysqlLegacyRemoteHosts[source]

Find the core databases matching the parameters.

Parameters
Raises

spicerack.mysql_legacy.MysqlLegacyError -- on invalid data or unexpected matching hosts.

Returns

an instance with the remote targets.

Return type

spicerack.mysql_legacy.MysqlLegacyRemoteHosts

get_core_masters_heartbeats(datacenter: str, heartbeat_dc: str) → Dict[str, datetime.datetime][source]

Get the current heartbeat values from core DB masters in DC for a given heartbeat DC.

Parameters

  • datacenter (str) -- the name of the datacenter from where to get the heartbeat values.

  • heartbeat_dc (str) -- the name of the datacenter for which to filter the heartbeat query.

Returns

a dictionary with the section name str as keys and their heartbeat datetime.datetime as values. For example:

{'s1': datetime.datetime(2018, 1, 2, 11, 22, 33, 123456)}

Return type

dict

Raises

spicerack.mysql_legacy.MysqlLegacyError -- on failure to gather the heartbeat or convert it into a datetime.

get_dbs(query: str) → spicerack.mysql_legacy.MysqlLegacyRemoteHosts[source]

Get a MysqlLegacyRemoteHosts instance for the matching hosts.

Parameters

query (str) -- the Remote query to use to fetch the DB hosts.

Returns

an instance with the remote targets.

Return type

spicerack.mysql_legacy.MysqlLegacyRemoteHosts

set_core_masters_readonly(datacenter: str) → None[source]

Set the core masters in read-only.

Parameters

datacenter (str) -- the name of the datacenter to filter for.

Raises
set_core_masters_readwrite(datacenter: str) → None[source]

Set the core masters in read-write.

Parameters

datacenter (str) -- the name of the datacenter to filter for.

Raises
verify_core_masters_readonly(datacenter: str, is_read_only: bool) → None[source]

Verify that the core masters are in read-only or read-write mode.

Parameters

  • datacenter (str) -- the name of the datacenter to filter for.

  • is_read_only (bool) -- whether the read-only mode should be set or not.

Raises

spicerack.mysql_legacy.MysqlLegacyError -- on failure.

heartbeat_query = "SELECT ts FROM heartbeat.heartbeat WHERE datacenter = '{dc}' and shard = '{section}' ORDER BY ts DESC LIMIT 1"

Query pattern to check the heartbeat for a given datacenter and section.

Type

str

class spicerack.mysql_legacy.MysqlLegacyRemoteHosts(remote_hosts: spicerack.remote.RemoteHosts)[source]

Bases: spicerack.remote.RemoteHostsAdapter

Custom RemoteHosts class to execute MySQL queries.

Initialize the instance.

Parameters

remote_hosts (spicerack.remote.RemoteHosts) -- the instance to act on the remote hosts.

run_query(query: str, database: str = '', success_threshold: float = 1.0, batch_size: Optional[Union[int, str]] = None, batch_sleep: Optional[float] = None, is_safe: bool = False) → Iterator[Tuple[ClusterShell.NodeSet.NodeSet, ClusterShell.MsgTree.MsgTreeElem]][source]

Execute the query via Remote.

Parameters

  • query (str) -- the mysql query to be executed. Double quotes must be already escaped.

  • database (str, optional) -- an optional MySQL database to connect to before executing the query.

  • success_threshold (float, optional) -- to consider the execution successful, must be between 0.0 and 1.0.

  • batch_size (int, str, optional) -- the batch size for cumin, either as percentage (e.g. 25%) or absolute number (e.g. 5).

  • batch_sleep (float, optional) -- the batch sleep in seconds to use in Cumin before scheduling the next host.

  • is_safe (bool, optional) -- whether the command is safe to run also in dry-run mode because it's a read-only command that doesn't modify the state.

Returns

as returned by cumin.transports.BaseWorker.get_results().

Return type

generator

Raises

RemoteExecutionError -- if the Cumin execution returns a non-zero exit code.

spicerack.mysql_legacy.ACTIVE_ACTIVE_SECTIONS: Tuple[str] = ('x2',)

active/active sections that should not go read-only

Type

tuple

spicerack.mysql_legacy.CORE_SECTIONS: Tuple[str, ...] = ('s1', 's2', 's3', 's4', 's5', 's6', 's7', 's8', 'x1', 'x2', 'es4', 'es5')

list of valid MySQL RW section names (external storage RO sections are not included here).

Type

tuple

spicerack.mysql_legacy.REPLICATION_ROLES: Tuple[str, ...] = ('master', 'slave', 'standalone')

list of valid replication roles.

Type

tuple