transferpy package

Subpackages

• transferpy.test package
  • Subpackages
    • transferpy.test.integration package
      • Submodules
      • transferpy.test.integration.test_transfer module
      • Module contents
    • transferpy.test.unit package
      • Submodules
      • transferpy.test.unit.test_transfer module
      • Module contents
  • Submodules
  • transferpy.test.utils module
    • hide_stderr
      • hide_stderr.FakeStderr
  • Module contents

Submodules

transferpy.Exceptions module

exception transferpy.Exceptions.Error

Bases: Exception

Base class for exceptions in this module.

exception transferpy.Exceptions.FirewallError(message)

Bases: Error

Exception raised for errors related to Firewall.

exception transferpy.Exceptions.TempDeletionError(message)

Bases: Error

Exception raised for errors in temp path deletion.

transferpy.Firewall module

class transferpy.Firewall.Firewall(target_host, remote_execution, parent_tmp_dir='/tmp')

Bases: object

Class for Transferer firewall related command execution

close(source_host)

Closes target port on iptables of target host.

Parameters:

• source_host – sender host

• target_port – port to be closed

Returns:

remote run exit code, successful(0)

find_available_port()

Checks port availability from a given range of ports on the target host and select one among them.

Returns:

available port if successful, else raises ValueError

find_pid(target_port)

Finds pid of the process based on the port it is using.

Parameters:

target_port – the port using by process

property find_used_ports_command

command to find used ports.

Returns:

command to find used ports

Type:

Property

kill_process(target_port)

Kill the process based on the port it is using.

Parameters:

target_port – the port using by process

Returns:

raises exception if not successful

open(source_host, target_port)

Opens target port on iptables of target host.

Parameters:

• source_host – sender host

• target_port – port to be opened

Returns:

raises exception if not successful

reserve_port(target_port)

Reserves target port by creating a directory.

Parameters:

target_port – port to be reserved

Returns:

True if reservation is successful

run_command(command)

Executes command on the target host.

Parameters:

command – command to run

Returns:

execution result (returncode, stdout, stderr)

unreserve_port(target_port)

Removes the reservation for target port by deleting a directory.

Parameters:

target_port – port to be unreserved

Returns:

True if port successfully unreserved

transferpy.MariaDB module

class transferpy.MariaDB.MariaDB(remote_execution)

Bases: object

Class for Transferer MariaDB related command execution

run_command(host, command)

Executes command on the target host.

Parameters:

• host – command execution target host

• command – command to be executed

Returns:

execution result (returncode, stdout, stderr)

start_replication(host, socket)

Starts slave on the instance of the given host and socket.

Parameters:

• host – MariaDB slave host

• socket – MariaDB slave socket

Returns:

remote run exit code, successful(0)

stop_replication(host, socket)

Stops slave on the instance of the given host and socket.

Parameters:

• host – MariaDB slave host

• socket – MariaDB slave socket

Returns:

remote run exit code, successful(0)

transferpy.Transferer module

class transferpy.Transferer.Transferer(source_host, source_path, target_hosts, target_paths, options={})

Bases: object

after_transfer_checks(result, target_host, target_path)

Post-transfer checks: Was the transfer really successful. Yes- return 0; No- return 1 or more.

attempt_temp_deletion(host, path, type)

Delete directory/file if it exist.

Parameters:

• host – host in which path need to be deleted

• path – path need to be deleted

• type – type of the path (file or dir)

calculate_checksum(host, path)

calculate_checksum_command(host, path)

clean_all_temps(target_host, target_tmp_dir)

Function to clean all the temporary paths if exists in case of any uncaught exception situations.

Parameters:

• target_host – last target host

• target_tmp_dir – temporary lock directory at the target

Returns:

property compress_command

copy_to(target_host, target_path, port)

Copies the source file or dir on the source host to ‘target_host’. ‘target_path’ is assumed to be a directory and the source file or directory will be copied inside.

create_temp_paths(lock_dir)

Update checksum file paths, and create a temporary directory at the source machine.

Parameters:

lock_dir – temporary lock directory at the target host

Returns:

None if successful, else Exception

property decompress_command

property decrypt_command

dir_is_empty(directory, host)

Returns true the given directory path is empty, false if it contains something (a file, a dir). If it is not a directory or does not exist, the result is undefined.

disk_usage(host, path, is_xtrabackup=False)

Returns the size used on the filesystem by the file path on the given host, or the aggregated size of all the files inside path and its subdirectories

property encrypt_command

file_exists(host, path)

Returns true if there is a file or a directory with such path on the remote host given

get_datadir_from_socket(socket)

has_available_disk_space(host, path, size)

host_exists(host)

Checks the availability of given host.

Parameters:

host – host to be checked

Returns:

remote execution run_command result

property is_decompress

is_dir(host, path)

is_socket(host, path)

property is_xtrabackup

property mbstream_command

netcat_listen_command(port)

netcat_send_command(target_host, port)

property parallel_checksum_source_command

command to make source checksum parallel to the file transfer.

Returns:

command to make the checksum

Type:

Property

property parallel_checksum_target_command

command to make target checksum parallel to the file transfer.

Returns:

command to make the checksum

Type:

Property

property password

read_checksum(host, path)

remove_temp_paths()

Remove temporary directories.

Returns:

None

run()

Transfers the file (or the directory and all its contents) given on source_path from the source_target machine to all target_hosts hosts, as fast as possible. Returns an array of exit codes, one per target host, indicating if the transfer was successful (0) or not (<> 0).

run_command(host, command)

sanity_checks()

Set of preflight checks for the transfer- raise an exception if they are not met.

property tar_command

property untar_command

property xtrabackup_command

transferpy.transfer module

class transferpy.transfer.RawOption(prog, indent_increment=2, max_help_position=24, width=None)

Bases: HelpFormatter

Class to format ArgumentParser help

transferpy.transfer.assign_default_options(options)

Assign default values if the given options dictionary is missing some required arguments.

Parameters:

options – given options

Returns:

transferpy.transfer.main()

Main of transfer framework.

Returns:

system exit

transferpy.transfer.option_parse()

Parses the input parameters and returns them as a list.

Returns:

sender host, sender path, receiver hosts, receiver paths, other options

transferpy.transfer.parse_arguments()

Parses the input parameters.

Returns:

parser object

transferpy.transfer.parse_configurations(config_file)

Parses the configuration file parameters.

Returns:

parser object

transferpy.transfer.setup_logger(verbose)

Setup a logger named transferpy. The logger level is set based on the verbose value. If verbose is true, set the logger to DEBUG level, else INFO level. This logger is globally available in the package and can be accessed using logging.getLogger(‘transferpy’).

Parameters:

verbose – verbose boolean variable

Returns:

transferpy.transfer.split_target(target)

Splits the target string to target hostname and path

Parameters:

target – string in the form of hostname:target-path

Returns:

if successful: target hostname, path else system exit

transferpy.transfer.to_bool(value)

Convert the given string to boolean value.

Parameters:

value – value needs to be converted

Returns:

boolean value if given value is convertible else ValueError

Module contents