Puppet Class: airflow

Defined in:
modules/airflow/manifests/init.pp

Overview

SPDX-License-Identifier: Apache-2.0

Class airflow

Base class for declaring airflow::instances. Installs the WMF airflow .deb package as a conda environment into /usr/lib/airflow.

Creates an 'airflow' control systemd service that all airflow::instance services declare themselves PartOf. This can be used for easy managing of all local airflow services.

Also installs a check_cmd used for NRPE checks when an airflow::instance has $monitoring_enabled => true.



12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
# File 'modules/airflow/manifests/init.pp', line 12

class airflow {
    # (mariadb-client needed to run airflow db shell.)
    ensure_packages(['airflow', 'wmf-mariadb104-client'])

    # Path to where airflow conda env is installed
    $airflow_prefix = '/usr/lib/airflow'

    # Generic nrpe check for running a command and checking its retval.
    # Used for custom Airflow CLI checks.
    file { '/usr/local/bin/check_cmd':
        ensure => 'present',
        mode   => '0555',
        owner  => 'root',
        group  => 'root',
        source => 'puppet:///modules/airflow/check_cmd.sh'
    }

    # Wrapper script around find to remove old log files and directories.
    # Each airflow::instance can set up a systemd timer to call this script
    # and clean its log files.
    file { '/usr/local/bin/clean_logs':
        ensure => 'present',
        mode   => '0555',
        owner  => 'root',
        group  => 'root',
        source => 'puppet:///modules/airflow/clean_logs.sh'
    }

    systemd::service { 'airflow':
        content => systemd_template('airflow'),
        restart => true,
    }

}