Puppet Class: profile::cpjobqueue

Defined in:
modules/profile/manifests/cpjobqueue.pp

Overview

Class: profile::cpjobqueue

profile::cpjobqueue is the profile class for ChangeProp for JobQueue, a special instance of ChangePropagation that is used to pick up job definitions from the main Kafka cluster and sends them for execution on the jobrunners.

Parameters

redis_path

The path to Redis' UNIX socket. Note that the password is taken directly from the private repository with secrets.

jobrunner_host

The address of the LVS end point for jobrunners.

videoscaler_host

The address of the LVS end point for videoscalers.

kafka_msg_max_bytes

The maximum allowed message size in Kafka. This value must match the brokers' configuration of the same name.

Parameters:

  • redis_path (Any) (defaults to: hiera('profile::cpjobqueue::redis_path'))
  • jobrunner_host (Any) (defaults to: hiera('profile::cpjobqueue::jobrunner_host'))
  • videoscaler_host (Any) (defaults to: hiera('profile::cpjobqueue::videoscaler_host'))
  • kafka_msg_max_bytes (Any) (defaults to: hiera('kafka_message_max_bytes'))


24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
# File 'modules/profile/manifests/cpjobqueue.pp', line 24

class profile::cpjobqueue(
    $redis_path          = hiera('profile::cpjobqueue::redis_path'),
    $jobrunner_host      = hiera('profile::cpjobqueue::jobrunner_host'),
    $videoscaler_host    = hiera('profile::cpjobqueue::videoscaler_host'),
    $kafka_msg_max_bytes = hiera('kafka_message_max_bytes'),
) {

    include ::passwords::redis
    require ::profile::changeprop::packages

    $kafka_config = kafka_config('main')

    service::node { 'cpjobqueue':
        port              => 7200,
        healthcheck_url   => '',
        has_spec          => true,
        deployment        => 'scap3',
        deployment_config => true,
        deployment_vars   => {
            broker_list      => $kafka_config['brokers']['string'],
            site             => $::site,
            redis_path       => $redis_path,
            redis_pass       => $::passwords::redis::main_password,
            jobrunner_host   => $jobrunner_host,
            videoscaler_host => $videoscaler_host,
            jobrunner_uri    => "${jobrunner_host}/rpc/RunSingleJob.php",
            videoscaler_uri  => "${videoscaler_host}/rpc/RunSingleJob.php",
            kafka_max_bytes  => $kafka_msg_max_bytes,
        },
        auto_refresh      => false,
        init_restart      => false,
        environment       => {
            'UV_THREADPOOL_SIZE' => 128,
        },
    }
}