Defined Type: service::node

Defined in:
modules/service/manifests/node.pp

Overview

Define: service::node

service::node provides a common wrapper for setting up Node.js services based on service-template-node on. Note that most of the facts listed as parameters to this class are set correctly via Hiera. The only required parameter is the port. Alternatively, config may be given as well if needed.

Parameters

enable

Whether or not the systemd unit for the service should be running. This is passed through to the underlying base::service_unit resource. Default: true.

port

Port on which to run the service

config

The individual service's config to use. It can be eaither a hash of key => value pairs, or a YAML-formatted string. Note that the complete configuration will be assembled using the bits given here and common node service configuration directives. If none is provided, we assume the config can be built from a template in a standard location

full_config

Whether the full config has been provided by the caller. If set to true, no config merging will take place and the caller is required to supply the 'config' parameter. If set to 'external', it will prevent service::node from performing any configuration. Default: false

no_workers

Number of workers to start. Default: 'ncpu' (i.e. start as many workers as there are CPUs)

heap_limit

Maximum amount of heap memory each worker is allowed to have in MBs. If surpassed, the worker will be killed and a new one will be spawned. Default: 300

heartbeat_to

Interval (in ms) used to monitor workers for activity. If $heartbeat_to milliseconds pass without a worker sending a heartbeat, it will be killed by the master. Default: 7500

no_file

Number of maximum allowed open files for the service, to be set by ulimit. Default: 10000

healthcheck_url

The url to monitor the service at. 200 OK is the expected answer. If has_spec it true, this is supposed to be the base url for the spec request

has_spec

If the service specifies a swagger spec, use it to thoroughly monitor it. Default: false

monitor_to

The maximum amount of time the service can take to respond to monitoring requests. It only has an effect if has_spec == true. Default: 5

repo

The name of the repo to use for deployment. Default: $title/deploy

starter_module

The service's starter module loaded by service-runner on start-up. Default: ./src/app.js

entrypoint

If the service's starter module exports a specific function that should be used on start-up, set this parameter to the function's name. Default: ''

starter_script

The script used for starting the service. Default: src/server.js

use_proxy

Whether the service needs to use the proxy to access external resources. Default: false

local_logging

Whether to store log entries on the target node as well. Default: true

logging_name

The logging name to send to logstash. Default: $title

statsd_prefix

The statsd metric prefix to use. Default: $title

auto_refresh

Whether the service should be automatically restarted after config changes. Default: true

init_restart

Whether the service should be respawned by the init system in case of crashes. Default: true

environment

Environment variables that should be set in the service systemd unit Default: undef

deployment

If this value is set to 'scap3' then deploy via scap3, otherwise, use trebuchet Default: undef

deployment_user

The user that will own the service code. Only applicable when $deployment =='scap3'. Default: $title

deployment_config

Whether Scap3 is used for deploying the config as well. Applicable only when $deployment == 'scap3'. Default: false

deployment_vars

Extra variables to include in the puppet-controlled variables file. This parameter's value is used only when the deployment method is Scap3 and the service's configuration is deployed with it as well. Default: {}

contact_groups

Contact groups for alerting. Default: lookup('contactgroups', => 'admins'), - use 'contactgroups'

hiera variable with a fallback to 'admins' if 'contactgroups' isn't set.
use_nodejs10

Deploy an apt component for nodejs 10 for hosts running Debian Stretch. Default: false

Examples

To set up a service named myservice on port 8520 and with a templated configuration, use:

service::node { 'myservice':
    port   => 8520,
    config => template('myservice/config.yaml.erb'),
}

Likewise, you can supply the configuration directly as a hash:

service::node { 'myservice':
    port   => 8520,
    config => {
        param1 => 'val1',
        param2 => $myvar
    },
}

You can supply additional enviroment variables for the service systemd unit

service::node { 'myservice':
    port   => 8520,
    environment => {
       FOO => "bar",
    },
}

Parameters:

  • port (Stdlib::Port)
  • enable (Boolean) (defaults to: true)
  • config (Variant[Hash,String]) (defaults to: {})
  • full_config (Variant[Enum['external'], Boolean]) (defaults to: false)
  • no_workers (Variant[Integer, Enum['ncpu']]) (defaults to: 'ncpu')
  • heap_limit (Integer) (defaults to: 300)
  • heartbeat_to (Integer) (defaults to: 7500)
  • no_file (Integer) (defaults to: 10000)
  • healthcheck_url (String) (defaults to: '/_info')
  • has_spec (Boolean) (defaults to: false)
  • monitor_to (Integer) (defaults to: 5)
  • repo (String) (defaults to: "${title}/deploy")
  • starter_module (String) (defaults to: './src/app.js')
  • entrypoint (String) (defaults to: '')
  • starter_script (String) (defaults to: 'src/server.js')
  • use_proxy (Boolean) (defaults to: false)
  • local_logging (Boolean) (defaults to: true)
  • logging_name (String) (defaults to: $title)
  • statsd_prefix (String) (defaults to: $title)
  • auto_refresh (Boolean) (defaults to: true)
  • init_restart (Boolean) (defaults to: true)
  • environment (Hash) (defaults to: {})
  • deployment (Optional[String]) (defaults to: undef)
  • deployment_user (String) (defaults to: 'deploy-service')
  • deployment_config (Boolean) (defaults to: false)
  • deployment_vars (Hash) (defaults to: {})
  • use_nodejs10 (Boolean) (defaults to: false)
  • contact_groups (String) (defaults to: lookup('contactgroups', {'default_value' => 'admins'}))


158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
# File 'modules/service/manifests/node.pp', line 158

define service::node(
    Stdlib::Port                       $port,
    Boolean                            $enable            = true,
    Variant[Hash,String]               $config            = {},
    Variant[Enum['external'], Boolean] $full_config       = false,
    Variant[Integer, Enum['ncpu']]     $no_workers        = 'ncpu',
    Integer                            $heap_limit        = 300,
    Integer                            $heartbeat_to      = 7500,
    Integer                            $no_file           = 10000,
    String                             $healthcheck_url   = '/_info',
    Boolean                            $has_spec          = false,
    Integer                            $monitor_to        = 5,
    String                             $repo              = "${title}/deploy",
    String                             $starter_module    = './src/app.js',
    String                             $entrypoint        = '',
    String                             $starter_script    = 'src/server.js',
    Boolean                            $use_proxy         = false,
    Boolean                            $local_logging     = true,
    String                             $logging_name      = $title,
    String                             $statsd_prefix     = $title,
    Boolean                            $auto_refresh      = true,
    Boolean                            $init_restart      = true,
    Hash                               $environment       = {},
    Optional[String]                   $deployment        = undef,
    String                             $deployment_user   = 'deploy-service',
    Boolean                            $deployment_config = false,
    Hash                               $deployment_vars   = {},
    Boolean                            $use_nodejs10      = false,
    # lint:ignore:wmf_styleguide
    String                             $contact_groups    = lookup('contactgroups',
                                                                  {'default_value' => 'admins'}),
    # lint:endignore
) {
    case $deployment {
        'git': {
            service::deploy::gitclone { $title:
                repository => $repo,
                before     => Base::Service_unit[$title],
            }
        }
        default: {
            if ! defined(Scap::Target[$repo]) {
                require service::deploy::common
                scap::target { $repo:
                    service_name => $title,
                    deploy_user  => $deployment_user,
                    before       => Base::Service_unit[$title],
                    manage_user  => true,
                }
            }
            # lint:ignore:wmf_styleguide
            include scap::conftool
            # lint:endignore
        }
    }

    # Import all common configuration
    include service::configuration

    # we do not allow empty names
    unless $title and size($title) > 0 {
        fail('No name for this resource given!')
    }

    # the local log directory
    $local_logdir = "${service::configuration::log_dir}/${title}"
    $local_logfile = "${local_logdir}/main.log"


    if $use_nodejs10 {
        if os_version('debian == stretch') {
            if !defined(Apt::Repository['wikimedia-node10']) {
                apt::repository { 'wikimedia-node10':
                    uri        => 'http://apt.wikimedia.org/wikimedia',
                    dist       => 'stretch-wikimedia',
                    components => 'component/node10',
                    before     => Package['nodejs'],
                }
            }
        } else {
            fail('The use_nodejs10 parameter is only usable on Debian Stretch.')
        }
    }

    # Software and the deployed code, firejail for containment
    if !defined(Package['nodejs']) {
        package { 'nodejs':
            ensure => present,
        }
    }
    if !defined(Package['nodejs-legacy']) and os_version('debian == jessie'){
        package { 'nodejs-legacy':
            ensure => present,
        }
    }
    if !defined(Package['firejail']) {
        package { 'firejail':
            ensure => present,
        }
    }

    # User/group
    group { $title:
        ensure => present,
        name   => $title,
        system => true,
        before => Service[$title],
    }

    user { $title:
        gid    => $title,
        home   => '/nonexistent',
        shell  => '/bin/false',
        system => true,
        before => Service[$title],
    }

    # Configuration, directories
    $conf_dir_gid = $deployment ? {
        'scap3' => $deployment_user,
        default => 'root',
    }
    file { "/etc/${title}":
        ensure => directory,
        owner  => 'root',
        group  => $conf_dir_gid,
        mode   => '0775',
    }

    if $full_config == 'external' {
        notice("Not configuring ${title} as configuration is handled independently")
    } elsif $deployment == 'scap3' and $deployment_config {
        # NOTE: this is a work-around need to switch config file deployments
        # to Scap3. The previous praxis was to make the config owned by root,
        # but that is not possible with Scap3, as it installs a symlink under
        # the $deployment_user user. chown'ing it will allow Scap3 to remove
        # the file and install its symlink
        $chown_user = "${deployment_user}:${deployment_user}"
        $chown_target = "/etc/${title}/config.yaml"
        exec { "chown ${chown_target}":
            command => "/bin/chown ${chown_user} ${chown_target}",
            # perform the chown only if root is the effective owner
            onlyif  => "/usr/bin/test -O ${chown_target}",
            require => [User[$deployment_user], Group[$deployment_user]]
        }
        service::node::config::scap3 { $title:
            port            => $port,
            no_workers      => $no_workers,
            heap_limit      => $heap_limit,
            heartbeat_to    => $heartbeat_to,
            repo            => $repo,
            starter_module  => $starter_module,
            entrypoint      => $entrypoint,
            logging_name    => $logging_name,
            statsd_prefix   => $statsd_prefix,
            auto_refresh    => $auto_refresh,
            deployment_vars => $deployment_vars,
            deployment_user => $deployment_user,
        }
    } else {
        service::node::config { $title:
            port           => $port,
            config         => $config,
            full_config    => $full_config,
            no_workers     => $no_workers,
            heap_limit     => $heap_limit,
            heartbeat_to   => $heartbeat_to,
            local_logging  => $local_logging,
            starter_module => $starter_module,
            entrypoint     => $entrypoint,
            logging_name   => $logging_name,
            statsd_prefix  => $statsd_prefix,
            auto_refresh   => $auto_refresh,
            use_proxy      => $use_proxy
        }
    }

    # set up redirecting of stdout/stderr to a file that will be readable by any user.
    systemd::syslog { $title:
        readable_by => 'all',
        base_dir    => $::service::configuration::log_dir,
        group       => 'root',
    }

    if $local_logging {
        # convenience script to pretty-print logs
        file { "/usr/local/bin/tail-${title}":
            content => template('service/node/tail-log.erb'),
            owner   => 'root',
            group   => 'root',
            mode    => '0755'
        }
        # we first placed tail-${title} in /usr/bin, so make sure
        # it's not there any more
        file { "/usr/bin/tail-${title}":
            ensure => absent,
        }

        # Ensure the local log directory is present before the service
        if $enable {
            File[$local_logdir] -> Service[$title]
        }
    }

    # service init script and activation
    base::service_unit { $title:
        ensure         => present,
        systemd        => systemd_template('node'),
        refresh        => $auto_refresh,
        service_params => {
            enable     => $enable,
            ensure     => ensure_service($enable),
            hasstatus  => true,
            hasrestart => true,
        },
    }

    # Basic firewall
    ferm::service { $title:
        proto => 'tcp',
        port  => $port,
    }

    # Monitoring
    $ensure_monitoring = $enable ? {
        true  => 'present',
        false => 'absent',
    }

    if $has_spec {
        # Advanced monitoring
        include service::monitoring

        $monitor_url = "http://${::ipaddress}:${port}${healthcheck_url}"
        file { "/usr/local/bin/check-${title}":
            content => template('service/check-service.erb'),
            owner   => 'root',
            group   => 'root',
            mode    => '0755',
        }
        nrpe::monitor_service{ "endpoints_${title}":
            ensure        => $ensure_monitoring,
            description   => "${title} endpoints health",
            nrpe_command  => "/usr/local/bin/check-${title}",
            subscribe     => File["/usr/local/bin/check-${title}"],
            contact_group => $contact_groups,
            notes_url     => "https://wikitech.wikimedia.org/wiki/Services/Monitoring/${title}",
        }
        # we also support smart-releases
        service::deployment_script { $name:
            monitor_url     => $monitor_url,
            has_autorestart => $auto_refresh,
        }
    } else {
        # Basic monitoring
        monitoring::service { $title:
            ensure        => $ensure_monitoring,
            description   => $title,
            check_command => "check_http_port_url!${port}!${healthcheck_url}",
            contact_group => $contact_groups,
            notes_url     => "https://wikitech.wikimedia.org/wiki/Services/Monitoring/${title}",
        }
    }

}