Resource Type: hdfs_file

Defined in:
modules/bigtop/lib/puppet/type/hdfs_file.rb
Providers:
hdfs_rsync

Summary

Manage a file on HDFS, supporting content/source, mode, owner, and group.

Overview

Use this type when individual files need to be managed on an HDFS file system. The file contents can be managed in a similar way to the regular file resource by using either the source or contents parameters.

The provider for this resource type depends on using the hdfs-rsync utility and it needs to be executed as the hdfs user, with a matching kerberos keytab present. Therefore only certain hosts are capable of realising these resources.

When ensuring that a resource is absent, the provider call the hdfs cli.

The HDFS path should be expressed as a normal posix path, so without an hdfs:// prefix. The path must start with a slash and contain only alphanumeric characters, plus hyphens and underscores. The file name element may also contain dots.

The resource will use the fs.defaultFS value from /etc/hadoop/conf/core-site.xml on the host, so we cannot yet specify a non-default HDFS file system.

Unless ensure is absent, the mode must be supplied and be an octal value of either 3 or 4 digits.

Similarly, unless ensure is absent, the owner and group parameters must be supplied.

Properties

  • content

    The desired inline content of the file.

  • ensure (defaults to: present)

    The basic property that the resource should be in.

    Supported values:
    • present
    • absent
  • group

    The desired group of the file.

  • mode

    The desired file mode (permissions).

  • owner

    The desired owner of the file.

  • source

    The source file from which to copy content.

Parameters

  • path (namevar)

    The path to the file on HDFS.

  • provider

    The specific backend to use for this hdfs_file resource. You will seldom need to specify this --- Puppet will usually discover the appropriate provider for your platform.