volume Block

The volume block allows the group to specify that it requires a given volume from the cluster.

The key of the block is the name of the volume as it will be exposed to task configuration.

job "docs" {
  group "example" {
    volume "certs" {
      type      = "host"
      source    = "ca-certificates"
      read_only = true
    }
  }
}

job "docs" {
  group "example" {
    volume "data" {
      type            = "csi"
      source          = "csi-volume"
      read_only       = true
      attachment_mode = "file-system"
      access_mode     = "single-node-writer"
      per_alloc       = true

      mount_options {
        fs_type     = "ext4"
        mount_flags = ["noatime"]
      }
    }
  }
}

The Nomad server will ensure that the allocations are only scheduled on hosts that have a set of volumes that meet the criteria specified in the volume blocks. These may be host volumes configured on the client, or CSI volumes dynamically mounted by CSI plugins.

The Nomad client will make the volumes available to tasks according to the volume_mount block in the task configuration.

volume Parameters

• type (string: "") - Specifies the type of a given volume. The valid volume types are "host" and "csi".

• source (string: <required>) - The name of the volume to request. When using host_volume's this should match the published name of the host volume. When using csi volumes, this should match the ID of the registered volume.

• read_only (bool: false) - Specifies that the group only requires read only access to a volume and is used as the default value for the volume_mount -> read_only configuration. This value is also used for validating host_volume ACLs and for scheduling when a matching host_volume requires read_only usage.

The following fields are only valid for volumes with type = "csi":

• access_mode (string: <required>) - Defines whether a volume should be available concurrently. The access_mode and attachment_mode together must exactly match one of the volume's capability blocks. Can be one of "single-node-reader-only", "single-node-writer", "multi-node-reader-only", "multi-node-single-writer", or "multi-node-multi-writer". Most CSI plugins support only single-node modes. Consult the documentation of the storage provider and CSI plugin.

• attachment_mode (string: <required>) - The storage API that will be used by the volume. The access_mode and attachment_mode together must exactly match one of the volume's capability blocks. Most storage providers will support "file-system", to mount volumes using the CSI filesystem API. Some storage providers will support "block-device", which will mount the volume with the CSI block device API within the container.

• per_alloc (bool: false) - Specifies that the source of the volume should have the suffix [n], where n is the allocation index. This allows mounting a unique volume per allocation, so long as the volume's source is named appropriately. For example, with the source myvolume and per_alloc = true, the allocation named myjob.mygroup.mytask[0] will require a volume ID myvolume[0].

  The per_alloc field cannot be true for system jobs, sysbatch jobs, or jobs that use canaries.

• mount_options - Options for mounting CSI volumes that have the file-system attachment mode. These options override the mount_options field from volume registration. Consult the documentation for your storage provider and CSI plugin as to whether these options are required or necessary.

  • fs_type: file system type (ex. "ext4")
  • mount_flags: the flags passed to mount (ex. ["ro", "noatime"])

Volume Interpolation

Because volumes represent state, many workloads with multiple allocations will want to mount specific volumes to specific tasks. The volume block is used to schedule workloads, so ${NOMAD_ALLOC_INDEX} can't be used directly in the volume.source field. The following job specification demonstrates how to use multiple volumes with multiple allocations, using the per_alloc field. This job specification also shows using HCL2 -variables interpolation to expose information to the task's environment.

variables {
  path = "test"
}

job "example" {
  datacenters = ["dc1"]

  group "cache" {

    count = 2

    volume "cache-volume" {
      type            = "csi"
      source          = "test-volume"
      attachment_mode = "file-system"
      access_mode     = "single-node-writer"
      per_alloc       = true
    }

    network {
      port "db" {
        to = 6379
      }
    }

    task "redis" {
      driver = "docker"
      config {
        image = "redis:7"
        ports = ["db"]
      }
      resources {
        cpu    = 500
        memory = 256
      }

      env {
        # this will be available as the MOUNT_PATH environment
        # variable in the task
        MOUNT_PATH = "${NOMAD_ALLOC_DIR}/${var.path}"
      }

      volume_mount {
        volume      = "cache-volume"
        destination = "${NOMAD_ALLOC_DIR}/${var.path}"
      }

    }
  }
}

The job that results from this job specification has two task groups, each one named for each of the two volumes. Each allocation has its own volume.

$ nomad job status example
ID            = example
...

Allocations
ID        Node ID   Task Group  Version  Desired  Status   Created  Modified
81d32909  352c6926  cache-1     0        run      running  4s ago   3s ago
ce6fbfc8  352c6926  cache-0     0        run      running  4s ago   3s ago

$ nomad volume status 'test-volume[0]'
...
Allocations
ID        Node ID   Task Group  Version  Desired  Status   Created  Modified
ce6fbfc8  352c6926  cache-0     0        run      running  29s ago  18s ago

$ nomad volume status 'test-volume[1]'
...
Allocations
ID        Node ID   Task Group  Version  Desired  Status   Created  Modified
81d32909  352c6926  cache-0     0        run      running  29s ago  18s ago