Configure in-memory volume mounts for jobs

This page describes how to configure a dedicated in-memory volume you can use for file reads and writes using Cloud Run volume mounts. Note that this feature differs from the built-in in-memory filesystem provided by Cloud Run.

When you mount the in-memory volume in Cloud Run, the in-memory volume shows up as files in the container file system. After you mount the in-memory volume, you access it as if it were a directory on your local file system, using your programming language's file system operations and libraries.

You can use in-memory volumes to do the following:

Limit the size of the in-memory volume. When you limit the size of a volume, writes to a full volume will fail, which is preferable to having Cloud Run terminate instances due to the volume consuming too much memory.
Share an in-memory volume between different containers in one Cloud Run instance. When Cloud Run scales out to multiple instances of a service, each instance will have its own in-memory volume shared by all the containers on that instance. This volume is available to all the containers when Cloud Run scales out to handle traffic.

Behavior

When creating an in-memory volume, we recommend specifying a size limit. If the volume reaches its size limit, further writes will fail with an out of memory error. Your instance can handle this error and keep running.

Note that the size limit is just a limit: it does not allocate additional space for your in-memory volume. Rather, your in-memory volume consumes the memory you configured for your containers. If you deploy multiple containers, the memory used by each write to the volume counts as memory usage for the container that wrote the data.

If you don't specify a size limit, it is automatically set to half of the total size of all the containers in your job or service. For example, emptyDir volume size = [Memory (Container A) + Memory (Container B) + Memory (Container N)]/2. This default behavior can result in the size limit of the in-memory volume being higher than the memory allocated for some of your containers. In this situation, if your container keeps writing memory to the volume, it will exceed its allocated memory and crash before the volume size limit is reached.

Although setting a size limit is optional, we recommend setting one to protect your containers from running out of memory and crashing.

Disallowed paths

Cloud Run does not allow you to mount a volume at /dev, /proc and /sys, or on their subdirectories.

Configure an in-memory volume

Any configuration change leads to the creation of a new revision. Subsequent revisions will also automatically get this configuration setting unless you make explicit updates to change it.

After you configure an in-memory volume for your Cloud Run service, an empty volume is created for every Cloud Run instance that is started, and the volume exists as long as that instance is running. When the instance stops running, the data in the volume is permanently deleted.

Console

In the Google Cloud console, go to the Cloud Run jobs page:

Go to Cloud Run
Click Deploy container and select Job to fill out the initial job settings page. If you are configuring an existing job, select the job, then click Edit.
Click Container, variables and secrets, connections, security to expand the job properties page.
Click the Volumes tab.
- Under Volumes:
  - Click Add volume.
  - In the Volume type drop-down, select In-memory as the volume type.
  - In the Volume name field, enter the name you want to use for the volume.
  - In the Size limit field, specify the memory limit you want to assign to the volume. This limit must be less than the total memory specified for your containers. Data stored in this volume consumes memory reserved by the container that wrote the data.
  - Click Done.
- Click the Container tab, then expand the container you are mounting the volume to, to edit the container.
- Click the Volume Mounts tab.
- Click Mount volume.
Click Create or Update.

gcloud

To add a volume and mount it:
```
gcloud run jobs update JOB \
  --add-volume=name=VOLUME_NAME,type=in-memory,size-limit=SIZE_LIMIT \
  --add-volume-mount=volume=VOLUME_NAME,mount-path=MOUNT_PATH
```
Replace:
- JOB with the name of your job.
- VOLUME_NAME with any name you want for your volume. The VOLUME_NAME value is used to map the volume to the volume mount.
- MOUNT_PATH with the relative path within the container file system where you want to mount this volume, for example, /mnt/my-volume.
- SIZE_LIMIT with the memory limit you want to assign to the volume, in MiB or GiB (specified as Mi or Gi), for example, 500Mi. This limit must be less than the total memory specified for your containers.

If you are using multiple containers, first specify the volumes, then specify the volume mounts for each container:

gcloud run jobs update JOB \
  --add-volume=name= VOLUME_NAME,type=in-memory,size-limit=SIZE_LIMIT \
  --container=CONTAINER_1 \
  --add-volume-mount=volume= VOLUME_NAME,mount-path=MOUNT_PATH \
  --container==CONTAINER_2 \
  --add-volume-mount=volume= VOLUME_NAME,mount-path=MOUNT_PATH2

YAML

If you are creating a new job, skip this step. If you are updating an existing job, download its YAML configuration:
```
gcloud run jobs describe JOB_NAME --format export > job.yaml
```
Configure the volumeMounts and volumes attributes as shown:
```
apiVersion: run.googleapis.com/v1
kind: Job
metadata:
  name: JOB_NAME
spec:
  template:
    spec:
      containers:
      - image: IMAGE_URL
        volumeMounts:
        - mountPath: MOUNT_PATH
          name: VOLUME_NAME
      volumes:
      - name: VOLUME_NAME
        emptyDir:
          sizeLimit: SIZE_LIMIT
          medium: Memory
```
Replace:
- IMAGE_URL with a reference to the container image, for example, us-docker.pkg.dev/cloudrun/container/hello:latest. If you use Artifact Registry, the repository REPO_NAME must already be created. The URL has the shape LOCATION-docker.pkg.dev/PROJECT_ID/REPO_NAME/PATH:TAG
- VOLUME_NAME with any name you want for your volume. The VOLUME_NAME value is used to map the volume to the volume mount.
- MOUNT_PATH with the relative path within the container file system where you want to mount this volume, for example, /mnt/my-volume.
- SIZE_LIMIT with the memory limit you want to assign to the volume, in MiB or GiB (specified as Mi or Gi), for example, 500Mi. This limit must be less than the total memory specified for your containers.
Create or update the job using the following command:
```
gcloud run jobs replace job.yaml
```

Reading and writing to a volume

If you use the Cloud Run volume mount feature, you access a mounted volume using the same libraries in your programming language that you use to read and write files on your local file system.

This is especially useful if you're using an existing container that expects data to be stored on the local file system and uses regular file system operations to access it.

The following snippets assume a volume mount with a mountPath set to /mnt/my-volume.

Nodejs

Use the File System module to create a new file or append to an existing in the volume, /mnt/my-volume:

var fs = require('fs');
fs.appendFileSync('/mnt/my-volume/sample-logfile.txt', 'Hello logs!', { flag: 'a+' });

Python

Write to a file kept in the volume, /mnt/my-volume:

f = open("/mnt/my-volume/sample-logfile.txt", "a")

Go

Use the os package to create a new file kept in the volume, /mnt/my-volume

f, err := os.Create("/mnt/my-volume/sample-logfile.txt")

Java

Use the Java.io.File class to create a log file in the volume, /mnt/my-volume:

import java.io.File;
File f = new File("/mnt/my-volume/sample-logfile.txt");