You might need to install additional packages or tools on Container-Optimized OS
for certain tasks, such as debugging. For example, debug node connectivity issues by
installing tcpdump
in toolbox.
Although Container-Optimized OS does not
include a package manager, you can use the pre-installed
toolbox
utility to install any additional packages or tools you require. Using
/usr/bin/toolbox
is the preferred method for installing and running one-off
debugging tools.
/usr/bin/toolbox
essentially provides you a shell in a Debian chroot-like
environment. When you invoke /usr/bin/toolbox
, it runs following commands:
docker pull
anddocker create
to set up the environment. These are only run the first time you invoke/usr/bin/toolbox
.systemd-nspawn
to run the given command or (in absence of any command) provides you a shell
toolbox
has some other properties to keep in mind:
- Invoking
toolbox
after the first invocation does not require a working Docker daemon, nor does it incur any network/disk overhead. - The
toolbox
environment is set up once for each user invoking it. Runningsudo toolbox
sets it up forroot
user. - The
toolbox
environment is created under/var/lib/toolbox
and is persistent across reboots. - You can access sections of the root filesystem, such as user home
directories, from inside the
toolbox
environment.
Customizing toolbox for your deployment
In the following examples, we assume that your user name is USER
and your
Container-Optimized OS instance name is cos-dev
.
You can customize the Docker image that toolbox
uses, as well as the paths
available to toolbox
in the root filesystem. These settings are located in the
file /etc/default/toolbox
. The default /etc/default/toolbox
file typically
resembles the following:
USER@cos-dev ~ $ cat /etc/default/toolbox
# Copyright 2016 The Chromium OS Authors. All rights reserved.
# Use of this source code is governed by a BSD-style license that can be
# found in the LICENSE file.
TOOLBOX_DOCKER_IMAGE="gcr.io/cos-cloud/toolbox"
TOOLBOX_DOCKER_TAG="v20220722"
TOOLBOX_BIND="--bind=/:/media/root/ --bind=/mnt/disks/:/media/root/mnt/disks/ --bind=/var/:/media/root/var/ --bind=/home:/media/root/home/ --bind=/etc/resolv.conf:/etc/resolv.conf"
: ${USER:=root}
- The
TOOLBOX_DOCKER_IMAGE
andTOOLBOX_DOCKER_TAG
variables specify the Docker image to be used. The defaultgcr.io/cos-cloud/toolbox
comes with some of the common tools like the Google Cloud CLI pre-installed. - The
TOOLBOX_BIND
variable specifies the paths from rootfs to be made available inside the toolbox environment.
To change the default settings, modify the /etc/default/toolbox
file, or
specify new values for the variables in ${HOME}/.toolboxrc
for the appropriate
user. For example, assuming that you want toolbox
to use fedora:latest
as
its container, you can run the following commands:
USER@cos-dev ~ $ echo "TOOLBOX_DOCKER_IMAGE=docker.io/library/fedora" > "${HOME}/.toolboxrc"
USER@cos-dev ~ $ echo "TOOLBOX_DOCKER_TAG=latest" >> "${HOME}/.toolboxrc"
USER@cos-dev ~ $ toolbox
In Milestone 93, toolbox will fail to pull custom toolbox image. To workaround
this problem, build custom toolbox docker image tar ball and export it via
either ${HOME}/.toolboxrc
file or /etc/default/toolbox
file.
USER@cos-dev ~ $ echo "TOOLBOX_DOCKER_IMAGE_TARBALL=[docker image tar ball]" > "${HOME}/.toolboxrc" USER@cos-dev ~ $ toolboxThe `/etc/default/toolbox` file can be modified as shown below.
USER@cos-dev ~ $ cat /etc/default/toolbox # Copyright 2016 The Chromium OS Authors. All rights reserved. # Use of this source code is governed by a BSD-style license that can be # found in the LICENSE file. TOOLBOX_DOCKER_IMAGE="gcr.io/cos-cloud/toolbox" TOOLBOX_DOCKER_TAG="v20220722" TOOLBOX_DOCKER_IMAGE_TARBALL=[location/to/tarball] TOOLBOX_BIND="--bind=/:/media/root/ --bind=/mnt/disks/:/media/root/mnt/disks/ --bind=/var/:/media/root/var/ --bind=/home:/media/root/home/" : ${USER:=root} USER@cos-dev ~ $ toolbox
Installing and running tools from toolbox
Once you've invoked the toolbox
utility to start the shell, you can use
apt-get
inside the resulting container to install packages. For example:
# Inside the toolbox shell
USER@cos-dev ~ $ toolbox
root@cos-dev:~# apt-get update && apt-get install -y htop psmisc
root@cos-dev:~# htop
root@cos-dev:~# pstree -p
root@cos-dev:~# exit
You do not always have to enter toolbox
to run commands in it. You can simply
prefix your commands with toolbox
. For example, to install and run the
strace
utility to trace the execution of the Docker daemon, you can do the
following:
USER@cos-dev ~ $ toolbox apt-get install -y strace
USER@cos-dev ~ $ toolbox strace -p `pidof dockerd`
The Google Cloud CLI is pre-installed in toolbox:
USER@cos-dev ~ $ toolbox
root@cos-dev:~# which gcloud
/google-cloud-sdk/bin/gcloud
# View installed components
root@cos-dev:~# gcloud components list
Your current gcloud CLI version is: 287.0.0
The latest available version is: 295.0.0
...
Keep in mind that the gcloud CLI can only access resources that the VM instance is authorized to access. Read the Compute Engine documentation to learn more about provisioning VM instances with access to other resources.
Read the Google Cloud CLI documentation to learn more about using the gcloud CLI.
Getting files into and out of toolbox
The root filesystem of the host is accessible inside toolbox
through the
/media/root
path. On the host, the root directory of toolbox
is accessible
through the following path:
/var/lib/toolbox/USER-gcr.io_cos-cloud_toolbox-VERSION/root
where USER
is your user name and VERSION
is the toolbox
version number
(for example, v20220722
).
The following examples show how to get files into and out of toolbox
:
# Access the host filesystem inside the toolbox
USER@cos-dev ~ $ toolbox
root@cos-dev:~# ls /media/root
bin boot dev etc home lib lib64 ...
root@cos-dev:~# cp /media/root/home/USER/some-file .
# Access toolbox directory from the host
USER@cos-dev ~ $ sudo cp some-file /var/lib/toolbox/USER-gcr.io_cos-cloud_toolbox-v20220722/root
# Run a command inside toolbox and save its output in your home directory
USER@cos-dev ~ $ toolbox strace -o /media/root/$HOME/ls.strace ls
USER@cos-dev ~ $ more $HOME/ls.strace
Cleaning up disk space used by toolbox
The toolbox installation consumes approximately 2-3 GB of disk
space in /var
(disk usage varies with the Container-Optimized OS version). To
free disk space used by toolbox, perform the following tasks depending on your
Container-Optimized OS version:
Milestone 89 and earlier
# remove files in /var/lib/toolbox
sudo rm -rf /var/lib/toolbox/*
# remove container artifacts
docker image rm $IMAGE_PATH:$TAG
You can run docker image ls
to determine these values. For example,
IMAGE_PATH=gcr.io/cos-cloud/toolbox
and
TAG=v20220722
.
Milestone 93 and later
# remove files in /var/lib/toolbox
sudo rm -rf /var/lib/toolbox/*
# remove container artifacts
sudo ctr images rm $IMAGE_NAME
You can run sudo ctr images ls -q
to determine this value. For example,
IMAGE_NAME=gcr.io/cos-cloud/toolbox:v20220722
.