# The data directory

> Where the shard daemon keeps its sandboxes, images, snapshots, secrets and policies, what each path under it holds, and what shard puts elsewhere on the host.

Source: https://useshards.com/docs/concepts/data-directory/

The data directory is where the daemon keeps everything it manages: the sandboxes, the images, the snapshots, the secrets and the network policies. It is `/var/lib/shard`, and `shard setup` creates it. To use another directory, start the daemon with `shard --root <dir> daemon` and pass the same `--root` to every command.

- **The daemon is its only writer.** Every command goes to the daemon, which changes the files. Nothing else writes there.
- **One daemon per data directory.** The daemon holds a lock on `daemon.lock`, so a second daemon on the same directory refuses to start.
- **One provider per data directory.** Each sandbox record names the provider that made it. A daemon refuses to start on a data directory that holds another provider's sandboxes, because it cannot run them.
- **Commands find the daemon through it.** A command on the host connects to the socket `shard.sock` in the data directory.

## What it holds

```text
/var/lib/shard/
├── shard.sock          the API socket
├── daemon.lock         keeps a second daemon off this directory
├── daemon.pid          the daemon's process ID
├── auth/               the signing key and token ledger of shard serve
├── .reserve            64 MiB held back, so a full disk can still free space
├── sandboxes/<id>/     one per sandbox: its record, logs and runtime files
├── names/              a link per sandbox name
├── checkpoints/        the saved memory of each paused sandbox
├── network/leases/     a file per IP address a sandbox holds
├── images/
│   ├── layout/         every pulled image, as an OCI image layout
│   ├── rootfs/         each image unpacked, once per digest
│   ├── erofs/          Firecracker: each image as a read-only EROFS file
│   └── disks/          vz: each image as an ext4 disk
├── snapshots/<id>/     one per snapshot: its record and the files it copied
├── snapshot-names/     a link per snapshot name
├── secrets/            a file per secret, mode 0600
├── policies/           the network policies
├── proxy/              the certificate authority of the egress proxy
├── egress.cursor       where the daemon left off in the kernel log
├── kernel/             Firecracker and vz: the guest kernel
├── runsc/              gVisor: the runtime's state per sandbox
├── runc/               runc: the runtime's state per sandbox
├── sysbox-runc/        Sysbox: the runtime's state per sandbox
├── exec/               gVisor, runc and Sysbox: each exec's scratch files
├── firecracker/        Firecracker: each microVM's files
├── jail/               Firecracker: the jail each microVM starts in
└── vz/                 vz: the VM helper and the guest's shard-init
```

- **Secrets stay in one place.** `secrets/` is the only place a secret value is written.
- **Only the provider in use makes its own paths.** The daemon downloads `kernel/` when it first starts, and installs `vz/` from its own binary.
- **Each sandbox and snapshot has a record.** `sandboxes/<id>/sandbox.json` holds a sandbox's name, state, provider, image and bounds, and `snapshots/<id>/snapshot.json` holds a snapshot's.
- **An image unpacks once per digest,** so two tags of one image unpack once.
- **A restart catches up.** `egress.cursor` lets a new daemon still log the packets the host dropped while it was down, and a new daemon clears the `exec/` files an earlier one left.

## Outside the data directory

shard also puts a few things elsewhere on a sandbox host:

- **`/usr/local/bin/shard`**, and on Linux **`/usr/local/bin/shard-init`**: the binaries that setup installs.
- **The service:** `/etc/systemd/system/shard.service` on Linux, or `/Library/LaunchDaemons/shard.daemon.plist` on a Mac.
- **The API key:** `/etc/shard/api-key` on Linux, when setup serves the HTTP API.
- **`/var/log/shard`**: the daemon's logs on a Mac. On Linux, the logs go to the journal.
- **`/var/run/shard/host.lock`**: on Linux, the lock that keeps a second daemon off the host, because every data directory shares one sandbox network.
- **The sandbox network:** on Linux, the bridge `shard0`, one `veth` pair per running sandbox, and the nft tables `inet shard` and `bridge shard`. On a host that runs firewalld, a `shard` zone holds the bridge.
- **`/var/lib/shard.xfs`**: on Firecracker, when the filesystem under `/var/lib/shard` cannot clone files, an XFS image mounted over `/var/lib/shard`, with its line in `/etc/fstab`.

A machine that only connects to a remote server keeps the CLI at `~/.local/bin/shard` and the saved connection in `~/.config/shard/config.json`.

For what `shard setup` installs, step by step, see [What setup does](https://useshards.com/docs/install/#what-setup-does).