How to give a sandbox an API key safely?

An agent needs your API key to call its model. If the key sits in the sandbox, anything there that reads the agent’s environment reads the key too. shard keeps the key on the host, so the sandbox never holds it.

Store a key on the host, grant it to api.openai.com, and look at what the sandbox holds:

CLI

printf '%s' "$OPENAI_API_KEY" | shard secret set --destination api.openai.com OPENAI_API_KEY
shard create --name agent --memory 512MiB --secret OPENAI_API_KEY python:3.12
shard exec agent sh -c 'echo $OPENAI_API_KEY'

TypeScript

import { Shard } from "useshards";

const shard = new Shard();
try {
  await shard.secrets.set("OPENAI_API_KEY", {
    value: process.env.OPENAI_API_KEY!,
    destinations: ["api.openai.com"],
  });
  const sandbox = await shard.create({
    name: "agent",
    image: "python:3.12",
    memoryMiB: 512,
    secrets: ["OPENAI_API_KEY"],
  });
  const result = await sandbox.exec(["sh", "-c", "echo $OPENAI_API_KEY"]);
  console.log(result.stdout);
} finally {
  shard.close();
}

Python

import os
from useshards import Shard

with Shard() as shard:
    shard.secrets.set(
        "OPENAI_API_KEY", value=os.environ["OPENAI_API_KEY"], destinations=["api.openai.com"]
    )
    sandbox = shard.create(
        "python:3.12", name="agent", memory_mib=512, secrets=["OPENAI_API_KEY"]
    )
    print(sandbox.exec(["sh", "-c", "echo $OPENAI_API_KEY"]).stdout)

The TypeScript and Python examples reach shard serve at the address in SHARD_REMOTE, with the API key in SHARD_API_KEY.

The sandbox prints mock-OPENAI_API_KEY. That placeholder is all the sandbox ever holds, and code in the sandbox uses it just as it would use the real key. When a request goes to api.openai.com over HTTPS, shard puts the real key into its headers on the host, outside the sandbox.

As a result, an agent that reads its environment, dumps its memory, or posts everything it finds to another host gives away only the placeholder.

Warning

Grant a key only to hosts that never send it back. shard does not inspect responses. A granted host that echoes back the headers it received, such as a debug page, returns the real key to the sandbox.

Keep the key out of your shell

shard secret set reads the key from a pipe, as in the example above. If stdin is a terminal and there is no value, it prompts for the key with echo turned off. Do not put the key on the command line: ps shows it while the command runs, and shard secret set warns you if you do.

The key is stored in a single file on the host that only the user the daemon runs as can read, which is root on Linux. shard secret list prints each name with its destinations and placeholder, but never the key.

Make the request the right way

shard puts the key in only when the request meets all of these conditions:

• It goes over HTTPS on port 443. A request over plain HTTP, or to any other port, carries the placeholder. The same goes for a database password on 5432 or SMTP on 587.
• It goes to a granted destination. A request to any other host carries the placeholder.
• The placeholder is in an end-to-end header, such as Authorization. shard never puts the key in the path, the query or the body, because a host can quote those back in an error. Connection headers, such as Proxy-Authorization or a header named by Connection, keep the placeholder. Use a client that sends the credential in an end-to-end header.
• The placeholder appears unchanged. HTTP Basic auth works too, as in https://api:mock-KEY@host. Any other encoding or signing of the key does not.

Choose the destinations

Repeat --destination for each host that needs the key. A destination may start with *. to cover every host under a domain that its owner controls:

printf '%s' "$GITHUB_TOKEN" | shard secret set --destination '*.github.com' GITHUB_TOKEN

shard refuses *.com, *.co.uk, *.github.io, a bare * and a * that is not the leftmost label.

A key whose shape the SDK checks

Some SDKs check the shape of a key before they send it, and never send mock-NAME. For those SDKs, set a placeholder of the right shape:

CLI

printf '%s' "$STRIPE_KEY" | shard secret set --destination api.stripe.com --placeholder sk_test_placeholder01 STRIPE_KEY

TypeScript

await shard.secrets.set("STRIPE_KEY", {
  value: process.env.STRIPE_KEY!,
  destinations: ["api.stripe.com"],
  placeholder: "sk_test_placeholder01",
});

Python

shard.secrets.set(
    "STRIPE_KEY",
    value=os.environ["STRIPE_KEY"],
    destinations=["api.stripe.com"],
    placeholder="sk_test_placeholder01",
)

A custom placeholder must be 8 characters or more, and contain only letters, digits, _, - and .. The default mock-NAME is exempt from those two checks. Neither kind may contain the key, and the key must not contain the placeholder.

Grant a key to a sandbox that exists

A grant needs a created or stopped sandbox. A running sandbox holds its environment in its processes, so the grant is refused with sandbox <name> is running: secret grant takes a created or stopped sandbox: stop it first:

CLI

shard stop agent
shard secret grant agent OPENAI_API_KEY
shard start agent

TypeScript

await sandbox.stop();
await shard.secrets.grant(sandbox, "OPENAI_API_KEY");
await sandbox.start();

Python

sandbox.stop()
shard.secrets.grant(sandbox, "OPENAI_API_KEY")
sandbox.start()

shard secret ungrant takes the grant back, and it too needs a created or stopped sandbox. A fork keeps the grants of its source. A snapshot holds no grants, so a create from a snapshot must name its own --secret.

With a network policy

A grant does not open the network to any host. If the sandbox has a network policy, that policy must also allow the granted host. See network egress.

Rotate a key

Run shard secret set again with the same name and the new key. The destinations and the placeholder stay the same unless you supply --destination or --placeholder. A running sandbox uses the new key on its next request, and the sandbox itself sees no change.

shard refuses to change the placeholder while a sandbox holds the secret, so ungrant the secret from that sandbox first.

Clients that do not trust the proxy

To put the key in, shard terminates the TLS connection on the host with its own certificate authority. It puts that certificate where the image reads its root certificates, and points SSL_CERT_FILE, REQUESTS_CA_BUNDLE, NODE_EXTRA_CA_CERTS and CURL_CA_BUNDLE at it. The usual clients for curl, Python, Node, Deno, Bun, Go, Ruby, PHP and .NET read that store. A client configured with its own roots can ignore it. These clients also need care:

• Java trusts its own keystore. On JDK 9 or later, import the last certificate in $SSL_CERT_FILE into it:

  awk '/-BEGIN CERTIFICATE-/{c=""} {c=c $0 "\n"} END{printf "%s", c}' "$SSL_CERT_FILE" > /tmp/shard-proxy-ca.pem
  keytool -importcert -noprompt -cacerts -storepass changeit -alias shard-proxy -file /tmp/shard-proxy-ca.pem

• Rust with webpki-roots compiles its root certificates into the binary. Configure rustls to use rustls-native-certs instead.

• A client that pins a certificate refuses the proxy. Its request fails with a certificate error, and the key never goes out.

What this does not stop

A sandbox that is allowed to call api.openai.com with the key can make any call the key allows, so a compromised agent can run up a bill or read whatever the key can read. Scope the key at the provider, because shard only keeps it out of the sandbox.

On the runc provider, nothing isolates the sandbox from the host: root in the sandbox is root on the host, and a process that escapes the sandbox can read the stored key. The secrets reference lists every rule.