Secrets

Code in a sandbox can use an API key without ever holding it. Store the key on the server, and give the sandbox a placeholder for it:

printf '%s' "$OPENAI_API_KEY" | shard secret set --destination api.openai.com OPENAI_API_KEY
shard create --name agent --memory 512MiB --secret OPENAI_API_KEY alpine:3.20
shard exec agent sh -c 'echo "$OPENAI_API_KEY"'
shard remove --force agent

import { Shard } from "useshards";

const shard = new Shard();
try {
  await shard.secrets.set("OPENAI_API_KEY", {
    value: process.env.OPENAI_API_KEY!,
    destinations: ["api.openai.com"],
  });
  const agent = await shard.create({
    name: "agent",
    image: "alpine:3.20",
    memoryMiB: 512,
    secrets: ["OPENAI_API_KEY"],
  });
  console.log((await agent.exec(["sh", "-c", 'echo "$OPENAI_API_KEY"'])).stdout);
  await agent.remove({ force: true });
} finally {
  shard.close();
}

import os

from useshards import Shard

with Shard() as shard:
    shard.secrets.set(
        "OPENAI_API_KEY",
        value=os.environ["OPENAI_API_KEY"],
        destinations=["api.openai.com"],
    )
    agent = shard.create(
        "alpine:3.20", name="agent", memory_mib=512, secrets=["OPENAI_API_KEY"]
    )
    print(agent.exec(["sh", "-c", 'echo "$OPENAI_API_KEY"']).stdout)
    agent.remove(force=True)

The TypeScript and Python examples reach shard serve at the address in SHARD_REMOTE, with the API key in SHARD_API_KEY.

The sandbox prints a placeholder, not the key:

mock-OPENAI_API_KEY

When code in the sandbox sends that placeholder in a header of an HTTPS request to api.openai.com, the host puts the real key in its place. Code that reads its environment, dumps its memory or posts every byte it finds somewhere else gets only the placeholder.

Store a secret

shard secret set writes the value on the server to <root>/secrets/<NAME>, with mode 0600, in a directory with mode 0700. That file is the only place shard writes the value.

The name is the environment variable the sandbox reads. It contains only uppercase letters, digits and _, must not start with a digit, and is at most 128 characters long.

--destination names a host the value may be sent to. It is required on the first set, and you repeat it for each host. --dest is the short form.

shard secret set takes the value in one of three ways:

on stdin, when stdin is a pipe or the value is -
at a prompt with echo off, when you run it in a terminal without a value
as an argument after the name

Use stdin in a script, so the value never appears in shell history or in the process list. A value given as an argument is still stored, but set prints a caution on stderr, because the value was visible in the process list while the command ran. set prints the same caution when the server refuses the value.

shard secret list prints the columns NAME, DESTINATIONS, PLACEHOLDER and UPDATED. It never prints a value. --format json prints the same fields as JSON.

shard secret remove <NAME> refuses while a sandbox holds the secret, and names those sandboxes. Ungrant it from them first, remove them, or pass --force.

Grant a secret to a sandbox

shard create --secret <NAME> gives the sandbox the placeholder as $NAME and records the grant in the sandbox record. shard inspect lists the grants under secrets. Every process that shard run starts in the sandbox sees the placeholder too.

A secret is granted to destinations, never to a sandbox alone. A request to a host that is not a destination carries only the placeholder.

A grant does not open any host. The sandbox’s policy decides what it can reach, so a sandbox with a policy needs an allow rule for each granted host. A grant does put the sandbox behind the egress proxy for every host it reaches. From then on, a TLS handshake with no server name is refused, and a request whose Host differs from the handshake name gets a 400. See egress.

A fork keeps its source’s grants. A snapshot holds no grants and no policy, so a create from a snapshot needs its own --secret and --policy.

Grant after the create

shard stop web
shard secret grant web OPENAI_API_KEY
shard start web

await web.stop();
await shard.secrets.grant(web, "OPENAI_API_KEY");
await web.start();

web.stop()
shard.secrets.grant(web, "OPENAI_API_KEY")
web.start()

shard secret grant <sandbox> <NAME> gives a sandbox the same placeholder that --secret gives at create. shard secret ungrant <sandbox> <NAME> takes it back. The sandbox picks up the change the next time it starts.

A grant also makes the sandbox trust the proxy’s certificate, and an ungrant leaves that trust in place. On a VM, both commands change the run message in the sandbox’s record, which the guest reads at its next start.

Both commands work only on a created or stopped sandbox. A running sandbox holds its environment in its processes, and a paused one holds it in its checkpoint, so on those the commands fail with stop it first. Granting the same secret again changes nothing, and so does ungranting a secret the sandbox does not hold.

A grant is refused when the sandbox’s environment already has a variable with that name, and the refused grant changes nothing. A secret named SSL_CERT_FILE, REQUESTS_CA_BUNDLE, NODE_EXTRA_CA_CERTS or CURL_CA_BUNDLE is refused both at create and at grant, because the proxy sets those variables.

Where the value goes in

The host puts the value in only when all of these are true:

the request goes over TLS, on port 443
the request goes to a destination of the secret
the placeholder appears exactly, in a header value

A placeholder in the path, the query or the body goes out unchanged, because an upstream can quote those fields back in an error, which would hand the value to the sandbox. Put the key in a header, where every SDK puts it.

A plain HTTP request never gets the value, even to a granted host, because it crosses the network in clear text. Send the credential over https://.

Only HTTPS on port 443 gets the value. A credential on any other port or protocol, such as a database password on 5432 or SMTP on 587, leaves the sandbox as the placeholder. The policy still decides whether the connection itself is allowed.

HTTP Basic auth works: the host decodes the header, puts the value in and encodes it again, so https://api:mock-KEY@host works. If the key is encoded any other way, or signed, the host does not put the value in.

Hop-by-hop headers keep the placeholder, because they belong to the connection rather than the upstream. They are Connection, Proxy-Connection, Keep-Alive, Proxy-Authenticate, Proxy-Authorization, Te, Trailer, Transfer-Encoding, Upgrade, and any header that Connection names.

Choose the placeholder

The default placeholder is mock-<NAME>. An SDK that checks the shape of a key before sending it will never send mock-<NAME>, so for such an SDK, choose a placeholder with the right shape:

shard secret set --destination api.stripe.com --placeholder sk_test_placeholder01 STRIPE_KEY

A placeholder you choose can contain only letters, digits, _, - and ., so no URL, JSON or base64 encoder changes it. It is refused when:

it is shorter than 8 characters
it contains any other character
it is inside the value, or the value is inside it
another secret owns it, as its own placeholder or as its default

The default mock-<NAME> skips the first two checks, so a short name still gets a placeholder, but it must still pass the last two. The first two checks apply only to a placeholder that the set names, so the placeholder a secret already has never blocks a new value. Every set, of any name, is refused while a secret in the store cannot be read, because that secret may own the placeholder.

You cannot change the placeholder while a sandbox holds the secret, because that sandbox already has the old one. Ungrant the secret first. The change is also refused while the sandbox records cannot be read, because then shard cannot tell whether a sandbox holds the secret.

Grant a wildcard

printf '%s' "$GITHUB_TOKEN" | shard secret set --destination '*.github.com' GITHUB_TOKEN

*.github.com grants the value to every host under github.com. The * must be the leftmost label, and the rest must be a domain that someone can register. *.github.com and *.openai.com are accepted, while *.*, *.com, api.github.*, a bare *, and a public suffix such as *.co.uk or *.github.io are refused. The public suffix list comes with the golang.org/x/net version that shard is built with.

Rotate a secret

Run shard secret set again with the same name to replace the value. The destinations and the placeholder stay the same unless --destination or --placeholder names new ones. The value is never cached: the proxy reads the store on every request, so a running sandbox uses the new value on its next request.

If you remove a secret with --force while a sandbox holds it, you cannot set it again until it is ungranted from that sandbox. The sandbox keeps the old placeholder, but nothing records that placeholder any more. Stop the sandbox, ungrant the secret, then set it and grant it again.

Clients that do not trust the proxy

The proxy reads HTTPS traffic, so the sandbox must trust the proxy’s certificate authority. shard adds that certificate to the file the image already reads, and points SSL_CERT_FILE, REQUESTS_CA_BUNDLE, NODE_EXTRA_CA_CERTS and CURL_CA_BUNDLE at that file. OpenSSL, curl, Python, Ruby, PHP, Go, .NET, Node, Deno and Bun all read it. Three kinds of client do not.

Java trusts its own keystore. keytool imports only the first certificate in a file, but the proxy’s certificate is the last one. To import it in the sandbox, on JDK 9 or later:

awk '/-BEGIN CERTIFICATE-/{c=""} {c=c $0 "\n"} END{printf "%s", c}' "$SSL_CERT_FILE" > /tmp/shard-proxy-ca.pem
keytool -importcert -noprompt -cacerts -storepass changeit -alias shard-proxy -file /tmp/shard-proxy-ca.pem

You can also point -Djavax.net.ssl.trustStore at a store that holds it. The server keeps the certificate on its own in <root>/proxy/ca.crt.

Rust code built with rustls and webpki-roots compiles its root certificates into the binary. Build with rustls-native-certs instead, which reads the same file.

A client that pins a certificate or a public key refuses the proxy. The call fails with a certificate error, and the value is never sent.

What it stops, and what it does not

A secret keeps the value out of the sandbox. Code in the sandbox can read its environment, dump its memory and search its disk, and all it finds is the placeholder. The only way the value can come back is through a granted host that echoes a header, which is why a secret should be granted only to hosts that never do that.

On the runc provider, the proxy is the only thing between the sandbox and the value. Root in the sandbox is root on the host, and code that escapes the sandbox can read the secret store just as the host does.

A secret does not stop misuse of the key: a sandbox that can call api.openai.com with the key can make any call the key allows. Limit what the key can do at the API provider, because shard only keeps the key out of the sandbox.