Give a sandbox an API key with credential brokering

You’ll give a sandbox an OpenAI key that never enters it. The key stays on the host, the sandbox holds a placeholder, and shard puts the real key into each HTTPS request to api.openai.com.

You need a shard server, a client, and an OpenAI key in OPENAI_API_KEY on your machine. See Create a sandbox, run commands and move files for the first two.

CLI

# Store the key on the host for api.openai.com. It reads the key from stdin.
printf '%s' "$OPENAI_API_KEY" |
  shard secret set --destination api.openai.com OPENAI_API_KEY

# Create a sandbox that holds the secret.
shard create --name agent --memory 512MiB --secret OPENAI_API_KEY python:3.12

# Print the key the sandbox sees. Prints mock-OPENAI_API_KEY.
shard exec agent sh -c 'echo $OPENAI_API_KEY'

# Call the API with it. Prints 200, because the host put the real key in.
shard exec agent sh -c 'curl -s -o /dev/null -w "%{http_code}\n" \
  -H "Authorization: Bearer $OPENAI_API_KEY" https://api.openai.com/v1/models'

# Remove the sandbox, then the secret.
shard remove --force agent
shard secret remove OPENAI_API_KEY

TypeScript

import { Shard } from "useshards";

const callApi = [
  "sh", "-c",
  'curl -s -o /dev/null -w "%{http_code}" ' +
    '-H "Authorization: Bearer $OPENAI_API_KEY" https://api.openai.com/v1/models',
];

const shard = new Shard();
try {
  // Store the key on the host for api.openai.com.
  await shard.secrets.set("OPENAI_API_KEY", {
    value: process.env.OPENAI_API_KEY!,
    destinations: ["api.openai.com"],
  });

  // Create a sandbox that holds the secret.
  const agent = await shard.create({
    name: "agent",
    image: "python:3.12",
    memoryMiB: 512,
    secrets: ["OPENAI_API_KEY"],
  });

  // Print the key the sandbox sees. Prints mock-OPENAI_API_KEY.
  console.log((await agent.exec("echo $OPENAI_API_KEY")).stdout);

  // Call the API with it. Prints 200, because the host put the real key in.
  console.log((await agent.exec(callApi)).stdout);

  // Remove the sandbox, then the secret.
  await agent.remove({ force: true });
  await shard.secrets.remove("OPENAI_API_KEY");
} finally {
  shard.close();
}

Python

import os

from useshards import Shard

call_api = [
    "sh", "-c",
    'curl -s -o /dev/null -w "%{http_code}" '
    '-H "Authorization: Bearer $OPENAI_API_KEY" https://api.openai.com/v1/models',
]

with Shard() as shard:
    # Store the key on the host for api.openai.com.
    shard.secrets.set(
        "OPENAI_API_KEY",
        value=os.environ["OPENAI_API_KEY"],
        destinations=["api.openai.com"],
    )

    # Create a sandbox that holds the secret.
    agent = shard.create(
        "python:3.12", name="agent", memory_mib=512, secrets=["OPENAI_API_KEY"]
    )

    # Print the key the sandbox sees. Prints mock-OPENAI_API_KEY.
    print(agent.exec("echo $OPENAI_API_KEY").stdout)

    # Call the API with it. Prints 200, because the host put the real key in.
    print(agent.exec(call_api).stdout)

    # Remove the sandbox, then the secret.
    agent.remove(force=True)
    shard.secrets.remove("OPENAI_API_KEY")

An agent that reads its environment, dumps its memory or posts everything it finds to another host gives away only the placeholder.

Warning

shard secret set replaces a secret of the same name without asking, and nothing can read the old value back. Pick a name that no other sandbox on the server uses.

Warning

Grant a key only to hosts that never send it back. shard does not inspect responses, so a granted host that echoes the headers it got, such as a debug page, returns the real key to the sandbox.

Store a key

shard secret set stores a key on the server, under a name and with the hosts it may go to:

• The name is the variable the sandbox reads. It contains only uppercase letters, digits and _, does not start with a digit, and is at most 128 characters long.
• --destination names a host the key may go to. It is required on the first set, and you repeat it for each host. --dest is the short form.
• The key comes from a pipe, or from a prompt. A value of - reads stdin too. With no pipe, shard secret set prompts for the key with echo turned off.

Never put the key on the command line: ps shows it while the command runs, and shard secret set warns you if you do, even when the server refuses the key. shard secret list prints the columns NAME, DESTINATIONS, PLACEHOLDER and UPDATED, or JSON with --format json, and never the key. shard inspect lists a sandbox’s grants under secrets.

Make the request the right way

shard puts the key in only when the request meets all of these:

• It goes over HTTPS on port 443. Plain HTTP, or any other port, carries the placeholder.
• It goes to a granted destination. A request to any other host carries the placeholder.
• The placeholder is in an end-to-end header, such as Authorization. shard never puts the key in the path, the query or the body, because a host can quote those back in an error.
• The placeholder appears unchanged. HTTP Basic auth works too, as in https://api:mock-KEY@host. Any other encoding or signing of the key does not.

For a table of example requests and whether each one gets the key, see where the value goes in.

Grant a key to a sandbox that exists

A grant needs a created or stopped sandbox, because a running one holds its environment in its processes:

CLI

# Stop the sandbox, so it can take a new variable.
shard stop agent

# Grant the secret. The sandbox gets the placeholder when it starts.
shard secret grant agent OPENAI_API_KEY

# Start it again.
shard start agent

TypeScript

// Stop the sandbox, so it can take a new variable.
await agent.stop();

// Grant the secret. The sandbox gets the placeholder when it starts.
await shard.secrets.grant(agent, "OPENAI_API_KEY");

// Start it again.
await agent.start();

Python

# Stop the sandbox, so it can take a new variable.
agent.stop()

# Grant the secret. The sandbox gets the placeholder when it starts.
shard.secrets.grant(agent, "OPENAI_API_KEY")

# Start it again.
agent.start()

shard secret ungrant agent OPENAI_API_KEY takes the grant back, also on a created or stopped sandbox.

• A running or paused sandbox is refused. A running sandbox holds its environment in its processes, and a paused one holds it in its checkpoint, so grant and ungrant fail with stop it first.
• A repeat changes nothing. Granting a secret the sandbox already holds is a no-op, and so is ungranting one it does not hold.
• A grant installs the proxy’s certificate. The sandbox then trusts the proxy, and an ungrant leaves that trust in place.
• A clash with the environment is refused. If the sandbox’s environment already has a variable with that name, shard refuses the grant and leaves the sandbox unchanged.
• Four names are reserved. A secret named SSL_CERT_FILE, REQUESTS_CA_BUNDLE, NODE_EXTRA_CA_CERTS or CURL_CA_BUNDLE is refused at create and at grant, because the proxy sets those variables.

A fork keeps the grants of its source. A snapshot holds none, so a create from a snapshot names its own --secret.

Grant a wildcard

A wildcard destination sends the key to every host under one domain:

CLI

# Store a GitHub token for every host under github.com.
printf '%s' "$GITHUB_TOKEN" |
  shard secret set --destination '*.github.com' GITHUB_TOKEN

TypeScript

// Store a GitHub token for every host under github.com.
await shard.secrets.set("GITHUB_TOKEN", {
  value: process.env.GITHUB_TOKEN!,
  destinations: ["*.github.com"],
});

Python

# Store a GitHub token for every host under github.com.
shard.secrets.set(
    "GITHUB_TOKEN",
    value=os.environ["GITHUB_TOKEN"],
    destinations=["*.github.com"],
)

*.github.com covers api.github.com and every other host under github.com, but not github.com itself. Add --destination github.com for that one.

The * must be the leftmost label, and the rest must be a domain that someone can register. *.github.com and *.openai.com are accepted, while *.*, *.com, api.github.*, a bare *, and a public suffix such as *.co.uk or *.github.io are refused. The public suffix list comes with the golang.org/x/net version that shard is built with.

Choose the placeholder

The default placeholder is mock-<NAME>. Some SDKs check the shape of a key before they send it, and never send mock-<NAME>. For such an SDK, choose a placeholder with the right shape:

CLI

# Store a Stripe key whose placeholder looks like a Stripe test key.
printf '%s' "$STRIPE_KEY" |
  shard secret set --destination api.stripe.com \
    --placeholder sk_test_placeholder01 STRIPE_KEY

TypeScript

// Store a Stripe key whose placeholder looks like a Stripe test key.
await shard.secrets.set("STRIPE_KEY", {
  value: process.env.STRIPE_KEY!,
  destinations: ["api.stripe.com"],
  placeholder: "sk_test_placeholder01",
});

Python

# Store a Stripe key whose placeholder looks like a Stripe test key.
shard.secrets.set(
    "STRIPE_KEY",
    value=os.environ["STRIPE_KEY"],
    destinations=["api.stripe.com"],
    placeholder="sk_test_placeholder01",
)

A placeholder you choose can contain only letters, digits, _, - and ., so no URL, JSON or base64 encoder changes it. It is refused when:

• it is shorter than 8 characters
• it contains any other character
• it is inside the value, or the value is inside it
• another secret owns it, as its own placeholder or as its default

The default mock-<NAME> skips the first two checks, so a short name still gets a placeholder. It must pass the last two. The first two checks apply only to a placeholder that a set names, so a secret’s existing placeholder never blocks a new value.

You cannot change the placeholder while a sandbox holds the secret, because that sandbox already has the old one. Ungrant the secret first.

Rotate a key

Run shard secret set again with the same name and the new key. The destinations and the placeholder stay unless --destination or --placeholder names new ones.

The proxy reads the store on every request and caches nothing, so a running sandbox uses the new key from its next request on. The sandbox itself sees no change.

Remove a key

shard secret remove OPENAI_API_KEY refuses while a sandbox holds the secret, and names those sandboxes. Ungrant it from them first, remove them, or pass --force.

A secret removed with --force cannot be set again while a sandbox still holds it. That sandbox keeps the old placeholder, but nothing records it any more. To bring the secret back:

CLI

# Stop the sandbox that still holds the old placeholder, and take the grant back.
shard stop agent
shard secret ungrant agent OPENAI_API_KEY

# Store the secret again.
printf '%s' "$OPENAI_API_KEY" |
  shard secret set --destination api.openai.com OPENAI_API_KEY

# Grant it again, and start the sandbox.
shard secret grant agent OPENAI_API_KEY
shard start agent

TypeScript

// Stop the sandbox that still holds the old placeholder, and take the grant back.
await agent.stop();
await shard.secrets.ungrant(agent, "OPENAI_API_KEY");

// Store the secret again.
await shard.secrets.set("OPENAI_API_KEY", {
  value: process.env.OPENAI_API_KEY!,
  destinations: ["api.openai.com"],
});

// Grant it again, and start the sandbox.
await shard.secrets.grant(agent, "OPENAI_API_KEY");
await agent.start();

Python

# Stop the sandbox that still holds the old placeholder, and take the grant back.
agent.stop()
shard.secrets.ungrant(agent, "OPENAI_API_KEY")

# Store the secret again.
shard.secrets.set(
    "OPENAI_API_KEY",
    value=os.environ["OPENAI_API_KEY"],
    destinations=["api.openai.com"],
)

# Grant it again, and start the sandbox.
shard.secrets.grant(agent, "OPENAI_API_KEY")
agent.start()

With a network policy

A grant opens no host. If the sandbox has a network policy, the policy must also allow the granted host, or the request is denied and the key never goes out. See Network policy and proxy.

If a client does not trust the proxy

The proxy reads HTTPS traffic, so the sandbox must trust the proxy’s certificate authority. shard adds that certificate to the file the image already reads, and points SSL_CERT_FILE, REQUESTS_CA_BUNDLE, NODE_EXTRA_CA_CERTS and CURL_CA_BUNDLE at that file.

OpenSSL, curl, Python, Ruby, PHP, Go, .NET, Node, Deno and Bun all read it. Three kinds of client do not.

Java trusts its own keystore. keytool imports only the first certificate in a file, but the proxy’s certificate is the last one. To import it in the sandbox, on JDK 9 or later:

# Copy the last certificate in the bundle, which is the proxy's, to its own file.
awk '/-BEGIN CERTIFICATE-/{c=""} {c=c $0 "\n"} END{printf "%s", c}' \
  "$SSL_CERT_FILE" > /tmp/shard-proxy-ca.pem

# Import it into the JDK's default trust store.
keytool -importcert -noprompt -cacerts -storepass changeit \
  -alias shard-proxy -file /tmp/shard-proxy-ca.pem

You can also point -Djavax.net.ssl.trustStore at a store that holds it. The server keeps the certificate on its own in <root>/proxy/ca.crt.

Rust code built with rustls and webpki-roots compiles its root certificates into the binary. Build with rustls-native-certs instead, which reads the same file.

A client that pins a certificate or a public key refuses the proxy. The call fails with a certificate error, and the key is never sent.

More

Secrets and network policies explains how a secret and a policy work together. For what brokering guarantees and every case of where the key goes in, see Secrets.