Prompt API best practices

Once an application fetches its prompts over the network instead of reading them from its own source, the prompt store becomes a production dependency. It sits in the request path, it can fail, and a careless change to it can alter behaviour for every caller at once.

These are the practices that keep that dependency boring.

Pin production to an explicit version

The default in most tools is to resolve "latest". It is convenient and it is the wrong setting for production.

Resolving latest means anyone publishing a prompt changes production behaviour immediately, including someone who was only trying something out. The blast radius of a publish becomes the entire production surface.

Point each environment at a chosen version, and move that pointer deliberately:

// The caller names the prompt and the environment, never a version number.
const { renderedContent } = await client.prompts.invoke(promptId, {
  environment: 'production',
  variables: { ticketBody },
});

Keeping the version out of application code is the point. The environment decides what resolves, so promoting or rolling back never requires a deploy.

Fail loudly on missing variables

When a required variable is absent, there are two possible behaviours, and only one of them is defensible.

Rendering the template anyway produces a prompt with a hole in it, Summarize the following ticket: followed by nothing. The model will answer. It will answer confidently. And the failure surfaces days later as "the summaries got weird" rather than as an error with a stack trace.

Fail the request instead. A missing required variable is a programming error, and it should look like one.

There is a legitimate place for lenient rendering: previewing a draft while authoring, where half-filled output is the point. Keep that mode clearly separate from the one production calls.

Scope keys narrowly, one workspace each

Treat prompt API keys exactly like database credentials, because that is what they are.

A key bound to one workspace limits what a leak reaches. A read-only key on the service that merely renders prompts means a compromised worker cannot rewrite what every other caller resolves.

Rotate on a schedule, and make sure revoking a key is something you can do in seconds without a deploy.

Cache the template, not the completion

Prompt fetches are network calls, and you do not want one on every request. But be precise about what is cacheable.

The resolved template is safely cacheable. A published version is immutable, so version 4 of a prompt is byte-identical forever. Cache it as long as you like.

The model completion is not. Different variables produce different output; caching across variable sets is a correctness bug, not an optimization.

The subtlety is invalidation. If you cache by prompt name, you must invalidate when the environment pointer moves, or a rollback will not take effect until your TTL expires, which is precisely when you least want to wait. Either cache by resolved version id, or keep the TTL short enough that a rollback is meaningfully fast.

Degrade deliberately when the store is unreachable

Any network dependency fails eventually. Decide in advance what happens, because the default, an unhandled exception in your request path, is rarely what you want.

The options, roughly in order of preference:

1. Serve the last known good template from cache. Versions are immutable, so a stale cached version is a valid prompt, not a corrupted one.
2. Fall back to a bundled default for paths that must never fail, accepting that it may be older.
3. Fail the request explicitly where a wrong prompt is worse than no answer.

What to avoid is silently substituting an empty or partial prompt. A degraded prompt that still calls the model spends money to produce output nobody should trust.

Keep the call surface stable

The last practice is the one that makes everything above possible: application code should name what it wants, never which revision.

One endpoint per prompt. Variables passed as structured data. Environment selection from configuration, not hardcoded. When those hold, every prompt change, publish, promote, roll back, happens without touching the application.

When they do not hold, you have re-coupled prompt changes to deploys, and you are back where you started with extra network calls.

Related reading

What is prompt versioning? explains the draft, version, and environment model these practices assume. How to organize prompts efficiently covers the authoring side.

The PromptCache API docs document the invoke surface, variable handling, and key scopes described here.