Protect Your AI API Keys: Server Proxies and Rate Limits

Putting an AI feature on a website creates a new spending endpoint. Anyone who can trigger a request may consume paid compute, and anyone who steals the provider credential may use it outside your application entirely. Hiding a key in a minified bundle does not change that risk.

A secure integration needs more than an environment variable. It needs a server boundary that controls identity, operations, budgets and failure behaviour. Treat the AI provider as an external dependency behind your application, not as an API the browser can freely configure.

Keep credentials outside the browser

Store the provider key in a server-side secret store or appropriately protected runtime configuration. Never include it in browser JavaScript, public environment variables, source maps, HTML attributes or client-visible error messages. Build tools often distinguish private and public variable prefixes; verify the resulting bundle rather than trusting the variable's name.

Use separate credentials for development, staging and production. Restrict their permissions and accessible projects where the provider supports that. Define an owner, rotation procedure and emergency revocation path. Removing a leaked key from the latest commit does not remove it from repository history or invalidate copies already collected.

The OWASP secrets management guidance provides a useful operational checklist. Include deployment logs, support tickets and observability exports in your review. Secrets can escape through infrastructure long before anyone opens the application in a browser.

Some services offer short-lived credentials for specific browser workflows. Use those only when explicitly supported and scoped for that purpose. A permanent general-purpose provider key should not be repackaged as a supposedly temporary browser token.

Make the proxy a policy boundary

Your server should expose product operations, not an unrestricted forwarding endpoint. Accept a support question or document summary request, then choose the provider, model, tools and system instructions on the server. Do not let a client supply an arbitrary destination URL or provider authorization header.

Require authentication and authorization independently. A valid login establishes identity; it does not establish access to every workspace or document. Resolve tenant membership from trusted session information and check each requested resource. Never accept a browser-supplied tenant identifier as sufficient proof of access.

A request flow can be expressed without tying it to one framework:

verify session and request origin policy
resolve authorised tenant and operation
validate input size and allowed fields
atomically reserve user and tenant budget
call the configured provider with server-held credentials
settle usage, return a bounded response, record safe telemetry

This order matters. Rejecting an oversized payload after paying for inference is too late. Similarly, a model deciding that a user “looks authorised” is not an access-control check. Keep permission decisions in deterministic server code.

If you build the feature with help from the AI coding assistants directory, review generated route handlers as carefully as handwritten ones. Scaffolding often demonstrates a happy path without implementing the controls a public endpoint requires.

Limit spend across several dimensions

A rate limit caps request frequency, but requests can vary dramatically in cost. Enforce input size, output-token ceilings, concurrent-request limits and cumulative usage budgets as well. A small number of expensive agent runs can cost more than thousands of short text responses.

Track per-user and per-tenant budgets, alongside an application-wide ceiling. Where users are anonymous, use layered controls such as constrained sessions and network-level signals. IP addresses alone are unreliable: shared offices can appear as one caller, while abusive clients can rotate addresses.

Reserve capacity atomically before invoking the provider, then reconcile against reported usage. Otherwise simultaneous requests can all pass a balance check and overspend the same allowance. Decide how reservations expire after worker crashes and how unknown provider outcomes are accounted for conservatively.

Apply stricter limits to expensive operations. Summarising one paragraph and analysing a large uploaded document should not share an identical allowance merely because both use the same route. The guide to LLM cost control explains how token budgets and model routing complement security limits.

Return a clear retry policy when a limit is reached. Do not let the frontend silently retry a denied request in a tight loop. Give legitimate users a recovery path without revealing internal thresholds in unnecessary detail.

Handle retries, streams and cancellation safely

Define timeout and retry budgets for each operation. Retry transient failures only when the operation's semantics permit it, use bounded backoff and avoid multiplying retries across the browser, proxy and worker. Provider timeouts can leave the outcome uncertain; a missing response does not prove no processing occurred.

For tools that change state, use idempotency protections where available. The function-calling guide describes why repeating an execution request can be more serious than repeating a text completion. A second attempt must not create a second order or issue another refund.

Streaming needs cancellation propagation. When a user stops generation or disconnects, attempt to cancel upstream work and release local resources. Cancellation may not immediately stop billing, so measure actual behaviour rather than promising that closing the tab makes a request free.

Use the API latency tester against a safe, non-sensitive health or staging endpoint to compare response timing before and after proxy changes. Do not paste production credentials into a public testing tool. A simple latency check is not a substitute for authenticated load tests or measuring time to first streamed output.

Detect abuse without collecting everything

Record safe telemetry such as operation type, tenant identifier, status, duration, token usage and a correlation identifier. Avoid logging raw authorization headers or full prompts by default. If content sampling is necessary, define access controls, redaction, retention and deletion rules before enabling it.

Alert on unusual spend, repeated authorization failures, sudden concurrency increases and unexpected provider regions or models. Test whether alerts actually reach someone who can act. A dashboard discovered after a weekend of abuse is an accounting tool, not an effective protection.

The OWASP API Security project frames risks including broken authorization and unrestricted resource consumption. Review your proxy against these ordinary API threats as well as AI-specific attacks. The MDN CORS documentation explains browser cross-origin restrictions. CORS is not authentication and does not stop a script running outside a browser from calling a public endpoint.

Prepare an incident runbook: disable the affected operation, revoke compromised credentials, preserve minimal relevant evidence and inspect provider usage. Check downstream actions and data access, not just the bill. Re-enable gradually after repairing the entry point and testing that the same abuse no longer succeeds.

Key takeaways

• Keep permanent provider credentials on trusted servers, never in browser bundles.
• Expose narrow product operations with independent authorization checks.
• Combine request limits with token, concurrency and cumulative spending controls.
• Bound retries and propagate cancellation without assuming billing stops instantly.
• Monitor safe metadata and rehearse credential revocation before an incident.