wtf( )unctionsystem design, drawn
← all problemsAI EngineeringMedium

The agent that runs code you didn't write

A data-analysis agent writes Python and runs it to answer questions about uploaded spreadsheets. That is the product, and it works.

Right now the code executes in the API process, with the service's own credentials and network access. Nothing malicious has to be involved: a generated script with a wrong path or an unbounded loop is enough to take the API down or reach something it shouldn't.

Put the execution somewhere a bad script can't hurt anything.
Components — tap one, then tap a slot on the diagram
!Generated code is executing inside the API process, with the API's own credentials.

Boundaries, outermost first: API: API (plans the work; FAILED: runs the code) No creds, no net: an empty slot for the run the code here Outside every boundary: Credentials (database, cloud), User (uploads a file) Connections: User calls API (step 1) Credentials controls API — never leaves the API API calls run the code here — code + data only (step 2) run the code here must NOT reach Credentials — no path to credentials

Credentialsdatabase, cloud
APIplans the workruns the code
Useruploads a file