How We Built a Sub-10ms Online Compiler Architecture

Online coding sandboxes often suffer from a frustrating 3 to 8 second cold-start latency. When a learner clicks Run, they expect instantaneous feedback—similar to their local terminal.

In this deep dive, we explore how Backbencher.codes achieves near-zero latency execution across 14 languages.

1. The Multi-Tier Execution Matrix

Not all programming languages need to run inside a heavy Linux container on remote servers. We stratified execution into three isolated tiers:

+--------------------------------------------------------------+
|                   Client Code Submission                     |
+--------------------------------------------------------------+
                               |
            +------------------+------------------+
            |                                     |
            v                                     v
  [Browser Tier: JS/TS/HTML/CSS]        [Cloud Sandbox Tier]
  - Double Null-Origin Iframe           - Warm Pool Containers
  - In-Memory WebAssembly SQLite       - Per-process Cgroups
  - Zero Network Roundtrip (<5ms)       - Sub-second I/O (<150ms)

Tier 1: Client-Side WebAssembly & Native Sandboxes

For frontend languages (HTML, CSS, JavaScript) and TypeScript, compilation and execution happen entirely on the user's hardware within a double-isolated null-origin iframe:

TypeScript: Fast transpilation via modern AST transforms in ~4ms.
SQL: An in-memory SQLite engine initialized via WebAssembly (sql.js). Users can run real CREATE TABLE, INSERT, and complex joins with structured table output, completely isolated from any server database.

Tier 2: Warm Pool Micro-Containers

For compiled languages like C++, Rust, and Java, cold VM boot times are unacceptable. We maintain a pool of pre-warmed container sandboxes with:

Pre-mounted toolchains (GCC 12, OpenJDK 17, rustc 1.78).
Read-only root filesystems.
Linux cgroups restricting CPU to 1 core and RAM to 128MB.
Strict 5-second process timeout enforced at kernel level.

2. Preventing Sandbox Escapes

Treating user code as untrusted is non-negotiable. Our security model enforces:

No Network Egress: Containers are strictly disconnected from external network access.
No Filesystem Persistence: All writes are directed to a temporary in-memory tmpfs capped at 10MB.
Resource Limits: ulimit controls prevent fork bombs (:(){ :|:& };:) from starving system threads.

Conclusion

By isolating browser-compatible workflows on the client and streamlining remote execution through pre-warmed Linux workers, Backbencher delivers a snappy, desktop-like development experience in the cloud.