feat: release Dever 0.1.5 with direct workers and long transactions

This commit is contained in:
Codex committed 2026-10-11 11:17:33 +00:00
1 parent 89629c2197
commit a900d7e8db
153 files changed
+3676 -3708

No files matched your search

+2 -2
View File
@@ -88,7 +88,7 @@ Test authority comes from compiler-assigned source origin, never a user-controll
For `module/<component>/<domain>/api.dever`, the route is `/<component>/<domain>/<action>`; recursive `api/<scopes>/<file>.dever` inserts scopes and file before action. `get/post/put/delete` bind same-domain App functions and inherit their wire-safe input/output shapes; GET/DELETE read query, POST/PUT read a unique-key JSON object. `rest` generates Model CRUD and can use checked owner/create/replace/search field bindings. Each successful request receives JSON `{code:0,message:"ok",data:<value>}` with HTTP 200; standard API errors map to fixed 400/401/403/404/409/429 envelopes and unknown failures become a redacted 500. A top-level POST `Upload` switches the statically known route to bounded multipart parsing and affine ownership. No environment-variable config or implicit database fallback is introduced.
Response Header/Cookie mutations are staged in request-local pending metadata. Generated dispatch serializes the response value, commits the database transaction, and only then promotes pending metadata for the server response; serialization, business, commit or rollback failure discards it. Cookie issue/clear uses the configured name, lifetime and attributes. Credential-changing requests require Origin and Host to exactly match the site's configured external HTTP/HTTPS scheme, host and effective port; a missing or cross-origin value is rejected without trusting forwarding headers. A write API call graph containing password verification or hashing does not receive an entry-wide transaction. Every reachable write must instead cross an explicit short `transaction` function, and password crypto is forbidden inside that transaction. Other write entries retain their generated transaction.
Response Header/Cookie mutations are staged in request-local pending metadata. Generated dispatch publishes metadata only after successful business execution and response serialization; failure discards pending metadata, rolls back ordinary entry transactions and cannot undo already committed long-entry App transactions. Cookie issue/clear uses the configured name, lifetime and attributes. Credential-changing requests require Origin and Host to exactly match the site's configured external HTTP/HTTPS scheme, host and effective port; a missing or cross-origin value is rejected without trusting forwarding headers. Ordinary writing HTTP, CMD and Job entries retain their automatic transaction through response encoding. The contextual `long` prefix on an entry explicitly disables only its entry-wide transaction and requires `transaction` functions around business writes and enqueue. Both backends obey the entry flag, never an inferred effect-based opt-out. `public long` preserves independent access control; `long` changes neither timeout nor scheduling. Generated REST keeps its own write boundary. External Worker, HTTP, file-transfer and password operations cannot occur inside a transaction. Database binding groups include ordinary automatic roots and explicit transactions, but not an entire long-entry orchestration spanning independent stages.
## 4. Validation & Error Matrix
@@ -145,7 +145,7 @@ Sensitivity belongs to the cached type graph in `types.rs`: Secret and private M
| User Job provider/site/reference, compiled permission, component or current role fails | Block with a redacted identity/component category; do not enter the business transaction |
| Credential-changing request has missing or non-matching Origin | JSON authentication error; no Cookie/Header is committed |
| Response serialization, business transaction or commit fails after metadata was staged | Discard staged response metadata |
| Password crypto API reaches a write outside an explicit transaction, or a transaction reaches password crypto | C014 before native generation |
| A long entry reaches a Model write or enqueue outside an explicit transaction, or an automatic/explicit transaction reaches external I/O, a Worker, wait/concurrency or password crypto | C014 before native generation |
## 5. Good / Base / Bad Cases
+5 -4
View File
@@ -81,7 +81,7 @@ Model operations, row-stream pulls and transaction begin/commit propagate `dever
Declare a business function with `transaction name(...) (...)`. The compiler propagates its database effects through ordinary calls and specialized handler bindings, then validates the concrete call graph. The generated hidden context owns one pool object at the outer boundary; nested transaction calls reuse it without savepoints.
Success commits. Error/fault rolls back before propagation. A rollback failure is appended as a cause and does not replace the original error. Commit and rollback consume the transaction, immediately returning a one-connection SQLite pool object after completion. SQLite write transactions use `BEGIN IMMEDIATE`. An API call graph containing password verification or hashing receives no entry-wide transaction; every write in that graph must be owned by an explicit short transaction reached after the expensive work, and that transaction must recheck the mutable state on which the write depends.
Success commits. Error/fault rolls back before propagation. A rollback failure is appended as a cause and does not replace the original error. Commit and rollback consume the transaction, immediately returning a one-connection SQLite pool object after completion. SQLite write transactions use `BEGIN IMMEDIATE`. Ordinary writing HTTP, CMD and Job roots own an entry-wide transaction through output encoding. Only explicitly declared `long` entries omit that transaction; every Model write and persistent enqueue in a long entry must cross an explicit transaction. Generated REST retains its own write transaction. Never infer a mode change from external work or elapsed time. External Worker, HTTP, file-transfer and password work belongs outside transactions. Recheck mutable write preconditions inside each transaction. In long entries, previously committed stages survive later errors, cancellation and response serialization failure; ordinary entries retain all-or-nothing rollback. Nested transactions reuse the entry owner when present.
Use one checked query planner and shared value model for both database executors. Backend-specific code is limited to SQL dialect, bind/row codecs, pool operations and catalog migration. Keep business transactions in the owning domain App; Model CRUD remains compiler-generated.
@@ -168,7 +168,7 @@ Checked Model/SQL/transaction lowering serves LLVM CMD, API, Job and case-local
Deployment settings still come only from executable-local `config/setting.json`; no connection environment variable or PATH language runtime is introduced. Ordinary resolution is explicit connection → component root → required default. Persistence ABI accepts only fixed storage scalar rows; business Model records, nullable fields, Related and DbError are concrete typed owners, not dynamic business values. Related ToMany groups at most 998 parent keys per SQL batch and retains the existing per-parent row limit. Migrations group by resolved database, create all tables before foreign keys and preserve existing history/Seed validation.
Writing CMD order is input decode → begin → sequential App/handler calls → output encode → commit → stdout. Nested transaction shares the outer owner, without savepoints; Task and parallel boundaries detach. Failed or cancelled unconfirmed operations discard the transaction connection. Rollback must move the original fault, retain its type/source/payload and only append cleanup cause; commit failure must release the already-initialized output. Drain children, stop every pool, await pool drivers, then destroy the runtime. Never reuse the process-global Settings/pools for repeated embedded roots.
Ordinary writing CMD order is input decode → begin → sequential App/handler calls → output encode → commit → stdout. A `long cmd` omits entry begin/commit and instead uses explicit App transaction stages. Nested transaction shares the outer owner, without savepoints; Task and parallel boundaries detach. Failed or cancelled unconfirmed operations discard the transaction connection. Rollback must move the original fault, retain its type/source/payload and only append cleanup cause; commit failure must release the already-initialized output. Drain children, stop every pool, await pool drivers, then destroy the runtime. Never reuse the process-global Settings/pools for repeated embedded roots.
### Validation & Error Matrix
@@ -177,7 +177,8 @@ Writing CMD order is input decode → begin → sequential App/handler calls →
| Invalid/misaligned output or typed callback slot | ABI fault before I/O, receiver advancement or fault consumption |
| Missing/invalid settings or missing tenant identity/readiness | Explicit root failure, no default connection fallback |
| Database error | Exact one of the ten checked DbError variants with owned Text payload |
| Output encoding or numeric fault after a write | Original runtime fault; rollback; no stdout |
| Ordinary entry business/output fault, or fault before a long stage commits | Original fault; rollback of the current owned transaction |
| Long-entry output encoding or later-stage fault after commit | Original fault; no successful stdout; previously committed rows remain |
| Captured PostgreSQL SQL constraint error | Current transaction remains aborted; no implicit savepoint/continued-query promise |
| Partial decode, stream stop/cancel or root exit | Paired typed cleanup and connection release/discard in its known state |
@@ -187,7 +188,7 @@ Base: a CMD creates, reads and deletes a row in its owned SQLite database. Good:
### Tests Required
Select root `llvm_database_source`, `runtime_entry_cleanup` and the canonical C database driver in `native_runtime_link`. Static cases must pass the existing checker and emit objects for all six targets. Opt-in native cases link the explicitly prepared runtime-only archive, execute with a cleared environment, and check output/fault plus allocation balance over warmup and 64 repeats. Cover CRUD/bulk/upsert, nullable/codec values, cursor/offset, stream cancellation with one connection, ToOne/ToMany, typed SQL, nested/sequential transactions, uncaught constraint/numeric/encoding rollback and successive schema/Seed revisions. Live PostgreSQL requires only the shared `database.postgres_test` isolated fixture, not an environment URL. These checks do not prove target runtime packs, throughput/RSS, tenant/API equivalence or injected commit/close failures.
Select root `llvm_database_source`, `runtime_entry_cleanup` and the canonical C database driver in `native_runtime_link`. Static cases must pass the existing checker and emit objects for all six targets. Opt-in native cases link the explicitly prepared runtime-only archive, execute with a cleared environment, and check output/fault plus allocation balance over warmup and 64 repeats. Cover CRUD/bulk/upsert, nullable/codec values, cursor/offset, stream cancellation with one connection, ToOne/ToMany, typed SQL, nested/sequential transactions, uncaught constraint/numeric rollback, committed state after encoding failure and successive schema/Seed revisions. Live PostgreSQL requires only the shared `database.postgres_test` isolated fixture, not an environment URL. These checks do not prove target runtime packs, throughput/RSS, tenant/API equivalence or injected commit/close failures.
### Wrong Versus Correct
+19 -19
View File
@@ -4,9 +4,9 @@
Application deployment targets belong to `toolchain::BuildTarget`, independently of the compiler host. Public `build --target linux-aarch64` may run on Linux x86_64; omitted targets and `run/test` stay native. Propagate the selected target through Lib preparation, CompileRequest, runtime pack selection, signature closure verification and both caches. Verify ELF machine/type for archive members and ordinary objects; a matching manifest label/hash alone does not prove architecture. A warm shared cache must still validate the selected target pack. Do not globally replace `platform_identity` or require an ARM compiler core for cross builds.
Target runtime/interpreter/extension/exec/sandbox assets must match the deployment target. Go build tools explicitly declare `build.host`, while Go output and stdlib headers match `build.goos/goarch` and version. Their sandbox uses host assets; emitted Workers use target assets. Product processes keep cleared environments; author-only cross linker settings or Go overlays must not become application configuration/environment knobs. Python sdist/npm hook execution requires a matching execution environment or a previously prepared complete target receipt; never relabel host native output.
Target runtime/interpreter/extension/exec/process-runtime assets must match the deployment target. Go build tools explicitly declare `build.host`, while Go output and stdlib headers match `build.goos/goarch` and version. Their managed process uses host assets; emitted Workers use target assets. Product processes clear non-allowlisted environments; author-only cross linker settings or Go overlays must not become application configuration/environment knobs. Python sdist/npm hook execution requires a matching execution environment or a previously prepared complete target receipt; never relabel host native output.
Executable modes are also part of the cross-target resource contract. Sandbox packaging and application embedding share `asset_executable`; classify both supported target loaders independently of the author's host, while retaining target ELF validation. Regressions must check actual packaged modes from non-executable source inputs and execute the embedded Worker. Host-only loader assertions and successful linking cannot establish target bootstrap behavior. Full-system sandbox fixtures must switch from initramfs rootfs to a normal root mount before bubblewrap uses `pivot_root`; do not loosen the production sandbox to accommodate a fixture mount error.
Executable modes are also part of the cross-target resource contract. Process runtime packaging and application embedding share `asset_executable`; classify both supported target loaders independently of the author's host, while retaining target ELF validation. Regressions must check actual packaged modes from non-executable source inputs and execute the embedded Worker. Host-only loader assertions and successful linking cannot establish target bootstrap behavior.
Applies to formatter/CLI, reference execution, native ownership and clause optimization, concurrent sequence actions and official source packages. JSON algorithms belong to `.dever`; HTTP reuses Hyper in the runtime, with source-owned public types and static business handlers. `intrinsic.rs` remains the exhaustive bridge for checker, native emission and reference evaluation.
@@ -24,7 +24,7 @@ Private `dever run/build/test` uses checked HIR → LLVM → object → in-proce
Native authoring uses the independent four-profile `runtime-pack` build; the old all-features debug archive remains a test fixture only. Signing prepared inputs and compiling source are separate operations. Do not move Cargo/tool discovery into the maker or application run/build.
Linux first installation reuses the release signature and machine version owner. Author `bootstrap` inputs provide launcher/daemon and their own complete private ELF library closure. The independently authenticated installer reads only its absolute root-owned `config/setting.json`; its public key must be supplied outside the downloaded release, then pinned atomically without key replacement. A bounded journal may name only the seven fixed bootstrap destinations, including the sandbox helper tree and dedicated AppArmor policy. Persist final file modes and child directories before publication; recover uncommitted changes, but never roll back a committed publication because cleanup failed. Stop a newly started service before removing its unit; restore previous service state after restoring files. Offline image installation never invokes systemctl. Incoming packages must match the supported service template; existing signed versions retain their original template and remain usable by pinned projects. Actual global activation and public release require separate operational authorization and evidence.
Linux first installation reuses the release signature and machine version owner. Author `bootstrap` inputs provide launcher/daemon and their own complete private ELF library closure. The independently authenticated installer reads only its absolute root-owned `config/setting.json`; its public key must be supplied outside the downloaded release, then pinned atomically without key replacement. A bounded journal may name only the five fixed bootstrap destinations: launcher/daemon bin, system service, command entry, enable link and active version. Persist final file modes and child directories before publication; recover uncommitted changes, but never roll back a committed publication because cleanup failed. Stop a newly started service before removing its unit; restore previous service state after restoring files. Offline image installation never invokes systemctl. Incoming packages must match the supported service template; existing signed versions retain their original template and remain usable by pinned projects. Actual global activation and public release require separate operational authorization and evidence.
Process main is separate from callable roots. Initialize Unix SIGPIPE policy before application threads, preserving stdout EPIPE as the existing located runtime error. Do not change an embedding host's signals from callable roots. Reuse typed fault render, indexed Test fault size/render/release, output cleanup and logger flush; Test main accepts exactly one nonempty unsigned decimal index below case count. Verify actual env-cleared CLI execution, relocated embedded Worker, selected-pack tampering on cache hits, broken stdout, and launcher/daemon dynamic dependencies. Six-target objects do not establish cross-platform execution.
@@ -85,7 +85,7 @@ Process main is separate from callable roots. Initialize Unix SIGPIPE policy bef
Developer onboarding uses one official source repository, `ssh://git@git.dever.cc:2222/shemic/dever.git`. Its ordinary `skills/` directory is committed and versioned with the compiler, without a submodule or separate skill repository; it distinguishes this language from Go Dever. Author `skill` is a required list of `{source,path,sha256}` entries published as signed `skills/dever-language/<path>` artifacts, including SKILL.md; at most 256 files, 1 MiB each and 8 MiB total. New-project source bytes come from the same skill templates; Markdown documentation wraps them without a second semantic copy. Keep the generated configuration CMD-only and free of database/auth prerequisites.
Official `install`, `update` and extension preparation resolve `https://git.dever.cc/api/v1/repos/shemic/dever/releases/latest` or `/tags/v<major.minor.patch>` anonymously. Neither project setting.json nor environment variables choose that origin. Resolve once per operation; the selected stable tag must match the signed manifest even for latest. Metadata is bounded to 2 MiB and 1–512 unique attachments with positive bounded sizes. Download the exact attachment name through its validated `browser_download_url`, enforcing the declared length and HTTPS same-origin redirects. Reject draft/prerelease, malformed tags, duplicate names, missing assets, foreign URLs, truncation and tag/signature mismatch before publication. `dever-release-v2` binds base `artifacts` and optional `extensions` in one signature. Each release publishes `dever-<platform>.manifest.json`, `.manifest.sig`, `.tar.zst`, plus `.ext-<kind>-<target>.tar.zst`. Kind is runtime-pip/npm/go, build-pip/npm, or target; target holds nonhost native runtime/CRT and sandbox. Validate closed IDs, path namespaces, duplicate/overlapping paths, digests and size/count limits in the shared catalog owner. Base retains all four host database profiles, compiler closure, bootstrap, host sandbox and skill. Extensions cannot override base files. Do not fabricate an unsigned merged manifest.
Official `install`, `update` and extension preparation resolve `https://git.dever.cc/api/v1/repos/shemic/dever/releases/latest` or `/tags/v<major.minor.patch>` anonymously. Neither project setting.json nor environment variables choose that origin. Resolve once per operation; the selected stable tag must match the signed manifest even for latest. Metadata is bounded to 2 MiB and 1–512 unique attachments with positive bounded sizes. Download the exact attachment name through its validated `browser_download_url`, enforcing the declared length and HTTPS same-origin redirects. Reject draft/prerelease, malformed tags, duplicate names, missing assets, foreign URLs, truncation and tag/signature mismatch before publication. `dever-release-v2` binds base `artifacts` and optional `extensions` in one signature. Each release publishes `dever-<platform>.manifest.json`, `.manifest.sig`, `.tar.zst`, plus `.ext-<kind>-<target>.tar.zst`. Kind is runtime-pip/npm/go, build-pip/npm, or target; target holds nonhost native runtime/CRT and process runtime libraries. Validate closed IDs, path namespaces, duplicate/overlapping paths, digests and size/count limits in the shared catalog owner. Base retains all four host database profiles, compiler closure, bootstrap, host process runtime libraries and skill. Extensions cannot override base files. Do not fabricate an unsigned merged manifest.
Authenticate the catalog before extraction. Zstandard is a single checksummed frame with window_log_max=27; verify complete compressed EOF/trailer and bounded expansion even after TAR EOF. USTAR contains only its signed ordinary artifact files, with no duplicate, escaping or extra entries; require complete terminator/padding and reject trailing content. Redirects remain on fixed official HTTPS hosts. Publication never replaces another writer's directory. First-install raw `.blob-<sha256>` assets include the launcher and every bootstrap/lib dependency, independently digest-checked before execution. The launcher owns extraction; Python must not implement a second TAR/Zstd parser or require system zstd.
@@ -127,7 +127,7 @@ The machine toolchain has one stable `dever` launcher and a signed version store
Linux cross-user IPC uses kernel `SO_PEERCRED`, verifies endpoint/server UID against the machine state owner and keeps the administrator token owner-only. Foreign users may transfer Lib artifacts and query metadata, never clean or trigger whole-cache hashing. Eight slots admit at most two foreign connections per UID and six total. Artifact transfer accepts at most 64 MiB and stores at most 2 GiB/4096 entries. Compiled entries have separate equal quotas and are inaccessible through ArtifactGet/Put: retrieving a program requires its complete canonical compilation request.
Compilation admission is two requests, with 384 MiB wire input, 4096/16 MiB sources and 4096/256 MiB resources (128 MiB each). The service verifies the installed release and complete runtime reference closure, including its native manifest and LLVM library, on every request before cache lookup. Identity binds canonical input and signed manifest; a persistent private install lock is shared through compilation/transfer and exclusive for install/uninstall, never unlinked. The whole request holds a cache operation; transmission retains its read lease. Only the signed core's hidden compile worker may produce a publishable program. It reads a service-owned 0600 request inside 0700 staging; trusted state/cache ancestors remain traversable and non-writable by other users. The worker reconstructs/checks sources and links, without loading deployment Settings, preparing Libs, executing App/Test or writing shared cache. Caller retains dependency preparation, baseline checks, program execution and per-case Test settings. Worker CPU/address-space/file/FD/core limits do not constitute an OS sandbox.
Compilation admission is two requests, with 384 MiB wire input, 4096/16 MiB sources and 4096/256 MiB resources (128 MiB each). The service verifies the installed release and complete runtime reference closure, including its native manifest and LLVM library, on every request before cache lookup. Identity binds canonical input and signed manifest; a persistent private install lock is shared while validating/compiling and exclusive for install/uninstall, never unlinked. Release that installation lease after the verified output is restored into owned memory, before returning the response, on both cache-hit and fresh-compile paths. Temporary staging cleanup must not leave a completed caller unable to switch versions. The whole request holds a cache operation; transmission retains its read lease. Only the signed core's hidden compile worker may produce a publishable program. It reads a service-owned 0600 request inside 0700 staging; trusted state/cache ancestors remain traversable and non-writable by other users. The worker reconstructs/checks sources and links, without loading deployment Settings, preparing Libs, executing App/Test or writing shared cache. Caller retains dependency preparation, baseline checks, program execution and per-case Test settings. Worker CPU/address-space/file/FD/core limits do not constitute an OS sandbox.
Compilation has a 180-second absolute service deadline, including output transfer, and a 190-second client download deadline. EOF or any byte following the exact request body cancels it. Every failure kills/waits for the owned child and removes staging; diagnostics are capped at 64 KiB and output at 320 MiB. This output budget applies only to trusted compilation receipts, never opaque ArtifactGet/Put. Apply remaining time before each write/read; a per-call socket timeout alone permits slow peers to retain version and cache locks indefinitely.
@@ -164,7 +164,7 @@ Good: complete relocatable interpreter plus stdlib, or Go tools plus every refer
`native_release` verifies deterministic archive/signature bytes, computed runtime identities, bad inputs and rollback. Explicit ecosystem acceptance must lock from a signed install, execute all three managed check/run/build paths, then run programs after removing source and machine stores inside a root with no system language installations. OS baseline libraries, test key, target and untested third-party registry behavior remain explicit.
Linux no-host acceptance must use a genuine mount-namespace root, shared by core, Worker and bootstrap fixtures. A plain `chroot` leaves the process root different from the namespace root; Linux then rejects the Worker's nested `CLONE_NEWUSER` with EPERM, even for root. Use the explicitly prepared bwrap/loader assets to bind only the owned OS root, private proc and synthetic devices. Do not remove the product's `--unshare-all` or add host library/tool mounts to make the fixture pass. A successful managed run does not replace source-free standalone execution under this boundary.
Linux no-host acceptance uses an explicitly prepared, test-only namespace fixture (`DEVER_TEST_ISOLATOR`) to prove standalone packaging without changing the host. This test harness is not a product dependency or Worker boundary. Source-free deployment and non-root execution remain separate checks; a managed run alone proves neither.
Managed Go compilation removes the invocation-owned stage prefix from recorded source paths with `compile -trimpath`. This applies to the SDK, Adapter and dependency packages through their single `Builder::compile` path. Preserve logical filenames/line numbers; do not strip diagnostics to hide path drift. The real `external_workers` Go fixture must prepare twice in fresh stages and compare every emitted resource identity, then execute after source removal. The signed three-ecosystem run/build test must publish exactly three compiled entries. A single successful Go execution does not prove reusable cache identity.
@@ -178,19 +178,19 @@ Application-test execution reuses checked HIR, specialization, runtime feature t
Port native dispatch is a direct call for one implementation or a closed match for several. `adapter.<port identity>.use` in setting.json selects an immutable startup binding; unknown choices/fields, duplicate keys and missing required settings fail before user code. A selected Adapter's contextual `setting` is a private record decoded with shared SettingInput wire helpers into a OnceLock; only that implementation requires values. Secret errors never include input values. Port-only binaries enable the wire feature without API/database runtime; a no-setting single implementation may omit setting.json. Runtime configuration uses the bounded duplicate-aware wire parser before typed config parsing.
External exec and managed dispatch remain behind the same selected Port binding. All executions require a verified embedded launch manifest, selected Worker tree and signed sandbox assets; no adjacent-file or bare-process fallback exists. CLI preparation reads sandbox leaves from the authenticated installed release, including for exec. Runtime clears the environment, maps only verified launch arguments, and uses length-framed stdio. Native and LLVM dispatch reuse one invocation settings snapshot for Adapter selection, business settings and deployment grants. Component tasks reuse the structured application Scope and bounded queues. `hello/ready/health` must finish before admission; request ids are sequential and exact; timeout/cancellation sends `cancel` and drains one matching `result` or `error`; application cleanup sends `shutdown`, joins, then uses a bounded kill only when the child misses the deadline.
External exec and managed dispatch remain behind the same selected Port binding. All executions require a verified embedded launch manifest, selected Worker tree and signed process runtime assets; no adjacent-file or bare-process fallback exists. CLI preparation reads process runtime leaves from the authenticated installed release, including for exec. Runtime clears the environment, maps only verified launch arguments, and uses length-framed stdio. Native and LLVM dispatch reuse one invocation settings snapshot for Adapter selection, business settings and command limits. Component tasks reuse the structured application Scope and bounded queues. `hello/ready/health` must finish before admission; request ids are sequential and exact; timeout/cancellation sends `cancel` and drains one matching `result` or `error`; application cleanup sends `shutdown`, joins, then uses a bounded kill only when the child misses the deadline.
Python SDK input uses an owned asyncio pipe transport and bounded message queue. Cancel and join its reader on normal shutdown and protocol failure, then close the transport. A daemon thread blocked on `sys.stdin.buffer` can abort CPython finalization even after a successful reply. The SDK regression must wait for successful process exit after the shutdown reply while the parent's stdin writer remains open; closing it before waiting masks this lifecycle defect. SDK source changes require re-embedding the source through the host compiler before deployed application verification.
`dever-sandbox` owns the Linux namespace/seccomp launch boundary for Workers and Go build tools. The signed release supplies bwrap, the safe Rust guard and a bounded, complete ELF library closure; the loader cannot select libraries through environment or cache, and a host preload configuration is explicitly unsupported. Mounts are opened with no-symlink `openat2` and passed as owned descriptors to bwrap bind-fd, preserving the authorized inode across rename/replacement. Never replace this with a path check followed by a path-based bind. Input trees are read-only; output grants cannot overlap executable inputs. The runtime additionally rejects writable grants overlapping `data/cache/lib`, including its ancestors. Sandbox descendants inherit the syscall filter; namespace termination stops even children that changed process groups.
`dever-process` owns direct Linux child execution and process groups, not an OS sandbox. Signed `process-runtime/<target>/lib` contains loader/OS ABI libraries only; no bwrap, guard, seccomp filter, mount grants or host AppArmor policy is shipped. Preserve authenticated resources, explicit paths, protocol validation, bounded queues, deadlines and cancellation. Pin the group leader with pidfd; terminate its group before reaping the leader, including normal exit and failure. Detached sessions are outside this lifecycle guarantee; trusted Workers must not daemonize. Never weaken the existing container/node security policy to run a Worker.
The signed bwrap must be a bounded static target ELF without an interpreter or dynamic dependencies and is executed directly. On Linux with restricted unprivileged user namespaces, a non-root launch requires the matching digest-addressed `/opt/dever/sandbox/<sha256>/bwrap`, root ownership of every ancestor, no shared writes/symlinks/setuid/setgid/file capabilities, and both dedicated AppArmor profiles loaded in enforce mode. The helper bytes must match the embedded signed asset. The runtime and real-host bootstrap require `kernel.apparmor_restrict_unprivileged_unconfined=1` before using/publishing this policy: at zero, unconfined callers can enter a capability-granting profile through `aa_change_profile`, independently of its executable attachment. Missing or invalid strict-policy state fails closed. The product only reads these flags; administrator-approved installation/loading owns host changes. Root or unrestricted-host execution uses the same embedded static helper, never an unsandboxed fallback. Test normal and immediate profile changes, with and without no-new-privileges, using the same-process negative probe; wrapper failure or unavailable setup is not a successful isolation check.
Managed children clear ambient application environment and inherit only the allocated GPU visibility/driver variables and `LD_LIBRARY_PATH`. No process discovers language runtimes or compiler tools through host PATH. `allow` remains static effect/handshake metadata, not a file/network/device enforcement mechanism. Only trusted Workers and dependency build hooks may execute; Pod UID, mounts, network and device policies own the real permission boundary.
With `process=true`, POSIX spawn may reset only the effective UID/GID to the guard's captured real identity: `setresuid(-1, uid, -1)` and `setresgid(-1, gid, -1)`. Compare all three arguments as 32-bit kernel IDs; changing real/saved IDs or choosing another effective ID remains denied. Without the process capability both calls remain denied. Keep clone3 returning ENOSYS so libc falls back to the flag-checked clone syscall. The actual C probe must cover both capability states, high-bit-equivalent IDs and rejected identity changes; an unrestricted setuid allowlist is not an acceptable build compatibility fix.
Deployment file grants live only at `adapter.<port>.files.<name> = {"path":"data/upload", "write":true}` in setting.json. Checked `allow file` is required, names are bounded lowercase identifiers, paths stay below the application, and write defaults to false. Workers see `/data/<name>`; grant configuration is not business `setting`. Network capability permits IP sockets and supplies only bounded resolver files; host Unix sockets and raw sockets stay denied. Process capability controls fork/process clones separately from threads. GPU and unsupported OS backends fail explicitly. Linux additionally requires host permission for user/mount/PID/network namespaces and openat2; root fixture success does not establish non-root deployment support under AppArmor.
Go tools consume a private read-only input tree, including parent-generated import/embed configuration, and a separate writable output tree. Parent reads of generated programs must use nofollow, regular-file and single-link checks with a byte limit; never follow a tool-created output symlink on the host. Runtime/SDK/build assets and source remain explicit, without host compiler discovery or implicit networking.
Adapter `files` and `gpu` grant fields are removed and rejected, not silently ignored. Pass real model/input/output paths as business `setting`; GPU devices and drivers come from the parent container without nested mounts. Managed C ABI remains the separate exec Worker SDK in `sdk/c`, never arbitrary dlopen in the Dever process or Dever raw pointers. A successful CPU invocation is not GPU/model validation.
Go tools consume a private input tree, including parent-generated import/embed configuration, and a separate writable output tree. Parent reads of generated programs must use nofollow, regular-file and single-link checks with a byte limit; never follow a tool-created output symlink on the host. Runtime/SDK/build assets and source remain explicit, without host compiler discovery or implicit networking.
When a build carries compiler-owned external resources, the native entry may include a deterministic resource table. `dever-runtime::external` releases it only below the executable's private `data/cache/lib/<digest>` directory, rejects symlinks/path escape and digest mismatches, and publishes a complete staging directory atomically. Existing entries are reverified; a concurrent publisher is reusable only after the same manifest verifies. This extraction path never downloads, consults `PATH`, or accepts runtime paths from environment variables.
@@ -252,7 +252,7 @@ Create a response-body cancellation owner only after the pending-to-committed st
- Project has a pinned version: `skill path <project>` resolves that exact installed version. Foreign users must read matching guidance without obtaining the administrator install lock.
- Test filter/timeout is missing, repeated, non-UTF-8 or malformed: reject before execution. No selected case: fail before compilation. Assertion, process error or timeout: retain labelled captured output, continue later selected cases and return nonzero. Cleanup only the case's process group; never reap unrelated children or leave adopted descendants to PID 1.
Cross build: unknown/duplicate/missing target arguments fail before execution; absent target packs, wrong ELF/archive machine, unsigned target leaves, incompatible lock/runtime/build receipt and changed target inputs fail before cache reuse or linking. Sandbox loader mode must survive both release packing and embedding; a hash-valid non-executable loader is not a usable Worker resource.
Cross build: unknown/duplicate/missing target arguments fail before execution; absent target packs, wrong ELF/archive machine, unsigned target leaves, incompatible lock/runtime/build receipt and changed target inputs fail before cache reuse or linking. Process runtime loader mode must survive both release packing and embedding; a hash-valid non-executable loader is not a usable Worker resource.
Native maker: wrong format/target/version, unknown or duplicate fields, missing profile, non-regular or escaping source, SHA mismatch, exposed/invalid signing key or payload alias to key → explicit failure, no output, owned staging removed. Existing or concurrently created destination → no overwrite. Correct bytes → complete signed v2 catalog with base and extension artifacts, without private key/configuration. Unsupported host publication → explicit failure, never a weaker rename fallback.
@@ -260,13 +260,13 @@ Managed input with duplicate fields/noncanonical encoding, unsafe paths or forge
External Lib declarations live only in the root `lib` array of `config/setting.json`; reject old `config/lib.json` rather than silently combining it. Canonical `dever-lock-v6` locks bind checked Port/Adapter/schema/operations/capabilities and independent Worker roots. Registry artifacts retain exact `{ecosystem,locator,filename}` sources. Build Receipt.inputs is the complete fixed LockFile: Python build requirements or npm's pre-pruning execution graph (no self-referential build receipt). Receipt.graph remains the post-build result. Doctor rejects stale bindings, missing/cyclic closure or within-environment version conflicts; independent Workers may select different versions. Embedding compares complete normalized provider metadata, including dependencies, runtime, schema and artifacts, not just an artifact subset.
`dever lib install <root>` requires a current canonical lock, rejects specs, and holds the existing project mutation lock. Restore exact Package ZIPs first, then check declaration/Worker closure and selected target before restoring Lib archives. It never invokes a version resolver or writes setting/lock. A true missing archive uses its recorded source with existing registry host/path/size rules and locked Go evidence. Missing build outputs replay fixed receipt inputs through the same bounded sandbox executor used by initial resolution; validate runtime/tools/frontend, result graph/outcomes, bytes and digest before publishing. A non-reproducible build is an explicit failure, not authority to rewrite the lock. Ordinary run/build and doctor/remove remain readonly; add/update/install and Package preparation ensure runtime resources before Worker binding, including no-Lib adapters. Build resources are ensured only when the source-build path actually requests them. Cached built outputs do not need build tools.
`dever lib install <root>` requires a current canonical lock, rejects specs, and holds the existing project mutation lock. Restore exact Package ZIPs first, then check declaration/Worker closure and selected target before restoring Lib archives. It never invokes a version resolver or writes setting/lock. A true missing archive uses its recorded source with existing registry host/path/size rules and locked Go evidence. Missing build outputs replay fixed receipt inputs through the same bounded managed executor used by initial resolution; validate runtime/tools/frontend, result graph/outcomes, bytes and digest before publishing. A non-reproducible build is an explicit failure, not authority to rewrite the lock. Ordinary run/build and doctor/remove remain readonly; add/update/install and Package preparation ensure runtime resources before Worker binding, including no-Lib adapters. Build resources are ensured only when the source-build path actually requests them. Cached built outputs do not need build tools.
Production registry access derives from a verified installed version core, not host tools or development fixtures. Signed assets `runtime/<ecosystem>/<target>/manifest.json` (`dever-registry-runtime-v1`) bind a RuntimePack identity and the adjacent `runtime.pack`; Python metadata additionally supplies target markers/wheel tags. Resolver owners implement PyPI PEP 508/440, npm semver/SRI and Go proxy/MVS with fixed HTTPS origins, disabled proxies and bounded work. Redirects are rejected except for Go `/@v/*.zip`: manually follow at most two hops into the exact HTTPS `storage.googleapis.com/proxy-golang-org-prod/` bucket. Validate normalized host/port/path and reject userinfo, fragments, missing/duplicate Location and loops before issuing a subsequent request; all hops share one 30-second deadline and diagnostics must not echo signed URLs. Loopback mirrors are explicit test transport, never project configuration. Unsupported extras/sdist, install hooks and Go sumdb are not silently approximated. Only explicit Lib commands resolve/download; run/build consume locked cached bytes.
`workers::prepare(root, Program, resources)` expands verified runtime packs (`dever-worker-runtime-v1`: ecosystem, target, executable, arguments; Go additionally has `build`) and owned Adapter sources into a per-contract `workers/<digest>` tree. Python uses exactly `-I -S -B`, installs the packaged SDK path and imports that trusted SDK before exposing business source/dependency paths, and reserves stdout for protocol; JavaScript binds ESM/CommonJS exports. Both bind each checked Port operation by name using `sdk_manifest()`. Go consumes fixed-target compiler/linker/source selector and stdlib importcfg from the same exact locked pack, compiles the SDK and checked-operation runner alongside target-selected Adapter files and locked module ZIPs, then embeds only the executable and checked contract. Go source exports one exact `func(context.Context, any, any) (any, error)` per operation; `send` maps to `Send`, dotted segments to PascalCase. Cgo, assembly and unlisted imports fail explicitly. Build tools run by absolute path with an empty environment and bounded lifetime; product compilation does not invoke host `go build`, `PATH`, `GOOS`, `GOARCH`, `GOCACHE` or `GOPATH`. `<entry>.dever-worker.json` (`dever-worker-launch-v1`: ecosystem, entry, executable, arguments, working_directory) accepts only manifest-bound resource arguments in that same tree and bounded literal flags. Runtime revalidates the entire private bundle before every launch/restart and clears the inherited environment. Discard consumed raw runtime/archive resources after expansion unless config roots or exec Workers still need them. A private fixed-target pack fixture proves the local compile/link path; signed release packs, Go sumdb and cross-platform distribution remain separate acceptance work.
Packaged exec Workers resolve only against the entry's embedded manifest. Bundle identity includes path, byte digest, length and executable bit. Validate all path segments, Unix owner/private permissions and file bytes on both cache reuse and every Worker restart; do not scan other bundles or fall back to source workers. Publish owner-only files/directories atomically before exposure. These protocol and integrity checks do not establish OS capability isolation; verify the sandbox launch boundary above separately.
Packaged exec Workers resolve only against the entry's embedded manifest. Bundle identity includes path, byte digest, length and executable bit. Validate all path segments, Unix owner/private permissions and file bytes on both cache reuse and every Worker restart; do not scan other bundles or fall back to source workers. Publish owner-only files/directories atomically before exposure. These protocol and integrity checks do not establish OS capability isolation; deployment container policies own that boundary.
`dever.crypto` uses the existing source Result choices and exhaustive Intrinsic bridge. Secret storage lives in the base runtime with zeroize; Argon2id, SHA/HMAC and OS randomness are in the optional crypto feature. Native runtime feature selection inspects already-pruned emitted crypto calls so direct system intrinsics, API roots and Test roots are covered. Password hashing uses RustCrypto Argon2 0.5.3, Argon2id v19 and default Params; PHC verification bounds algorithm/version/memory/time/parallelism/output before computing. Failure messages contain no input. Hash/verify use the existing blocking-worker effect boundary. `dever.process` exposes arguments only; application settings never come from environment variables.
@@ -360,7 +360,7 @@ Native release changes select `native_release` plus affected private pack/instal
Managed compilation selects `packages::managed_compile`, the installed Package execution fixture, affected `shared_toolchain` cache/install/IPC cases, and private `llvm_cli`. Cover secret-free snapshots, reserved/strict source checks, cross-UID reuse, changed sources, failure recovery, fake-only Test execution, signature reference omissions, cache-hit tampering, limits, real cancellation/disconnect/deadline reaping, and shared install lock lifetime. Tests use owned daemons and temporary signing keys. Check final `dever`/`deverd` DT_NEEDED independently: empty-environment success on an LLVM-equipped host cannot establish absence of a dependency.
Runtime ABI changes select `native_runtime_abi` and opt-in `native_runtime_link`: exact Decimal word round-trip, checked errors/IEEE floats, borrowed UTF-8/Bytes including null-empty and invalid input, real C-header/staticlib linkage, LLVM scalar/out-pointer calls and matching buffer release. Managed changes include canonical C typed lifecycle callbacks, nested rows, COW aliases, cursors/early release, duplicate Map keys, split descriptor identity and Bytes conversion/slice. Build the offline runtime-only archive explicitly in `target/native-runtime-abi`; do not recursively build it from tests or silently use an embedded-compiler archive. Opt-in `llvm_managed_native` and `llvm_result_native` reuse the shared runtime-linked runner to verify real checked-source execution and repeated allocation balance; this is not a general memory profiler or capacity test. Typed lowering/bridge tests and target-pack limitations are specified in `compiler-contracts.md`.
Runtime ABI changes select `native_runtime_abi` and opt-in `native_runtime_link`: exact Decimal word round-trip, checked errors/IEEE floats, borrowed UTF-8/Bytes including null-empty and invalid input, real C-header/staticlib linkage, LLVM scalar/out-pointer calls and matching buffer release. Managed changes include canonical C typed lifecycle callbacks, nested rows, COW aliases, cursors/early release, duplicate Map keys, split descriptor identity and Bytes conversion/slice. Build the offline runtime-only archive explicitly, then set `DEVER_RUNTIME_ABI_ARCHIVE` to its absolute path for `native_runtime_link`; release checks must select the actual candidate archive rather than an unrelated fixed-path cache. Do not recursively build it from tests or silently use an embedded-compiler archive. Opt-in `llvm_managed_native` and `llvm_result_native` reuse the shared runtime-linked runner to verify real checked-source execution and repeated allocation balance; this is not a general memory profiler or capacity test. Typed lowering/bridge tests and target-pack limitations are specified in `compiler-contracts.md`.
File/Stream/coroutine ABI changes additionally select `llvm_resource_native`, `llvm_async_native` and `llvm_async_source`. Keep hand-written LLVM/C ABI smoke separate from checked-source lowering evidence; execute both against the actual archive. Assert all initial/pending/final destroy paths, actual Pending/backpressure, typed error transfer, sync input borrowing, non-null creation consumption, nullable row/End, Scope drain and repeat allocation balance. Use an atomic allocation ledger for concurrent workers. Do not run broad database/service/performance checks or claim complete application/target-pack acceptance from these tests.
@@ -402,11 +402,11 @@ LLVM external validation selects `llvm_external_source`, the original `external_
Runtime config owns the shared Job bounds: timeout <= 3,600,000 ms and lease in 100..7,200,000 ms. Tests use the maximum lease so every legal timeout fits SQLite without bypassing production policy validation.
Ordinary Linux commands use `workers/command.rs` with the shared `workers/elf.rs` dependency-closure checker. Prepare the domain-owned ELF and flat adjacent `lib/` from either the project or locked Package; never scan host libraries or require a Lib lock for command-only projects. Validate target, GNU interpreter, recursive DT_NEEDED, SONAME, symbol versions and loader hooks. All command RPATH/RUNPATH expansions must remain inside the logical `program` tree, which maps to `/worker`; accepting an escape such as `$ORIGIN/../data/grant` would allow a file grant to replace a validated dependency at execution time. Private libraries cannot shadow signed sandbox libraries. Dynamic launch uses the signed target loader with `--inhibit-cache --library-path lib:/lib`; user argv is appended literally after mapping only compiler-owned resource arguments. Commands use a distinct `dever-command-launch-v1` manifest and `commands/` tree under the existing verified bundle/extraction owner.
Ordinary Linux commands use `workers/command.rs` with the shared `workers/elf.rs` dependency-closure checker. Prepare the domain-owned ELF and flat adjacent `lib/` from the project or locked Package; never scan host libraries or require a Lib lock for command-only projects. Validate target, GNU interpreter, recursive DT_NEEDED, SONAME, symbol versions and loader hooks. RPATH/RUNPATH must remain in the packaged program tree and private libraries cannot shadow signed process runtime libraries. The launch manifest names the actual executable; `dever-process::Launch` executes it directly through the kernel, with private Worker/runtime library paths plus inherited driver library paths. The deployment container supplies the matching Linux/glibc OS ABI. Do not launch language executables through the GNU loader: Node/Rust self-executable discovery and subprocess spawning would resolve the loader instead. Signed process-runtime libraries remain build-time target dependency inputs; do not inject their old libc under a different container loader. User argv is appended literally, not interpreted as shell syntax. Commands use `dever-command-launch-v1` and the `commands/` tree under the existing verified bundle owner.
Command limits are `adapter.<port>.command.timeout_ms` (default 30000, 1..3600000) and `output_limit` (default 1048576, 1..8388608 combined stdout/stderr); stdin is bounded at 8388608 bytes and argv at 65536 including NUL terminators, with embedded NUL rejected. A call owns one process, closes absent stdin, drains both output pipes concurrently and never replays. Nonzero program exit is output; failure before target exec must remain a runtime fault and cannot be inferred from stderr or a reserved exit code. Invocation shutdown and caller cancellation must wait for actual sandbox completion. Queued cancellation has no child to drain and must not wait behind an unrelated active call. Ordinary application tests remain fake-only. Focused evidence belongs to `external_command`, command cases in `external_workers`/`llvm_external_source`, and the opt-in signed native-release command acceptance; physical ARM execution must not be inferred from cross-target metadata tests.
Command limits are `adapter.<port>.command.timeout_ms` (default 30000, 1..3600000) and `output_limit` (default 1048576, 1..8388608 combined stdout/stderr); stdin is bounded at 8388608 bytes and argv at 65536 including NUL terminators, with embedded NUL rejected. A call owns one process, closes absent stdin, drains both output pipes concurrently and never replays. Invalid resources and OS spawn errors are runtime faults. A successfully started executable or loader returns its actual exit status; do not infer a special failure from stderr or a reserved exit code. Invocation shutdown and caller cancellation terminate the owned process group and await the leader; this is not containment for detached sessions. Queued cancellation has no child to drain and must not wait behind an unrelated active call. Ordinary application tests remain fake-only. Focused evidence belongs to `external_command`, command cases in `external_workers`/`llvm_external_source`, and the opt-in signed native-release command acceptance; physical ARM execution must not be inferred from cross-target metadata tests.
Command execution observes guard exec through a private CLOEXEC pipe. Bind the PID namespace init through bwrap `--info-fd`, keep it blocked with `--block-fd` until pidfd acquisition and PPid verification, and wait for that pinned pidfd before releasing caller-scope completion. Pipe EOF (`--sync-fd`) is not sufficient: Linux closes init descriptors before tearing down descendants. Cleanup always reaps the owned launcher even if namespace observation fails, retaining both errors. Old Worker launches keep their existing handshake path.
Command and Worker cleanup use the same pidfd-pinned process-group owner. Kill the group before reaping its leader so a reused PID cannot target unrelated processes. Test timeout, cancellation, explicit shutdown, root-scope close and successful leader exit with a descendant still holding output pipes. Invalid executable input must fail before any invocation; static checks and ignored fixtures are not execution evidence.
## 7. Wrong vs Correct
Generated
+15 -57
View File
@@ -161,12 +161,6 @@ version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "cfg_aliases"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f079e83a288787bcd14a6aea84cee5c87a67c5a3e660c30f557a3d24761b3527"
[[package]]
name = "chacha20"
version = "0.10.2"
@@ -193,16 +187,6 @@ version = "0.5.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c9ea0ac24bc397ab3c98583a3c9ba74fa56b09a4449bbe172b9b1ddb016027a"
[[package]]
name = "command-fds"
version = "0.3.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1b60b5124979fccd9addd89d8b97a1d6eebb4950694520c75ddd722535ea443f"
dependencies = [
"nix",
"thiserror 2.0.20",
]
[[package]]
name = "const-oid"
version = "0.9.6"
@@ -430,7 +414,7 @@ dependencies = [
[[package]]
name = "dever-backend-bridge"
version = "0.1.4"
version = "0.1.5"
dependencies = [
"cc",
"dever-core",
@@ -443,14 +427,14 @@ dependencies = [
[[package]]
name = "dever-cli"
version = "0.1.4"
version = "0.1.5"
dependencies = [
"base64 0.22.1",
"csv",
"dever-backend-bridge",
"dever-core",
"dever-process",
"dever-runtime",
"dever-sandbox",
"flate2",
"fs2",
"goblin",
@@ -471,7 +455,7 @@ dependencies = [
[[package]]
name = "dever-core"
version = "0.1.4"
version = "0.1.5"
dependencies = [
"dever-runtime",
"pulldown-cmark",
@@ -479,9 +463,17 @@ dependencies = [
"sha2 0.10.9",
]
[[package]]
name = "dever-process"
version = "0.1.5"
dependencies = [
"goblin",
"rustix",
]
[[package]]
name = "dever-runtime"
version = "0.1.4"
version = "0.1.5"
dependencies = [
"argon2",
"base64 0.22.1",
@@ -490,7 +482,7 @@ dependencies = [
"deadpool-postgres",
"deadpool-sqlite",
"dec",
"dever-sandbox",
"dever-process",
"futures-util",
"getrandom 0.2.17",
"hmac 0.12.1",
@@ -512,22 +504,9 @@ dependencies = [
"zeroize",
]
[[package]]
name = "dever-sandbox"
version = "0.1.4"
dependencies = [
"command-fds",
"goblin",
"libc",
"nix",
"rustix",
"seccompiler",
"sha2 0.10.9",
]
[[package]]
name = "dever-tests"
version = "0.1.4"
version = "0.1.5"
dependencies = [
"bytes",
"dever-core",
@@ -1189,18 +1168,6 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "nix"
version = "0.31.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf20d2fde8ff38632c426f1165ed7436270b44f199fc55284c38276f9db47c3d"
dependencies = [
"bitflags",
"cfg-if",
"cfg_aliases",
"libc",
]
[[package]]
name = "node-semver"
version = "2.2.0"
@@ -1667,15 +1634,6 @@ dependencies = [
"syn 3.0.4",
]
[[package]]
name = "seccompiler"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a4ae55de56877481d112a559bbc12667635fdaf5e005712fd4e2b2fa50ffc884"
dependencies = [
"libc",
]
[[package]]
name = "security-framework"
version = "3.7.0"
+2 -2
View File
@@ -1,9 +1,9 @@
[workspace]
members = ["crates/dever-core", "crates/dever-runtime", "crates/dever-cli", "crates/dever-backend-bridge", "crates/dever-sandbox", "test/dever-tests"]
members = ["crates/dever-core", "crates/dever-runtime", "crates/dever-cli", "crates/dever-backend-bridge", "crates/dever-process", "test/dever-tests"]
resolver = "3"
[workspace.package]
version = "0.1.4"
version = "0.1.5"
edition = "2024"
rust-version = "1.95"
publish = false
+5 -5
View File
@@ -433,7 +433,7 @@ TLS 使用 tokio-rustls 0.26.5、rustls 0.23.44 的 ring 后端和 webpki-roots
模块 API 的站点归属由检查器把 API 物理父目录与 `setting.json` 的 `sites.path` 做分段前缀匹配,文件名不参与匹配,站点也不改写 URL。配置引用的 verify 必须解析为一个只读 App 根;它和其数据库读依赖进入生产可达图及原生初始化,不能在运行时按字符串反射调用。
原生路由先由 runtime 校验站点绑定的 Bearer 或 Secret Cookie,再把结构化 Claims 交给该静态 verify。verify 输出的 App-owned record 经生成桥接转换为 runtime Identity,随后使用 Tokio task-local 包住真实 handler。源码只能通过 `dever.auth`、`dever.site` 读取复制值,不能构造或保存上下文本体。JWT 固定 HS256,并校验 algorithm、issuer/provider、audience/site、时间边界及签名;public 路由忽略浏览器自动附带的 Cookie,但显式错误 Authorization 仍拒绝。Cookie 写入和 Cookie 认证的非安全方法要求 Origin 与 Host 精确匹配站点配置的外部 HTTP/HTTPS origin,不根据内部监听协议推断,也不信任转发 Header。响应 Header/Cookie 先暂存,只在 handler、wire 编码与事务提交全部成功后发布。含密码 hash/verify 的 HTTP 调用链不生成覆盖整个 handler 的事务;检查器要求全部写入位于显式短 transaction 中,并禁止 transaction 到达密码计算,避免 SQLite writer 锁和 WAL 读快照升级错误。
原生路由先由 runtime 校验站点绑定的 Bearer 或 Secret Cookie,再把结构化 Claims 交给该静态 verify。verify 输出的 App-owned record 经生成桥接转换为 runtime Identity,随后使用 Tokio task-local 包住真实 handler。源码只能通过 `dever.auth`、`dever.site` 读取复制值,不能构造或保存上下文本体。JWT 固定 HS256,并校验 algorithm、issuer/provider、audience/site、时间边界及签名;public 路由忽略浏览器自动附带的 Cookie,但显式错误 Authorization 仍拒绝。Cookie 写入和 Cookie 认证的非安全方法要求 Origin 与 Host 精确匹配站点配置的外部 HTTP/HTTPS origin,不根据内部监听协议推断,也不信任转发 Header。响应 Header/Cookie 先暂存,只在 handler、wire 编码与事务提交全部成功后发布。普通写 API/CMD/Job 恢复入口级事务;显式 `long` 入口不生成外层事务,其 Model 写入及 Job 入队必须位于 transaction 函数中。两种事务都禁止外部 Worker、密码计算、文件/网络 I/O、等待和并发。入口策略由源码声明固定,检查器不推断切换;`public long` 仍只由 public 控制匿名访问。
权限目录由编译后的 API 自动生成并同步到核心 RBAC 表,受保护入口按站点、租户、用户和角色校验精确权限,不由 verify 返回权限字符串,也不要求逐个调用 `auth.require`。verify 只提供经验证的用户/租户/会话身份,public 声明显式允许匿名;业务数据所有权是独立边界。官方 `dever.api.Error` 由后端按确切类型与变体映射为固定响应,未知业务失败只记录调用元数据并返回脱敏 500。日志在 runtime 追加站点和已验证身份字段,不记录 token、Cookie 或 JWT payload。
@@ -441,7 +441,7 @@ TLS 使用 tokio-rustls 0.26.5、rustls 0.23.44 的 ring 后端和 webpki-roots
runtime `component` 是唯一 Worker owner。它从当前可执行程序目录解析编译器规范化的相对 entry,以清空环境、固定 `--dever-component` 参数和专用管道直接启动子进程,不使用 shell 或 `PATH`。协议帧为四字节大端长度加 UTF-8 JSON,继续使用 16 MiB、64 层、65536 值以及重复字段拒绝规则。`hello/ready` 绑定版本、Port、Adapter、schema hash、操作、capability 和 setting;随后使用严格递增 request id 的 `call/result/error/cancel`,并以 `health` admission 和 `shutdown` 收口。业务 `error` 必须属于 Port 已声明 variant;协议、超时、崩溃和非法 payload 都是运行 fault,不伪造业务错误,也不自动重放请求。
每个 Worker 的有界命令队列、进程、stderr drain 和 supervisor 都注册到现有结构化 Scope。超时或调用任务取消会发送 `cancel`,随后必须消费同一 id 的一个 `result` 或 `error` 才能继续;错误 id、未知 kind、截断/超限 frame 会终止该 Worker 并关闭 pending 调用。崩溃后的下一次新调用可以在固定启动频率内重建,但失败中的调用不重放。生成入口在应用成功、业务失败和运行 fault 后都走同一 cleanup,先完成 Component shutdown,再关闭数据库,并保留主错误及 cleanup cause。allow 集合参与编译期 effect 和握手精确匹配;Linux 启动还统一进入 `dever-sandbox` 的 namespace/seccomp/只读运行树与显式文件授权边界。GPU、其他 OS 和不允许 namespace 的宿主明确失败,不绕过隔离。
每个 Worker 的有界命令队列、进程、stderr drain 和 supervisor 都注册到现有结构化 Scope。当前源码改为 `dever-process` 直接执行与 pidfd/进程组回收,不再建立内层 namespace/seccomp/AppArmor 沙箱;权限继承容器,allow 只保留编译及握手效果契约。超时或调用任务取消会发送 `cancel`,随后必须消费同一 id 的一个 `result` 或 `error` 才能继续;错误 id、未知 kind、截断/超限 frame 会终止该 Worker 并关闭 pending 调用。崩溃后的下一次新调用可以在固定启动频率内重建,但失败中的调用不重放。生成入口在应用成功、业务失败和运行 fault 后都走同一 cleanup,先完成 Component shutdown,再关闭数据库,并保留主错误及 cleanup cause。allow 集合参与编译期 effect 和握手精确匹配;Linux 启动还统一进入 `dever-process` 的 namespace/seccomp/只读运行树与显式文件授权边界。Linux NVIDIA GPU 由 `adapter.<port>.gpu` 显式授权具体设备和只读驱动文件;guard 在执行 Worker 前核对设备 inode/device/rdev,驱动挂载采用原有 descriptor pinning。没有声明、配置、父容器权限或 namespace 支持时明确失败。其他 OS 仍不支持,不绕过隔离。C ABI SDK 与 exec 原生动态库闭包打包复用同一 Worker owner,未新增 Rust unsafe 或进程内用户 FFI。
External Lib 的项目边界由 `dever-cli::libs` 统一拥有:`config/setting.json` 根 `lib` 数组保存显式请求,canonical v5 `dever.lock` 保存精确版本、传递依赖、runtime/schema/artifact 摘要、源码构建收据和每个 checked Worker 的绑定。每个 Worker 独立校验环境闭包与版本冲突,打包精确比对整份 provider 元数据,不允许锁中多写未嵌入资源。`dever lib add/update` 才能解析,`run/build` 只离线消费锁。全部 exec 候选与锁资源形成嵌入清单,bundle identity 包含路径、SHA-256、长度、可执行位;runtime 原子提取到私有 `data/cache/lib/<digest>`,每次 Worker 启动重新核对清单、字节与 Unix 所有权/权限,不扫描别的 bundle。
@@ -467,7 +467,7 @@ File 的 create_new/open/read/write/close 与共享别名、同步 Stream 的共
`llvm::emit_application` 已接通独立 CMD 与 HTTP/REST 应用入口。CMD 保持完整名称和单个 JSON 对象输入;混合 HTTP/CMD 项目执行 CMD 时不绑定监听器。`llvm/api.rs`、`api_context.rs`、`rest.rs` 与 `application_commands.rs` 生成静态路由、具体输入/输出、Claims/Identity 及权限描述符,复用已有 JWT/业务 verify、站点、RBAC、database-only 租户、Cookie/Header、Upload 与日志实现,不另建协议栈或向业务传入动态 Value。非 public 路由缺少认证或精确权限时,源码与私有 ABI 两处均拒绝;权限 key 仍不含 HTTP method。
完整应用和可达日志的 CMD 共用 invocation-owned `application::Session`,只读取一次 `config/setting.json`。数据库、迁移临时池、租户 manager 和 lifecycle 均由该 owner 持有;子任务先排空,再在同一 runtime 中关闭连接池、刷新日志。POST/PUT/DELETE 输入解码后开始事务,输出编码后提交,成功才发布响应 metadata。租户管理入口沿用 migrate/owner/component 合同,缺少 ready marker、租户身份或启用组件不能回退平台库。Upload 经 Port/Adapter 消费到存储,部分解码失败清理 affine owner。私有 CLI 和 Linux managed daemon 编译已接 LLVM;签名发行包提供 Linux x86_64 编译资源,Linux ARM64 应用资源通过显式 target 扩展准备。
完整应用和可达日志的 CMD 共用 invocation-owned `application::Session`,只读取一次 `config/setting.json`。数据库、迁移临时池、租户 manager 和 lifecycle 均由该 owner 持有;子任务先排空,再在同一 runtime 中关闭连接池、刷新日志。普通写 API/CMD/Job 持有入口级事务,响应编码成功后提交;`long` 入口中的最外层 transaction 正常返回即提交,后续阶段或响应编码失败不撤销此前提交。自动 REST 保留自己短事务;响应 metadata 仍仅在 handler 与 wire 编码成功后发布。租户管理入口沿用 migrate/owner/component 合同,缺少 ready marker、租户身份或启用组件不能回退平台库。Upload 经 Port/Adapter 消费到存储,部分解码失败清理 affine owner。私有 CLI 和 Linux managed daemon 编译已接 LLVM;签名发行包提供 Linux x86_64 编译资源,Linux ARM64 应用资源通过显式 target 扩展准备。
`llvm/external.rs` 接通 checked external Adapter;选中的 Setting 成功解码后才启动 Worker,具名输入/输出和声明业务错误复用 typed wire codec。缺失可空输出字段、额外字段、错类型和未声明错误均为运行故障。`llvm::emit_application_with_resources` 消费原打包器的 `EmbeddedResource`,与 Rust backend 共用摘要/去重,运行时复用安全提取和受管 launch;不查找宿主解释器。`runtime-external` 与编译器 SDK 分离。`component::Session` 由每次 task 根持有,传播到子任务、blocking 和 API;supervisor 挂在根 Scope,短命调用者退出不关闭共享 Worker。根退出先停止接收和关闭 Worker,再排空 Scope,typed 原始错误保留并附加清理原因。显式 shutdown 被取消时仍保留 Task owner,根可继续排空;普通 Test 仍只有 fake。
@@ -477,7 +477,7 @@ File 的 create_new/open/read/write/close 与共享别名、同步 Stream 的共
`llvm/database.rs`、`database_rows.rs`、`orm*.rs`、`database_schema.rs` 和 `transaction.rs` 接通 Model;API 应用通过 Session 同时支持平台库与租户物理库。SQL/DDL 与关联查询复用 `native/orm.rs` 的纯构造函数,不生成 Rust、不复制数据库驱动。持久化固定标量在 ABI 边界转换为既有 `orm::Value`,业务记录和错误仍保持具体类型。Row/Rows/RowStream/Related 是不透明所有者;ToOne 保留可空关联,ToMany 按 998 个父键分块并使用既有 per-parent 限额 SQL。启动按实际数据库分组先迁移全部 Model,再建立外键;数据迁移与 Seed 复用原历史校验、事务和顺序规则。
`database::Session` 持有本次入口的 Settings 和连接池,复用原配置校验和驱动初始化,不写入全局 OnceLock。`task::run_entry_with_typed_cleanup` 先排空 root Scope,再在同一个 Tokio runtime 内停止所有连接池、等待驱动关闭,最后销毁 runtime。需要租户的应用使用上述 application Session 与 tenant owner,不能回退到平台库。写入 CMD 在输入解码后开事务,普通调用和顺序 handler 继承隐藏事务,Task/parallel 回调使用 detached 上下文;嵌套 transaction 复用外层、不制造 savepoint。输出编码成功后才提交,提交后才发布 stdout;失败/取消清理 typed 输出并回滚或丢弃状态未知的连接。rollback 保留原始 fault,可附加 cleanup cause;DbError 十个变体使用静态 typed 回调,不按消息猜类别。PostgreSQL 的 SQL 约束失败会使当前事务失败,不能承诺捕获后继续使用该事务;NotFound 等非 SQL 失败仍可正常捕获。
`database::Session` 持有本次入口的 Settings 和连接池,复用原配置校验和驱动初始化,不写入全局 OnceLock。`task::run_entry_with_typed_cleanup` 先排空 root Scope,再在同一个 Tokio runtime 内停止所有连接池、等待驱动关闭,最后销毁 runtime。需要租户的应用使用上述 application Session 与 tenant owner,不能回退到平台库。普通写 API/CMD/Job 建立入口事务上下文;`long` 入口初始上下文为空,由显式 transaction 建立局部事务。普通调用继承上下文,嵌套 transaction 复用外层、不制造 savepoint;只有拥有事务的最外层边界提交。失败或取消清理当前未提交事务,保留原始 fault 和 cleanup cause。编码成功后才发布 stdout;普通入口编码失败回滚,`long` 的编码失败不撤销此前提交的阶段。DbError 十个变体使用静态 typed 回调,不按消息猜类别。PostgreSQL 的 SQL 约束失败会使当前事务失败,不能承诺捕获后继续使用该事务;NotFound 等非 SQL 失败仍可正常捕获。
`llvm/failures.rs` 接通业务 `fail`、默认传播和 `result(named_call)`,直接复用 `specialize::failures` 与 `capture::targets`。fault 原前五字段不变,末尾追加 nominal type(编译期 ID+1,0 表示无载荷)、variant 与按可达错误类型最大布局对齐的内联 Choice;Business code 为 4。载荷用具体类型 store/load 和静态 drop,不是动态 Value 或序列化消息。捕获支持零/单/多输出、直接 error 与一层 error-only Choice 包装;显式 raised outer variant 保留身份。匹配后移交载荷并清空 fault,未匹配的数值/RuntimeAbi 故障继续追加调用帧传播。BytesToText/BytesFromInts/BytesSlice 复用既有 C ABI:status 1 的原始消息转换为标准 Failed(Text),status 2 保持 RuntimeAbi fault;成功句柄与错误 Buffer 各自准确转交/释放。真实执行覆盖捕获后重失败、非零 variant 根释放、受管 COW 与精确源调用链。
@@ -491,7 +491,7 @@ Linux `deverd` 已接可信编译 IPC。managed core 提交逻辑源码、无秘
原生发行包作者入口 `sdk/native-release.rs` 从独立作者根的 `config/setting.json` 读取显式输入及摘要,复用既有 runtime manifest、验证器和 Ed25519 发行协议。制作器复制并复核四 profile、CRT 和核心/LLVM 文件,在私有 staging 中签名,再以不覆盖方式原子发布;私钥不进入产物。`runtime-pack` Cargo profile 为 base/sqlite/postgres/both 分别生成保留链接符号的优化 archive,共享一个构建缓存。相同输入和密钥的重复打包一致,不等同于跨机器源码构建一致。
作者配置的可选 `runtimes` 已接 Python/Node/Go 完整运行/构建输入,制作器与 Worker 共用 manifest 和文件闭包校验。排序且固定元数据的 gzip/tar、registry 描述统一进入顶层发行签名;Go SDK、Adapter 和依赖的编译共用 `-trimpath`,避免随机临时目录改变 Worker 与共享缓存身份。LLVM 的二进制资源附件复用既有摘要去重,在 verify 前初始化常量,IR 仍限8 MiB;附件最多4096项、单项128 MiB/总256 MiB,共享编译请求384 MiB、产物320 MiB,不透明 artifact 仍64 MiB。Linux x86_64 三生态签名安装、受管 `check/run/build`、三条共享缓存及移除源码/机器安装后无系统语言环境的独立执行已通过,包含Python/Node源码构建原生依赖与离线收据重放。Linux签名首装、旧签名模板升级、篡改拒绝、双项目及真实双UID共享核心亦已通过自有完整验收。Linux root 沙箱、真实第三方高级依赖、双 CMS/PG 和 LLVM 容量报告已补齐;受限 AppArmor 非 root 入口和定向 CI 质量门也已完成。公开 Linux x86_64 v0.1.3 发行包的匿名下载、在线首装和更新已验收;Linux ARM64 应用交叉构建已支持,其他宿主平台的产品代码及完整发行资产仍待适配。长时间压力测试不能由这些定向结果替代。
作者配置的可选 `runtimes` 已接 Python/Node/Go 完整运行/构建输入,制作器与 Worker 共用 manifest 和文件闭包校验。排序且固定元数据的 gzip/tar、registry 描述统一进入顶层发行签名;Go SDK、Adapter 和依赖的编译共用 `-trimpath`,避免随机临时目录改变 Worker 与共享缓存身份。LLVM 的二进制资源附件复用既有摘要去重,在 verify 前初始化常量,IR 仍限8 MiB;附件最多4096项、单项128 MiB/总256 MiB,共享编译请求384 MiB、产物320 MiB,不透明 artifact 仍64 MiB。Linux x86_64 三生态签名安装、受管 `check/run/build`、三条共享缓存及移除源码/机器安装后无系统语言环境的独立执行已通过,包含Python/Node源码构建原生依赖与离线收据重放。Linux签名首装、旧签名模板升级、篡改拒绝、双项目及真实双UID共享核心亦已通过自有完整验收。历史发行曾验证 Linux root 沙箱及受限 AppArmor 非 root 入口;当前源码已移除该执行机制,历史沙箱结果不能作为新版直执行链的验收。真实第三方高级依赖、双 CMS/PG 和 LLVM 容量报告属于原有验收范围。公开 Linux x86_64 v0.1.3 发行包的匿名下载、在线首装和更新已验收;Linux ARM64 应用交叉构建已支持,其他宿主平台的产品代码及完整发行资产仍待适配。长时间压力测试不能由这些定向结果替代。
database 租户模式把 verify 返回的内部 tenant ModelId 放入同一 task-local 调用域。`global type` 继续使用平台连接,其他 Model、自动 REST 与 Job 通过 `StorageBinding` 选择已显式迁移且 fingerprint 匹配的租户物理库;缺少上下文或 ready marker 直接失败。runtime tenant manager 对 `(connection, tenant_id)` 池设置总量与 idle 回收,生成的 `tenant migrate` 入口按稳定顺序迁移全部 tenant Model/Job 后才写本地和 control marker;PostgreSQL 先通过控制库幂等建租户库。User Job 持久化 Provider/站点/session/租户声明和已通过的能力名,Worker 重新调用静态 verify 并使用最新权限复核,不能把历史任务降级为 System。table/field 模式尚未实现,不能在固定表 SQL 上只补一个配置分支。
+28 -9
View File
@@ -148,7 +148,7 @@ cmd version = app.version
command Port 只有一个操作,操作名由业务决定;输入固定为 `args: List<Text>`,可选第二项 `stdin: Bytes?`,输出固定为 `output: dever.process.Output`,错误为 `dever.process.Error`。只需 argv 时可省略整项 stdin 参数。Output 有 `code: Int`、`stdout: Bytes`、`stderr: Bytes`;二进制输出不做隐式文本转换,正常非零退出码也是结果。每次调用启动一次进程,不重试,不使用 shell;空格、引号、`$()` 等按参数原样传递。未提供 stdin 时立即发送 EOF。启动失败、超时和输出超限产生源位置 runtime fault。
调用限额只从 `config/setting.json` 读取:`{"adapter":{"media.tool":{"command":{"timeout_ms":30000,"output_limit":1048576}}}}`。示例值就是默认值;timeout_ms 最大 3600000,output_limit 为 stdout/stderr 合计字节数、最大 8388608,均必须为正数。argv 含终止符总计最多 65536 字节且不含 NUL,stdin 最多 8388608 字节。超时、取消和退出会回收本次沙箱进程树。需要文件、网络或子进程时仍在声明中写 `allow file/network/process` 并使用已有部署授权;command 不接受 `setting` 或 `lib` 声明。
调用限额只从 `config/setting.json` 读取:`{"adapter":{"media.tool":{"command":{"timeout_ms":30000,"output_limit":1048576}}}}`。示例值就是默认值;timeout_ms 最大 3600000,output_limit 为 stdout/stderr 合计字节数、最大 8388608,均必须为正数。argv 含终止符总计最多 65536 字节且不含 NUL,stdin 最多 8388608 字节。超时、取消和退出会回收本次受管进程组;不保证清理主动脱离进程组的守护程序。需要文件、网络或子进程时仍在声明中写 `allow file/network/process`,实际权限继承所在容器;command 不接受 `setting` 或 `lib` 声明。
`run/build` 离线嵌入 ELF 和相邻 `bin/lib/` 下显式随附的普通动态库文件;静态程序可直接使用,动态程序使用签名发行包的 GNU loader 和 OS 库。不扫描宿主库,不接受符号链接或 lib 子目录;缺库、错架构、SONAME/版本不匹配、越界搜索路径在构建期失败。Linux x86_64 与 ARM64 必须提供各自架构的二进制和库,Dever 不转换第三方二进制。单独使用 command 不需要 `dever lib` 或 `dever.lock`;资源可随 Dever Package 分发,Package 自身仍须锁定。测试继续用 Port fake,不执行真实命令。
@@ -165,7 +165,7 @@ external exec "worker/mail" {
}
```
entry 是所属领域目录内的可移植相对路径;上例构建时收集 `module/notification/mail/worker/mail`,将全部候选 exec Worker 的字节、SHA-256 和可执行位嵌入程序。入口原子提取到私有 `data/cache/lib/<bundle>`,每次启动 Worker 重新验证精确资源清单、摘要和 Unix 所有权/权限;不扫描其他 bundle,不依赖部署目录保留源码 Worker。路径不接受绝对形式、`.`、`..`、反斜杠或盘符,也不通过 shell、`PATH`、环境变量或外部命令字符串查找。capability 名称为 `network`、`file`、`process`、`gpu`;重复值和未知值在检查期拒绝。Worker 握手必须精确回显已编译的 Port、Adapter、schema、操作和 capability 集合。Linux 的文件、网络、进程权限还由统一 OS 沙箱执行;文件授权仅来自 `config/setting.json` 的 Adapter 配置。GPU 与其他 OS 沙箱尚不支持,明确失败。宿主必须允许所需 namespace 和 seccomp;不会自动放宽 AppArmor 等主机策略。
entry 是所属领域目录内的可移植相对路径;上例构建时收集 `module/notification/mail/worker/mail`,将全部候选 exec Worker 的字节、SHA-256 和可执行位嵌入程序。入口原子提取到私有 `data/cache/lib/<bundle>`,每次启动 Worker 重新验证精确资源清单、摘要和 Unix 所有权/权限;不扫描其他 bundle,不依赖部署目录保留源码 Worker。路径不接受绝对形式、`.`、`..`、反斜杠或盘符,也不通过 shell、`PATH`、环境变量或外部命令字符串查找。capability 名称为 `network`、`file`、`process`、`gpu`;重复值和未知值在检查期拒绝。Worker 握手必须精确回显已编译的 Port、Adapter、schema、操作和 capability 集合。Linux Worker 直接执行,不创建内层 OS 沙箱。`allow` 是编译期效果及协议声明,不限制操作系统访问;实际文件、网络、GPU 权限由所在容器的用户、挂载、网络和设备策略决定。旧 Adapter `files` / `gpu` 授权字段已删除,真实路径通过业务 setting 传入。GPU 继承父容器设备和必要驱动环境,不重挂载设备,不修改主机安全策略。仅执行可信代码;配置与受管 C ABI 用法见 `sdk/README.md`。
`external` 只在 Adapter role 中有意义,且一个文件只能包含该声明和至多一个 `setting`。它完整实现且只能实现同领域一个 Port identity;不能同时写 Dever Adapter 函数或 helper。应用测试仍必须提供 case-local Dever Port fake,不启动生产 Worker,也不读取生产 Adapter setting。reference 后端明确不执行外部程序,正式执行只走 native 后端。
@@ -188,7 +188,7 @@ Go Worker 使用锁定的目标专用 build pack 内的 compiler、linker 和标
显式 `dever lib add/update` 可用签名 build pack 构建 PEP 517 sdist、执行 npm registry 包的 `preinstall/install/postinstall` 和原生 addon 编译。build pack 必须提供目标运行时、头文件、编译器、sysroot、固定 shell 和工具闭包;缺少的工具或库明确报错,不寻找宿主环境。Python 的静态与动态 build requirements 使用独立精确锁,console scripts 安装到隔离构建 prefix;metadata 与 wheel hook 分进程执行,最终 wheel 必须匹配父进程保存的 metadata。npm publishing hooks 不作为 registry 安装钩子执行,可选依赖构建失败会裁剪其不可用闭包,必需依赖失败则整个准备失败。
源码构建收据绑定源码摘要、运行时、工具包、固定 frontend、构建依赖与输出摘要;npm 还绑定完整安装图。`run/build` 离线消费已锁定输出,不再次执行上述安装钩子或源码构建。Node addon 在仅有目标 Node 和运行库的独立沙箱中逐个验证;所有 `.node` 文件必须有成功或明确拒绝记录,运行树只安装成功项,其他平台/ABI 的变体保留在原锁定归档中。本次源码生成的 addon 验证失败会使准备失败,不能改用纯 JavaScript 回退并声称原生构建成功。Linux 构建使用只读工具与输入、独占可写工作区并禁止网络。构建期限为五分钟,stdout/stderr 各限 16 MiB;工作区每 250 ms 检查 512 MiB/65,536 项预算并在超限时回收子进程,这不是内核磁盘 quota,检查间隔内可暂时超限。最终安装树仍限 256 MiB/65,536 项。普通 Worker 的能力由同一 OS 沙箱执行,其他平台不能借宿主工具绕过缺失的发行支持。
源码构建收据绑定源码摘要、运行时、工具包、固定 frontend、构建依赖与输出摘要;npm 还绑定完整安装图。`run/build` 离线消费已锁定输出,不再次执行上述安装钩子或源码构建。Node addon 在仅有目标 Node 和运行库的独立进程中逐个验证;所有 `.node` 文件必须有成功或明确拒绝记录,运行树只安装成功项,其他平台/ABI 的变体保留在原锁定归档中。本次源码生成的 addon 验证失败会使准备失败,不能改用纯 JavaScript 回退并声称原生构建成功。Linux 构建使用明确的签名工具与输入、独立临时工作区;构建钩子直接继承容器权限,不再宣称文件/网络隔离。构建期限为五分钟,stdout/stderr 各限 16 MiB;工作区每 250 ms 检查 512 MiB/65,536 项预算并在超限时回收子进程,这不是内核磁盘 quota,检查间隔内可暂时超限。最终安装树仍限 256 MiB/65,536 项。普通 Worker 由同一进程监管执行;其他平台不能借宿主工具绕过缺失的发行支持。
Dever Package 的请求放在 `config/setting.json.package`:`registry` 是显式 HTTPS registry origin,`use` 是 `name@version-range` 数组。`dever package add/update` 解析版本和传递依赖;Package 与 Lib 共用 `dever.lock` 和机器摘要缓存。Package 的源码归属于 `module/<name>/`,Worker 资源归属于同组件路径或 `worker/<name>/`;其他组件路径不能混入归档,也不能与项目本地组件重名。`check/test/run/build` 自动加载锁定源码,无须增加 import 或注册方法;缺失的锁定 Worker 必须失败,不能使用项目同名文件替代。`remove` 离线裁剪已验证锁,不重新选择剩余依赖版本。
@@ -1054,11 +1054,11 @@ public post greeting = app.greet
cmd greeting = app.greet
```
前两项对应 `GET`、`POST /user/account/greeting`;CMD 名称为 `user.account.greeting`,输入是单个 JSON 对象,输出同样使用 `{"code":0,"message":"ok","data":...}`。HTTP 声明默认要求认证,只有明确的登录、健康检查等单条 HTTP 声明可加 `public`;`public cmd` 和 `public rest` 非法。public 入口忽略浏览器自动携带的旧 Cookie,使过期会话不会阻止重新登录;客户端显式提交的无效 Authorization 仍返回 401,不能降级成匿名请求。`get/post/put/delete <action> = app.<function>` 只绑定本领域无歧义的 App 函数,继承其输入输出签名,输出名不必叫 `response`。GET 的完整调用链不能写数据库或入队 Job;普通写 HTTP/CMD/Job 调用链在入口建立事务。含 password hash/verify 的 HTTP 调用链不建立覆盖密码计算的外层事务,其写入必须全部进入显式短 `transaction`,且事务内禁止密码计算;短事务必须重查写入依赖的账户、租户或成员状态。API role 不写函数体或旧 `get_` 前缀。GET/DELETE 从 query 读取标量输入,POST/PUT 从 `Content-Type: application/json` 的对象正文读取;命令也读取严格 JSON 对象。缺少必填值、额外或重复字段均拒绝。输入输出使用同一静态 wire codec,当前不支持 Choice、Bytes 等类型。
前两项对应 `GET`、`POST /user/account/greeting`;CMD 名称为 `user.account.greeting`,输入是单个 JSON 对象,输出同样使用 `{"code":0,"message":"ok","data":...}`。HTTP 声明默认要求认证,只有明确的登录、健康检查等单条 HTTP 声明可加 `public`;`public cmd` 和 `public rest` 非法。public 入口忽略浏览器自动携带的旧 Cookie,使过期会话不会阻止重新登录;客户端显式提交的无效 Authorization 仍返回 401,不能降级成匿名请求。`get/post/put/delete <action> = app.<function>` 只绑定本领域无歧义的 App 函数,继承其输入输出签名,输出名不必叫 `response`。GET 的完整调用链不能写数据库或入队 Job。普通写 HTTP/CMD/Job 由入口建立整体事务;涉及外部 I/O、Worker、等待、并发或密码计算的写入口必须显式加 `long`,并将写库与入队放进 `transaction` 函数。`long` 不建立入口级事务,已提交阶段不因后续失败撤销;写入时须在事务内重查账户、租户或成员状态。编译器只验证声明,不根据调用图自动切换事务模式。API role 不写函数体或旧 `get_` 前缀。GET/DELETE 从 query 读取标量输入,POST/PUT 从 `Content-Type: application/json` 的对象正文读取;命令也读取严格 JSON 对象。缺少必填值、额外或重复字段均拒绝。输入输出使用同一静态 wire codec,当前不支持 Choice、Bytes 等类型。
简单领域使用 `api.dever`;复杂领域可同时使用 `api.dever` 与 `api/<scope>/<resource>.dever`,topic 依次进入 URL,例如 `module/user/account/api/admin/session.dever` 中的 `get status = app.status` 对应 `/user/account/admin/session/status`。`sites.<key>.path` 匹配这里的 API 父目录:`admin` 匹配 `api/admin/**`,不匹配同级文件 `api/admin.dever`,也不改写 URL。每条 HTTP API 必须且只能归属一个站点;目录遗漏、path 前缀重叠和同一路径同方法重复都会在运行前报错。`rest` 可为同领域唯一 Model 生成 GET 列表/详情、POST、PUT、DELETE;多 Model 时写 `rest model` 或 `rest model.<topic>` 消歧。列表只接受有界 `page`、`size` 和至多一个显式 `search` 条件,不开放任意查询。自动 REST 只序列化公开字段;`private owner user_id: ... = dever.auth.user_id()` 在创建时写入可信身份,并把同一 owner 条件放入全部读写 SQL。客户端不能提交 owner 或无 `input` 的服务端字段。需要不同资源策略或跨 Model 编排时仍使用 App,不把业务规则塞进 REST 声明。
HTTP 绑定的 App 调用链可用 `dever.api.request_id()`、`method()`、`path()`、`client_address()`、`header(name)`、`cookie(name)`、`secret_cookie(name)` 读取请求元数据,并用 `set_header(name, value)`、`set_cookie(name, value, options)`、`set_secret_cookie(name, secret, options)` 设置响应元数据;无需传递 Context 参数。`client_address()` 只返回直接 TCP peer 的地址,不信任转发 Header。`secure_cookie_options()` 提供 `Path=/`、`SameSite=Lax`、`Secure`、`HttpOnly` 的起点,业务可构造 `CookieOptions` 调整 Domain 和 Max-Age。Secret Cookie 使用 `v1.` 加无填充 base64url 存储原始 Secret 字节,读写不会把 Secret 转成业务 Text;非此格式的值读作缺席。敏感请求头不能经 `header` 读为 Text,Secret Cookie 必须使用 Secure 与 HttpOnly;响应 Header 不允许覆盖 Set-Cookie、Content-Type、Content-Length 等运行时字段或注入换行。Header/Cookie 先暂存在请求作用域,只有 handler 成功、响应序列化成功且数据库提交成功后才发布;失败和回滚会丢弃它们。Cookie 写请求和使用 Cookie 认证的非安全方法必须携带 Origin,并要求 Origin 与 Host 都精确匹配 `sites.<key>.origin` 配置的外部 HTTP/HTTPS scheme、规范化主机和有效端口;运行时不从内部明文监听协议猜外部 HTTPS,也不信任转发 Header。CMD/Job 调用链不能使用 HTTP 请求能力,脱离请求的并发任务也不能调用这些能力。
HTTP 绑定的 App 调用链可用 `dever.api.request_id()`、`method()`、`path()`、`client_address()`、`header(name)`、`cookie(name)`、`secret_cookie(name)` 读取请求元数据,并用 `set_header(name, value)`、`set_cookie(name, value, options)`、`set_secret_cookie(name, secret, options)` 设置响应元数据;无需传递 Context 参数。`client_address()` 只返回直接 TCP peer 的地址,不信任转发 Header。`secure_cookie_options()` 提供 `Path=/`、`SameSite=Lax`、`Secure`、`HttpOnly` 的起点,业务可构造 `CookieOptions` 调整 Domain 和 Max-Age。Secret Cookie 使用 `v1.` 加无填充 base64url 存储原始 Secret 字节,读写不会把 Secret 转成业务 Text;非此格式的值读作缺席。敏感请求头不能经 `header` 读为 Text,Secret Cookie 必须使用 Secure 与 HttpOnly;响应 Header 不允许覆盖 Set-Cookie、Content-Type、Content-Length 等运行时字段或注入换行。Header/Cookie 先暂存在请求作用域,只有 handler 成功并且响应序列化成功后才发布;失败会丢弃待发布元数据;普通入口回滚未提交事务,`long` 入口不会撤销先前已经提交的 App 事务。Cookie 写请求和使用 Cookie 认证的非安全方法必须携带 Origin,并要求 Origin 与 Host 都精确匹配 `sites.<key>.origin` 配置的外部 HTTP/HTTPS scheme、规范化主机和有效端口;运行时不从内部明文监听协议猜外部 HTTPS,也不信任转发 Header。CMD/Job 调用链不能使用 HTTP 请求能力,脱离请求的并发任务也不能调用这些能力。
POST App 可以直接接收一个顶层 `Upload` 及普通标量,此时请求必须是 `multipart/form-data`。Runtime 按 `http.upload` 的总大小、单文件、字段、part、header 和 filename 上限流式写入 `data/tmp`;filename 仅作展示值,不能成为路径。`Upload` 是 affine 请求资源,不能嵌套、复制、作为输出或进入 GET/CMD/helper;App 可读取 metadata 后显式 `close_upload`,或一次性传给同领域 Port,再由 Adapter 调用 `dever.storage.put`。未消费资源及解析失败、handler 失败和取消路径会清理临时文件。
@@ -1205,7 +1205,26 @@ transaction register(email: Text, display_name: Text) (account: model.Account) {
}
```
编译器沿完整调用图推导数据库 effect。最外层 transaction 成功时提交;error、fault 或取消时先回滚再传播原失败。同一连接的嵌套 transaction 复用外层事务。无数据库操作、跨连接,或在事务中通过 `run`、`parallel`、流式游标、Channel 启动可逃逸工作都会在构建前被拒绝。`blocking(call())` 可以在事务中使用,但仍须满足其目标非挂起、无数据库和无并发的既有检查,调用者会等待它完成。
编译器沿完整调用图推导数据库 effect。普通写 HTTP、CMD 和 Job 在入口建立整体事务,响应编码成功后才提交;同一入口内的显式 `transaction` 复用外层事务。`long` 入口不建立整体事务,业务写库和持久入队必须跨入显式 `transaction`;自动 REST 写操作保持原来的事务边界。最外层 transaction 成功时提交;error、fault 或取消时先回滚当前事务再传播原失败。同一连接的嵌套 transaction 复用外层事务。无数据库操作、跨连接,以及事务内的并发、流式游标、HTTP、文件传输、外部 Worker 和密码计算都会在构建前被拒绝。`blocking(call())` 仅在目标不包含这些操作并满足原有非挂起、无数据库约束时可用。事务外可以等待或编排多个独立事务;写入前需在事务内重新核验状态。`long` 中先前已经提交的事务不会因为后续阶段或响应序列化失败而回滚;重试的幂等性、失败状态和外部产物清理由业务定义,数据库回滚不撤销外部调用。
`long` 只修饰 HTTP、CMD 或 Job 入口,不修饰 App 函数、`transaction` 或 `rest`;与 `public` 同用时写 `public long`。声明形式如下(App 与请求类型按各自领域定义):
```dever
post create = app.create
long post generate = app.generate
long cmd generate = app.generate
public long post login = app.login
```
Job 保留原来的重试和超时声明:
```dever
long job generate(input: app.Request) () retry(3) timeout(30000) {
app.run(input)
}
```
`long` 表示入口不自动持有整体事务,不代表允许长事务、自动异步、自动延长超时或按运行秒数分类。普通自动事务中出现外部操作,或 `long` 调用链存在事务外写库,均在编译期报 C014;仅添加外部调用不能静默改变原子性。GET 即使声明 `long` 也不能写库或入队。
数据库错误默认传播,保留 `dever.database.Error` 的类别和消息:Pool、PoolExhausted、Connection、Timeout、Cancelled、Database、Constraint、NotFound、InvalidData、Migration。要主动捕获一个返回 Account 的 App 调用,可以声明下面的结果类型;`result(load_account(...))` 的成功载荷和完整错误集合必须与调用一致。
@@ -1429,7 +1448,7 @@ Dever 用分句、类型化失败和集合操作表达控制流程。以下写
- `if`/`else`、`switch`、`match` 和条件表达式;条件永久使用 function 分句。
- `for`、`while`、`break`、`continue` 和用户递归;重复使用集合/Stream 组合器。
- class、继承、interface、method、annotation、macro、unsafe 和 FFI。
- class、继承、interface、method、annotation、macro、unsafe 和进程内裸指针 FFI;受管 C ABI 通过隔离的 `external exec` Worker 提供,不新增业务源码中的指针类型。
- lambda、闭包、一般化一等 function、用户泛型 type/function。
- exception、`throw` 和 `catch`;可恢复错误使用推导式默认传播、`result(call)` 和 `fail(...)`。
@@ -1442,10 +1461,10 @@ Linux x86_64 已提供签名安装包、机器共享编译服务和配套 skill
- 通用动态 Json 值、Set 和 BigInt;持久化 JSON 使用 Model 的 `Json` 字段,普通 JSON 处理使用 `dever.json.Document`。
- CONNECT、非空 trailer、服务器推送、WebSocket-over-HTTP/2,以及 WebSocket 子协议、压缩扩展和关闭帧元数据。
- table/field 多租户隔离模式,以及按租户业务配置动态选择已编译 Adapter;当前只支持 database 隔离和进程级 Adapter 选择。
- GPU capability、macOS/Windows 外部程序沙箱和完整宿主发行;当前也不提供 ARM64 宿主编译器安装包。
- 跨 GPU 厂商的设备验收、macOS/Windows 受管外部程序和完整宿主发行;当前也不提供 ARM64 宿主编译器安装包。GPU 支持不包含驱动安装、设备调度、模型分发或自动放宽容器权限。
- 编译器自举。
未支持的能力明确失败,不使用默认值或关闭隔离来继续运行。具体安装与沙箱条件见 [安装指引](skills/references/install.md)。
未支持的能力明确失败,不使用默认值伪装支持。具体安装与执行条件见 [安装指引](skills/references/install.md)。
## 17. 开发建议
+1 -1
View File
@@ -12,7 +12,7 @@ workspace = true
[dependencies]
dever-backend-bridge = { path = "../dever-backend-bridge", features = ["embedded"] }
dever-core.workspace = true
dever-sandbox = { path = "../dever-sandbox" }
dever-process = { path = "../dever-process" }
dever-runtime = { workspace = true, features = ["database"] }
serde_json.workspace = true
serde.workspace = true
+3 -2
View File
@@ -117,11 +117,11 @@ fn test_profile(program: &Program) -> RuntimeProfile {
pub(super) fn worker_compile(
program: &Program,
sources: &SourceMap,
request: &CompileRequest,
request: &mut CompileRequest,
build: &Path,
output: &Path,
) -> Result<(), String> {
let resources = request.embedded_resources();
let resources = request.take_embedded_resources();
let (profile, ir) = match request.kind {
CompileKind::Application => {
let profile = program.validate_database_bindings(
@@ -209,6 +209,7 @@ fn link_ir(
.open(&path)
.and_then(|mut file| file.write_all(&object))
.map_err(|error| format!("cannot write application object: {error}"))?;
drop(object);
let snapshot = pack.snapshot(build)?;
let inputs = snapshot
.start
+15 -14
View File
@@ -779,13 +779,13 @@ impl build::Inputs for InstalledRegistry {
}
fn assets(&self) -> Result<Vec<(String, Vec<u8>)>, String> {
self.sandbox_resources(crate::toolchain::BuildTarget::host()?)?
self.process_runtime_resources(crate::toolchain::BuildTarget::host()?)?
.into_iter()
.map(|resource| {
let path = resource
.path
.strip_prefix("sandbox/")
.ok_or("invalid sandbox resource path")?
.strip_prefix("process-runtime/")
.ok_or("invalid process_runtime resource path")?
.to_owned();
Ok((path, resource.bytes))
})
@@ -861,11 +861,11 @@ impl InstalledRegistry {
self.resources.read(name)
}
fn sandbox_resources(
fn process_runtime_resources(
&self,
target: crate::toolchain::BuildTarget,
) -> Result<Vec<dever_core::native::EmbeddedResource>, String> {
let prefix = format!("sandbox/{}/", target.platform());
let prefix = format!("process-runtime/{}/", target.platform());
let mut resources = Vec::new();
let mut files = Vec::new();
for artifact in self.resources.artifacts(&prefix)? {
@@ -875,13 +875,13 @@ impl InstalledRegistry {
let bytes = self.signed_artifact(Path::new(&artifact.path))?;
files.push((relative.to_owned(), bytes.clone()));
resources.push(dever_core::native::EmbeddedResource {
path: format!("sandbox/{relative}"),
path: format!("process-runtime/{relative}"),
sha256: artifact.sha256.clone(),
bytes,
executable: dever_sandbox::asset_executable(relative),
executable: dever_process::asset_executable(relative),
});
}
dever_sandbox::validate_assets_for_target(&files, target.platform())?;
dever_process::validate_assets_for_target(&files, target.platform())?;
Ok(resources)
}
}
@@ -1488,13 +1488,14 @@ pub fn prepare_resources(
let mut host_assets = Vec::new();
if program.has_external_adapters() {
let installed = InstalledRegistry::current(target)?;
resources.extend(installed.sandbox_resources(target)?);
resources.extend(installed.process_runtime_resources(target)?);
if program
.external_worker_contracts()
.iter()
.any(|worker| worker.ecosystem == "go")
{
host_assets = installed.sandbox_resources(crate::toolchain::BuildTarget::host()?)?;
host_assets =
installed.process_runtime_resources(crate::toolchain::BuildTarget::host()?)?;
}
}
let packaged = crate::workers::prepare_for_target(
@@ -1725,9 +1726,9 @@ pub fn doctor_with_go_verifier(lock: &LockFile, verifier: &sumdb::Verifier) -> R
.iter()
.any(|node| node.source.archive == lib.spec)
}))
&& !fixtures
&& fixtures
.package(&lib.spec)
.is_some_and(|package| package.locked() == *lib)
.is_none_or(|package| package.locked() != *lib)
{
return Err(
"npm archive requires an exact instance graph, not flat dependencies".into(),
@@ -1735,9 +1736,9 @@ pub fn doctor_with_go_verifier(lock: &LockFile, verifier: &sumdb::Verifier) -> R
}
if lib.spec.ecosystem == Ecosystem::Go
&& !go.contains(&lib.spec.name, &lib.spec.version)
&& !fixtures
&& fixtures
.package(&lib.spec)
.is_some_and(|package| package.locked() == *lib)
.is_none_or(|package| package.locked() != *lib)
{
return Err(format!(
"{} lacks authenticated Go sumdb evidence",
+24 -18
View File
@@ -1,27 +1,33 @@
// Fixed registry-package lifecycle frontend. Runs only in the build sandbox.
const fs = require('node:fs');
const path = require('node:path');
const runScript = require('/worker/npm/frontend/node_modules/npm/node_modules/@npmcli/run-script');
const binLinks = require('/worker/npm/frontend/node_modules/npm/node_modules/bin-links');
const runScript = require(path.join(__dirname, 'npm/frontend/node_modules/npm/node_modules/@npmcli/run-script'));
const binLinks = require(path.join(__dirname, 'npm/frontend/node_modules/npm/node_modules/bin-links'));
const shellQuote = value => "'" + value.replace(/'/g, "'\\''") + "'";
async function main() {
const request = JSON.parse(fs.readFileSync('/worker/request.json', 'utf8'));
const root = '/data/build/output/install';
fs.mkdirSync('/data/build/output');
fs.cpSync('/worker/install', root, { recursive: true, dereference: false, errorOnExist: true });
fs.mkdirSync('/data/build/tmp');
fs.mkdirSync('/data/build/home');
const work = fs.realpathSync(process.argv[2]);
const toolRoot = path.join(__dirname, 'rootfs');
const tools = path.join(toolRoot, 'usr/bin');
const request = JSON.parse(fs.readFileSync(path.join(__dirname, 'request.json'), 'utf8'));
const root = path.join(work, 'output/install');
fs.mkdirSync(path.join(work, 'output'));
fs.cpSync(path.join(__dirname, 'install'), root, { recursive: true, dereference: false, errorOnExist: true });
fs.mkdirSync(path.join(work, 'tmp'));
fs.mkdirSync(path.join(work, 'home'));
Object.assign(process.env, {
PATH: '/worker/runtime/bin:/usr/bin',
CC: '/usr/bin/cc', CXX: '/usr/bin/c++', AR: '/usr/bin/ar',
PATH: [path.join(__dirname, 'runtime/bin'), tools].join(path.delimiter),
CC: `${shellQuote(path.join(tools, 'cc'))} ${shellQuote('--sysroot=' + toolRoot)}`,
CXX: `${shellQuote(path.join(tools, 'c++'))} ${shellQuote('--sysroot=' + toolRoot)}`,
AR: path.join(tools, 'ar'),
PYTHON: request.python,
npm_config_python: request.python,
npm_config_nodedir: '/worker/npm',
npm_config_node_gyp: '/worker/npm/frontend/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js',
npm_config_nodedir: path.join(__dirname, 'npm'),
npm_config_node_gyp: path.join(__dirname, 'npm/frontend/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js'),
npm_config_build_from_source: 'true',
npm_config_cache: '/data/build/npm-cache',
TMPDIR: '/data/build/tmp', SOURCE_DATE_EPOCH: '0',
HOME: '/data/build/home',
npm_config_cache: path.join(work, 'npm-cache'),
TMPDIR: path.join(work, 'tmp'), SOURCE_DATE_EPOCH: '0',
HOME: path.join(work, 'home'),
});
const nodes = new Map(request.instances.map(node => [node.path, node]));
for (const node of request.instances) {
@@ -48,7 +54,7 @@ async function main() {
}
const directory = path.join(root, name);
for (const event of ['preinstall', 'install', 'postinstall']) {
await runScript({ event, path: directory, stdio: 'inherit', scriptShell: '/bin/sh' });
await runScript({ event, path: directory, stdio: 'inherit', scriptShell: path.join(tools, 'sh') });
}
const native = path.join(directory, 'build', 'Release');
if (fs.existsSync(native)) {
@@ -97,7 +103,7 @@ async function main() {
fs.rmSync(path.join(root, node.path, 'build', 'node_gyp_bins'), { recursive: true, force: true });
}
fs.rmSync(path.join(root, 'node_modules', '.bin'), { recursive: true, force: true });
fs.writeFileSync('/data/build/output/result.json', JSON.stringify({ failed: [...failed].sort(), native: [...nativeEntries].filter(file => retained.has(file.split('/build/Release/')[0])).sort() }));
fs.writeFileSync(path.join(work, 'output/result.json'), JSON.stringify({ failed: [...failed].sort(), native: [...nativeEntries].filter(file => retained.has(file.split('/build/Release/')[0])).sort() }));
}
main().catch(error => { process.stderr.write(String(error.stack || error) + '\n'); process.exitCode = 1; });
@@ -70,7 +70,7 @@ impl Environment {
descriptor.validate(runtime, Some(&python))?;
let (manifest, tools) = pack::unpack(&bytes, &descriptor)?;
let assets = inputs.assets()?;
dever_sandbox::validate_assets(&assets)?;
dever_process::validate_assets(&assets)?;
pack::validate_tools(&manifest, &tools, &assets)?;
let mut files = BTreeMap::new();
for (path, bytes) in node_files {
@@ -122,9 +122,9 @@ impl Environment {
}
for (path, bytes) in &self.assets {
stage.write(
&format!("sandbox/{path}"),
&format!("process-runtime/{path}"),
bytes,
dever_sandbox::asset_executable(path),
dever_process::asset_executable(path),
)?;
}
for (path, bytes) in files {
@@ -137,7 +137,7 @@ impl Environment {
let request = serde_json::json!({
"instances":graph.instances, "required":super::required_nodes(graph)?,
"roots":graph.roots.iter().map(|root|format!("node_modules/{}",root.name)).collect::<Vec<_>>(),
"python":format!("/worker/{}", self.python),
"python":input.join(&self.python).to_string_lossy(),
});
stage.write(
"input/request.json",
@@ -147,31 +147,20 @@ impl Environment {
stage.write("input/frontend.js", include_bytes!("../npm.js"), false)?;
let work = stage.0.join("work");
fs::create_dir(&work).map_err(|error| error.to_string())?;
let grants = [dever_sandbox::Grant {
source: work.clone(),
destination: "/data/build".into(),
writable: true,
}];
let arguments = [input.join("frontend.js").into_os_string()];
let arguments = [
input.join("frontend.js").into_os_string(),
work.clone().into_os_string(),
];
let executable = input.join(&self.node);
let assets = stage.0.join("sandbox");
let launch = dever_sandbox::Launch {
let assets = stage.0.join("process-runtime");
let launch = dever_process::Launch {
assets: &assets,
worker: &input,
executable: &executable,
arguments: &arguments,
working_directory: &input,
capabilities: dever_sandbox::Capabilities {
network: false,
process: true,
},
grants: &grants,
};
stage.run(
launch.command_for_build(&input.join("rootfs"))?,
"isolated npm lifecycle",
&work,
)?;
stage.run(launch.command()?, "managed npm lifecycle", &work)?;
inputs.diagnostic(&stage.diagnostics()?);
let mut outputs = super::super::python::output_files(&work.join("output"))?;
let response: ResultMetadata = serde_json::from_slice(
@@ -210,7 +199,7 @@ impl Environment {
rejections: BTreeMap::new(),
};
// Release the large compiler tree before staging the much smaller
// runtime-only native admission sandbox.
// runtime-only native admission process_runtime.
drop(stage);
self.admit_native(inputs, &mut output)?;
Ok(output)
@@ -244,7 +233,7 @@ impl Environment {
files.extend(
self.assets
.iter()
.map(|(path, bytes)| (format!("sandbox/{path}"), bytes.as_slice())),
.map(|(path, bytes)| (format!("process-runtime/{path}"), bytes.as_slice())),
);
files.extend(
output
@@ -254,21 +243,21 @@ impl Environment {
);
let stage = Stage::new()?;
for (path, bytes) in &files {
let relative = if path.starts_with("sandbox/") {
let relative = if path.starts_with("process-runtime/") {
path.clone()
} else {
format!("input/{path}")
};
let executable = path == &self.node
|| path
.strip_prefix("sandbox/")
.is_some_and(dever_sandbox::asset_executable);
.strip_prefix("process-runtime/")
.is_some_and(dever_process::asset_executable);
stage.write(&relative, bytes, executable)?;
}
stage.write("input/probe.cjs", include_bytes!("probe.cjs"), false)?;
let input = stage.0.join("input");
let executable = input.join(&self.node);
let assets = stage.0.join("sandbox");
let assets = stage.0.join("process-runtime");
for (path, bytes) in candidates {
let rejection = match goblin::elf::Elf::parse(bytes) {
Err(_) => Some(NativeRejection::Format),
@@ -293,14 +282,12 @@ impl Environment {
input.join("probe.cjs").into_os_string(),
input.join(path).into_os_string(),
];
let launch = dever_sandbox::Launch {
let launch = dever_process::Launch {
assets: &assets,
worker: &input,
executable: &executable,
arguments: &arguments,
working_directory: &input,
capabilities: dever_sandbox::Capabilities::default(),
grants: &[],
};
let admitted = stage.probe(launch.command()?)?;
inputs.diagnostic(&stage.diagnostics()?);
+1 -1
View File
@@ -156,7 +156,7 @@ pub fn validate_tools(
.map(|(path, bytes)| (path.clone(), bytes.as_slice()))
.collect::<BTreeMap<_, _>>();
for (path, bytes) in assets {
supplied.insert(format!("sandbox/{path}"), bytes);
supplied.insert(format!("process-runtime/{path}"), bytes);
}
crate::workers::python_wheel::native::validate_build_tools(
&manifest.target,
+48 -10
View File
@@ -51,8 +51,6 @@ impl Stage {
let path = self.0.join(relative);
fs::create_dir_all(path.parent().expect("validated relative path"))
.map_err(|error| format!("cannot create build input directory: {error}"))?;
// This tree is private and never a writable child mount. A second write
// must not silently replace a previously verified input.
use std::io::Write;
let mut file = fs::OpenOptions::new()
.write(true)
@@ -89,7 +87,7 @@ impl Stage {
pub(crate) fn probe(&self, command: Command) -> Result<bool, String> {
match self
.execute(command, "isolated Node addon probe", None)?
.execute(command, "managed Node addon probe", None)?
.code()
{
Some(0) => Ok(true),
@@ -107,8 +105,6 @@ impl Stage {
operation: &str,
writable: Option<&Path>,
) -> Result<std::process::ExitStatus, String> {
// Captures are outside every child-writable grant. Namespace teardown
// reaps descendants when the owned bootstrap is killed on any exit path.
let stdout = self.0.join("tool-stdout");
let stderr = self.0.join("tool-stderr");
let child = command
@@ -121,7 +117,7 @@ impl Stage {
))
.spawn()
.map_err(|error| format!("cannot execute {operation}: {error}"))?;
let mut child = OwnedChild(child);
let mut child = OwnedChild::new(child)?;
let deadline = Instant::now() + DEADLINE;
let mut next_scratch_check = Instant::now();
let within_budget = || -> Result<(), String> {
@@ -141,7 +137,6 @@ impl Stage {
next_scratch_check = Instant::now() + Duration::from_millis(250);
}
if let Some(status) = child
.0
.try_wait()
.map_err(|error| format!("cannot wait for {operation}: {error}"))?
{
@@ -176,11 +171,54 @@ impl Drop for Stage {
}
}
struct OwnedChild(Child);
struct OwnedChild {
child: Child,
#[cfg(target_os = "linux")]
group: Option<dever_process::ProcessGroup>,
}
impl OwnedChild {
fn new(mut child: Child) -> Result<Self, String> {
#[cfg(target_os = "linux")]
match dever_process::ProcessGroup::pin(child.id()) {
Ok(group) => Ok(Self {
child,
group: Some(group),
}),
Err(error) => {
let _ = child.kill();
let _ = child.wait();
Err(error)
}
}
#[cfg(not(target_os = "linux"))]
{
let _ = child.kill();
let _ = child.wait();
Err("managed builds require Linux".into())
}
}
fn try_wait(&mut self) -> Result<Option<std::process::ExitStatus>, String> {
#[cfg(target_os = "linux")]
if let Some(group) = &self.group {
if !group.exited()? {
return Ok(None);
}
group.terminate()?;
self.group.take();
}
self.child.try_wait().map_err(|error| error.to_string())
}
}
impl Drop for OwnedChild {
fn drop(&mut self) {
let _ = self.0.kill();
let _ = self.0.wait();
#[cfg(target_os = "linux")]
if let Some(group) = self.group.take() {
let _ = group.terminate();
}
let _ = self.child.kill();
let _ = self.child.wait();
}
}
@@ -64,5 +64,5 @@ fn visit(path: &Path, depth: usize, budget: &mut Budget) -> Result<(), String> {
#[cfg(not(unix))]
fn visit(_path: &Path, _depth: usize, _budget: &mut Budget) -> Result<(), String> {
Err("managed build scratch accounting requires its Linux sandbox".into())
Err("managed build scratch accounting requires its Linux process_runtime".into())
}
+17 -12
View File
@@ -1,4 +1,4 @@
"""Fixed PEP 517 frontend, executed only inside the managed build sandbox."""
"""Fixed PEP 517 frontend executed with explicit build inputs."""
import importlib
import json
@@ -7,8 +7,11 @@ from pathlib import Path
import re
import shutil
import sys
import shlex
import tomllib
INPUT = Path(__file__).resolve().parent
def build_system(source):
project = source / "pyproject.toml"
@@ -82,7 +85,7 @@ def execute(request, source, output):
return {"metadata": str(prepared.relative_to(output)) if prepared else None}
prepared = None
if request.get("metadata") is not None:
shutil.copytree("/worker/prepared", output / "prepared")
shutil.copytree(INPUT / "prepared", output / "prepared")
prepared = output / "prepared" / request["metadata"]
if not prepared.resolve().is_relative_to((output / "prepared").resolve()):
raise ValueError("prepared metadata path escapes its snapshot")
@@ -93,22 +96,24 @@ def execute(request, source, output):
def main():
request = json.loads(Path("/worker/request.json").read_text("utf-8"))
work = Path("/data/build")
request = json.loads((INPUT / "request.json").read_text("utf-8"))
work = Path(sys.argv[1]).resolve(strict=True)
tool_root = INPUT / "rootfs"
tools = tool_root / "usr/bin"
source = work / "source"
shutil.copytree("/worker/source", source)
shutil.copytree(INPUT / "source", source)
output = work / "output"
output.mkdir()
(work / "tmp").mkdir()
# These values are part of the fixed frontend, never project/host settings.
os.environ.update({
"PATH": "/worker/runtime/bin:/usr/bin",
"CC": "/usr/bin/cc", "CXX": "/usr/bin/c++",
"AR": "/usr/bin/ar", "RANLIB": "/usr/bin/ranlib",
"LDSHARED": "/usr/bin/cc -shared",
"CFLAGS": "-fPIC -ffile-prefix-map=/data/build/source=.",
"CPPFLAGS": "-I/worker/runtime/include/python" + request["python_version"],
"TMPDIR": "/data/build/tmp", "SOURCE_DATE_EPOCH": "0",
"PATH": os.pathsep.join([str(INPUT / "runtime/bin"), str(tools)]),
"CC": shlex.join([str(tools / "cc"), "--sysroot=" + str(tool_root)]), "CXX": shlex.join([str(tools / "c++"), "--sysroot=" + str(tool_root)]),
"AR": str(tools / "ar"), "RANLIB": str(tools / "ranlib"),
"LDSHARED": shlex.join([str(tools / "cc"), "--sysroot=" + str(tool_root), "-shared"]),
"CFLAGS": shlex.join(["-fPIC", "-ffile-prefix-map=" + str(source) + "=."]),
"CPPFLAGS": shlex.join(["-I" + str(INPUT / "runtime/include" / ("python" + request["python_version"]))]),
"TMPDIR": str(work / "tmp"), "SOURCE_DATE_EPOCH": "0",
})
os.chdir(source)
result = execute(request, source, output)
+8 -21
View File
@@ -358,7 +358,7 @@ impl Environment {
descriptor.validate(requested, None)?;
let (tools, tool_files) = pack::unpack(&bytes, &descriptor)?;
let assets = session.inputs.assets()?;
dever_sandbox::validate_assets(&assets)?;
dever_process::validate_assets(&assets)?;
pack::validate_tools(&tools, &tool_files, &assets)?;
for (path, bytes) in tool_files {
let executable = tools.executables.contains(&path);
@@ -402,9 +402,9 @@ impl Environment {
}
for (path, bytes) in &self.assets {
stage.write(
&format!("sandbox/{path}"),
&format!("process-runtime/{path}"),
bytes,
dever_sandbox::asset_executable(path),
dever_process::asset_executable(path),
)?;
}
for (path, bytes) in &source.files {
@@ -472,7 +472,7 @@ impl Environment {
.map(|(path, (bytes, _))| (path.clone(), bytes.as_slice()))
.collect::<BTreeMap<_, _>>();
for (path, bytes) in &self.assets {
supplied.insert(format!("sandbox/{path}"), bytes);
supplied.insert(format!("process-runtime/{path}"), bytes);
}
for (path, bytes) in &installed {
supplied.insert(path.clone(), bytes);
@@ -494,35 +494,22 @@ impl Environment {
stage.write("input/frontend.py", include_bytes!("python.py"), false)?;
let work = stage.0.join("work");
fs::create_dir(&work).map_err(|error| error.to_string())?;
let grants = [dever_sandbox::Grant {
source: work.clone(),
destination: "/data/build".into(),
writable: true,
}];
let arguments = [
OsString::from("-I"),
OsString::from("-B"),
input.join("frontend.py").into_os_string(),
work.clone().into_os_string(),
];
let executable = input.join(&self.executable);
let assets = stage.0.join("sandbox");
let launch = dever_sandbox::Launch {
let assets = stage.0.join("process-runtime");
let launch = dever_process::Launch {
assets: &assets,
worker: &input,
executable: &executable,
arguments: &arguments,
working_directory: &input,
capabilities: dever_sandbox::Capabilities {
network: false,
process: true,
},
grants: &grants,
};
stage.run(
launch.command_for_build(&input.join("rootfs"))?,
"isolated Python build hook",
&work,
)?;
stage.run(launch.command()?, "isolated Python build hook", &work)?;
inputs.diagnostic(&stage.diagnostics()?);
let outputs = output_files(&work.join("output"))?;
let response = serde_json::from_slice(
+1 -1
View File
@@ -56,7 +56,7 @@ impl InstalledRegistryPack {
if self.format != "dever-registry-runtime-v1"
|| self.runtime.target != target
|| (ecosystem == &Ecosystem::Pip) != self.runtime.python_markers.is_some()
|| (ecosystem == &Ecosystem::Pip) != !self.runtime.python_wheel_tags.is_empty()
|| (ecosystem == &Ecosystem::Pip) == self.runtime.python_wheel_tags.is_empty()
{
return Err(format!(
"signed {} runtime manifest does not match this target",
+20 -25
View File
@@ -11,7 +11,6 @@ use sha2::{Digest, Sha256};
use super::release::{self, InstallLock};
use super::{Artifact, Layout, MachineManager, ReleaseManifest, Version};
mod sandbox;
mod service;
mod transaction;
pub use service::{RecoveryAction, recovery_actions};
@@ -45,11 +44,6 @@ pub fn install(installer_root: &Path) -> Result<Version, String> {
})?;
absolute_root(&settings.system_root)?;
trusted_directory(&settings.system_root)?;
if settings.system_root == Path::new("/") {
// Publishing an AppArmor file can trigger a later system-wide reload,
// even when this installer does not load profiles or activate services.
dever_sandbox::validate_host_policy()?;
}
if settings.activate_service && settings.system_root != Path::new("/") {
return Err(
"service activation requires the actual system root; image installs are offline".into(),
@@ -96,7 +90,7 @@ pub fn install(installer_root: &Path) -> Result<Version, String> {
// The private copy is checked again by the version owner before any health
// command executes. A writable download cannot race signed bytes into it.
let manifest = MachineManager::verify_package(&package, None, &key)?;
validate_payload(&package, &manifest.artifacts, true)?;
validate_payload(&package, &manifest.artifacts, true, false)?;
pin_key(&layout, &key)?;
let manager = MachineManager::new(layout.clone());
if manager
@@ -140,6 +134,7 @@ pub(crate) fn validate_payload(
root: &Path,
artifacts: &[Artifact],
current_template: bool,
previous_layout: bool,
) -> Result<(), String> {
let scoped = artifacts
.iter()
@@ -154,10 +149,20 @@ pub(crate) fn validate_payload(
})
})
.collect::<Vec<_>>();
validate_files(&root.join("bootstrap"), &scoped, current_template)
validate_files(
&root.join("bootstrap"),
&scoped,
current_template,
previous_layout,
)
}
fn validate_files(root: &Path, artifacts: &[Artifact], new_payload: bool) -> Result<(), String> {
fn validate_files(
root: &Path,
artifacts: &[Artifact],
new_payload: bool,
previous_layout: bool,
) -> Result<(), String> {
let bytes = artifacts
.iter()
.try_fold(0u64, |total, artifact| total.checked_add(artifact.bytes))
@@ -166,11 +171,12 @@ fn validate_files(root: &Path, artifacts: &[Artifact], new_payload: bool) -> Res
return Err("bootstrap closure exceeds its 64-file/64-MiB budget".into());
}
for artifact in artifacts {
if !matches!(
let supported = matches!(
artifact.path.as_str(),
"dever" | "deverd" | "deverd.service" | "dever-sandbox"
) && !artifact.path.starts_with("lib/")
{
"dever" | "deverd" | "deverd.service"
) || artifact.path.starts_with("lib/")
|| previous_layout && !new_payload && artifact.path == "dever-sandbox";
if !supported {
return Err("bootstrap has an unsupported artifact".into());
}
let path = super::runtime_pack::checked_path(root, &artifact.path, true)?;
@@ -191,17 +197,6 @@ fn validate_files(root: &Path, artifacts: &[Artifact], new_payload: bool) -> Res
"bootstrap service definition is absent or differs from the supported contract".into(),
);
}
if new_payload
&& (!artifacts
.iter()
.any(|artifact| artifact.path == "dever-sandbox")
|| read_bounded(&root.join("dever-sandbox"), 8192)?
!= dever_sandbox::APPARMOR_PROFILE.as_bytes())
{
return Err(
"bootstrap AppArmor profile is absent or differs from the supported contract".into(),
);
}
super::core_libraries::validate_entries(root, artifacts, &["dever", "deverd"])
}
@@ -245,7 +240,7 @@ fn validate_bin(root: &Path, key: &[u8]) -> Result<Vec<u8>, String> {
})
})
.collect::<Vec<_>>();
validate_files(root, &artifacts, false)?;
validate_files(root, &artifacts, false, manifest.uses_sandbox_layout())?;
let expected = artifacts
.iter()
.map(|artifact| artifact.path.as_str())
@@ -1,138 +0,0 @@
//! Preserve signed content-addressed helpers across compiler upgrades/removal.
use std::fs;
use std::os::unix::fs::PermissionsExt;
use std::path::Path;
use super::{
MachineManager, ReleaseManifest, digest, read_bounded, release, trusted_directory, trusted_file,
};
const MAX_HELPERS: usize = 24;
const MAX_HELPER_BYTES: u64 = 32 * 1024 * 1024;
fn artifact_path() -> String {
format!(
"sandbox/{}/bin/bwrap",
crate::toolchain::platform_identity()
)
}
pub(super) fn stage(
installed: &Path,
next: &Path,
source: &Path,
manifest: &ReleaseManifest,
key: &[u8],
) -> Result<(), String> {
fs::create_dir(next).map_err(|error| error.to_string())?;
fs::set_permissions(next, fs::Permissions::from_mode(0o755))
.map_err(|error| error.to_string())?;
let mut hashes = std::collections::BTreeSet::new();
if installed.exists() {
trusted_directory(installed)?;
for entry in fs::read_dir(installed).map_err(|error| error.to_string())? {
let entry = entry.map_err(|error| error.to_string())?;
let hash = entry
.file_name()
.into_string()
.map_err(|_| "sandbox helper identity is not UTF-8")?;
if hashes.len() >= MAX_HELPERS || !valid_hash(&hash) {
return Err("installed sandbox helper store has unsupported entries or exceeds its version bound".into());
}
validate_installed(&entry.path(), &hash, key)?;
let destination = next.join(&hash);
copy_helper(&entry.path().join("bwrap"), &entry.path(), &destination)?;
hashes.insert(hash);
}
}
if let Some(artifact) = manifest
.artifacts
.iter()
.find(|artifact| artifact.path == artifact_path())
&& !hashes.contains(&artifact.sha256)
{
if hashes.len() >= MAX_HELPERS {
return Err("sandbox helper version store is full; retained applications must be retired before removing old helpers".into());
}
let bytes = read_bounded(&source.join(&artifact.path), MAX_HELPER_BYTES)?;
if bytes.len() as u64 != artifact.bytes
|| dever_sandbox::validate_bwrap(&bytes)? != artifact.sha256
{
return Err("sandbox helper differs from its signed release".into());
}
let destination = next.join(&artifact.sha256);
copy_helper(&source.join(&artifact.path), source, &destination)?;
validate_installed(&destination, &artifact.sha256, key)?;
}
release::sync_tree(next)
}
fn copy_helper(helper: &Path, signed_release: &Path, destination: &Path) -> Result<(), String> {
fs::create_dir(destination).map_err(|error| error.to_string())?;
fs::set_permissions(destination, fs::Permissions::from_mode(0o755))
.map_err(|error| error.to_string())?;
release::copy_file(helper, &destination.join("bwrap"), true)?;
for name in ["manifest.json", "manifest.sig"] {
release::copy_file(&signed_release.join(name), &destination.join(name), false)?;
}
Ok(())
}
fn valid_hash(hash: &str) -> bool {
hash.len() == 64
&& hash
.bytes()
.all(|byte| byte.is_ascii_digit() || (b'a'..=b'f').contains(&byte))
}
fn validate_installed(root: &Path, hash: &str, key: &[u8]) -> Result<(), String> {
use std::os::unix::fs::MetadataExt;
trusted_directory(root)?;
let names = fs::read_dir(root)
.map_err(|error| error.to_string())?
.map(|entry| entry.map(|entry| entry.file_name()))
.collect::<Result<std::collections::BTreeSet<_>, _>>()
.map_err(|error| error.to_string())?;
let expected = ["bwrap", "manifest.json", "manifest.sig"]
.map(std::ffi::OsString::from)
.into_iter()
.collect();
if names != expected {
return Err("installed sandbox helper contains unsigned entries".into());
}
for name in ["bwrap", "manifest.json", "manifest.sig"] {
trusted_file(&root.join(name))?;
}
let helper = root.join("bwrap");
if fs::metadata(&helper)
.map_err(|error| error.to_string())?
.mode()
& 0o6111
!= 0o111
{
return Err("installed sandbox helper has unsupported executable permissions".into());
}
match rustix::fs::lgetxattr(&helper, "security.capability", &mut [0u8; 64]) {
Err(rustix::io::Errno::NODATA | rustix::io::Errno::OPNOTSUPP) => {}
Ok(_) => return Err("installed sandbox helper must not have file capabilities".into()),
Err(error) => {
return Err(format!(
"cannot inspect installed sandbox helper capabilities: {error}"
));
}
}
let manifest = MachineManager::signed_manifest(root, key)?;
let artifact = manifest
.artifacts
.iter()
.find(|artifact| artifact.path == artifact_path())
.ok_or("installed sandbox helper has no signed artifact")?;
let bytes = read_bounded(&helper, MAX_HELPER_BYTES)?;
if artifact.sha256 != hash || artifact.bytes != bytes.len() as u64 || digest(&bytes) != hash {
return Err(
"installed sandbox helper differs from its signature or directory identity".into(),
);
}
dever_sandbox::validate_bwrap(&bytes)?;
Ok(())
}
@@ -95,8 +95,8 @@ fn property(name: &str) -> Result<String, String> {
.map_err(|_| "service state output is not UTF-8")?
.trim()
.to_owned();
if !status.success()
&& !(status.code() == Some(1) && name == "LoadState" && value == "not-found")
if !(status.success()
|| status.code() == Some(1) && name == "LoadState" && value == "not-found")
{
return Err(format!("cannot inspect Dever service {name}: {status}"));
}
@@ -22,17 +22,13 @@ const MAX_TREE_BYTES: u64 = 80 * 1024 * 1024;
#[serde(rename_all = "snake_case")]
enum Target {
Bin,
Sandbox,
Apparmor,
Service,
Entry,
Enable,
Active,
}
const TARGETS: [Target; 7] = [
Target::Sandbox,
Target::Apparmor,
const TARGETS: [Target; 5] = [
Target::Bin,
Target::Service,
Target::Entry,
@@ -44,8 +40,6 @@ impl Target {
fn relative(self) -> &'static str {
match self {
Self::Bin => "opt/dever/bin",
Self::Sandbox => "opt/dever/sandbox",
Self::Apparmor => "etc/apparmor.d/dever-sandbox",
Self::Service => "etc/systemd/system/deverd.service",
Self::Entry => "usr/local/bin/dever",
Self::Enable => "etc/systemd/system/multi-user.target.wants/deverd.service",
@@ -136,16 +130,6 @@ pub(super) fn install(
"bootstrap cannot replace an unrelated public entry or service link".into(),
);
}
} else if target == Target::Apparmor && target.path(root).exists() {
trusted_file(&target.path(root))?;
let installed_profile = layout.bin().join("dever-sandbox");
if previous_service.is_none()
|| !installed_profile.exists()
|| read_bounded(&target.path(root), 8192)?
!= read_bounded(&installed_profile, 8192)?
{
return Err("bootstrap cannot replace an unrelated AppArmor profile".into());
}
} else if target == Target::Service && target.path(root).exists() {
trusted_file(&target.path(root))?;
if previous_service.as_deref()
@@ -220,18 +204,6 @@ pub(super) fn install(
}
fn stage(root: &Path, source: &Path, manifest: &ReleaseManifest, key: &[u8]) -> Result<(), String> {
super::sandbox::stage(
&Target::Sandbox.path(root),
&Target::Sandbox.next(root),
source,
manifest,
key,
)?;
write_new(
&Target::Apparmor.next(root),
dever_sandbox::APPARMOR_PROFILE.as_bytes(),
0o644,
)?;
let bin = Target::Bin.next(root);
fs::create_dir(&bin).map_err(|error| error.to_string())?;
for artifact in &manifest.artifacts {
@@ -276,9 +248,6 @@ fn stage(root: &Path, source: &Path, manifest: &ReleaseManifest, key: &[u8]) ->
}
fn publish(root: &Path, change: &Change) -> Result<(), String> {
if change.target == Target::Apparmor && root == Path::new("/") {
dever_sandbox::validate_host_policy()?;
}
let path = change.target.path(root);
let next = change.target.next(root);
if identity(&path, change.target.link())? != change.before
@@ -169,13 +169,13 @@ impl CompileRequest {
)
}
pub fn embedded_resources(&self) -> Vec<EmbeddedResource> {
pub fn take_embedded_resources(&mut self) -> Vec<EmbeddedResource> {
self.resources
.iter()
.drain(..)
.map(|resource| EmbeddedResource {
path: resource.path.clone(),
bytes: resource.bytes.clone(),
sha256: resource.sha256.clone(),
path: resource.path,
bytes: resource.bytes,
sha256: resource.sha256,
executable: resource.executable,
})
.collect()
@@ -52,7 +52,7 @@ pub(super) fn validate_entries(
entries: &[&str],
) -> Result<(), String> {
runtime_pack::host_target()?;
dever_sandbox::require_no_loader_preload()?;
dever_process::require_no_loader_preload()?;
let mut files = BTreeMap::new();
let mut total = 0u64;
for artifact in artifacts {
+8 -3
View File
@@ -55,7 +55,7 @@ impl ExtensionId {
format!("ext-{}.tar.zst", self.name())
}
fn permits(&self, path: &str) -> bool {
fn permits(&self, path: &str, process_root: &str) -> bool {
let target = self.target.platform();
match &self.kind {
ExtensionKind::Runtime(ecosystem) => {
@@ -66,7 +66,7 @@ impl ExtensionId {
}
ExtensionKind::Target => {
path.starts_with(&format!("runtime/{target}/"))
|| path.starts_with(&format!("sandbox/{target}/"))
|| path.starts_with(&format!("{process_root}/{target}/"))
}
}
}
@@ -113,6 +113,11 @@ pub fn validate_catalog(manifest: &ReleaseManifest) -> Result<(), String> {
}
let mut paths = BTreeSet::new();
validate_artifacts(&manifest.artifacts, &mut paths)?;
let process_root = if manifest.uses_sandbox_layout() {
"sandbox"
} else {
"process-runtime"
};
let mut ids = BTreeSet::new();
for extension in &manifest.extensions {
let id = extension.id();
@@ -126,7 +131,7 @@ pub fn validate_catalog(manifest: &ReleaseManifest) -> Result<(), String> {
if extension
.artifacts
.iter()
.any(|artifact| !id.permits(&artifact.path))
.any(|artifact| !id.permits(&artifact.path, process_root))
{
return Err(format!(
"extension '{}' contains a file outside its resource namespace",
+8 -8
View File
@@ -22,7 +22,7 @@ static NEXT_STAGING: AtomicU64 = AtomicU64::new(0);
mod bootstrap;
mod builds;
mod ecosystems;
mod sandbox;
mod process_runtime;
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
@@ -45,7 +45,7 @@ struct Settings {
#[serde(default)]
runtimes: ecosystems::RuntimeInputs,
#[serde(default)]
sandbox: Option<Vec<LinkInput>>,
process_runtime: Option<Vec<LinkInput>>,
#[serde(default)]
builds: builds::Inputs,
#[serde(default)]
@@ -53,7 +53,7 @@ struct Settings {
#[serde(default)]
build_targets: BTreeMap<BuildTarget, builds::Inputs>,
#[serde(default)]
sandbox_targets: BTreeMap<BuildTarget, Vec<LinkInput>>,
process_runtime_targets: BTreeMap<BuildTarget, Vec<LinkInput>>,
}
#[derive(Deserialize)]
@@ -232,14 +232,14 @@ fn assemble(
});
}
}
if let Some(inputs) = settings.sandbox {
artifacts.extend(sandbox::assemble(&mut writer, inputs, host)?);
if let Some(inputs) = settings.process_runtime {
artifacts.extend(process_runtime::assemble(&mut writer, inputs, host)?);
}
for (target, inputs) in settings.sandbox_targets {
for (target, inputs) in settings.process_runtime_targets {
let selected = target_artifacts
.get_mut(&target)
.ok_or("target sandbox requires a matching native target")?;
selected.extend(sandbox::assemble(&mut writer, inputs, target)?);
.ok_or("target process_runtime requires a matching native target")?;
selected.extend(process_runtime::assemble(&mut writer, inputs, target)?);
}
for (target, artifacts) in target_artifacts {
extensions.push(Extension {
@@ -39,26 +39,17 @@ pub(super) fn assemble(
#[cfg(target_os = "linux")]
{
use sha2::{Digest, Sha256};
for (path, bytes) in [
(
"bootstrap/deverd.service",
crate::toolchain::bootstrap::SERVICE.as_bytes(),
),
(
"bootstrap/dever-sandbox",
dever_sandbox::APPARMOR_PROFILE.as_bytes(),
),
] {
let destination = writer.reserve(path)?;
std::fs::write(&destination, bytes).map_err(|error| error.to_string())?;
super::set_mode(&destination, false)?;
artifacts.push(Artifact {
path: path.into(),
bytes: bytes.len() as u64,
sha256: format!("{:x}", Sha256::digest(bytes)),
});
}
crate::toolchain::bootstrap::validate_payload(writer.staging, &artifacts, true)?;
let path = "bootstrap/deverd.service";
let bytes = crate::toolchain::bootstrap::SERVICE.as_bytes();
let destination = writer.reserve(path)?;
std::fs::write(&destination, bytes).map_err(|error| error.to_string())?;
super::set_mode(&destination, false)?;
artifacts.push(Artifact {
path: path.into(),
bytes: bytes.len() as u64,
sha256: format!("{:x}", Sha256::digest(bytes)),
});
crate::toolchain::bootstrap::validate_payload(writer.staging, &artifacts, true, false)?;
}
Ok(artifacts)
}
@@ -110,22 +110,30 @@ pub(super) fn assemble(
files.sort_by(|left, right| left.0.cmp(&right.0));
manifest.validate(&files, &descriptor)?;
let mut assets = Vec::new();
for directory in ["bin", "lib"] {
for entry in fs::read_dir(writer.staging.join(format!("sandbox/{target}/{directory}")))
.map_err(|error| format!("build pack requires prepared sandbox assets: {error}"))?
for directory in ["lib"] {
for entry in fs::read_dir(
writer
.staging
.join(format!("process-runtime/{target}/{directory}")),
)
.map_err(|error| format!("build pack requires prepared process_runtime assets: {error}"))?
{
let entry = entry.map_err(|error| error.to_string())?;
let name = entry
.file_name()
.into_string()
.map_err(|_| "sandbox asset name is not UTF-8")?;
.map_err(|_| "process_runtime asset name is not UTF-8")?;
assets.push((
format!("{directory}/{name}"),
read_bounded(&entry.path(), 32 * 1024 * 1024, "sandbox build asset")?,
read_bounded(
&entry.path(),
32 * 1024 * 1024,
"process_runtime build asset",
)?,
));
}
}
dever_sandbox::validate_assets_for_target(&assets, &target)?;
dever_process::validate_assets_for_target(&assets, &target)?;
pack::validate_tools(&manifest, &files, &assets)?;
let prefix = format!("build/{}/{target}", ecosystem.as_str());
let archive_name = format!("{prefix}/build.pack");
@@ -1,5 +1,3 @@
//! Sandbox tools and OS libraries are authenticated by the existing release.
use super::{Artifact, LinkInput, PayloadWriter, read_bounded};
use sha2::{Digest, Sha256};
@@ -9,30 +7,30 @@ pub(super) fn assemble(
target: crate::toolchain::BuildTarget,
) -> Result<Vec<Artifact>, String> {
if inputs.len() > 128 {
return Err("sandbox pack contains too many files".into());
return Err("process_runtime pack contains too many files".into());
}
let mut files = Vec::new();
let mut remaining = 32 * 1024 * 1024;
for input in inputs {
super::runtime_pack::relative_path(&input.path)?;
let source = writer.source(&input.source, &input.path, &input.sha256)?;
let bytes = read_bounded(&source, remaining, "sandbox asset")?;
let bytes = read_bounded(&source, remaining, "process_runtime asset")?;
if format!("{:x}", Sha256::digest(&bytes)) != input.sha256 {
return Err("sandbox asset changed during packaging".into());
return Err("process_runtime asset changed during packaging".into());
}
remaining -= bytes.len() as u64;
files.push((input.path, bytes));
}
files.sort_by(|left, right| left.0.cmp(&right.0));
dever_sandbox::validate_assets_for_target(&files, target.platform())?;
dever_process::validate_assets_for_target(&files, target.platform())?;
// Leaf artifacts avoid a second manifest/signature or extraction protocol.
let mut artifacts = Vec::new();
for (path, bytes) in files {
let destination = format!("sandbox/{}/{path}", target.platform());
let destination = format!("process-runtime/{}/{path}", target.platform());
let output = writer.reserve(&destination)?;
std::fs::write(&output, &bytes)
.map_err(|error| format!("cannot write sandbox asset: {error}"))?;
super::set_mode(&output, dever_sandbox::asset_executable(&path))?;
.map_err(|error| format!("cannot write process_runtime asset: {error}"))?;
super::set_mode(&output, dever_process::asset_executable(&path))?;
artifacts.push(Artifact {
path: destination,
bytes: bytes.len() as u64,
+33 -5
View File
@@ -78,6 +78,10 @@ pub struct ReleaseManifest {
}
impl ReleaseManifest {
pub(super) fn uses_sandbox_layout(&self) -> bool {
version_components(self.version.as_str()) < [0, 1, 5]
}
pub fn new(version: Version, artifacts: Vec<Artifact>) -> Self {
Self {
format: "dever-release-v2".into(),
@@ -703,7 +707,12 @@ impl MachineManager {
{
// Older installed versions retain their own signed service template.
// Only the explicit bootstrap installer applies the current template.
super::bootstrap::validate_payload(root, &manifest.artifacts, false)?;
super::bootstrap::validate_payload(
root,
&manifest.artifacts,
false,
manifest.uses_sandbox_layout(),
)?;
}
Ok(manifest)
}
@@ -947,16 +956,35 @@ pub(super) fn ensure_real_directory(path: &Path, label: &str) -> Result<(), Stri
}
fn create_private_directory(path: &Path) -> Result<(), String> {
match fs::symlink_metadata(path) {
Ok(_) => return ensure_real_directory(path, "machine directory"),
Err(error) if error.kind() == io::ErrorKind::NotFound => {}
Err(error) => return Err(format!("cannot inspect '{}': {error}", path.display())),
}
if let Some(parent) = path.parent()
&& !parent.as_os_str().is_empty()
{
create_private_directory(parent)?;
}
let mut builder = fs::DirBuilder::new();
builder.recursive(true);
builder.recursive(false);
#[cfg(unix)]
{
use std::os::unix::fs::DirBuilderExt;
builder.mode(0o755);
}
builder
.create(path)
.map_err(|error| format!("cannot create '{}': {error}", path.display()))?;
match builder.create(path) {
Ok(()) => {
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
fs::set_permissions(path, fs::Permissions::from_mode(0o755))
.map_err(|error| format!("cannot protect '{}': {error}", path.display()))?;
}
}
Err(error) if error.kind() == io::ErrorKind::AlreadyExists => {}
Err(error) => return Err(format!("cannot create '{}': {error}", path.display())),
}
ensure_real_directory(path, "machine directory")
}
@@ -84,6 +84,7 @@ pub(super) fn serve(layout: &Layout, stream: &mut UnixStream, bytes: u64) -> Res
.map_err(|error| format!("cannot monitor compilation caller: {error}"))?;
require_connected(stream, started)?;
if let Some(artifact) = store.restore(&identity)? {
drop(compiler);
return send_output(stream, artifact.as_bytes(), started);
}
let directory = store.compilation_directory()?;
@@ -104,6 +105,7 @@ pub(super) fn serve(layout: &Layout, stream: &mut UnixStream, bytes: u64) -> Res
let artifact = store
.restore(&identity)?
.ok_or("compiled output was not published")?;
drop(compiler);
send_output(stream, artifact.as_bytes(), started)
}
@@ -131,16 +133,11 @@ fn verify_pack(
));
}
}
// This is the embedded compiler's fixed Linux LLVM dependency, not a PATH lookup.
if cfg!(target_os = "linux")
&& !compiler
.manifest
.artifacts
.iter()
.any(|artifact| artifact.path == "lib/libLLVM.so.18.1")
{
return Err("signed release is missing lib/libLLVM.so.18.1".into());
}
let core_root = compiler
.core
.parent()
.ok_or("installed compiler has no release directory")?;
super::super::core_libraries::validate(core_root, &compiler.manifest.artifacts)?;
Ok(())
}
+2 -2
View File
@@ -28,7 +28,7 @@ pub(super) fn execute() -> Result<(), String> {
.read_to_end(&mut bytes)
})
.map_err(|error| format!("cannot read compile request: {error}"))?;
let request = CompileRequest::decode(&bytes)?;
let mut request = CompileRequest::decode(&bytes)?;
drop(bytes);
if request.version != version {
return Err("compile request version does not match worker".into());
@@ -63,7 +63,7 @@ pub(super) fn execute() -> Result<(), String> {
crate::compile::worker_compile(
&program,
&sources,
&request,
&mut request,
&directory,
&directory.join(WORKER_OUTPUT_FILE),
)
+9 -33
View File
@@ -127,7 +127,13 @@ pub fn prepare_for_target(
}
for contract in workers {
if contract.ecosystem == "exec" {
output.extend(prepare_exec(&contract, &supplied, target)?);
output.extend(command::prepare(
project_root,
&contract,
&package_files,
&supplied,
target,
)?);
continue;
}
let lock = lock.as_ref().expect("managed Worker has a lock");
@@ -173,36 +179,6 @@ pub fn prepare_for_target(
Ok(output)
}
fn prepare_exec(
contract: &ExternalWorkerContract,
supplied: &BTreeMap<&str, &EmbeddedResource>,
target: crate::toolchain::BuildTarget,
) -> Result<Vec<EmbeddedResource>, String> {
let entry = supplied
.get(contract.entry.as_str())
.ok_or("exec Worker is missing from verified resources")?;
if !entry.executable || entry.bytes.is_empty() {
return Err("exec Worker entry must be a nonempty executable resource".into());
}
runtime::validate_binary(&contract.entry, &entry.bytes, target.platform())?;
let identity =
serde_json::to_vec(&contract.sdk_manifest()).map_err(|error| error.to_string())?;
let tree = format!("workers/{}", digest(&identity));
let executable = format!("{tree}/worker");
let manifest = json!({
"format": "dever-worker-launch-v1", "ecosystem": "exec", "entry": contract.entry,
"executable": executable, "arguments": [], "working_directory": tree,
});
Ok(vec![
resource(executable, entry.bytes.clone(), true),
resource(
format!("{}.dever-worker.json", contract.entry),
serde_json::to_vec(&manifest).map_err(|error| error.to_string())?,
false,
),
])
}
fn prepare_worker(
project_root: &Path,
contract: &ExternalWorkerContract,
@@ -301,7 +277,7 @@ fn prepare_worker(
.build
.as_ref()
.expect("validated Go build descriptor"),
sandbox: build_assets,
process_runtime: build_assets,
},
&mut tree_files,
target,
@@ -696,7 +672,7 @@ fn validate_worker_native(
files.extend(
supplied
.iter()
.filter(|(path, _)| path.starts_with("sandbox/lib/"))
.filter(|(path, _)| path.starts_with("process-runtime/lib/"))
.map(|(path, resource)| ((*path).to_owned(), resource.bytes.as_slice())),
);
python_wheel::validate_native(
+44 -29
View File
@@ -20,12 +20,26 @@ pub(super) fn prepare(
supplied: &BTreeMap<&str, &EmbeddedResource>,
target: BuildTarget,
) -> Result<Vec<EmbeddedResource>, String> {
let worker = contract.ecosystem == "exec";
let tree_kind = if worker { "workers" } else { "commands" };
let binary_name = if worker { "worker" } else { "tool" };
let prefix = format!(
"commands/{}",
"{tree_kind}/{}",
digest(&serde_json::to_vec(&contract.sdk_manifest()).map_err(|error| error.to_string())?)
);
let mut tree = Tree::new(prefix.clone());
let source = command_files(root, &contract.entry, package_files)?;
let entry = if worker {
let entry = supplied
.get(contract.entry.as_str())
.ok_or("exec Worker is missing from verified resources")?;
if !entry.executable || entry.bytes.is_empty() {
return Err("exec Worker entry must be a nonempty executable resource".into());
}
Some(entry.bytes.as_slice())
} else {
None
};
let source = command_files(root, &contract.entry, package_files, entry)?;
let binary = &source["program/tool"];
runtime::validate_binary(&contract.entry, binary, target.platform())?;
let machine = match target {
@@ -44,19 +58,17 @@ pub(super) fn prepare(
files.extend(
supplied
.iter()
.filter(|(path, _)| path.starts_with("sandbox/lib/"))
.filter(|(path, _)| path.starts_with("process-runtime/lib/"))
.map(|(path, entry)| ((*path).to_owned(), entry.bytes.as_slice())),
);
let closure = elf::Closure {
files: &files,
machine,
directories: vec!["program/lib".into(), "sandbox/lib".into()],
directories: vec!["program/lib".into(), "process-runtime/lib".into()],
};
for (path, bytes) in &source {
let object = elf::object(path, bytes, machine)?;
reject_loader_hooks(&object)?;
// The logical program root becomes /worker in the sandbox. A path
// escaping it could otherwise resolve a library from a file grant.
for fields in [&object.rpaths, &object.runpaths] {
for directory in elf::origin_directories(path, fields)? {
if directory != "program" && !directory.starts_with("program/") {
@@ -65,9 +77,9 @@ pub(super) fn prepare(
}
}
if let Some(name) = path.strip_prefix("program/lib/") {
if supplied.contains_key(format!("sandbox/lib/{name}").as_str()) {
if supplied.contains_key(format!("process-runtime/lib/{name}").as_str()) {
return Err(format!(
"command library '{name}' overrides a signed sandbox library"
"command library '{name}' overrides a signed process_runtime library"
));
}
if object.header.e_type != header::ET_DYN
@@ -83,38 +95,38 @@ pub(super) fn prepare(
}
for (path, bytes) in source {
let name = path.strip_prefix("program/").expect("owned command tree");
tree.add(name, bytes, name == "tool")?;
tree.add(
if name == "tool" { binary_name } else { name },
bytes,
name == "tool",
)?;
}
let (executable, arguments) = if let Some(interpreter) = interpreter {
if worker {
tree.add(
"contract.json",
serde_json::to_vec(&contract.sdk_manifest()).map_err(|error| error.to_string())?,
false,
)?;
}
if let Some(interpreter) = interpreter {
let name = interpreter
.rsplit('/')
.next()
.expect("validated GNU loader");
let loader = supplied
.get(format!("sandbox/lib/{name}").as_str())
.get(format!("process-runtime/lib/{name}").as_str())
.ok_or("command requires the signed target GNU loader")?;
runtime::validate_binary(name, &loader.bytes, target.platform())?;
tree.add("loader", loader.bytes.clone(), true)?;
(
format!("{prefix}/loader"),
vec![
json!({"literal": "--inhibit-cache"}),
json!({"literal": "--library-path"}),
json!({"literal": "lib:/lib"}),
json!({"resource": format!("{prefix}/tool")}),
],
)
} else {
(format!("{prefix}/tool"), Vec::new())
};
}
let suffix = if worker { "worker" } else { "command" };
let manifest = json!({
"format": "dever-command-launch-v1", "ecosystem": "command",
"entry": contract.entry, "executable": executable,
"arguments": arguments, "working_directory": prefix,
"format": format!("dever-{suffix}-launch-v1"), "ecosystem": contract.ecosystem,
"entry": contract.entry, "executable": format!("{prefix}/{binary_name}"),
"arguments": [], "working_directory": prefix,
});
let mut output = tree.finish();
output.push(resource(
format!("{}.dever-command.json", contract.entry),
format!("{}.dever-{suffix}.json", contract.entry),
serde_json::to_vec(&manifest).map_err(|error| error.to_string())?,
false,
));
@@ -143,6 +155,7 @@ fn command_files(
root: &Path,
entry: &str,
package_files: &BTreeMap<String, Vec<u8>>,
verified: Option<&[u8]>,
) -> Result<BTreeMap<String, Vec<u8>>, String> {
valid_path(entry)?;
let parent = entry
@@ -151,7 +164,9 @@ fn command_files(
.ok_or("command entry must belong to a domain")?;
let library_prefix = format!("{parent}/lib/");
let packaged = crate::packages::owns_worker(root, entry)?;
let binary = if packaged {
let binary = if let Some(bytes) = verified {
bytes.to_vec()
} else if packaged {
package_files
.get(entry)
.cloned()
+1 -1
View File
@@ -46,7 +46,7 @@ pub(crate) fn origin_directories(path: &str, fields: &[&str]) -> Result<Vec<Stri
}
}
let normalized = normalized.join("/");
if normalized.is_empty() || normalized.starts_with("sandbox/") {
if normalized.is_empty() || normalized.starts_with("process-runtime/") {
return Err("native library path escapes its Worker installation".into());
}
if !result.contains(&normalized) {
+14 -27
View File
@@ -43,7 +43,6 @@ struct Builder {
stage: Stage,
input: PathBuf,
assets: PathBuf,
grants: Vec<dever_sandbox::Grant>,
tools: GoBuildManifest,
entry: String,
stdlib: BTreeMap<String, PathBuf>,
@@ -55,7 +54,7 @@ struct Builder {
pub(super) struct Inputs<'a> {
pub files: &'a [(String, Vec<u8>)],
pub manifest: &'a GoBuildManifest,
pub sandbox: &'a [EmbeddedResource],
pub process_runtime: &'a [EmbeddedResource],
}
pub(super) fn compile_worker(
@@ -69,7 +68,7 @@ pub(super) fn compile_worker(
let Inputs {
files: pack_files,
manifest: build,
sandbox: build_assets,
process_runtime: build_assets,
} = inputs;
let references = build.validate_files(pack_files, target)?;
if build.host != crate::toolchain::BuildTarget::host()? {
@@ -77,24 +76,19 @@ pub(super) fn compile_worker(
}
let stage = Stage::new()?;
let input = stage.0.join("input");
let assets = stage.0.join("sandbox");
let assets = stage.0.join("process-runtime");
for resource in build_assets
.iter()
.filter(|resource| resource.path.starts_with("sandbox/"))
.filter(|resource| resource.path.starts_with("process-runtime/"))
{
if digest(&resource.bytes) != resource.sha256 {
return Err("Go build sandbox asset has a digest mismatch".into());
return Err("Go build process_runtime asset has a digest mismatch".into());
}
stage.write(&resource.path, &resource.bytes, resource.executable)?;
}
let build_directory = stage.0.join("build");
fs::create_dir(&build_directory)
.map_err(|error| format!("cannot create Go build output: {error}"))?;
let grants = vec![dever_sandbox::Grant {
source: build_directory,
destination: "/data/build".into(),
writable: true,
}];
for (path, bytes) in pack_files {
stage.write(
&format!("input/{path}"),
@@ -164,7 +158,6 @@ pub(super) fn compile_worker(
stage,
input,
assets,
grants,
tools: build.clone(),
entry: entry_name(&contract.entry)?.into(),
stdlib,
@@ -235,9 +228,7 @@ impl GoBuildManifest {
}
for tool in [&self.compiler, &self.linker, &self.analyzer] {
if !tool.starts_with("bin/")
|| !pack
.get(tool.as_str())
.is_some_and(|bytes| !bytes.is_empty())
|| pack.get(tool.as_str()).is_none_or(|bytes| bytes.is_empty())
{
return Err(format!("locked Go build tool '{tool}' is missing"));
}
@@ -259,7 +250,7 @@ impl GoBuildManifest {
.ok_or("invalid locked Go standard library importcfg")?;
valid_import(name)?;
if path != format!("stdlib/{name}.a")
|| !pack.get(path).is_some_and(|bytes| !bytes.is_empty())
|| pack.get(path).is_none_or(|bytes| bytes.is_empty())
|| stdlib.insert(name.into(), path.into()).is_some()
{
return Err(
@@ -503,34 +494,30 @@ impl Builder {
fn run(&self, executable: &Path, args: &[&str]) -> Result<Vec<u8>, String> {
let arguments = args.iter().map(OsString::from).collect::<Vec<_>>();
self.stage.run(
self.sandbox(executable, &arguments).command()?,
self.process_runtime(executable, &arguments).command()?,
"locked Go build tool",
&self.stage.0.join("build"),
)
}
fn sandbox<'a>(
fn process_runtime<'a>(
&'a self,
executable: &'a Path,
arguments: &'a [OsString],
) -> dever_sandbox::Launch<'a> {
dever_sandbox::Launch {
) -> dever_process::Launch<'a> {
dever_process::Launch {
assets: &self.assets,
worker: &self.input,
executable,
arguments,
working_directory: &self.input,
capabilities: dever_sandbox::Capabilities::default(),
grants: &self.grants,
}
}
fn visible_path(&self, path: &Path) -> Result<String, String> {
self.sandbox(&self.input, &[])
.argument_path(path)?
.into_os_string()
.into_string()
.map_err(|_| "Go sandbox path is not UTF-8".into())
path.to_str()
.map(str::to_owned)
.ok_or_else(|| "Go build path is not UTF-8".into())
}
}
+1 -1
View File
@@ -463,7 +463,7 @@ impl Wheel {
{
let newline = bytes.iter().position(|byte| *byte == b'\n').unwrap();
bytes = [
format!("#!/worker/{interpreter} -I\n").as_bytes(),
scripts::launcher(interpreter)?.as_bytes(),
&bytes[newline + 1..],
]
.concat();
@@ -5,7 +5,7 @@ use goblin::elf::header;
use std::collections::{BTreeMap, BTreeSet};
/// `files` contains Worker-relative installation files and separately verified
/// OS ABI files at sandbox/lib/*. No host directory is searched or executed.
/// OS ABI files at process-runtime/lib/*. No host directory is searched or executed.
pub fn validate_native(
target: &str,
suffixes: &[String],
@@ -36,7 +36,7 @@ pub fn validate_native(
let closure = Closure {
files,
machine,
directories: vec!["sandbox/lib".into()],
directories: vec!["process-runtime/lib".into()],
};
let interpreter_elf = object(
interpreter,
@@ -121,7 +121,7 @@ pub(crate) fn validate_build_tools(
machine,
directories: vec![
format!("rootfs/usr/lib/{multiarch}"),
"sandbox/lib".into(),
"process-runtime/lib".into(),
"rootfs/usr/lib".into(),
],
};
@@ -146,7 +146,7 @@ pub(crate) fn validate_build_tools(
}
let elf = object(path, bytes, machine)?;
if elf.interpreter.is_some_and(|value| value != interpreter) {
return Err("build tool interpreter differs from signed sandbox loader".into());
return Err("build tool interpreter differs from signed process_runtime loader".into());
}
closure.follow(path, vec![], &BTreeMap::new())?;
}
@@ -14,6 +14,23 @@ fn identifier(path: &str) -> bool {
})
}
pub(super) fn launcher(interpreter: &str) -> Result<String, String> {
let name = interpreter
.rsplit('/')
.next()
.ok_or("missing Python interpreter name")?;
if name.is_empty()
|| !name
.bytes()
.all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'.' | b'_' | b'-'))
{
return Err("invalid Python interpreter file name".into());
}
Ok(format!(
"#!/bin/sh\n'''exec' \"${{0%/*}}/{name}\" -I \"$0\" \"$@\"\n' '''\n"
))
}
pub(super) fn entry_points(
bytes: &[u8],
prefix: &str,
@@ -82,7 +99,8 @@ pub(super) fn entry_points(
}else{value.trim()};
let (module,function)=value.split_once(':').ok_or("wheel console script must name a callable")?;
if !identifier(module) || !identifier(function) { return Err("wheel script has an invalid Python entry point".into()); }
let source=format!("#!/worker/{interpreter} -I\nimport importlib\nimport re\nimport sys\nentry = importlib.import_module({module:?})\nfor part in {function:?}.split('.'):\n entry = getattr(entry, part)\nsys.argv[0] = re.sub(r'(-script\\.pyw|\\.exe)?$', '', sys.argv[0])\nsys.exit(entry())\n");
let launcher = launcher(interpreter)?;
let source=format!("{launcher}import importlib\nimport re\nimport sys\nentry = importlib.import_module({module:?})\nfor part in {function:?}.split('.'):\n entry = getattr(entry, part)\nsys.argv[0] = re.sub(r'(-script\\.pyw|\\.exe)?$', '', sys.argv[0])\nsys.exit(entry())\n");
Ok(InstalledFile {path:format!("{prefix}/bin/{name}"),bytes:source.into_bytes(),executable:true})
}).collect()
}
+35 -44
View File
@@ -99,11 +99,13 @@ pub(super) fn validate_rest_bindings(program: &Program, errors: &mut Vec<Diagnos
ExpressionKind::Call {
target: CallTarget::Function(target),
..
} => {
if !rest_function_allowed(program, *target, &model_domain) {
errors.push(Diagnostic::error("C013", "REST field bindings call only pure same-domain Domain functions or pure standard library functions", value.span));
}
} if !rest_function_allowed(program, *target, &model_domain) => {
errors.push(Diagnostic::error("C013", "REST field bindings call only pure same-domain Domain functions or pure standard library functions", value.span));
}
ExpressionKind::Call {
target: CallTarget::Function(_),
..
} => {}
ExpressionKind::Call { .. }
| ExpressionKind::ModelOperation { .. }
| ExpressionKind::JobEnqueue { .. }
@@ -270,57 +272,44 @@ fn rest_intrinsic_allowed(operation: crate::intrinsic::Intrinsic) -> bool {
}
pub(super) fn validate_write_transactions(program: &Program, errors: &mut Vec<Diagnostic>) {
let write_routes = program
let roots = program
.api_routes
.iter()
.filter(|route| route.method != "GET")
.map(|route| route.function)
.map(|route| (route.function, route.long))
.chain(
program
.api_commands
.iter()
.map(|command| (command.function, command.long)),
)
.chain(program.jobs.iter().map(|job| (job.function, job.long)))
.collect::<std::collections::BTreeSet<_>>();
for function in &write_routes {
let root = crate::specialize::Specialization {
function: *function,
handlers: Vec::new(),
};
let password_crypto = crate::specialize::uses_intrinsic(
program,
&root,
crate::intrinsic::Intrinsic::CryptoPasswordVerify,
) || crate::specialize::uses_intrinsic(
program,
&root,
crate::intrinsic::Intrinsic::CryptoPasswordHash,
);
if password_crypto
&& let Some(span) = crate::specialize::write_outside_transaction(program, &root)
{
let mut error = Diagnostic::error(
"C014",
"password crypto API writes must be enclosed in an explicit transaction function",
program.functions[*function].span,
);
error.related.push(crate::diagnostic::Label {
span,
message: "database write occurs outside the short transaction boundary".into(),
});
errors.push(error);
}
}
let roots = write_routes
.into_iter()
.chain(program.api_commands.iter().map(|command| command.function))
.chain(program.jobs.iter().map(|job| job.function))
.collect::<std::collections::BTreeSet<_>>();
for function in roots {
for (function, long) in roots {
let root = crate::specialize::Specialization {
function,
handlers: Vec::new(),
};
if crate::specialize::writes_database(program, &root)
if long {
if let Some(span) = crate::specialize::write_outside_transaction(program, &root) {
let mut error = Diagnostic::error(
"C014",
"long entry database writes must be enclosed in an explicit transaction function",
program.functions[function].span,
);
error.related.push(crate::diagnostic::Label {
span,
message: "database write occurs outside the explicit transaction boundary"
.into(),
});
errors.push(error);
}
} else if crate::specialize::writes_database(program, &root)
&& crate::specialize::starts_transaction_unsafe_work(program, &root)
{
errors.push(Diagnostic::error(
"C014",
"an API, CMD, or Job database transaction cannot start concurrent work",
"automatic entry transactions cannot contain external I/O, Workers, waits, concurrency, or password crypto; declare the entry 'long' and enclose its writes in explicit transaction functions",
program.functions[function].span,
));
}
@@ -526,6 +515,7 @@ pub(super) fn register(
let path = format!("/{prefix}/{}", binding.action.text);
if occupy(&mut routes, (method, path.clone()), binding.span, "HTTP route", errors) {
entries.routes.push(ApiRoute { method, path, function: id, inputs,
long: binding.long,
directory: directory.clone(), anonymous: binding.anonymous,
component: component.clone(), domain: domain.clone(),
action: binding.action.text.clone(), auth: None, permission: None,
@@ -536,7 +526,7 @@ pub(super) fn register(
} else {
let name = format!("{domain_name}.{}", binding.action.text);
if occupy(&mut commands, name.clone(), binding.span, "command", errors) {
entries.commands.push(ApiCommand { name, function: id, inputs, components: Vec::new() });
entries.commands.push(ApiCommand { name, function: id, long: binding.long, inputs, components: Vec::new() });
}
}
}
@@ -981,6 +971,7 @@ fn legacy_routes(
inputs,
directory: Vec::new(),
anonymous: false,
long: false,
span: clause.span,
component: function.name.split('.').next().unwrap_or("app").to_owned(),
domain: function.name.split('.').nth(1).unwrap_or("main").to_owned(),
+14 -11
View File
@@ -113,18 +113,19 @@ pub(super) fn register(
let mut scheduled = BTreeSet::new();
for declaration in &package.declarations {
match declaration {
Declaration::Database(binding) if is_job => {
if connection
.replace(ConnectionSelector::Explicit(binding.name.text.clone()))
.is_some()
{
errors.push(Diagnostic::error(
"C002",
"duplicate Job database binding",
binding.span,
));
}
Declaration::Database(binding)
if is_job
&& connection
.replace(ConnectionSelector::Explicit(binding.name.text.clone()))
.is_some() =>
{
errors.push(Diagnostic::error(
"C002",
"duplicate Job database binding",
binding.span,
));
}
Declaration::Database(_) if is_job => {}
Declaration::Function(clause) => {
let declared = matches!(clause.kind, FunctionKind::Job { .. });
if is_job != declared {
@@ -159,6 +160,7 @@ pub(super) fn register(
let FunctionKind::Job {
attempts,
timeout_ms,
long,
} = entry.kind
else {
continue;
@@ -210,6 +212,7 @@ pub(super) fn register(
};
jobs.push(Job {
function,
long,
connection: connection.clone(),
payload,
attempts,
+12 -16
View File
@@ -525,23 +525,19 @@ fn validate_model_declarations(
let mut sql = BTreeSet::new();
for declaration in &package.declarations {
match declaration {
Declaration::Database(binding) => {
if database.replace(binding.span).is_some() {
errors.push(Diagnostic::error(
"C002",
"a Model may declare only one database binding",
binding.span,
));
}
Declaration::Database(binding) if database.replace(binding.span).is_some() => {
errors.push(Diagnostic::error(
"C002",
"a Model may declare only one database binding",
binding.span,
));
}
Declaration::Seed(value) => {
if seed.replace(value.span).is_some() {
errors.push(Diagnostic::error(
"C002",
"a Model may declare only one seed block",
value.span,
));
}
Declaration::Seed(value) if seed.replace(value.span).is_some() => {
errors.push(Diagnostic::error(
"C002",
"a Model may declare only one seed block",
value.span,
));
}
Declaration::Migration(migration) => {
if !migrations.insert(&migration.name.text) {
+1 -6
View File
@@ -548,12 +548,7 @@ pub(super) fn referenced_types(ty: &Type, visit: &mut impl FnMut(usize)) {
| Type::Stream(element)
| Type::AsyncStream(element)
| Type::RowStream(element)
| Type::Related(element)
| Type::Channel(element) => {
if !matches!(ty, Type::Related(_)) {
referenced_types(element, visit)
}
}
| Type::Channel(element) => referenced_types(element, visit),
Type::Map(key, value) | Type::MapEntry(key, value) => {
referenced_types(key, visit);
referenced_types(value, visit);
+2 -2
View File
@@ -588,7 +588,7 @@ fn validate_transactions(program: &Program, summaries: &[Effects], errors: &mut
if concurrent {
errors.push(Diagnostic::error(
"C014",
"transaction functions cannot start concurrent work",
"transaction functions cannot start concurrent work or external I/O",
function.span,
));
}
@@ -669,7 +669,7 @@ fn validate_transactions(program: &Program, summaries: &[Effects], errors: &mut
for function in concurrent {
errors.push(Diagnostic::error(
"C014",
"transaction functions cannot start concurrent work",
"transaction functions cannot start concurrent work or external I/O",
program.functions[function].span,
));
}
+7
View File
@@ -117,6 +117,9 @@ impl Formatter {
if binding.anonymous {
parts.extend([self.token("public"), Doc::space()]);
}
if binding.long {
parts.extend([self.token("long"), Doc::space()]);
}
parts.extend(vec![
self.token(binding.kind.keyword()),
Doc::space(),
@@ -377,6 +380,9 @@ impl Formatter {
parts.extend([self.token("transaction"), Doc::space()]);
}
if matches!(function.kind, FunctionKind::Job { .. }) {
if matches!(function.kind, FunctionKind::Job { long: true, .. }) {
parts.extend([self.token("long"), Doc::space()]);
}
parts.extend([self.token("job"), Doc::space()]);
}
for (index, segment) in function.name.text.split('.').enumerate() {
@@ -394,6 +400,7 @@ impl Formatter {
if let FunctionKind::Job {
attempts,
timeout_ms,
..
} = function.kind
{
for (name, value) in [("retry", attempts), ("timeout", timeout_ms)] {
+3
View File
@@ -44,6 +44,7 @@ pub(crate) struct AuthHook {
#[derive(Clone, Debug)]
pub(crate) struct Job {
pub function: usize,
pub long: bool,
pub connection: crate::model::ConnectionSelector,
pub payload: Option<crate::wire::Schema>,
pub attempts: u32,
@@ -148,6 +149,7 @@ impl TestCase {
#[derive(Clone, Debug)]
pub(crate) struct ApiRoute {
pub multipart: bool,
pub long: bool,
pub method: &'static str,
pub path: String,
pub function: usize,
@@ -166,6 +168,7 @@ pub(crate) struct ApiRoute {
#[derive(Clone, Debug)]
pub(crate) struct ApiCommand {
pub name: String,
pub long: bool,
pub function: usize,
pub inputs: Vec<String>,
pub components: Vec<String>,
+1 -7
View File
@@ -292,13 +292,7 @@ impl Module<'_> {
};
let mut emitter = FunctionEmitter::new(self, instance);
let parameters = emitter.begin_generated_function(&wrapper);
let password = [
Intrinsic::CryptoPasswordHash,
Intrinsic::CryptoPasswordVerify,
]
.iter()
.any(|operation| specialize::uses_intrinsic(emitter.module.program, &target, *operation));
let transaction = !password && emitter.begin_command_transaction(&target, route.span);
let transaction = !route.long && emitter.begin_command_transaction(&target, route.span);
let arguments = app
.parameters
.iter()
+1 -1
View File
@@ -752,7 +752,7 @@ impl FunctionEmitter<'_, '_> {
args.push((ty.clone(), value));
}
let output_ty = output_type(&function.outputs);
let transactional = self.begin_command_transaction(&instance, span);
let transactional = !command.long && self.begin_command_transaction(&instance, span);
let (status, row) = self.invoke_values(&instance, &args, &output_ty, span);
self.propagate_status(&status, span, false);
let value = self.temp();
@@ -213,6 +213,7 @@ impl FunctionEmitter<'_, '_> {
.program
.api_commands
.iter()
.filter(|command| !command.long)
.map(|command| Specialization {
function: command.function,
handlers: Vec::new(),
@@ -224,7 +225,7 @@ impl FunctionEmitter<'_, '_> {
.program
.api_routes
.iter()
.filter(|route| matches!(route.method, "POST" | "PUT" | "DELETE"))
.filter(|route| !route.long && matches!(route.method, "POST" | "PUT" | "DELETE"))
.map(|route| Specialization {
function: route.function,
handlers: Vec::new(),
@@ -248,6 +249,7 @@ impl FunctionEmitter<'_, '_> {
.program
.jobs
.iter()
.filter(|job| !job.long)
.map(|job| Specialization {
function: job.function,
handlers: Vec::new(),
+2 -1
View File
@@ -188,7 +188,8 @@ impl Module<'_> {
)
})
.collect::<Vec<_>>();
let transaction = emitter.begin_command_transaction(&target, function.span);
let transaction = !emitter.module.program.jobs[index].long
&& emitter.begin_command_transaction(&target, function.span);
if specialize::suspends(emitter.module.program, &target) {
let (status, _) =
emitter.invoke_values(&target, &arguments, &Type::Unit, function.span);
+15 -2
View File
@@ -699,9 +699,22 @@ fn emit_database_setup(
for function in program
.api_routes
.iter()
.filter(|route| !route.long)
.map(|route| route.function)
.chain(program.api_commands.iter().map(|command| command.function))
.chain(program.jobs.iter().map(|job| job.function))
.chain(
program
.api_commands
.iter()
.filter(|command| !command.long)
.map(|command| command.function),
)
.chain(
program
.jobs
.iter()
.filter(|job| !job.long)
.map(|job| job.function),
)
{
if function_writes_database(program, function) {
transaction_roots.insert(Specialization {
+4 -11
View File
@@ -352,16 +352,7 @@ fn emit_handler(
function: route.function,
handlers: Vec::new(),
};
let password_crypto = specialize::uses_intrinsic(
program,
&target,
crate::intrinsic::Intrinsic::CryptoPasswordVerify,
) || specialize::uses_intrinsic(
program,
&target,
crate::intrinsic::Intrinsic::CryptoPasswordHash,
);
let transaction_database = (!password_crypto)
let transaction_database = (!route.long)
.then(|| super::transaction_database(program, route.function))
.flatten();
if let Some(database) = &transaction_database {
@@ -478,7 +469,9 @@ pub(super) fn emit_commands(program: &Program, output: &mut String) {
function: command.function,
handlers: Vec::new(),
};
let transaction_database = super::transaction_database(program, command.function);
let transaction_database = (!command.long)
.then(|| super::transaction_database(program, command.function))
.flatten();
if let Some(database) = &transaction_database {
writeln!(output, "let transaction_database = ({database}).map_err(|error| AppError::database(\"command\", error))?;\nlet transaction = transaction_database.begin().await.map_err(|error| AppError::database(\"command\", error))?;")
.expect("string formatting");
+3 -1
View File
@@ -121,7 +121,9 @@ pub(super) fn definitions(
function: *target,
handlers: Vec::new(),
};
let transaction_database = super::transaction_database(emitter.program, *target);
let transaction_database = (!job.long)
.then(|| super::transaction_database(emitter.program, *target))
.flatten();
if let Some(database) = &transaction_database {
writeln!(output, "let transaction_database = match {database} {{ Ok(database) => database, Err(_) => return Outcome::Retry }};\nlet transaction = match transaction_database.begin().await {{ Ok(value) => value, Err(_) => return Outcome::Retry }};")
.expect("string formatting");
+31 -3
View File
@@ -195,6 +195,30 @@ impl Parser {
let mut errors = Vec::new();
while !self.at(&Kind::End) {
let visibility = self.take(&Kind::Public).map(|token| token.span);
let long = if self.at_name("long")
&& self
.tokens
.get(self.cursor + 1)
.is_some_and(|token| token.kind != Kind::LeftParen)
{
Some(self.advance().span)
} else {
None
};
let long_entry = match layout.role() {
Some(crate::source::SourceRole::Api) => ["get", "post", "put", "delete", "cmd"]
.iter()
.any(|name| self.at_name(name)),
Some(crate::source::SourceRole::Job) => self.at_name("job"),
_ => false,
};
if let Some(span) = long.filter(|_| !long_entry) {
errors.push(Diagnostic::error(
"P001",
"'long' is allowed only before an HTTP, CMD, or Job entry; place 'public' before 'long'",
span,
));
}
let anonymous_http = layout.role() == Some(crate::source::SourceRole::Api)
&& ["get", "post", "put", "delete"]
.iter()
@@ -264,8 +288,9 @@ impl Parser {
FunctionKind::Job {
attempts: 0,
timeout_ms: 0,
long: long.is_some(),
},
Some(start),
Some(long.unwrap_or(start)),
)
.map(Declaration::Function)
} else if self.at_name("transaction") {
@@ -280,7 +305,8 @@ impl Parser {
Ok(mut declaration) => {
if let Declaration::Api(binding) = &mut declaration {
binding.anonymous = visibility.is_some() && anonymous_http;
if let Some(start) = visibility {
binding.long = long.is_some();
if let Some(start) = visibility.or(long) {
binding.span = start.through(binding.span);
}
}
@@ -409,6 +435,7 @@ impl Parser {
self.line_end(&Kind::End)?;
Ok(Declaration::Api(ApiBinding {
anonymous: false,
long: false,
kind,
action,
target,
@@ -925,13 +952,14 @@ impl Parser {
let inputs = self.separated(Kind::RightParen, Self::input)?;
self.expect(&Kind::LeftParen, "the named output list '('")?;
let outputs = self.separated(Kind::RightParen, Self::field)?;
if matches!(kind, FunctionKind::Job { .. }) {
if let FunctionKind::Job { long, .. } = kind {
let attempts = self.job_policy("retry", 100)?;
let timeout_ms =
self.job_policy("timeout", dever_runtime::config::MAX_JOB_TIMEOUT_MS)?;
kind = FunctionKind::Job {
attempts,
timeout_ms,
long,
};
}
let (pure, recovery) = self.function_contracts()?;
+12 -5
View File
@@ -172,9 +172,6 @@ pub(crate) fn uses_intrinsic(
})
}
/// Finds a database write reachable without first crossing an explicit
/// transaction function. Password crypto APIs use this to keep slow work out
/// of SQLite transactions while preserving a short, explicit write boundary.
pub(crate) fn write_outside_transaction(
program: &crate::hir::Program,
root: &Specialization,
@@ -237,7 +234,7 @@ pub(crate) fn function_starts_transaction_unsafe_work(
program: &crate::hir::Program,
function: usize,
) -> bool {
let mut unsafe_work = false;
let mut unsafe_work = program.functions[function].external.is_some();
crate::check::visit(&program.functions[function], |expression| {
unsafe_work |= match &expression.kind {
ExpressionKind::RunCall { .. }
@@ -259,7 +256,17 @@ pub(crate) fn function_starts_transaction_unsafe_work(
}
ExpressionKind::Intrinsic { operation, .. } => {
operation.is_async()
&& !matches!(operation, crate::intrinsic::Intrinsic::UploadStore)
|| matches!(
operation,
crate::intrinsic::Intrinsic::CryptoPasswordHash
| crate::intrinsic::Intrinsic::CryptoPasswordVerify
| crate::intrinsic::Intrinsic::FileOpen
| crate::intrinsic::Intrinsic::TimeSleep
| crate::intrinsic::Intrinsic::FileCreate
| crate::intrinsic::Intrinsic::FileRead
| crate::intrinsic::Intrinsic::FileWrite
| crate::intrinsic::Intrinsic::FileChunks
)
}
ExpressionKind::Collection {
operation,
+6 -1
View File
@@ -106,6 +106,7 @@ impl ApiKind {
#[derive(Clone, Debug, PartialEq)]
pub struct ApiBinding {
pub anonymous: bool,
pub long: bool,
pub kind: ApiKind,
pub action: Name,
pub target: Path,
@@ -277,7 +278,11 @@ pub struct FunctionClause {
pub enum FunctionKind {
Ordinary,
Transaction,
Job { attempts: u32, timeout_ms: u32 },
Job {
attempts: u32,
timeout_ms: u32,
long: bool,
},
}
#[derive(Clone, Debug, PartialEq)]
@@ -1,5 +1,5 @@
[package]
name = "dever-sandbox"
name = "dever-process"
version.workspace = true
edition.workspace = true
rust-version.workspace = true
@@ -9,20 +9,11 @@ publish.workspace = true
workspace = true
[dependencies]
sha2.workspace = true
goblin = { version = "=0.10.7", default-features = false, features = ["elf32", "elf64", "endian_fd", "std"] }
[target.'cfg(target_os = "linux")'.dependencies]
seccompiler = { version = "=0.5.0", default-features = false }
libc = "=0.2.189"
command-fds = "=0.3.3"
nix = { version = "=0.31.3", default-features = false }
rustix = { version = "=1.1.2", features = ["fs", "process"] }
[[bin]]
name = "dever-sandbox-guard"
path = "src/main.rs"
[[test]]
name = "isolation"
path = "../../test/dever-sandbox-tests/isolation.rs"
name = "execution"
path = "../../test/dever-process-tests/execution.rs"
@@ -5,17 +5,11 @@ use std::fs;
use std::io::Read;
use std::path::Path;
use goblin::elf::{Elf, dynamic, header, program_header};
use sha2::{Digest, Sha256};
use goblin::elf::{Elf, header};
const MAX_ASSET_BYTES: usize = 32 * 1024 * 1024;
const MAX_ASSET_FILES: usize = 128;
/// The first executable must be independent of every host loader and library.
pub fn validate_bwrap(bytes: &[u8]) -> Result<String, String> {
validate_bwrap_for_target(bytes, host_target())
}
fn host_target() -> &'static str {
if cfg!(target_arch = "aarch64") {
"linux-aarch64"
@@ -36,36 +30,10 @@ fn target_abi(target: &str) -> Result<(u16, &'static str, &'static str), String>
"ld-linux-aarch64.so.1",
"/lib/ld-linux-aarch64.so.1",
)),
_ => Err(format!("unsupported sandbox target '{target}'")),
_ => Err(format!("unsupported process_runtime target '{target}'")),
}
}
pub fn validate_bwrap_for_target(bytes: &[u8], target: &str) -> Result<String, String> {
let elf = Elf::parse(bytes).map_err(|error| format!("invalid sandbox bwrap ELF: {error}"))?;
let (machine, _, _) = target_abi(target)?;
if bytes.len() > MAX_ASSET_BYTES
|| !elf.is_64
|| !elf.little_endian
|| elf.header.e_machine != machine
|| !matches!(elf.header.e_type, header::ET_DYN | header::ET_EXEC)
|| elf
.program_headers
.iter()
.any(|segment| segment.p_type == program_header::PT_INTERP)
|| elf.dynamic.as_ref().is_some_and(|table| {
table
.dyns
.iter()
.any(|entry| entry.d_tag == dynamic::DT_NEEDED)
})
|| !elf.rpaths.is_empty()
|| !elf.runpaths.is_empty()
{
return Err("sandbox bwrap must be a static target ELF without PT_INTERP, DT_NEEDED or library search paths".into());
}
Ok(format!("{:x}", Sha256::digest(bytes)))
}
/// Used both before signing a pack and when launching its verified extraction.
/// Content authentication belongs to the release/bundle owner; this validates
/// that the loader can resolve every dependency without a host-library search.
@@ -78,30 +46,24 @@ pub fn validate_assets_for_target(files: &[(String, Vec<u8>)], target: &str) ->
let total = files
.iter()
.try_fold(0usize, |total, (_, bytes)| total.checked_add(bytes.len()))
.ok_or("sandbox asset size overflow")?;
.ok_or("process_runtime asset size overflow")?;
if files.len() > MAX_ASSET_FILES || total > MAX_ASSET_BYTES {
return Err("sandbox assets exceed their file or byte budget".into());
return Err("process_runtime assets exceed their file or byte budget".into());
}
let mut supplied = BTreeMap::new();
for (name, bytes) in files {
let valid = matches!(name.as_str(), "bin/bwrap" | "bin/guard")
|| name.strip_prefix("lib/").is_some_and(|name| {
!name.is_empty() && name != "." && name != ".." && !name.contains(['/', '\\', '\0'])
});
let valid = name.strip_prefix("lib/").is_some_and(|name| {
!name.is_empty() && name != "." && name != ".." && !name.contains(['/', '\\', '\0'])
});
if !valid || supplied.insert(name.as_str(), bytes.as_slice()).is_some() {
return Err("sandbox asset has an invalid or repeated path".into());
return Err("process_runtime asset has an invalid or repeated path".into());
}
}
let loader = format!("lib/{loader_name}");
for required in ["bin/bwrap", "bin/guard", &loader] {
if !supplied.contains_key(required) {
return Err(format!("sandbox asset '{required}' is missing"));
}
if !supplied.contains_key(loader.as_str()) {
return Err(format!("process_runtime asset '{loader}' is missing"));
}
validate_bwrap_for_target(supplied["bin/bwrap"], target)?;
let mut reached = BTreeSet::new();
// The shared OS ABI set also serves packaged language runtimes. Validate
// every supplied library even when the two bootstrap helpers do not use it.
let mut remaining = supplied
.keys()
.map(|name| (*name).to_owned())
@@ -112,29 +74,33 @@ pub fn validate_assets_for_target(files: &[(String, Vec<u8>)], target: &str) ->
}
let bytes = supplied
.get(name.as_str())
.ok_or_else(|| format!("sandbox dependency '{name}' is missing"))?;
let elf =
Elf::parse(bytes).map_err(|error| format!("invalid sandbox ELF asset: {error}"))?;
.ok_or_else(|| format!("process_runtime dependency '{name}' is missing"))?;
let elf = Elf::parse(bytes)
.map_err(|error| format!("invalid process_runtime ELF asset: {error}"))?;
if !elf.is_64
|| !elf.little_endian
|| elf.header.e_machine != machine
|| !matches!(elf.header.e_type, header::ET_DYN | header::ET_EXEC)
{
return Err("sandbox ELF asset has an incompatible target or object type".into());
return Err(
"process_runtime ELF asset has an incompatible target or object type".into(),
);
}
// Reject paths that can outrank the explicit bootstrap library path.
if !elf.rpaths.is_empty() || !elf.runpaths.is_empty() {
return Err("sandbox bootstrap assets must not declare RPATH or RUNPATH".into());
return Err(
"process_runtime bootstrap assets must not declare RPATH or RUNPATH".into(),
);
}
if let Some(actual) = elf.interpreter
&& actual != interpreter
{
return Err("sandbox ELF interpreter differs from the packaged loader".into());
return Err("process_runtime ELF interpreter differs from the packaged loader".into());
}
for library in elf.libraries {
if library.is_empty() || library.contains(['/', '\\']) || matches!(library, "." | "..")
{
return Err("sandbox dependency is not a plain library name".into());
return Err("process_runtime dependency is not a plain library name".into());
}
remaining.push(format!("lib/{library}"));
}
@@ -142,10 +108,10 @@ pub fn validate_assets_for_target(files: &[(String, Vec<u8>)], target: &str) ->
Ok(())
}
pub(super) fn verify_directory(root: &Path) -> Result<String, String> {
pub(super) fn verify_directory(root: &Path) -> Result<(), String> {
let mut files = Vec::new();
let mut remaining = MAX_ASSET_BYTES;
for directory in ["bin", "lib"] {
for directory in ["lib"] {
let directory_path = super::real_path(&root.join(directory))?;
for entry in fs::read_dir(directory_path).map_err(|error| error.to_string())? {
let entry = entry.map_err(|error| error.to_string())?;
@@ -156,27 +122,24 @@ pub(super) fn verify_directory(root: &Path) -> Result<String, String> {
.is_file()
|| files.len() >= MAX_ASSET_FILES
{
return Err("sandbox assets contain a non-regular file or too many files".into());
return Err(
"process_runtime assets contain a non-regular file or too many files".into(),
);
}
let mut bytes = Vec::new();
fs::File::open(path)
.and_then(|file| file.take(remaining as u64 + 1).read_to_end(&mut bytes))
.map_err(|error| format!("cannot read sandbox asset: {error}"))?;
.map_err(|error| format!("cannot read process_runtime asset: {error}"))?;
remaining = remaining
.checked_sub(bytes.len())
.ok_or("sandbox assets exceed their byte budget")?;
.ok_or("process_runtime assets exceed their byte budget")?;
let name = entry
.file_name()
.into_string()
.map_err(|_| "sandbox asset name is not UTF-8")?;
.map_err(|_| "process_runtime asset name is not UTF-8")?;
files.push((format!("{directory}/{name}"), bytes));
}
}
validate_assets(&files)?;
let bytes = &files
.iter()
.find(|(name, _)| name == "bin/bwrap")
.ok_or("sandbox bwrap is missing")?
.1;
Ok(format!("{:x}", Sha256::digest(bytes)))
Ok(())
}
+160
View File
@@ -0,0 +1,160 @@
use std::ffi::OsString;
use std::path::{Component, Path, PathBuf};
use std::process::Command;
mod assets;
pub use assets::{validate_assets, validate_assets_for_target};
pub fn asset_executable(path: &str) -> bool {
matches!(
path,
"lib/ld-linux-x86-64.so.2" | "lib/ld-linux-aarch64.so.1"
)
}
pub struct Launch<'a> {
pub assets: &'a Path,
pub worker: &'a Path,
pub executable: &'a Path,
pub arguments: &'a [OsString],
pub working_directory: &'a Path,
}
pub fn require_no_loader_preload() -> Result<(), String> {
match std::fs::symlink_metadata("/etc/ld.so.preload") {
Ok(_) => {
Err("private executable does not support host loader preload configuration".into())
}
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(()),
Err(error) => Err(format!(
"cannot inspect host loader preload configuration: {error}"
)),
}
}
impl Launch<'_> {
pub fn command(&self) -> Result<Command, String> {
if !cfg!(all(
target_os = "linux",
any(target_arch = "x86_64", target_arch = "aarch64")
)) {
return Err("managed external programs currently require Linux amd64 or arm64".into());
}
let assets = real_path(self.assets)?;
assets::verify_directory(&assets)?;
require_no_loader_preload()?;
let worker = real_path(self.worker)?;
let executable = real_path(self.executable)?;
let directory = real_path(self.working_directory)?;
if !executable.starts_with(&worker) || !directory.starts_with(&worker) {
return Err("managed execution path escapes its verified resource tree".into());
}
let bytes = std::fs::read(&executable).map_err(|error| error.to_string())?;
if !bytes.starts_with(b"#!/bin/sh\n") {
goblin::elf::Elf::parse(&bytes)
.map_err(|error| format!("invalid executable ELF: {error}"))?;
}
let mut paths = vec![worker.join("lib"), worker.join("runtime/lib")];
if let Some(inherited) = std::env::var_os("LD_LIBRARY_PATH") {
paths.extend(
std::env::split_paths(&inherited).filter(|path| !path.as_os_str().is_empty()),
);
}
let libraries = std::env::join_paths(paths).map_err(|error| error.to_string())?;
let mut command = Command::new(&executable);
command.args(self.arguments).current_dir(directory);
configure(&mut command);
command.env("LD_LIBRARY_PATH", libraries);
Ok(command)
}
}
pub fn configure(command: &mut Command) {
command.env_clear();
for name in [
"CUDA_VISIBLE_DEVICES",
"NVIDIA_VISIBLE_DEVICES",
"NVIDIA_DRIVER_CAPABILITIES",
"CUDA_DEVICE_ORDER",
"LD_LIBRARY_PATH",
] {
if let Some(value) = std::env::var_os(name) {
command.env(name, value);
}
}
#[cfg(unix)]
{
use std::os::unix::process::CommandExt;
command.process_group(0);
}
}
pub fn loader_name() -> &'static str {
if cfg!(target_arch = "aarch64") {
"ld-linux-aarch64.so.1"
} else {
"ld-linux-x86-64.so.2"
}
}
fn real_path(path: &Path) -> Result<PathBuf, String> {
if !path.is_absolute()
|| path
.components()
.any(|part| !matches!(part, Component::RootDir | Component::Normal(_)))
{
return Err("execution path must be absolute and normalized".into());
}
for ancestor in path.ancestors() {
let metadata = std::fs::symlink_metadata(ancestor).map_err(|error| error.to_string())?;
if metadata.file_type().is_symlink() {
return Err("execution input must not contain symbolic links".into());
}
}
Ok(path.to_owned())
}
#[cfg(target_os = "linux")]
pub struct ProcessGroup {
pid: rustix::process::Pid,
descriptor: std::os::fd::OwnedFd,
}
#[cfg(target_os = "linux")]
impl ProcessGroup {
pub fn pin(pid: u32) -> Result<Self, String> {
use rustix::process::{Pid, PidfdFlags, pidfd_open};
let pid = i32::try_from(pid)
.ok()
.and_then(Pid::from_raw)
.ok_or("invalid child PID")?;
let descriptor =
pidfd_open(pid, PidfdFlags::NONBLOCK).map_err(|error| error.to_string())?;
Ok(Self { pid, descriptor })
}
pub fn exited(&self) -> Result<bool, String> {
use rustix::process::{WaitId, WaitIdOptions, waitid};
use std::os::fd::AsFd;
waitid(
WaitId::PidFd(self.descriptor.as_fd()),
WaitIdOptions::EXITED | WaitIdOptions::NOHANG | WaitIdOptions::NOWAIT,
)
.map(|status| status.is_some())
.map_err(|error| error.to_string())
}
pub fn terminate(&self) -> Result<(), String> {
match rustix::process::kill_process_group(self.pid, rustix::process::Signal::KILL) {
Ok(()) | Err(rustix::io::Errno::SRCH) => Ok(()),
Err(error) => Err(format!("cannot stop owned process group: {error}")),
}
}
}
#[cfg(target_os = "linux")]
impl std::os::fd::AsRawFd for ProcessGroup {
fn as_raw_fd(&self) -> std::os::fd::RawFd {
std::os::fd::AsRawFd::as_raw_fd(&self.descriptor)
}
}
+2 -2
View File
@@ -13,14 +13,14 @@ default = []
auth-crypto = ["dep:sha2", "dep:hmac", "dep:subtle"]
crypto = ["auth-crypto", "dep:argon2", "dep:getrandom"]
wire = ["dep:serde", "dep:serde_json", "serde_json/raw_value", "dep:base64", "dep:getrandom"]
external = ["wire", "dep:sha2", "dep:dever-sandbox"]
external = ["wire", "dep:sha2", "dep:dever-process"]
api = ["wire", "auth-crypto"]
database = ["wire", "dep:getrandom"]
sqlite = ["database", "dep:deadpool-sqlite"]
postgres = ["database", "dep:deadpool-postgres", "dep:tokio-postgres", "dep:tokio-postgres-rustls"]
[dependencies]
dever-sandbox = { path = "../dever-sandbox", optional = true }
dever-process = { path = "../dever-process", optional = true }
dec.workspace = true
indexmap.workspace = true
tokio.workspace = true
+29 -18
View File
@@ -5,7 +5,13 @@ use std::sync::Arc;
use std::time::{Duration, Instant};
use tokio::io::{AsyncRead, AsyncReadExt, AsyncWrite, AsyncWriteExt};
use tokio::process::{Child, ChildStderr, ChildStdin, ChildStdout, Command};
#[cfg(not(feature = "external"))]
use tokio::process::Child;
use tokio::process::{ChildStderr, ChildStdin, ChildStdout, Command};
#[cfg(feature = "external")]
mod child;
#[cfg(feature = "external")]
use child::Child;
use tokio::sync::{Mutex, mpsc, oneshot};
mod command;
@@ -99,7 +105,6 @@ fn session() -> Result<Arc<Session>, String> {
current().ok_or_else(|| "external Adapter requires a Dever invocation".into())
}
/// One settings snapshot owns both Adapter selection and deployment grants.
pub fn adapter_settings() -> Result<Arc<crate::config::Settings>, String> {
#[cfg(feature = "api")]
if let Some(application) = crate::application::current() {
@@ -359,14 +364,11 @@ async fn supervise(
let mut command = worker_command(&definition)?;
command
.arg("--dever-component")
.env_clear()
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.kill_on_drop(true);
let mut child = command
.spawn()
.map_err(|error| format!("cannot start external Adapter worker: {error}"))?;
let mut child = spawn_child(&mut command)?;
let Some(stdin) = child.stdin.take() else {
terminate_child(&mut child).await?;
return Err("external Adapter worker has no stdin".into());
@@ -395,7 +397,7 @@ fn worker_command(definition: &Definition) -> Result<Command, String> {
#[cfg(not(feature = "external"))]
{
let _ = definition;
Err("external Worker sandbox runtime feature is not enabled".into())
Err("external Worker process_runtime runtime feature is not enabled".into())
}
}
@@ -530,21 +532,30 @@ async fn wait_for_exit(child: &mut Child) -> Result<(), String> {
}
}
async fn terminate_child(child: &mut Child) -> Result<(), String> {
match child.try_wait() {
Ok(Some(_)) => return Ok(()),
Ok(None) => {}
Err(error) => return Err(format!("cannot inspect external Adapter worker: {error}")),
}
if let Err(error) = child.start_kill()
&& !matches!(child.try_wait(), Ok(Some(_)))
fn spawn_child(command: &mut Command) -> Result<Child, String> {
let child = command
.spawn()
.map_err(|error| format!("cannot start external program: {error}"))?;
#[cfg(feature = "external")]
{
return Err(format!("cannot stop external Adapter worker: {error}"));
Child::new(child)
}
#[cfg(not(feature = "external"))]
{
Ok(child)
}
}
async fn terminate_child(child: &mut Child) -> Result<(), String> {
if child.id().is_some() {
child
.start_kill()
.map_err(|error| format!("cannot stop external program: {error}"))?;
}
match tokio::time::timeout(SHUTDOWN_GRACE, child.wait()).await {
Ok(Ok(_)) => Ok(()),
Ok(Err(error)) => Err(format!("cannot wait for external Adapter worker: {error}")),
Err(_) => Err("external Adapter worker could not be reaped before its deadline".into()),
Ok(Err(error)) => Err(format!("cannot reap external program: {error}")),
Err(_) => Err("external program exceeded its cleanup deadline".into()),
}
}
@@ -0,0 +1,76 @@
use std::io;
use std::ops::{Deref, DerefMut};
use std::process::ExitStatus;
pub(super) struct Child {
child: tokio::process::Child,
#[cfg(target_os = "linux")]
group: Option<tokio::io::unix::AsyncFd<dever_process::ProcessGroup>>,
}
impl Child {
pub(super) fn new(mut child: tokio::process::Child) -> Result<Self, String> {
#[cfg(target_os = "linux")]
{
let result =
dever_process::ProcessGroup::pin(child.id().ok_or("new child has no PID")?)
.and_then(|group| {
tokio::io::unix::AsyncFd::new(group).map_err(|error| error.to_string())
});
match result {
Ok(group) => Ok(Self {
child,
group: Some(group),
}),
Err(error) => {
let _ = child.start_kill();
Err(error)
}
}
}
#[cfg(not(target_os = "linux"))]
{
let _ = child.start_kill();
Err("managed external programs currently require Linux".into())
}
}
fn stop_group(&mut self) -> io::Result<()> {
#[cfg(target_os = "linux")]
if let Some(group) = self.group.take() {
group.get_ref().terminate().map_err(io::Error::other)?;
}
Ok(())
}
pub(super) fn start_kill(&mut self) -> io::Result<()> {
self.stop_group()?;
self.child.start_kill()
}
pub(super) async fn wait(&mut self) -> io::Result<ExitStatus> {
#[cfg(target_os = "linux")]
if let Some(group) = &self.group {
let _ready = group.readable().await?;
}
self.stop_group()?;
self.child.wait().await
}
}
impl Deref for Child {
type Target = tokio::process::Child;
fn deref(&self) -> &Self::Target {
&self.child
}
}
impl DerefMut for Child {
fn deref_mut(&mut self) -> &mut Self::Target {
&mut self.child
}
}
impl Drop for Child {
fn drop(&mut self) {
let _ = self.stop_group();
}
}
+17 -46
View File
@@ -1,14 +1,13 @@
//! One ordinary child per request, owned by the existing Adapter supervisor.
use super::{Definition, Reply, Request, adapter_settings, terminate_child, worker_command};
#[cfg(feature = "external")]
use super::spawn_child;
use super::{Child, Definition, Reply, Request, adapter_settings, terminate_child, worker_command};
use crate::bytes::Bytes;
use crate::config::CommandLimits;
use tokio::io::{AsyncReadExt, AsyncWriteExt};
use tokio::process::{Child, ChildStderr, ChildStdin, ChildStdout};
use tokio::process::{ChildStderr, ChildStdin, ChildStdout};
use tokio::sync::{mpsc, oneshot};
mod observation;
use observation::Observation;
#[cfg(feature = "external")]
const ARGUMENT_BYTES: usize = 65_536;
#[cfg(feature = "external")]
@@ -71,7 +70,6 @@ pub(super) async fn supervise(
) -> Result<(), String> {
let prepared: Result<CommandLimits, String> = (|| {
let limits = adapter_settings()?.command_limits(&definition.port)?;
// Validate resources and grants without starting the user's executable.
worker_command(&definition)?;
Ok(limits)
})();
@@ -155,31 +153,27 @@ fn input(payload: &str) -> Result<Input, String> {
}
#[cfg(feature = "external")]
fn spawn(definition: &Definition, payload: &str) -> Result<(Input, Child, Observation), String> {
fn spawn(definition: &Definition, payload: &str) -> Result<(Input, Child), String> {
let input = input(payload)?;
let (command, observation) = crate::external::observed_command(definition)?;
let observation = Observation::new(observation)?;
let command = crate::external::worker_command(definition)?;
let mut command = tokio::process::Command::from(command);
// Resource arguments were mapped by the sandbox owner. Call arguments
// are opaque values, including absolute sandbox paths and shell syntax.
// Resource arguments were mapped by the process_runtime owner. Call arguments
// are opaque values, including absolute process_runtime paths and shell syntax.
command
.args(&input.arguments)
.env_clear()
.stdin(std::process::Stdio::piped())
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.kill_on_drop(true);
let child = command
.spawn()
.map_err(|error| format!("cannot start external command: {error}"))?;
let child = spawn_child(&mut command)?;
// Dropping Command closes the parent's copies of mapped pipe descriptors.
drop(command);
Ok((input, child, observation))
Ok((input, child))
}
#[cfg(not(feature = "external"))]
fn spawn(_: &Definition, _: &str) -> Result<(Input, Child, Observation), String> {
Err("external command sandbox runtime feature is not enabled".into())
fn spawn(_: &Definition, _: &str) -> Result<(Input, Child), String> {
Err("external command process_runtime runtime feature is not enabled".into())
}
async fn execute(
@@ -188,7 +182,7 @@ async fn execute(
request: &mut Request,
stopping: &mut oneshot::Receiver<()>,
) -> Execution {
let (input, mut child, mut observation) = match spawn(definition, &request.payload) {
let (input, mut child) = match spawn(definition, &request.payload) {
Ok(prepared) => prepared,
Err(error) => {
return Execution {
@@ -198,12 +192,9 @@ async fn execute(
};
}
};
let wrapper = child
.id()
.expect("newly spawned command launcher has a PID");
let pipes = (child.stdin.take(), child.stdout.take(), child.stderr.take());
let (Some(stdin), Some(stdout), Some(stderr)) = pipes else {
let cleanup_error = cleanup(&mut child, &mut observation, wrapper).await.err();
let cleanup_error = terminate_child(&mut child).await.err();
return Execution {
reply: Err(cleanup_error
.clone()
@@ -215,8 +206,7 @@ async fn execute(
let mut stopped = false;
let result = {
let completion = async {
let (_, _, (stdout, stderr), status) = tokio::try_join!(
observation.started(wrapper),
let (_, (stdout, stderr), status) = tokio::try_join!(
write_stdin(stdin, input.stdin),
capture(stdout, stderr, limits.output_limit),
async {
@@ -228,7 +218,7 @@ async fn execute(
)?;
let code = status
.code()
.ok_or("external command sandbox terminated without an exit code")?;
.ok_or("external command process_runtime terminated without an exit code")?;
output(code, stdout, stderr)
};
tokio::select! {
@@ -243,7 +233,7 @@ async fn execute(
}
}
};
let cleanup_error = cleanup(&mut child, &mut observation, wrapper).await.err();
let cleanup_error = terminate_child(&mut child).await.err();
let reply = match (result, &cleanup_error) {
(result, None) => result,
(Ok(_), Some(cleanup)) => Err(cleanup.clone()),
@@ -256,25 +246,6 @@ async fn execute(
}
}
async fn cleanup(
child: &mut Child,
observation: &mut Observation,
wrapper: u32,
) -> Result<(), String> {
let namespace = tokio::time::timeout(super::SHUTDOWN_GRACE, observation.terminate(wrapper))
.await
.unwrap_or_else(|_| Err("external command namespace exceeded its cleanup deadline".into()));
// Reap the owned launcher even if namespace observation itself failed.
// Both stages have independent finite deadlines; neither error is lost.
let launcher = terminate_child(child).await;
match (namespace, launcher) {
(Err(namespace), Err(launcher)) => {
Err(format!("{namespace}; launcher cleanup failed: {launcher}"))
}
(namespace, launcher) => namespace.and(launcher),
}
}
async fn write_stdin(mut stdin: ChildStdin, input: Option<Bytes>) -> Result<(), String> {
if let Some(input) = input {
match stdin.write_all(input.values()).await {
@@ -1,135 +0,0 @@
#[cfg(all(feature = "external", target_os = "linux"))]
use tokio::io::AsyncReadExt;
pub(super) struct Observation {
#[cfg(all(feature = "external", target_os = "linux"))]
startup: tokio::net::unix::pipe::Receiver,
#[cfg(all(feature = "external", target_os = "linux"))]
information: tokio::net::unix::pipe::Receiver,
#[cfg(all(feature = "external", target_os = "linux"))]
information_bytes: Vec<u8>,
#[cfg(all(feature = "external", target_os = "linux"))]
release: Option<std::fs::File>,
#[cfg(all(feature = "external", target_os = "linux"))]
namespace: Option<tokio::io::unix::AsyncFd<dever_sandbox::CommandNamespace>>,
}
impl Observation {
#[cfg(all(feature = "external", target_os = "linux"))]
pub(super) fn new(observation: dever_sandbox::CommandObservation) -> Result<Self, String> {
use tokio::net::unix::pipe::Receiver;
Ok(Self {
startup: Receiver::from_file(observation.startup).map_err(|error| error.to_string())?,
information: Receiver::from_file(observation.namespace)
.map_err(|error| error.to_string())?,
information_bytes: Vec::new(),
release: Some(observation.release),
namespace: None,
})
}
#[cfg(all(feature = "external", not(target_os = "linux")))]
pub(super) fn new(_: dever_sandbox::CommandObservation) -> Result<Self, String> {
Err("external commands require Linux".into())
}
#[cfg(all(feature = "external", target_os = "linux"))]
async fn pin(&mut self, wrapper: u32) -> Result<(), String> {
if self.namespace.is_some() {
return Ok(());
}
let mut buffer = [0; 1024];
loop {
let count = self
.information
.read(&mut buffer)
.await
.map_err(|error| format!("cannot read command namespace identity: {error}"))?;
if count == 0 {
break;
}
if self.information_bytes.len() + count > 4096 {
return Err("command namespace identity exceeds its byte limit".into());
}
self.information_bytes.extend_from_slice(&buffer[..count]);
}
let text = std::str::from_utf8(&self.information_bytes)
.map_err(|_| "command namespace identity is not UTF-8")?;
let node = crate::wire::parse(text)?;
let pid = node
.object()?
.get("child-pid")
.ok_or("command namespace identity is missing")?
.int()?;
let namespace = dever_sandbox::CommandNamespace::pin(
i32::try_from(pid).map_err(|_| "invalid command namespace PID")?,
wrapper,
)?;
self.namespace = Some(
tokio::io::unix::AsyncFd::new(namespace)
.map_err(|error| format!("cannot observe command namespace exit: {error}"))?,
);
Ok(())
}
pub(super) async fn started(&mut self, wrapper: u32) -> Result<(), String> {
#[cfg(all(feature = "external", target_os = "linux"))]
{
self.pin(wrapper).await?;
// EOF releases bwrap only after its namespace is pinned. Neither
// normal cancellation nor a very short process can race PID reuse.
drop(self.release.take());
let mut status = Vec::new();
(&mut self.startup)
.take(4096)
.read_to_end(&mut status)
.await
.map_err(|error| format!("cannot read command startup status: {error}"))?;
match status.as_slice() {
b"R" => Ok(()),
[] => Err("external command sandbox failed before starting its guard".into()),
status => {
let message = status
.strip_prefix(b"RE")
.or_else(|| status.strip_prefix(b"E"));
Err(match message {
Some(message) => format!(
"external command startup failed: {}",
String::from_utf8_lossy(message)
),
None => "invalid external command startup status".into(),
})
}
}
}
#[cfg(not(all(feature = "external", target_os = "linux")))]
{
let _ = wrapper;
Err("external commands require Linux".into())
}
}
pub(super) async fn terminate(&mut self, wrapper: u32) -> Result<(), String> {
#[cfg(all(feature = "external", target_os = "linux"))]
{
self.pin(wrapper).await?;
let namespace = self
.namespace
.as_ref()
.expect("namespace pinned before cleanup");
namespace.get_ref().terminate()?;
// pidfd readiness follows kernel namespace teardown, unlike pipe
// EOF, which occurs while the init is still closing its files.
let _exited = namespace
.readable()
.await
.map_err(|error| format!("cannot wait for command namespace exit: {error}"))?;
Ok(())
}
#[cfg(not(all(feature = "external", target_os = "linux")))]
{
let _ = wrapper;
Err("external commands require Linux".into())
}
}
}
+1 -72
View File
@@ -11,7 +11,7 @@ pub use command::CommandLimits;
pub use compilation::{CompilationBindings, CompilationSite, DatabaseDriver};
const DATA_DIRECTORIES: [&str; 5] = ["db", "upload", "log", "cache", "tmp"];
const ADAPTER_FIELDS: &[&str] = &["use", "setting", "files", "command"];
const ADAPTER_FIELDS: &[&str] = &["use", "setting", "command"];
static SETTINGS: OnceLock<Settings> = OnceLock::new();
pub const MAX_JOB_TIMEOUT_MS: u32 = 3_600_000;
pub const MAX_JOB_LEASE_MS: i64 = 2 * MAX_JOB_TIMEOUT_MS as i64;
@@ -764,77 +764,6 @@ impl Settings {
Ok((index, setting))
}
/// File access is a deployment grant for the selected Port, not Worker
/// business settings. Paths stay below the application and appear at /data.
#[cfg(feature = "external")]
pub fn external_grants(
&self,
identity: &str,
capabilities: &[String],
) -> Result<Vec<dever_sandbox::Grant>, String> {
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct FileGrant {
path: PathBuf,
#[serde(default)]
write: bool,
}
let Some(adapter) = &self.adapter else {
return Ok(Vec::new());
};
let node = crate::wire::parse(adapter.get())?;
let Some(binding) = node.object()?.get(identity) else {
return Ok(Vec::new());
};
let binding = binding.fields(ADAPTER_FIELDS)?;
let Some(files) = binding.get("files") else {
return Ok(Vec::new());
};
if !capabilities.iter().any(|capability| capability == "file") {
return Err("Adapter file grants require its checked file capability".into());
}
let files: BTreeMap<String, FileGrant> = serde_json::from_str(files.raw())
.map_err(|_| "Adapter files must map names to {path, write} grants")?;
if files.len() > 64 {
return Err("Adapter has too many file grants".into());
}
let mut grants = Vec::new();
for (name, grant) in files {
if name.is_empty()
|| name.len() > 64
|| !name
.bytes()
.all(|byte| byte.is_ascii_lowercase() || byte.is_ascii_digit() || byte == b'_')
{
return Err("Adapter file grant names must contain lowercase letters, digits or underscores".into());
}
if grant.path.as_os_str().is_empty()
|| grant.path.is_absolute()
|| grant
.path
.components()
.any(|part| !matches!(part, Component::Normal(_)))
{
return Err("Adapter file grant path must be relative to the application".into());
}
let executable_cache = Path::new("data/cache/lib");
if grant.write
&& (grant.path.starts_with(executable_cache)
|| executable_cache.starts_with(&grant.path))
{
return Err(
"Adapter writable file grant overlaps the executable resource cache".into(),
);
}
grants.push(dever_sandbox::Grant {
source: self.executable_directory.join(grant.path),
destination: Path::new("/data").join(name),
writable: grant.write,
});
}
Ok(grants)
}
pub fn initialize_data_directories(&self) -> Result<(), String> {
for name in DATA_DIRECTORIES {
let path = self.executable_directory.join("data").join(name);
+6 -39
View File
@@ -87,59 +87,26 @@ struct LiteralArgument {
pub fn worker_command(
definition: &crate::component::Definition,
) -> Result<std::process::Command, String> {
sandbox_launch(definition, |launch| launch.command())
}
pub(crate) fn observed_command(
definition: &crate::component::Definition,
) -> Result<(std::process::Command, dever_sandbox::CommandObservation), String> {
sandbox_launch(definition, |launch| launch.observed_command())
}
fn sandbox_launch<T>(
definition: &crate::component::Definition,
build: impl FnOnce(dever_sandbox::Launch<'_>) -> Result<T, String>,
) -> Result<T, String> {
let session =
crate::component::current().ok_or("external resources require a Dever invocation")?;
let bundle = session
.resources
.get()
.ok_or("external Worker resources and sandbox were not prepared")?;
.ok_or("external Worker resources were not prepared")?;
let launch = verified_worker_launch(
&bundle.directory,
&bundle.resources,
&definition.entry,
&definition.ecosystem,
)?;
let settings = crate::component::adapter_settings()?
.external_grants(&definition.port, &definition.capabilities)?;
if definition
.capabilities
.iter()
.any(|capability| capability == "gpu")
{
return Err("GPU Worker requires a supported device isolation backend".into());
}
let arguments = launch.arguments;
build(dever_sandbox::Launch {
assets: &bundle.directory.join("sandbox"),
dever_process::Launch {
assets: &bundle.directory.join("process-runtime"),
worker: &launch.working_directory,
executable: &launch.executable,
arguments: &arguments,
arguments: &launch.arguments,
working_directory: &launch.working_directory,
capabilities: dever_sandbox::Capabilities {
network: definition
.capabilities
.iter()
.any(|capability| capability == "network"),
process: definition
.capabilities
.iter()
.any(|capability| capability == "process"),
},
grants: &settings,
})
}
.command()
}
pub fn verified_worker_launch(
-322
View File
@@ -1,322 +0,0 @@
//! Fixed allowlist for packaged Linux runtimes. Mounts own file visibility;
//! this filter owns syscall categories that could bypass that visibility.
use super::Capabilities;
use seccompiler::{
BpfProgram, SeccompAction, SeccompCmpArgLen, SeccompCmpOp, SeccompCondition, SeccompFilter,
SeccompRule,
};
use std::collections::BTreeMap;
fn condition(argument: u8, op: SeccompCmpOp, value: u64) -> Result<SeccompCondition, String> {
SeccompCondition::new(argument, SeccompCmpArgLen::Qword, op, value)
.map_err(|error| error.to_string())
}
fn rule(conditions: Vec<SeccompCondition>) -> Result<SeccompRule, String> {
SeccompRule::new(conditions).map_err(|error| error.to_string())
}
fn install(
rules: BTreeMap<i64, Vec<SeccompRule>>,
default: SeccompAction,
matched: SeccompAction,
) -> Result<(), String> {
let architecture = std::env::consts::ARCH
.try_into()
.map_err(|error: seccompiler::BackendError| error.to_string())?;
let filter = SeccompFilter::new(rules, default, matched, architecture)
.map_err(|error| error.to_string())?;
let program: BpfProgram = filter
.try_into()
.map_err(|error: seccompiler::BackendError| error.to_string())?;
seccompiler::apply_filter(&program)
.map_err(|error| format!("cannot enforce Worker syscall policy: {error}"))
}
pub(super) fn apply(capabilities: Capabilities) -> Result<(), String> {
// clone3 carries flags behind a pointer. ENOSYS makes supported libc use
// clone, whose scalar flags can be checked without user-memory races.
install(
BTreeMap::from([(libc::SYS_clone3, vec![])]),
SeccompAction::Allow,
SeccompAction::Errno(libc::ENOSYS as u32),
)?;
let mut rules = BTreeMap::new();
for syscall in [
libc::SYS_read,
libc::SYS_write,
libc::SYS_readv,
libc::SYS_writev,
libc::SYS_pread64,
libc::SYS_pwrite64,
libc::SYS_preadv,
libc::SYS_pwritev,
libc::SYS_close,
libc::SYS_close_range,
libc::SYS_lseek,
libc::SYS_fcntl,
libc::SYS_dup,
libc::SYS_dup3,
libc::SYS_pipe2,
libc::SYS_openat,
libc::SYS_fstat,
libc::SYS_statx,
libc::SYS_faccessat,
libc::SYS_faccessat2,
libc::SYS_readlinkat,
// copy2/copytree inspect extended metadata on their visible source.
// These calls only read mounted files; writes remain independently
// governed by the read-only inputs and explicit writable grants.
libc::SYS_listxattr,
libc::SYS_llistxattr,
libc::SYS_flistxattr,
libc::SYS_getxattr,
libc::SYS_lgetxattr,
libc::SYS_fgetxattr,
libc::SYS_getdents64,
libc::SYS_getcwd,
libc::SYS_chdir,
libc::SYS_fchdir,
libc::SYS_mkdirat,
libc::SYS_unlinkat,
libc::SYS_renameat,
libc::SYS_renameat2,
libc::SYS_linkat,
libc::SYS_symlinkat,
libc::SYS_ftruncate,
libc::SYS_truncate,
libc::SYS_fchmod,
libc::SYS_fchmodat,
libc::SYS_fchown,
libc::SYS_fchownat,
libc::SYS_flock,
libc::SYS_fsync,
libc::SYS_fdatasync,
libc::SYS_sync_file_range,
libc::SYS_fallocate,
libc::SYS_fstatfs,
libc::SYS_statfs,
libc::SYS_umask,
libc::SYS_utimensat,
libc::SYS_sendfile,
libc::SYS_copy_file_range,
libc::SYS_mmap,
libc::SYS_mprotect,
libc::SYS_munmap,
libc::SYS_mremap,
libc::SYS_madvise,
libc::SYS_brk,
libc::SYS_msync,
libc::SYS_rt_sigaction,
libc::SYS_rt_sigprocmask,
libc::SYS_rt_sigreturn,
libc::SYS_rt_sigsuspend,
libc::SYS_rt_sigtimedwait,
libc::SYS_sigaltstack,
libc::SYS_tgkill,
libc::SYS_kill,
libc::SYS_getpid,
libc::SYS_getppid,
libc::SYS_gettid,
libc::SYS_getuid,
libc::SYS_geteuid,
libc::SYS_getgid,
libc::SYS_getegid,
libc::SYS_getgroups,
libc::SYS_getpgid,
libc::SYS_getsid,
libc::SYS_setpgid,
libc::SYS_setsid,
libc::SYS_set_tid_address,
libc::SYS_set_robust_list,
libc::SYS_get_robust_list,
libc::SYS_futex,
libc::SYS_rseq,
libc::SYS_sched_yield,
libc::SYS_sched_getaffinity,
libc::SYS_sched_getparam,
libc::SYS_sched_getscheduler,
libc::SYS_clock_gettime,
libc::SYS_clock_getres,
libc::SYS_clock_nanosleep,
libc::SYS_nanosleep,
libc::SYS_gettimeofday,
libc::SYS_times,
libc::SYS_getrusage,
libc::SYS_getrlimit,
libc::SYS_prlimit64,
libc::SYS_uname,
libc::SYS_sysinfo,
libc::SYS_getrandom,
libc::SYS_epoll_create1,
libc::SYS_epoll_ctl,
libc::SYS_epoll_pwait,
libc::SYS_epoll_pwait2,
libc::SYS_ppoll,
libc::SYS_pselect6,
libc::SYS_eventfd2,
libc::SYS_timerfd_create,
libc::SYS_timerfd_settime,
libc::SYS_timerfd_gettime,
libc::SYS_inotify_init1,
libc::SYS_inotify_add_watch,
libc::SYS_inotify_rm_watch,
libc::SYS_recvfrom,
libc::SYS_sendto,
libc::SYS_recvmsg,
libc::SYS_sendmsg,
libc::SYS_recvmmsg,
libc::SYS_sendmmsg,
libc::SYS_shutdown,
libc::SYS_getsockopt,
libc::SYS_setsockopt,
libc::SYS_getsockname,
libc::SYS_getpeername,
libc::SYS_execve,
libc::SYS_exit,
libc::SYS_exit_group,
libc::SYS_wait4,
libc::SYS_waitid,
libc::SYS_clone3,
] {
rules.insert(syscall, vec![]);
}
#[cfg(target_arch = "x86_64")]
for syscall in [
libc::SYS_arch_prctl,
libc::SYS_open,
libc::SYS_stat,
libc::SYS_lstat,
libc::SYS_newfstatat,
libc::SYS_access,
libc::SYS_readlink,
libc::SYS_pipe,
libc::SYS_dup2,
libc::SYS_poll,
libc::SYS_select,
libc::SYS_epoll_wait,
libc::SYS_epoll_create,
libc::SYS_mkdir,
libc::SYS_rmdir,
libc::SYS_unlink,
libc::SYS_rename,
libc::SYS_link,
libc::SYS_symlink,
libc::SYS_chmod,
libc::SYS_chown,
libc::SYS_time,
libc::SYS_alarm,
libc::SYS_getpgrp,
] {
rules.insert(syscall, vec![]);
}
#[cfg(target_arch = "aarch64")]
{
rules.insert(libc::SYS_newfstatat, vec![]);
}
// New namespaces and CLONE_PARENT could escape the owned PID lifecycle.
let forbidden_clone = (libc::CLONE_NEWCGROUP
| libc::CLONE_NEWIPC
| libc::CLONE_NEWNET
| libc::CLONE_NEWNS
| libc::CLONE_NEWPID
| libc::CLONE_NEWUSER
| libc::CLONE_NEWUTS
| libc::CLONE_PARENT
| libc::CLONE_UNTRACED) as u64;
let mut clone = vec![condition(0, SeccompCmpOp::MaskedEq(forbidden_clone), 0)?];
if !capabilities.process {
let required = (libc::CLONE_THREAD | libc::CLONE_VM | libc::CLONE_SIGHAND) as u64;
clone.push(condition(
0,
SeccompCmpOp::MaskedEq(required | 255),
required,
)?);
}
rules.insert(libc::SYS_clone, vec![rule(clone)?]);
if capabilities.process {
// POSIX_SPAWN_RESETIDS restores the caller's real identity before exec.
// Permit that exact reset only; real/saved IDs and group membership
// cannot change. Linux uid_t/gid_t consume the low 32 argument bits.
for (syscall, identity) in [
(libc::SYS_setresuid, rustix::process::getuid().as_raw()),
(libc::SYS_setresgid, rustix::process::getgid().as_raw()),
] {
let conditions = [u32::MAX, identity, u32::MAX]
.into_iter()
.enumerate()
.map(|(argument, value)| {
SeccompCondition::new(
argument as u8,
SeccompCmpArgLen::Dword,
SeccompCmpOp::Eq,
u64::from(value),
)
.map_err(|error| error.to_string())
})
.collect::<Result<Vec<_>, _>>()?;
rules.insert(syscall, vec![rule(conditions)?]);
}
}
#[cfg(target_arch = "x86_64")]
if capabilities.process {
rules.insert(libc::SYS_fork, vec![]);
rules.insert(libc::SYS_vfork, vec![]);
}
// Anonymous socket pairs cannot reach another process. They are needed by
// language runtime event loops even without the network capability.
rules.insert(
libc::SYS_socketpair,
vec![rule(vec![condition(
0,
SeccompCmpOp::Eq,
libc::AF_UNIX as u64,
)?])?],
);
if capabilities.network {
let mut sockets = Vec::new();
for domain in [libc::AF_INET, libc::AF_INET6] {
for kind in [libc::SOCK_STREAM, libc::SOCK_DGRAM] {
sockets.push(rule(vec![
condition(0, SeccompCmpOp::Eq, domain as u64)?,
condition(
1,
SeccompCmpOp::MaskedEq(
!(libc::SOCK_NONBLOCK | libc::SOCK_CLOEXEC) as u32 as u64,
),
kind as u64,
)?,
])?);
}
}
rules.insert(libc::SYS_socket, sockets);
for syscall in [
libc::SYS_connect,
libc::SYS_bind,
libc::SYS_listen,
libc::SYS_accept,
libc::SYS_accept4,
] {
rules.insert(syscall, vec![]);
}
}
// ioctl truncates request to unsigned int. Compare that same width so a
// caller cannot hide a forbidden request behind nonzero upper bits.
let terminal_requests = [libc::TIOCSTI, libc::TIOCLINUX]
.into_iter()
.map(|request| {
SeccompCondition::new(1, SeccompCmpArgLen::Dword, SeccompCmpOp::Ne, request)
.map_err(|error| error.to_string())
})
.collect::<Result<Vec<_>, _>>()?;
rules.insert(libc::SYS_ioctl, vec![rule(terminal_requests)?]);
// The allowlist excludes namespace, mount, ptrace/process_vm, io_uring,
// device creation, kernel administration, BPF and all x32 syscall numbers.
install(
rules,
SeccompAction::Errno(libc::EPERM as u32),
SeccompAction::Allow,
)
}
-321
View File
@@ -1,321 +0,0 @@
//! The fixed Linux execution boundary shared by Lib runtime and build owners.
//! Callers verify signed assets and own every child; this crate never discovers tools.
use std::collections::BTreeSet;
use std::ffi::OsString;
use std::path::{Component, Path, PathBuf};
use std::process::Command;
mod assets;
mod mounts;
mod observation;
mod trusted_helper;
pub use assets::{
validate_assets, validate_assets_for_target, validate_bwrap, validate_bwrap_for_target,
};
pub use observation::CommandNamespace;
pub use trusted_helper::{APPARMOR_PROFILE, TRUSTED_HELPER_ROOT, validate_host_policy};
/// Executable modes are part of the bootstrap resource contract. The private
/// loader is needed by the guard and packaged language runtimes. Authoring and
/// embedding can run on a different host; target validation belongs to assets.
pub fn asset_executable(path: &str) -> bool {
matches!(
path,
"bin/bwrap" | "bin/guard" | "lib/ld-linux-x86-64.so.2" | "lib/ld-linux-aarch64.so.1"
)
}
#[cfg(target_os = "linux")]
mod filter;
#[derive(Clone, Copy, Debug, Default)]
pub struct Capabilities {
pub network: bool,
pub process: bool,
}
/// The source is a parent-validated deployment grant, never a Worker request.
#[derive(Debug)]
pub struct Grant {
pub source: PathBuf,
pub destination: PathBuf,
pub writable: bool,
}
pub struct Launch<'a> {
pub assets: &'a Path,
pub worker: &'a Path,
pub executable: &'a Path,
pub arguments: &'a [OsString],
pub working_directory: &'a Path,
pub capabilities: Capabilities,
pub grants: &'a [Grant],
}
/// Guard exec confirmation and PID-namespace lifetime, separate from user I/O.
pub struct CommandObservation {
pub startup: std::fs::File,
pub namespace: std::fs::File,
pub release: std::fs::File,
}
/// A private ELF closure cannot honor additional host preload libraries.
/// Applies to compiler launches as well as the sandbox bootstrap.
pub fn require_no_loader_preload() -> Result<(), String> {
match std::fs::symlink_metadata("/etc/ld.so.preload") {
Ok(_) => {
Err("private executable does not support host loader preload configuration".into())
}
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(()),
Err(error) => Err(format!(
"cannot inspect host loader preload configuration: {error}"
)),
}
}
impl Launch<'_> {
pub fn command(&self) -> Result<Command, String> {
self.command_with_tools(None, None)
}
#[cfg(target_os = "linux")]
pub fn observed_command(&self) -> Result<(Command, CommandObservation), String> {
use std::os::fd::OwnedFd;
let (startup, startup_writer) = std::io::pipe().map_err(|error| error.to_string())?;
let (namespace, namespace_writer) = std::io::pipe().map_err(|error| error.to_string())?;
let (blocked, release) = std::io::pipe().map_err(|error| error.to_string())?;
let command = self.command_with_tools(
None,
Some([
std::fs::File::from(OwnedFd::from(startup_writer)),
std::fs::File::from(OwnedFd::from(namespace_writer)),
std::fs::File::from(OwnedFd::from(blocked)),
]),
)?;
Ok((
command,
CommandObservation {
startup: OwnedFd::from(startup).into(),
namespace: OwnedFd::from(namespace).into(),
release: OwnedFd::from(release).into(),
},
))
}
#[cfg(not(target_os = "linux"))]
pub fn observed_command(&self) -> Result<(Command, CommandObservation), String> {
Err("external command sandbox is not supported on this platform".into())
}
/// Only the explicit package-build owner may supply this verified signed
/// root. Ordinary Worker launches never acquire a compiler or /usr mount.
pub fn command_for_build(&self, tool_root: &Path) -> Result<Command, String> {
if self.capabilities.network || !self.capabilities.process {
return Err(
"package builds require isolated network and child-process capability".into(),
);
}
self.command_with_tools(Some(tool_root), None)
}
fn command_with_tools(
&self,
tool_root: Option<&Path>,
observation: Option<[std::fs::File; 3]>,
) -> Result<Command, String> {
if !cfg!(all(
target_os = "linux",
any(target_arch = "x86_64", target_arch = "aarch64")
)) {
return Err("external Worker sandbox is not supported on this platform".into());
}
let assets = real_path(self.assets)?;
let bwrap_hash = assets::verify_directory(&assets)?;
require_no_loader_preload()?;
let worker = real_path(self.worker)?;
let tool_root = tool_root.map(real_path).transpose()?;
let executable = mapped_path(&worker, self.executable)?;
let working_directory = mapped_path(&worker, self.working_directory)?;
let loader = assets.join("lib").join(loader_name());
for file in [
&loader,
&assets.join("bin/bwrap"),
&assets.join("bin/guard"),
] {
if !real_path(file)?.is_file() {
return Err("sandbox executable asset is not a regular file".into());
}
}
// A direct static exec is required for the trusted AppArmor attachment.
let bwrap = trusted_helper::executable(&assets, &bwrap_hash)?;
let mut command = Command::new(bwrap);
command.env_clear().args([
"--unshare-all",
"--die-with-parent",
"--new-session",
"--cap-drop",
"ALL",
]);
if self.capabilities.network {
command.arg("--share-net");
}
let mut mounts = mounts::Mounts::default();
let startup_fd = if let Some([startup, namespace, blocked]) = observation {
let startup = mounts.pass(startup)?;
let namespace = mounts.pass(namespace)?;
let blocked = mounts.pass(blocked)?;
command
.arg("--info-fd")
.arg(namespace.to_string())
.arg("--block-fd")
.arg(blocked.to_string());
Some(startup)
} else {
None
};
mounts.bind(&mut command, &assets, Path::new("/sandbox"), false)?;
mounts.bind(&mut command, &assets.join("lib"), Path::new("/lib"), false)?;
command.args(["--symlink", "lib", "/lib64"]);
mounts.bind(&mut command, &worker, Path::new("/worker"), false)?;
if let Some(root) = &tool_root {
mounts.bind(&mut command, &root.join("usr"), Path::new("/usr"), false)?;
command.args(["--symlink", "usr/bin", "/bin"]);
}
command.args([
"--proc", "/proc", "--dev", "/dev", "--tmpfs", "/tmp", "--dir", "/data",
]);
if self.capabilities.network {
// DNS is an OS service input, never application configuration.
// Expose only these bounded resolver files, not the host's /etc.
for name in ["resolv.conf", "hosts"] {
let source = Path::new("/etc")
.join(name)
.canonicalize()
.map_err(|error| format!("cannot resolve network resolver input: {error}"))?;
let source = real_path(&source)?;
let metadata = std::fs::metadata(&source).map_err(|error| error.to_string())?;
if !metadata.is_file() || metadata.len() > 65_536 {
return Err("network resolver input is not a bounded regular file".into());
}
mounts.bind(&mut command, &source, &Path::new("/etc").join(name), false)?;
}
}
let mut destinations = BTreeSet::new();
for grant in self.grants {
let source = real_path(&grant.source)?;
if grant.writable
&& [&assets, &worker]
.into_iter()
.chain(tool_root.iter())
.any(|protected| {
source.starts_with(protected) || protected.starts_with(&source)
})
{
return Err("sandbox writable grant overlaps executable inputs".into());
}
let destination = &grant.destination;
normalized_absolute(destination)?;
if !destination.starts_with("/data") || destination == Path::new("/data") {
return Err("sandbox file grant must have a path below /data".into());
}
if destinations.iter().any(|prior: &&Path| {
prior.starts_with(destination) || destination.starts_with(prior)
}) {
return Err("sandbox file grants overlap".into());
}
destinations.insert(destination.as_path());
mounts.bind(&mut command, &source, destination, grant.writable)?;
}
let flags =
u8::from(self.capabilities.network) | (u8::from(self.capabilities.process) << 1);
command
.arg("--chdir")
.arg(working_directory)
.args(["--", "/sandbox/bin/guard"]);
if let Some(fd) = startup_fd {
command.arg("--observe").arg(fd.to_string());
}
command.arg(flags.to_string()).arg(executable);
for argument in self.arguments {
// Only resource arguments are absolute; fixed flags stay literal.
// Other absolute arguments cannot silently expose a host pathname.
let path = Path::new(argument);
command.arg(if path.is_absolute() {
self.argument_path(path)?.into_os_string()
} else {
argument.clone()
});
}
mounts.attach(&mut command)?;
Ok(command)
}
/// Build manifests need the same names as command arguments. Output files
/// may not exist yet; their owning mount was already validated at launch.
pub fn argument_path(&self, path: &Path) -> Result<PathBuf, String> {
normalized_absolute(path)?;
for grant in self.grants {
if let Ok(relative) = path.strip_prefix(&grant.source) {
return Ok(grant.destination.join(relative));
}
}
let relative = path
.strip_prefix(self.worker)
.map_err(|_| "sandbox argument escapes its owned mounts")?;
Ok(Path::new("/worker").join(relative))
}
}
pub fn loader_name() -> &'static str {
if cfg!(target_arch = "aarch64") {
"ld-linux-aarch64.so.1"
} else {
"ld-linux-x86-64.so.2"
}
}
fn normalized_absolute(path: &Path) -> Result<(), String> {
if !path.is_absolute()
|| path
.components()
.any(|part| !matches!(part, Component::RootDir | Component::Normal(_)))
{
return Err("sandbox path must be absolute and normalized".into());
}
Ok(())
}
fn real_path(path: &Path) -> Result<PathBuf, String> {
normalized_absolute(path)?;
for ancestor in path.ancestors() {
let metadata = std::fs::symlink_metadata(ancestor)
.map_err(|error| format!("cannot inspect sandbox input: {error}"))?;
if metadata.file_type().is_symlink() {
return Err("sandbox input must not contain symbolic links".into());
}
}
Ok(path.to_owned())
}
fn mapped_path(worker: &Path, path: &Path) -> Result<PathBuf, String> {
let path = real_path(path)?;
let relative = path
.strip_prefix(worker)
.map_err(|_| "sandbox execution path escapes the Worker tree")?;
Ok(Path::new("/worker").join(relative))
}
/// Called only by the single-threaded guard immediately before exec. The filter
/// is inherited by exec and every subsequently created thread or child.
pub fn restrict(capabilities: Capabilities) -> Result<(), String> {
#[cfg(target_os = "linux")]
{
filter::apply(capabilities)
}
#[cfg(not(target_os = "linux"))]
{
let _ = capabilities;
Err("sandbox is not supported on this platform".into())
}
}
-79
View File
@@ -1,79 +0,0 @@
//! Private signed helper. It accepts a fixed capability mask, never filter code.
#[cfg(target_os = "linux")]
fn execute() -> Result<(), String> {
use std::io::Write;
use std::os::unix::process::CommandExt;
let mut arguments = std::env::args_os().skip(1);
let first = arguments.next();
let mut observation = if first.as_deref() == Some(std::ffi::OsStr::new("--observe")) {
let fd = arguments
.next()
.and_then(|value| value.to_str().and_then(|value| value.parse::<i32>().ok()))
.filter(|fd| *fd >= 3)
.ok_or("invalid sandbox observation descriptor")?;
// Opening proc creates a safe owned CLOEXEC duplicate. Close the
// inherited raw descriptor before exec so neither channel leaks.
let file = std::fs::OpenOptions::new()
.write(true)
.open(format!("/proc/self/fd/{fd}"))
.map_err(|error| format!("cannot open sandbox observation: {error}"))?;
nix::unistd::close(fd)
.map_err(|error| format!("cannot close inherited observation: {error}"))?;
Some(file)
} else {
None
};
let first = if observation.is_some() {
arguments.next()
} else {
first
};
let result = (|| {
let flags = first
.and_then(|value| value.to_str().and_then(|value| value.parse::<u8>().ok()))
.filter(|value| *value <= 3)
.ok_or("invalid sandbox capability mask")?;
let executable = arguments.next().ok_or("missing sandbox executable")?;
if !std::path::Path::new(&executable).starts_with("/worker") {
return Err("sandbox executable must be inside its Worker tree".into());
}
let mut command = std::process::Command::new(executable);
command.args(arguments).env_clear();
dever_sandbox::restrict(dever_sandbox::Capabilities {
network: flags & 1 != 0,
process: flags & 2 != 0,
})?;
if let Some(observation) = &mut observation {
observation
.write_all(b"R")
.map_err(|error| format!("cannot confirm sandbox guard: {error}"))?;
}
Err(format!(
"cannot execute sandboxed Worker: {}",
command.exec()
))
})();
if let (Some(observation), Err(error)) = (&mut observation, &result) {
observation
.write_all(b"E")
.and_then(|_| observation.write_all(error.as_bytes()))
.map_err(|write| format!("{error}; cannot report sandbox startup: {write}"))?;
}
result
}
#[cfg(not(target_os = "linux"))]
fn execute() -> Result<(), String> {
Err("sandbox is not supported on this platform".into())
}
fn main() -> std::process::ExitCode {
match execute() {
Ok(()) => std::process::ExitCode::SUCCESS,
Err(error) => {
eprintln!("{error}");
std::process::ExitCode::FAILURE
}
}
}
-87
View File
@@ -1,87 +0,0 @@
//! Pin deployment grants until bwrap has mounted the exact checked inode.
use std::path::Path;
use std::process::Command;
#[derive(Default)]
pub(super) struct Mounts {
#[cfg(target_os = "linux")]
descriptors: Vec<command_fds::FdMapping>,
}
impl Mounts {
#[cfg(target_os = "linux")]
pub(super) fn pass(&mut self, file: std::fs::File) -> Result<i32, String> {
let child_fd =
i32::try_from(self.descriptors.len()).map_err(|_| "too many sandbox descriptors")? + 3;
self.descriptors.push(command_fds::FdMapping {
parent_fd: file.into(),
child_fd,
});
Ok(child_fd)
}
#[cfg(not(target_os = "linux"))]
pub(super) fn pass(&mut self, _: std::fs::File) -> Result<i32, String> {
Err("sandbox descriptors are not supported on this platform".into())
}
#[cfg(target_os = "linux")]
pub(super) fn bind(
&mut self,
command: &mut Command,
source: &Path,
destination: &Path,
writable: bool,
) -> Result<(), String> {
use rustix::fs::{Mode, OFlags, ResolveFlags, openat2};
let descriptor = openat2(
rustix::fs::CWD,
source,
OFlags::PATH | OFlags::CLOEXEC,
Mode::empty(),
ResolveFlags::NO_SYMLINKS | ResolveFlags::NO_MAGICLINKS,
)
.map_err(|error| format!("cannot pin sandbox mount without following links: {error}"))?;
let child_fd =
i32::try_from(self.descriptors.len()).map_err(|_| "too many sandbox mounts")? + 3;
command
.arg(if writable {
"--bind-fd"
} else {
"--ro-bind-fd"
})
.arg(child_fd.to_string())
.arg(destination);
self.descriptors.push(command_fds::FdMapping {
parent_fd: descriptor,
child_fd,
});
Ok(())
}
#[cfg(target_os = "linux")]
pub(super) fn attach(self, command: &mut Command) -> Result<(), String> {
use command_fds::CommandFdExt;
command
.fd_mappings(self.descriptors)
.map_err(|error| format!("cannot pass sandbox mount descriptors: {error}"))?;
Ok(())
}
#[cfg(not(target_os = "linux"))]
pub(super) fn bind(
&mut self,
_: &mut Command,
_: &Path,
_: &Path,
_: bool,
) -> Result<(), String> {
Err("sandbox mounts are not supported on this platform".into())
}
#[cfg(not(target_os = "linux"))]
pub(super) fn attach(self, _: &mut Command) -> Result<(), String> {
Err("sandbox mounts are not supported on this platform".into())
}
}
-48
View File
@@ -1,48 +0,0 @@
//! A pinned namespace init is the completion boundary for all descendants.
#[cfg(target_os = "linux")]
use std::os::fd::{AsRawFd, RawFd};
pub struct CommandNamespace {
#[cfg(target_os = "linux")]
descriptor: std::fs::File,
}
impl CommandNamespace {
#[cfg(target_os = "linux")]
pub fn pin(pid: i32, wrapper: u32) -> Result<Self, String> {
use rustix::process::{Pid, PidfdFlags, pidfd_open};
let pid = Pid::from_raw(pid).ok_or("invalid sandbox namespace PID")?;
let descriptor = pidfd_open(pid, PidfdFlags::NONBLOCK)
.map_err(|error| format!("cannot pin sandbox namespace: {error}"))?;
// --block-fd holds the namespace before guard/target execution. If
// setup failed and its PID was reused, do not acquire authority over
// that process: only this bwrap's direct child belongs to this launch.
let status = std::fs::read_to_string(format!("/proc/{}/status", pid.as_raw_pid()))
.map_err(|error| format!("cannot verify sandbox namespace parent: {error}"))?;
let parent = status
.lines()
.find_map(|line| line.strip_prefix("PPid:"))
.and_then(|value| value.trim().parse::<u32>().ok());
if parent != Some(wrapper) {
return Err("sandbox namespace no longer belongs to its launcher".into());
}
Ok(Self {
descriptor: descriptor.into(),
})
}
#[cfg(target_os = "linux")]
pub fn terminate(&self) -> Result<(), String> {
match rustix::process::pidfd_send_signal(&self.descriptor, rustix::process::Signal::KILL) {
Ok(()) | Err(rustix::io::Errno::SRCH) => Ok(()),
Err(error) => Err(format!("cannot stop owned command namespace: {error}")),
}
}
}
#[cfg(target_os = "linux")]
impl AsRawFd for CommandNamespace {
fn as_raw_fd(&self) -> RawFd {
self.descriptor.as_raw_fd()
}
}
-130
View File
@@ -1,130 +0,0 @@
//! The only host integration is a signed, administrator-installed static helper.
use std::path::{Path, PathBuf};
pub const TRUSTED_HELPER_ROOT: &str = "/opt/dever/sandbox";
pub const APPARMOR_PROFILE: &str = include_str!("../../../sdk/apparmor/dever-sandbox");
#[cfg(all(test, target_os = "linux"))]
#[path = "../../../test/dever-sandbox-tests/trusted_helper.rs"]
mod tests;
pub(super) fn executable(assets: &Path, expected: &str) -> Result<PathBuf, String> {
#[cfg(target_os = "linux")]
if rustix::process::geteuid().as_raw() != 0 && validate_host_policy()? {
let path = Path::new(TRUSTED_HELPER_ROOT).join(expected).join("bwrap");
verify_helper(&path, expected)?;
require_profiles()?;
return Ok(path);
}
let _ = expected;
Ok(assets.join("bin/bwrap"))
}
/// Validate the host prerequisite shared by non-root launch and real-host
/// policy installation. This only reads kernel state; administrators own it.
pub fn validate_host_policy() -> Result<bool, String> {
#[cfg(target_os = "linux")]
{
host_policy(|path| std::fs::read_to_string(path))
}
#[cfg(not(target_os = "linux"))]
{
Ok(false)
}
}
#[cfg(target_os = "linux")]
fn host_policy(read: impl Fn(&str) -> std::io::Result<String>) -> Result<bool, String> {
let restricted = policy_flag(&read, "apparmor_restrict_unprivileged_userns")?.unwrap_or(false);
if !restricted {
return Ok(false);
}
if policy_flag(&read, "apparmor_restrict_unprivileged_unconfined")? != Some(true) {
return Err("Dever AppArmor deployment requires kernel.apparmor_restrict_unprivileged_unconfined=1 while user namespace restriction is enabled; an administrator must approve strict profile transitions before installing or loading the Dever policy".into());
}
Ok(true)
}
#[cfg(target_os = "linux")]
fn policy_flag(
read: &impl Fn(&str) -> std::io::Result<String>,
name: &str,
) -> Result<Option<bool>, String> {
match read(&format!("/proc/sys/kernel/{name}")) {
Ok(value) => match value.trim() {
"0" => Ok(Some(false)),
"1" => Ok(Some(true)),
_ => Err(format!(
"invalid AppArmor kernel flag '{name}': expected 0 or 1"
)),
},
Err(error) if error.kind() == std::io::ErrorKind::NotFound => Ok(None),
Err(error) => Err(format!(
"cannot inspect AppArmor kernel flag '{name}': {error}"
)),
}
}
#[cfg(target_os = "linux")]
fn verify_helper(path: &Path, expected: &str) -> Result<(), String> {
use std::io::Read;
use std::os::unix::fs::MetadataExt;
for (index, ancestor) in path.ancestors().enumerate() {
let metadata = std::fs::symlink_metadata(ancestor).map_err(|error| format!(
"trusted sandbox helper is missing or inaccessible at '{}': {error}; install the matching signed Dever sandbox assets", path.display()))?;
if metadata.file_type().is_symlink()
|| metadata.uid() != 0
|| metadata.mode() & 0o022 != 0
|| (index == 0
&& (!metadata.is_file()
|| metadata.mode() & 0o6111 != 0o111
|| metadata.nlink() != 1))
|| (index != 0 && !metadata.is_dir())
{
return Err("trusted sandbox helper requires real root-owned paths without shared writes, setuid or setgid".into());
}
}
match rustix::fs::lgetxattr(path, "security.capability", &mut [0u8; 64]) {
Err(rustix::io::Errno::NODATA | rustix::io::Errno::OPNOTSUPP) => {}
Ok(_) => return Err("trusted sandbox helper must not have file capabilities".into()),
Err(error) => {
return Err(format!(
"cannot inspect sandbox helper capabilities: {error}"
));
}
}
let mut bytes = Vec::new();
std::fs::File::open(path)
.and_then(|file| file.take(32 * 1024 * 1024 + 1).read_to_end(&mut bytes))
.map_err(|error| format!("cannot read trusted sandbox helper: {error}"))?;
if super::validate_bwrap(&bytes)? != expected {
return Err("trusted sandbox helper differs from the embedded signed asset".into());
}
Ok(())
}
#[cfg(target_os = "linux")]
fn require_profiles() -> Result<(), String> {
// The profiles list requires policy-admin privilege; the policy filesystem
// exposes each profile's name/mode to ordinary users without changing state.
let entries = std::fs::read_dir("/sys/kernel/security/apparmor/policy/profiles")
.map_err(|error| format!("cannot verify the Dever AppArmor profiles: {error}; an administrator must load /etc/apparmor.d/dever-sandbox"))?;
let mut required = std::collections::BTreeSet::from(["dever-bwrap", "dever-bwrap-child"]);
for entry in entries.take(4096) {
let entry = entry.map_err(|error| format!("cannot inspect AppArmor profiles: {error}"))?;
let name = std::fs::read_to_string(entry.path().join("name"))
.map_err(|error| format!("cannot read AppArmor profile name: {error}"))?;
if required.contains(name.trim()) {
let mode = std::fs::read_to_string(entry.path().join("mode"))
.map_err(|error| format!("cannot read AppArmor profile mode: {error}"))?;
if mode.trim() != "enforce" {
return Err("Dever AppArmor profiles must be loaded in enforce mode".into());
}
required.remove(name.trim());
}
if required.is_empty() {
return Ok(());
}
}
Err("Dever AppArmor profile is not loaded; an administrator must load /etc/apparmor.d/dever-sandbox".into())
}
@@ -1 +1 @@
post upload = app.store
long post upload = app.store
@@ -11,6 +11,10 @@ store(file: Upload) (result: StoreResult) {
content_type = dever.api.upload_content_type(file)
size = dever.api.upload_size(file)
key = port.store(file)
result = record(key, filename, content_type, size)
}
transaction record(key: Uuid, filename: Text, content_type: Text, size: Int) (result: StoreResult) {
asset = model.create(
{
storage_key = key
@@ -1 +1 @@
cmd bootstrap = app.bootstrap
long cmd bootstrap = app.bootstrap
@@ -1,3 +1,3 @@
public post login = app.login
public long post login = app.login
post logout = app.logout
@@ -1,3 +1,3 @@
public post login = app.login
public long post login = app.login
post logout = app.logout
@@ -33,10 +33,20 @@ bootstrap(
email: Text,
display_name: Text,
password: Secret
) (result: BootstrapView) {
hash = blocking(dever.crypto.password_hash(password))
result = register_account(tenant_key, tenant_name, email, display_name, hash)
}
transaction register_account(
tenant_key: Text,
tenant_name: Text,
email: Text,
display_name: Text,
hash: Text
) (result: BootstrapView) {
tenant = platform.tenant.provision(tenant_key, tenant_name)
input = domain.prepare(email, display_name)
hash = blocking(dever.crypto.password_hash(password))
principal = model.principal.create(
{
display_name = input.name
@@ -14,5 +14,5 @@
- 声明:`post upload`
```dever
post upload = app.store
long post upload = app.store
```
@@ -7,10 +7,12 @@
- `StoreResult`
- 公开方法:
- `store`
- `record`
- `require_ready`
- `ready`
- 使用:
- `media.asset.store(file)`
- `media.asset.record(key, filename, content_type, size)`
- `media.asset.require_ready(id)`
- `media.asset.ready(available)`
@@ -52,6 +54,25 @@ store(file: Upload) (result: StoreResult) {
content_type = dever.api.upload_content_type(file)
size = dever.api.upload_size(file)
key = port.store(file)
result = record(key, filename, content_type, size)
}
```
## 保存元数据
文件传输结束后,在短事务中保存资源元数据与审计记录。
- 函数:`record`
- 输入:
- `key: Uuid`:已保存文件的存储标识
- `filename: Text`:文件名称
- `content_type: Text`:文件类型
- `size: Int`:文件大小
- 输出:
- `result: StoreResult`:资源公开元数据
```dever
transaction record(key: Uuid, filename: Text, content_type: Text, size: Int) (result: StoreResult) {
asset = model.create(
{
storage_key = key
@@ -14,5 +14,5 @@
- 声明:`cmd bootstrap`
```dever
cmd bootstrap = app.bootstrap
long cmd bootstrap = app.bootstrap
```
@@ -14,7 +14,7 @@
- 声明:`post login`
```dever
public post login = app.login
public long post login = app.login
```
## 登出
@@ -14,7 +14,7 @@
- 声明:`post login`
```dever
public post login = app.login
public long post login = app.login
```
## 登出
@@ -11,6 +11,7 @@
- `Identity`
- 公开方法:
- `bootstrap`
- `register_account`
- `login`
- `establish_login`
- `logout`
@@ -20,6 +21,7 @@
- `require_user`
- 使用:
- `user.account.bootstrap(tenant_key, tenant_name, email, display_name, password)`
- `user.account.register_account(tenant_key, tenant_name, email, display_name, hash)`
- `user.account.login(credential_id, password, tenant)`
- `user.account.establish_login(credential_id, tenant, site)`
- `user.account.logout()`
@@ -137,10 +139,36 @@ bootstrap(
email: Text,
display_name: Text,
password: Secret
) (result: BootstrapView) {
hash = blocking(dever.crypto.password_hash(password))
result = register_account(tenant_key, tenant_name, email, display_name, hash)
}
```
## 提交平台账户
密码计算完成后,原子提交租户、账户和成员关系。
- 函数:`register_account`
- 输入:
- `tenant_key: Text`:租户键
- `tenant_name: Text`:租户名称
- `email: Text`:邮箱
- `display_name: Text`:显示名
- `hash: Text`:密码摘要
- 输出:
- `result: BootstrapView`:初始化结果
```dever
transaction register_account(
tenant_key: Text,
tenant_name: Text,
email: Text,
display_name: Text,
hash: Text
) (result: BootstrapView) {
tenant = platform.tenant.provision(tenant_key, tenant_name)
input = domain.prepare(email, display_name)
hash = blocking(dever.crypto.password_hash(password))
principal = model.principal.create(
{
display_name = input.name
+35 -8
View File
@@ -1,25 +1,28 @@
# Python、JavaScript 与 Go SDK
# 外部 Worker SDK
SDK 用于把现成的 Python、JavaScript 或 Go 能力接入 Dever。应用先声明 Port,再用外部 Adapter 对接;Dever 会管理 Worker 进程和通信。
SDK 用于把 Python、JavaScript、Go 或 C ABI 能力接入 Dever。应用先声明 Port,再用外部 Adapter 对接;Dever 管理 Worker 进程和通信。C ABI 运行在独立受管进程中,不是 Dever 主进程的任意 `dlopen` 或裸指针调用。
开发应用先看 [三种语言的完整例子](../skills/references/external-lib.md)。那里包含同一 Port、三种 Adapter、处理函数和 fake 测试,不需要手写协议或维护另一份接口注册表。
## 处理函数怎么写
函数名与 Port 操作名对应。输入、输出和业务错误的类型由 Dever 编译器生成,SDK 负责验证。
函数名与 Port 操作名对应。输入、输出和业务错误的类型由 Dever 编译器生成;Python/JavaScript/Go SDK 负责解码,C shim 自行处理借用的 wire JSON,所有结果还须通过 Runtime 的 Port 类型校验。
| 语言 | 处理函数 | 加载生成的接口定义 |
| --- | --- | --- |
| Python | 异步函数 | `Worker.from_manifest(manifest, handlers)` |
| JavaScript | 函数接收 `AbortSignal`;支持 ESM 导出或 CommonJS 属性 | `Worker.fromManifest(manifest, handlers)` |
| Go | 函数接收 `context.Context` | `NewFromManifest(manifest)`,再 `RegisterHandlers(handlers)` |
| C/C++ | `dever_ffi_handler` 的同步 C ABI 回调 | `dever_ffi_serve` 读取编译器打包的 `contract.json` |
处理函数返回能用 JSON 表达的值,业务失败返回已声明的 `BusinessError`。不要往 stdout 写日志;stdout 用于 Worker 通信,日志写 stderr。
取消请求会产生一次 `dever.cancelled` 错误,已取消调用的结果会被丢弃。如果处理函数一秒内不响应取消,Worker 失败并停止接收新调用。Dever 管理进程与 stdin;自行在 Go 进程内调用 `Serve` 时,调用方负责关闭或解除自己的 `io.Reader` 阻塞。
Python/JavaScript/Go 的协作取消会产生一次 `dever.cancelled` 错误,已取消调用的结果会被丢弃。如果处理函数一秒内不响应取消,Worker 失败并停止接收新调用。Dever 管理进程与 stdin;自行在 Go 进程内调用 `Serve` 时,调用方负责关闭或解除自己的 `io.Reader` 阻塞。C ABI 的同步调用终止边界见后文。
## SDK 维护与验证
C ABI 的独立协议/动态库验收:`/usr/bin/python3 -B test/external-sdk/test_c_ffi.py --cc /absolute/path/to/cc`。测试在自有临时目录编译并调用真实 `.so`,结束清理;不代表 GPU 或完整发行已验收。
维护 SDK 时,显式指定本机运行环境的绝对路径:
```text
@@ -34,23 +37,47 @@ SDK 用于把现成的 Python、JavaScript 或 Go 能力接入 Dever。应用先
SDK 不依赖第三方包,也不读取环境变量或 PATH。托管 Worker 只读取已打包的 manifest。Python 固定使用 `-I -S -B`,先加载打包的 SDK,再把 Adapter 与依赖加入 `sys.path`。
## 受管 C ABI
复用 `external exec "worker/native" {}`,不引入另一套 FFI 语法或运行器。C/C++ shim 包含 `c/dever_ffi.h`,与 `c/dever_ffi.c` 一起预编译为对应平台 ELF;在 `invoke` 中调用现成库的 C ABI。编译由开发/发行环境显式完成,Dever `run/build` 不查找宿主编译器。不要把 C++ 异常跨过 C ABI 边界。
- `abi` 必须为 `DEVER_FFI_ABI_V1`;`open(context, setting)` 在握手时初始化一次,`invoke` 串行处理操作,`close` 在正常退出或可处理故障时释放状态。每个 Worker 生命周期可复用已加载状态,但不保证跨独立 Dever invocation 保温。
- `operation` 是借用的操作名字节;`setting`、`payload` 是借用的 JSON 切片,均不保证 NUL 结尾,只在当前回调有效,不得留存指针。
- 输出缓冲由 SDK 分配,shim 只写 `data[0..capacity]` 并设置 `size`,不得替换指针/容量、释放缓冲或越界。业务值使用 Port 的 wire JSON;Runtime 按已编译 Port 校验实际输出类型、声明错误及深度/大小,不接受动态接口签名。
- 返回 `DEVER_FFI_OK` 时输出为 JSON 结果;`DEVER_FFI_ERROR` 时输出为 `{"error":"已声明错误身份","payload":...}`;其他状态关闭通道并报告运行故障。源码中的 Secret 不得写 stdout/stderr,stdout 仅作协议通信。
- v1 同步原生调用不承诺协作取消;调用卡死、崩溃或超过 supervisor 期限时由现有进程监管终止整个 Worker,不在主进程强杀线程,不自动重放失败调用。强制终止不保证执行 `close`,外部副作用应幂等。
静态 ELF 可直接打包;动态依赖放在入口旁的 `lib/`,只接收正常文件,不接收软链接。构建验证目标、SONAME、递归依赖和 ABI 版本;非系统依赖来自该目录;Linux 的解释器/基础系统 ABI 由运行容器提供并须匹配构建目标,不扫描宿主 Python、Node、Go 或编译器。例如 `worker/native` 对应 `worker/lib/libengine.so`。编译器同时打包唯一的 `contract.json`,不手写第二份 Port hash。C++ STL、跨语言对象、回调指针、可变参数及跨进程零拷贝不属于 v1 接口;需要时在 shim 内转为明确的 C 数据与 JSON 值。
开发验收在 `test/dever-cli-tests/external_workers/ffi.rs`:显式指定绝对路径 `DEVER_FFI_CC`(C 编译器)、`DEVER_PROCESS_RUNTIME`(已验证的目标 loader/libc 等运行库目录,仅读取 `lib/`),运行 `cargo test --offline -p dever-cli --test external_workers ffi::real_c_abi_direct_worker_errors_deadlines_and_cancellation -- --ignored --exact`。验证真实 `.so` 打包、Dever 应用执行、业务错误、原生崩溃、阻塞调用、取消和期限处理;测试不替换已安装编译器,不算 GPU/模型验收。原生调用不响应取消、导致协议清理超时时,当前调用报告运行故障,而不是伪装成成功的取消或超时回退。
发行验收还须用 `DEVER_RUNTIME_ABI_ARCHIVE` 指向实际候选归档并运行 `native_runtime_link`,尤其是 Bytes wire codec 用例;源码测试通过不能替代配套发行运行时的真实链接与执行。
## 执行边界与 GPU
Worker 和依赖构建都直接在当前容器中运行,由内核启动真实可执行文件(不以 GNU loader 充当进程入口,保留 Python/Node/Rust 的自身可执行路径和子进程行为)。目标容器须提供匹配的 Linux/glibc 基础 ABI,签名 `process-runtime` 库保留用于目标依赖核验,不用旧版 libc 替换容器加载器的配套 libc。不再创建 Bubblewrap/user/mount/PID/network namespace、附加 seccomp 或安装 Dever 专用 AppArmor 策略。`dever-process` 只负责明确的执行路径、运行库和进程组;超时、取消、退出会清理本次进程组。自行 `setsid` 脱离监管的守护进程不受此保证,Worker 不应自行后台化。
`allow file/network/process/gpu` 保留为编译期效果声明和握手契约,**不是操作系统访问控制**。Worker 具有当前容器用户已有的文件、网络和设备权限。旧 `adapter.<port>.files` / `.gpu` 挂载授权字段已移除,配置会明确拒绝;真实模型路径和输出目录放入业务 `setting`,由 Pod volume、UID/GID、NetworkPolicy 和设备分配实施边界。只运行可信代码,不用这套进程监管执行不可信租户程序。
GPU 使用父容器已经分配的设备和驱动,不再由 Dever 重新挂载。除已打包的私有库路径外,仅透传 `CUDA_VISIBLE_DEVICES`、`NVIDIA_VISIBLE_DEVICES`、`NVIDIA_DRIVER_CAPABILITIES`、`CUDA_DEVICE_ORDER` 与 `LD_LIBRARY_PATH`;其他宿主环境不自动传入。CUDA/PyTorch 等依赖仍需匹配目标平台。是否有 GPU、是否可加载驱动以及显存是否足够由真实部署决定,不会绕过 Kubernetes 调度或容器权限。设备验收须单独取得 GPU 准入;本次源码改造不等于已部署或模型推理通过。
## 依赖与跨平台构建
`dever lib install <project>` 按现有锁恢复依赖,不重新选版本、不修改配置或锁。`dever-lock-v6` 记录 Python extras/wheel METADATA、npm 嵌套依赖及 peers/optional/bundled 依赖、Go 模块校验证据。原始包保留来源与文件名,构建记录保留完整输入,包括 npm 剪除可选构建依赖前的依赖图。
丢失的构建结果由同一沙箱重建,必须与记录的摘要一致。Python 原生 wheel 安装到配套解释器中,必须满足签名声明的标签、扩展 ABI 和动态库依赖要求。
丢失的构建结果由同一受管构建链重建,必须与记录的摘要一致。Python 原生 wheel 安装到配套解释器中,必须满足签名声明的标签、扩展 ABI 和动态库依赖要求。
生成 ARM64 应用时,先运行 `dever lib update <project> --target linux-aarch64`,再运行 `dever build <project> --target linux-aarch64 --output <new-file>`。Port 和 Worker 源码无需改写;锁、解释器、原生依赖和沙箱资源必须匹配目标平台。缺少目标资源时失败,不借用宿主运行环境。`run/test` 仍在本机执行。
生成 ARM64 应用时,先运行 `dever lib update <project> --target linux-aarch64`,再运行 `dever build <project> --target linux-aarch64 --output <new-file>`。Port 和 Worker 源码无需改写;锁、解释器、原生依赖和进程运行库资源必须匹配目标平台。缺少目标资源时失败,不借用宿主运行环境。`run/test` 仍在本机执行。
Go pack 的 `build.host` 描述编译器、链接器与分析器的运行平台;`build.goos/goarch` 和标准库对象头描述输出平台。工具在清空的环境中执行,目标必须预先正确配置。
## 第三方源码构建限制
只有显式准备 Lib 时才构建 Python sdist 或运行 npm 安装钩子。构建使用签名工具包和 Linux 沙箱,输入只读、网络关闭,写入仅限本次临时目录。缺少工具时不回退宿主 PATH。
只有显式准备 Lib 时才构建 Python sdist 或运行 npm 安装钩子。构建使用签名工具包,在独立临时工作目录内直接运行受管子进程;不再提供内层文件/网络隔离,因此只运行可信依赖及构建钩子。缺少工具时不回退宿主 PATH。
Python PEP 517 的构建依赖单独锁定,每个 hook 使用新进程,最终 wheel 按保留的元数据核对。npm 的配套运行器处理 `preinstall/install/postinstall`、依赖 bin 和 node-gyp。
构建记录绑定源码、目标运行环境、工具、构建前端、依赖图和输出摘要。Worker 准备阶段离线恢复结果,不重跑安装钩子。npm `.node` 产物须通过目标检查、动态加载检查和独立 Node 加载探测,才能进入只读 Worker 目录;线程和子进程也受此限制。不匹配平台或 ABI 的其他变体保留在锁定原始包中,不加入运行目录。运行时原生依赖只能来自 Worker 目录或签名 OS 库,不能借用编译器 sysroot。
构建记录绑定源码、目标运行环境、工具、构建前端、依赖图和输出摘要。Worker 准备阶段离线恢复结果,不重跑安装钩子。npm `.node` 产物须通过目标检查、动态加载检查和独立 Node 加载探测,才能进入 Worker 资源目录。不匹配平台或 ABI 的其他变体保留在锁定原始包中,不加入运行目录。运行时非系统依赖来自 Worker 目录,系统 ABI 由目标容器提供,不能借用编译器 sysroot。
生成的原生 addon 验证失败,整个构建就失败;使用纯 JS 备用路径不能证明原生构建成功。当前限制:
-45
View File
@@ -1,45 +0,0 @@
# Based on AppArmor 4.0 extras/bwrap-userns-restrict (GPL-2.0-only).
# Only signed static helpers published by the root-owned Dever installer attach.
# The exec transition is stacked under no_new_privs; child capabilities are denied.
# On restricted hosts, kernel.apparmor_restrict_unprivileged_unconfined=1 is
# mandatory BEFORE installation/loading: attachment paths do not constrain an
# explicit change_profile by an unconfined caller. Never enable this policy on
# such a host with that flag unset. Rollback removes this policy before restoring
# the previous strict flag. Dever never changes host sysctls automatically.
abi <abi/4.0>,
@{DEVER_SANDBOX_HASH} = [0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]
profile dever-bwrap /opt/dever/sandbox/@{DEVER_SANDBOX_HASH}/bwrap flags=(attach_disconnected,mediate_deleted) {
allow capability,
allow file rwlkm /{**,},
allow network,
allow unix,
allow ptrace,
allow signal,
allow mqueue,
allow io_uring,
allow userns,
allow mount,
allow umount,
allow pivot_root,
allow dbus,
allow px /** -> dever-bwrap//&dever-bwrap-child,
}
profile dever-bwrap-child flags=(attach_disconnected,mediate_deleted) {
allow file rwlkm /{**,},
allow network,
allow unix,
allow ptrace,
allow signal,
allow mqueue,
allow io_uring,
allow userns,
allow mount,
allow umount,
allow pivot_root,
allow dbus,
allow pix /** -> &dever-bwrap-child,
audit deny capability,
}
+310
View File
@@ -0,0 +1,310 @@
#include "dever_ffi.h"
#include <ctype.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#define MAX_BYTES (16U * 1024U * 1024U)
#define MAX_TOKENS 65536U
#define MAX_DEPTH 64U
typedef struct {
size_t start, end, after, count;
char kind;
} token;
typedef struct {
const char *data;
size_t length, cursor, used;
token *tokens;
} document;
static void whitespace(document *doc) {
while (doc->cursor < doc->length && strchr(" \t\r\n", doc->data[doc->cursor]) != NULL)
doc->cursor++;
}
static int string_end(document *doc) {
if (doc->data[doc->cursor++] != '"') return 0;
while (doc->cursor < doc->length) {
unsigned char byte = (unsigned char)doc->data[doc->cursor++];
if (byte == '"') return 1;
if (byte < 32) return 0;
if (byte == '\\') {
if (doc->cursor == doc->length) return 0;
byte = (unsigned char)doc->data[doc->cursor++];
if (byte == 'u') {
for (unsigned digit = 0; digit < 4; digit++)
if (doc->cursor == doc->length || !isxdigit((unsigned char)doc->data[doc->cursor++])) return 0;
} else if (strchr("\"\\/bfnrt", byte) == NULL) return 0;
}
}
return 0;
}
static int number_end(document *doc) {
if (doc->data[doc->cursor] == '-') doc->cursor++;
if (doc->cursor == doc->length) return 0;
if (doc->data[doc->cursor] == '0') doc->cursor++;
else {
if (doc->data[doc->cursor] < '1' || doc->data[doc->cursor] > '9') return 0;
while (doc->cursor < doc->length && isdigit((unsigned char)doc->data[doc->cursor])) doc->cursor++;
}
if (doc->cursor < doc->length && doc->data[doc->cursor] == '.') {
size_t start = ++doc->cursor;
while (doc->cursor < doc->length && isdigit((unsigned char)doc->data[doc->cursor])) doc->cursor++;
if (doc->cursor == start) return 0;
}
if (doc->cursor < doc->length && strchr("eE", doc->data[doc->cursor]) != NULL) {
doc->cursor++;
if (doc->cursor < doc->length && strchr("+-", doc->data[doc->cursor]) != NULL) doc->cursor++;
size_t start = doc->cursor;
while (doc->cursor < doc->length && isdigit((unsigned char)doc->data[doc->cursor])) doc->cursor++;
if (doc->cursor == start) return 0;
}
return 1;
}
static int value(document *doc, unsigned depth) {
whitespace(doc);
if (depth > MAX_DEPTH || doc->used == MAX_TOKENS || doc->cursor == doc->length) return 0;
token *item = &doc->tokens[doc->used++];
item->start = doc->cursor;
item->kind = doc->data[doc->cursor];
if (item->kind == '"') {
if (!string_end(doc)) return 0;
} else if (item->kind == '{' || item->kind == '[') {
char end = item->kind == '{' ? '}' : ']';
doc->cursor++;
whitespace(doc);
if (doc->cursor == doc->length) return 0;
if (doc->data[doc->cursor] != end) {
for (;;) {
whitespace(doc);
if (item->kind == '{') {
if (doc->cursor == doc->length || doc->data[doc->cursor] != '"' || !value(doc, depth + 1)) return 0;
whitespace(doc);
if (doc->cursor == doc->length || doc->data[doc->cursor++] != ':') return 0;
}
if (!value(doc, depth + 1)) return 0;
item->count++;
whitespace(doc);
if (doc->cursor == doc->length) return 0;
if (doc->data[doc->cursor] == end) break;
if (doc->data[doc->cursor++] != ',') return 0;
}
}
doc->cursor++;
} else {
const char *literal = item->kind == 't' ? "true" : item->kind == 'f' ? "false" : item->kind == 'n' ? "null" : NULL;
if (literal != NULL) {
size_t size = strlen(literal);
if (size > doc->length - doc->cursor || memcmp(doc->data + doc->cursor, literal, size) != 0) return 0;
doc->cursor += size;
} else if (!number_end(doc)) return 0;
}
item->end = doc->cursor;
item->after = doc->used;
return 1;
}
static int parse(document *doc, const char *data, size_t size) {
*doc = (document){.data = data, .length = size};
if (size == 0 || size > MAX_BYTES || memchr(data, 0, size) != NULL) return 0;
doc->tokens = calloc(MAX_TOKENS, sizeof(token));
if (doc->tokens == NULL) return 0;
if (value(doc, 0)) {
whitespace(doc);
if (doc->cursor == size) return 1;
}
free(doc->tokens);
doc->tokens = NULL;
return 0;
}
static dever_ffi_slice slice(const document *doc, size_t index) {
const token *item = &doc->tokens[index];
return (dever_ffi_slice){doc->data + item->start, item->end - item->start};
}
static int text_is(const document *doc, size_t index, const char *text) {
if (index >= doc->used || doc->tokens[index].kind != '"') return 0;
dever_ffi_slice part = slice(doc, index);
return part.size == strlen(text) + 2 && memcmp(part.data + 1, text, (size_t)part.size - 2) == 0;
}
static size_t field(const document *doc, const char *name) {
size_t found = MAX_TOKENS;
if (doc->tokens[0].kind != '{') return found;
for (size_t index = 1; index < doc->tokens[0].after;) {
size_t entry = doc->tokens[index].after;
if (text_is(doc, index, name)) {
if (found != MAX_TOKENS) return MAX_TOKENS;
found = entry;
}
index = doc->tokens[entry].after;
}
return found;
}
static int same(const document *left, size_t left_index, const document *right, size_t right_index) {
if (left_index >= left->used || right_index >= right->used) return 0;
const token *left_item = &left->tokens[left_index], *right_item = &right->tokens[right_index];
if (left_item->kind != right_item->kind || left_item->count != right_item->count) return 0;
if (left_item->kind == '[') {
for (size_t offset = 1; offset <= left_item->count; offset++)
if (!same(left, left_index + offset, right, right_index + offset)) return 0;
return 1;
}
dever_ffi_slice left_slice = slice(left, left_index), right_slice = slice(right, right_index);
return left_slice.size == right_slice.size && memcmp(left_slice.data, right_slice.data, (size_t)left_slice.size) == 0;
}
static int append(dever_ffi_output *output, dever_ffi_slice input) {
if (input.size > output->capacity - output->size) return 0;
memcpy(output->data + output->size, input.data, (size_t)input.size);
output->size += input.size;
return 1;
}
static int literal(dever_ffi_output *output, const char *text) {
return append(output, (dever_ffi_slice){text, strlen(text)});
}
static int send(dever_ffi_slice data) {
if (data.size == 0 || data.size > MAX_BYTES) return 0;
unsigned char size[4] = {(unsigned char)(data.size >> 24), (unsigned char)(data.size >> 16), (unsigned char)(data.size >> 8), (unsigned char)data.size};
return fwrite(size, 1, 4, stdout) == 4 && fwrite(data.data, 1, (size_t)data.size, stdout) == data.size && fflush(stdout) == 0;
}
static char *receive(size_t *length) {
unsigned char size[4];
if (fread(size, 1, 4, stdin) != 4) return NULL;
*length = ((uint32_t)size[0] << 24) | ((uint32_t)size[1] << 16) | ((uint32_t)size[2] << 8) | size[3];
if (*length == 0 || *length > MAX_BYTES) return NULL;
char *data = malloc(*length);
if (data == NULL) return NULL;
if (fread(data, 1, *length, stdin) != *length) { free(data); return NULL; }
return data;
}
static int identifier(const document *doc, size_t index, uint64_t *result) {
if (index >= doc->used) return 0;
dever_ffi_slice input = slice(doc, index);
uint64_t number = 0;
for (uint64_t offset = 0; offset < input.size; offset++) {
unsigned digit = (unsigned char)input.data[offset] - '0';
if (digit > 9 || number > ((uint64_t)INT64_MAX - digit) / 10) return 0;
number = number * 10 + digit;
}
*result = number;
return 1;
}
static int serve(const dever_ffi_handler *handler, const document *manifest, int *opened) {
size_t length = 0;
char *input = receive(&length);
document hello = {0};
char *buffer = malloc(MAX_BYTES), *returned = malloc(MAX_BYTES);
int okay = input != NULL && buffer != NULL && returned != NULL && parse(&hello, input, length);
const char *fields[] = {"port", "schema", "adapter", "capabilities", "operations"};
if (okay) okay = hello.tokens[0].count == 8 && text_is(&hello, field(&hello, "kind"), "hello") && text_is(&hello, field(&hello, "version"), "dever-component-1");
for (size_t index = 0; okay && index < 5; index++) okay = same(&hello, field(&hello, fields[index]), manifest, field(manifest, fields[index]));
size_t setting = okay ? field(&hello, "setting") : MAX_TOKENS;
if (okay) okay = setting < hello.used;
if (okay && handler->open != NULL) okay = handler->open(handler->context, slice(&hello, setting)) == DEVER_FFI_OK;
dever_ffi_output output = {buffer, MAX_BYTES, 0};
if (okay) {
*opened = 1;
okay = literal(&output, "{\"kind\":\"ready\",\"version\":\"dever-component-1\"");
for (size_t index = 0; okay && index < 5; index++)
okay = literal(&output, ",\"") && literal(&output, fields[index]) && literal(&output, "\":") && append(&output, slice(manifest, field(manifest, fields[index])));
okay = okay && literal(&output, "}") && send((dever_ffi_slice){output.data, output.size});
}
free(hello.tokens);
free(input);
uint64_t next_id = 1;
int cancelled = 0, stopped = 0;
while (okay && !stopped) {
input = receive(&length);
document message = {0};
okay = input != NULL && parse(&message, input, length);
if (!okay) { free(input); break; }
size_t kind = field(&message, "kind"), id_index = field(&message, "id");
uint64_t request_id = 0;
output.size = 0;
if (text_is(&message, kind, "shutdown")) {
okay = message.tokens[0].count == 1 && send((dever_ffi_slice){"{\"kind\":\"shutdown\"}", 19});
stopped = 1;
} else if (text_is(&message, kind, "health")) {
okay = message.tokens[0].count == 2 && identifier(&message, id_index, &request_id) && request_id == 0 && send((dever_ffi_slice){input, length});
} else if (text_is(&message, kind, "cancel")) {
okay = message.tokens[0].count == 2 && identifier(&message, id_index, &request_id) && request_id > 0 && request_id == next_id - 1 && !cancelled;
cancelled = 1;
} else if (text_is(&message, kind, "call")) {
size_t operation = field(&message, "operation"), payload = field(&message, "payload");
okay = message.tokens[0].count == 4 && identifier(&message, id_index, &request_id) && request_id == next_id && payload < message.used && operation < message.used && message.tokens[operation].kind == '"';
size_t operations = field(manifest, "operations");
int found = 0;
for (size_t index = operations + 1; okay && index < manifest->tokens[operations].after; index = manifest->tokens[index].after)
if (same(&message, operation, manifest, index)) found = 1;
okay = okay && found;
if (okay) {
next_id++;
cancelled = 0;
dever_ffi_slice operation_name = slice(&message, operation);
operation_name.data++;
operation_name.size -= 2;
dever_ffi_output result = {returned, MAX_BYTES - 256, 0};
int status = handler->invoke(handler->context, operation_name, slice(&message, payload), &result);
document result_doc = {0};
okay = result.data == returned && result.capacity == MAX_BYTES - 256 && result.size <= result.capacity && parse(&result_doc, returned, (size_t)result.size);
if (okay && status == DEVER_FFI_OK) {
okay = literal(&output, "{\"kind\":\"result\",\"id\":") && append(&output, slice(&message, id_index)) && literal(&output, ",\"payload\":") && append(&output, (dever_ffi_slice){returned, result.size});
} else if (okay && status == DEVER_FFI_ERROR) {
size_t error = field(&result_doc, "error"), error_payload = field(&result_doc, "payload");
okay = result_doc.tokens[0].count == 2 && error < result_doc.used && result_doc.tokens[error].kind == '"' && error_payload < result_doc.used;
okay = okay && literal(&output, "{\"kind\":\"error\",\"id\":") && append(&output, slice(&message, id_index)) && literal(&output, ",\"error\":") && append(&output, slice(&result_doc, error)) && literal(&output, ",\"payload\":") && append(&output, slice(&result_doc, error_payload));
} else okay = 0;
okay = okay && literal(&output, "}") && send((dever_ffi_slice){output.data, output.size});
free(result_doc.tokens);
}
} else okay = 0;
free(message.tokens);
free(input);
}
free(buffer);
free(returned);
return okay && stopped;
}
int dever_ffi_serve(const dever_ffi_handler *handler) {
if (handler == NULL || handler->abi != DEVER_FFI_ABI_V1 || handler->invoke == NULL) return 1;
FILE *file = fopen("contract.json", "rb");
if (file == NULL) return 1;
char *data = malloc(MAX_BYTES + 1);
if (data == NULL) { fclose(file); return 1; }
size_t size = fread(data, 1, MAX_BYTES + 1, file);
int okay = !ferror(file) && size <= MAX_BYTES;
fclose(file);
document manifest = {0};
okay = okay && parse(&manifest, data, size);
if (okay) {
size_t operations = field(&manifest, "operations");
okay = operations < manifest.used && manifest.tokens[operations].kind == '[' && manifest.tokens[operations].count > 0;
const char *names[] = {"port", "schema", "adapter"};
for (size_t index = 0; okay && index < 3; index++) {
size_t entry = field(&manifest, names[index]);
okay = entry < manifest.used && manifest.tokens[entry].kind == '"';
}
size_t capabilities = field(&manifest, "capabilities");
okay = okay && capabilities < manifest.used && manifest.tokens[capabilities].kind == '[';
}
int opened = 0;
okay = okay && serve(handler, &manifest, &opened);
if (opened && handler->close != NULL) handler->close(handler->context);
free(manifest.tokens);
free(data);
if (!okay) fputs("native FFI Worker failed\n", stderr);
return okay ? 0 : 1;
}
+40
View File
@@ -0,0 +1,40 @@
#ifndef DEVER_FFI_H
#define DEVER_FFI_H
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
#define DEVER_FFI_ABI_V1 1
#define DEVER_FFI_OK 0
#define DEVER_FFI_ERROR 1
#define DEVER_FFI_FAULT 2
typedef struct {
const char *data;
uint64_t size;
} dever_ffi_slice;
typedef struct {
char *data;
uint64_t capacity;
uint64_t size;
} dever_ffi_output;
typedef struct {
uint32_t abi;
void *context;
int (*open)(void *context, dever_ffi_slice setting);
int (*invoke)(void *context, dever_ffi_slice operation, dever_ffi_slice payload, dever_ffi_output *output);
void (*close)(void *context);
} dever_ffi_handler;
int dever_ffi_serve(const dever_ffi_handler *handler);
#ifdef __cplusplus
}
#endif
#endif
+1 -1
View File
@@ -54,7 +54,7 @@ def packages(manifest):
if kind["type"] == "target":
if target == manifest["platform"]:
raise ValueError("host native resources belong to the base package")
prefixes = (f"runtime/{target}/", f"sandbox/{target}/")
prefixes = (f"runtime/{target}/", f"process-runtime/{target}/")
else:
prefixes = (f"{kind['type']}/{kind['ecosystem']}/{target}/",)
if any(not artifact["path"].startswith(prefixes) for artifact in extension["artifacts"]):
+2 -2
View File
@@ -20,9 +20,9 @@ GET/DELETE 从 URL query 读标量参数,POST/PUT 从 JSON body 读参数。
`dever.api.Error` 中 Invalid/Unauthorized/Forbidden/NotFound/Conflict/TooManyRequests 对应 400/401/403/404/409/429;未知内部失败记录脱敏日志并返回安全的 500。
写 HTTP、CMD 和 Job 默认由入口建立事务;GET 不能写库或入队。
普通写 HTTP、CMD 和 Job 默认由入口建立整体事务。需要事务外编排时,在入口写 `long post generate = app.generate`、`long cmd generate = app.generate` 或 `long job ...`,其业务写库和 Job 入队必须进入显式 `transaction`。`public long` 的匿名权限与事务模式独立;自动 REST 保持原事务行为,GET 不能写库或入队。
密码 hash/verify 除外:计算期间不能长期占用数据库事务。实际写入要放在显式的短 `transaction` 内,并重新检查相关状态。不要因为“默认有事务”删掉这里的短事务。
HTTP、文件传输、外部 Worker 和密码计算放在事务外;事务内重新检查写入依赖的状态。需要业务记录与入队原子提交时,将两者放在同一个 `transaction`。`long` 中前一阶段已经提交的写入不会因为后续处理或响应编码失败而撤销,重试须保证业务幂等;普通入口仍整体回滚。`long` 不延长超时、不自动转后台;编译器遇到边界违规报错,不自动切换模式。
## 站点、身份与权限
+1 -1
View File
@@ -42,7 +42,7 @@ Python 使用 `Worker.from_manifest`,JS 使用 `Worker.fromManifest`,Go 由
业务拒绝使用 SDK 的 `BusinessError`:本例的 identity 是 `document.markup.RenderResult.Rejected`,payload 是 `{"message":"..."}`。未知异常属于运行失败,不自动变成业务拒绝,也不返回假成功。
stdout 专用于协议,日志写 stderr。Python 处理函数要响应取消,JS 使用 signal,Go 使用 context。需要网络、文件、子进程或 GPU 时,通过 `allow` 声明并在配置中授权;本例纯文档转换不需要这些能力。缺少权限时不能关闭沙箱。
stdout 专用于协议,日志写 stderr。Python 处理函数要响应取消,JS 使用 signal,Go 使用 context。需要网络、文件、子进程或 GPU 时,通过 `allow` 声明效果;实际权限继承容器用户,由部署侧管理,不再使用 Dever 内层沙箱或 Adapter 挂载授权。真实文件路径通过业务 setting 传入。只运行可信 Worker,本例纯文档转换不需要这些能力。
## 准备与测试
+2 -2
View File
@@ -14,7 +14,7 @@ Dever 安装包和编辑器插件从 [Releases](https://git.dever.cc/shemic/deve
curl -fsSL https://git.dever.cc/shemic/dever/raw/branch/main/install | bash
```
无需先克隆源码或安装 skill。安装器下载最新正式版本,安装到 `/opt/dever`,提供 `/usr/local/bin/dever`,并安装、启动机器共享服务及必要的沙箱策略。无需系统 zstd。没有匹配平台的发行包或下载校验失败时明确报错。
无需先克隆源码或安装 skill。安装器下载最新正式版本,安装到 `/opt/dever`,提供 `/usr/local/bin/dever`,并安装、启动机器共享服务。无需系统 zstd。没有匹配平台的发行包或下载校验失败时明确报错。
将命令尾部的 `bash` 换为 `bash -s -- --help` 可查看参数,换为 `bash -s -- --version <准确版本>` 可指定版本。`--no-start` 只安装、不启动服务;应用编译仍需要共享服务可用。已下载脚本也可用 `bash install --help` 查看帮助。
@@ -52,7 +52,7 @@ dever build <project-root> --target linux-aarch64 --output <新文件>
依赖和目标准备会显式联网,后续 run/build 离线;缺资源时按提示准备,不扫描宿主 Python、Node、Go 或 PATH 充当运行环境。省略目标时使用本机架构,run/test 始终在本机执行。ARM64 应用目标不表示提供 ARM64 宿主编译器,macOS、Windows 宿主安装暂未提供。
Linux 外部调用要求系统支持所需 namespace 与 openat2。受限 AppArmor 环境需要安装器配置的专用策略;策略不满足要求时明确失败,不关闭沙箱继续执行。
Linux 外部调用使用受管子进程和 pidfd,不再依赖嵌套 namespace、Bubblewrap 或 Dever 专用 AppArmor 策略。代码直接继承容器用户权限,只运行可信依赖;镜像、设备、挂载与网络权限由部署侧管理。
## 更新与选择版本
+1 -1
View File
@@ -49,7 +49,7 @@ execute(args: List<Text>, stdin: Bytes?) (output: dever.process.Output) fails de
external command "bin/tool" {}
```
将相应架构 ELF 放在领域的 bin/tool,App 调用 `port.execute(["--version"], null)`。结果有 code、stdout、stderr,字节显式转文本;参数不是 shell 字符串。非零退出码仍是结果,启动失败/超时/超限是错误。文件、网络、子进程能力显式声明且受配置授权,不能关闭沙箱来掩盖部署问题。
将相应架构 ELF 放在领域的 bin/tool,App 调用 `port.execute(["--version"], null)`。结果有 code、stdout、stderr,字节显式转文本;参数不是 shell 字符串。非零退出码仍是结果,启动失败/超时/超限是错误。文件、网络、子进程能力仍显式声明,但不构成 OS 沙箱;执行权限继承容器用户。不要把可信 Worker 当作不可信代码隔离环境。
测试必须使用 Port fake,不执行真实第三方程序。跨到 ARM 时必须提供目标架构的库和二进制,Dever 不转换已有 x86 ELF。
+19 -2
View File
@@ -44,10 +44,27 @@
替换路径,勿直接复制示例运行。下文 CI 字段均位于 `performance.linux_ci`。`inputs` 列出固定原始作者输入及准备好的资产文件/目录,逐文件计 SHA-256;工具、协议产物、CMS manifest/可执行文件/配置和 `performance.network_peer/live_peer` 也自动计摘要,`linux_ci` 对象不作为 peer 路径解析。输出目录不得包含在 inputs 中。报告只存摘要,不复制数据库密码。
`required_inputs` 为 `author`、`native`、`postgres`、`sandbox`、`bounded-perf`、`soak` 各提供非空绝对路径列表,必须落在 `inputs` 声明范围内。缺路径记录 blocked,不算 pass。作者 fixture 的具体固定路径/准备命令仍以 `test/ecosystem-release/prepare.py`、`test/ecosystem-release/build_inputs.py` 和对应 ignored 原因文本为准;gate 不把现存文件冒充由当前源码重建的 provenance。
`required_inputs` 为 `author`、`native`、`postgres`、`process-runtime`、`bounded-perf`、`soak` 各提供非空绝对路径列表,必须落在 `inputs` 声明范围内。缺路径记录 blocked,不算 pass。作者 fixture 的具体固定路径/准备命令仍以 `test/ecosystem-release/prepare.py`、`test/ecosystem-release/build_inputs.py` 和对应 ignored 原因文本为准;gate 不把现存文件冒充由当前源码重建的 provenance。
显式 LLVM Worker 验收读取 `target/native-release-inputs/component-fixture`:先用当前源码构建 `dever-tests` 的 `component-fixture` bin,再将程序复制到该准备目录,记录源码/程序摘要,并把固定副本列入 `inputs` 和 `required_inputs.native`。不能将 `target/debug/component-fixture` 当作不可变输入:Cargo 在 default/all-feature 组合间会重新链接这个输出。普通 Worker 测试仍使用 Cargo 为各自组合生成的 `CARGO_BIN_EXE_component-fixture`,不改质量门的输入变更拒绝规则。
原生发行验收还需在同一准备目录提供四个独立构建的 `base.a`、`sqlite.a`、`postgres.a`、`both.a`,`core-libraries/` 放与候选编译器对应的完整非系统动态库闭包,`native/` 放 CRT、静态链接库和匹配的 `runtime.a`。这些输入必须为真实文件;不再假定作者容器是某个 Ubuntu/GCC/LLVM 安装布局。`DEVER_PROCESS_RUNTIME` 显式指向已准备的进程 ABI 目录。测试使用自有签名、安装根和守护进程,不改正式信任密钥或现有安装。
同一目录的 `bin/` 准备当前源码构建的 `dever`、`deverd` 和 `native-release`,验收使用这些固定副本而非猜测 Cargo 输出路径。需要在另一容器的自有临时目录执行时,可用 `DEVER_NATIVE_ACCEPTANCE_WORKSPACE` 指向包含相同准备目录及所需 C 夹具的根。显式设置 `DEVER_NATIVE_ACCEPTANCE_OUTPUT` 为不存在的绝对目录,对应验收全部通过后才导出未压缩的签名输入目录、测试公钥与验证结果;测试私钥不导出,不能当作正式发行签名。
保留候选发行附件也必须复用 `sdk/release-assets.py`,不把基础包、语言扩展和验证记录重新合成总安装包。基础 `dever-<平台>.tar.zst` 仅含签名清单的 `artifacts`;Python、Node.js、Go 分别生成 `ext-runtime-pip/npm/go-<目标>.tar.zst`,按需下载。清单和签名为独立附件;测试公钥及验证记录不放进安装包。示例:
```sh
python3 -B sdk/release-assets.py /absolute/candidate/ecosystems/release \
--output /absolute/candidate/assets --zstd /absolute/author/bin/zstd
```
拆包测试复用 `test/onboarding/test_installer.py`,默认作者工具为 `/usr/bin/zstd`;不同工具位置通过测试专用的 `DEVER_TEST_ZSTD` 显式指定。该测试同时核验三种语言扩展互不混入、基础包内容不变且压缩字节不因新增扩展变化,不安装全局编译器。
完整版本验收使用 `test/onboarding/verify_release_update.py`,显式传入 `--previous-release`、`--release`、`--assets`、`--trusted-key`、`--extension-helper`,以及不存在的绝对 `--work`、`--report` 路径。在独立验证容器中以 root 运行,不挂载业务数据、不启动系统服务;使用容器自有临时空间,不占用正在服务的 dev/deverd 临时卷。它先验证基础包首装、正式项目模板、四档运行库、`long` 和按需语言扩展,再验证真实旧版本升级与回退。旧、新版本必须具有各自实际构建的二进制,不能只改清单版本模拟升级;临时测试密钥的结果也不能算作正式签名验收。
升级核验按签名版本读取旧版原始资源路径和 bootstrap 元数据,不恢复旧沙箱执行;0.1.5 起的新包不得携带已移除的 bootstrap 沙箱配置。旧文件仍须通过原签名中的大小和摘要校验。
`author_outputs` 精确声明 author 会制作的 pack、下载结果和 receipt 路径。author 阶段只将其余原始输入当作不可变输入,另外记录完整输出摘要;后续验收按准备后完整输入摘要绑定。不要把整个作者输入根排除掉。显式 `author_network: true` 才能执行已分类的固定官方下载/源码构建。新 ignored 必须人工更新 `ignored.json`;inventory 核对源码声明及实际 all-feature libtest 名单,执行时一律使用完整 `--exact` 名称。内部 probe 记录 parent-only,只由所属父用例运行。
资源准备归外部托管者:自有空父启用 memory/cpu,限制总 CPU<=2、memory<=4GiB、swap=0;runner/Cargo/负载客户端放 `control` 子组,自有 PostgreSQL 可放兄弟子组,64/128MiB 服务子组由现有 performance owner 创建。gate 只读检查现有父限制与自身归属,不创建/修改主机策略。作者工具所需固定 PATH/动态库路径可由外部托管者显式准备,不写入产品配置。
@@ -64,7 +81,7 @@ python3 -B test/ci/run.py run --stage python
python3 -B test/ci/run.py run --stage sdk
python3 -B test/ci/run.py run --stage native
python3 -B test/ci/run.py run --stage postgres
python3 -B test/ci/run.py run --stage sandbox
python3 -B test/ci/run.py run --stage process-runtime
python3 -B test/ci/run.py run --stage bounded-perf
```
+2 -2
View File
@@ -41,7 +41,7 @@ def classify_cases(names, manifest, target):
def audit_classification():
manifest = json.loads((ROOT / "test/ci/ignored.json").read_text())
stages = {"probe", "author", "native", "postgres", "sandbox", "bounded-perf"}
stages = {"probe", "author", "native", "postgres", "process-runtime", "bounded-perf"}
if any(row["stage"] not in stages for row in manifest):
raise RuntimeError("ignored classification contains an unknown execution stage")
expected = {(row["source"], row["name"], row["reason"]) for row in manifest}
@@ -292,7 +292,7 @@ def soak_stage(stage, config, directory, result):
def run_stage(stage, document, config, directory, result):
if stage == "inventory":
inventory(config, directory, result)
elif stage in ("author", "native", "postgres", "sandbox", "bounded-perf"):
elif stage in ("author", "native", "postgres", "process-runtime", "bounded-perf"):
ignored_stage(stage, document, config, directory, result)
else:
soak_stage(stage, config, directory, result)
+109 -133
View File
@@ -1,24 +1,8 @@
[
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "unprivileged_launcher_probe",
"reason": "internal non-root launcher probe, invoked by enforces_unprivileged_namespace",
"stage": "probe",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/trusted_helper.rs",
"name": "rejects_untrusted_helper_paths_bytes_modes_owners_and_capabilities",
"reason": "requires root and the prepared static author bwrap; touches only owned target files",
"stage": "sandbox",
"package": "dever-sandbox",
"target": "dever_sandbox"
},
{
"source": "test/native_runtime_link.rs",
"name": "canonical_c_header_links_and_executes_the_real_runtime_archive",
"reason": "build runtime-abi-only staticlib in target/native-runtime-abi before running",
"reason": "requires an explicit runtime-only DEVER_RUNTIME_ABI_ARCHIVE",
"stage": "native",
"package": "dever-backend-bridge",
"target": "native_runtime_link"
@@ -26,7 +10,7 @@
{
"source": "test/native_runtime_link.rs",
"name": "llvm_object_calls_the_versioned_runtime_abi_without_rust_layouts",
"reason": "build runtime-abi-only staticlib in target/native-runtime-abi before running",
"reason": "requires an explicit runtime-only DEVER_RUNTIME_ABI_ARCHIVE",
"stage": "native",
"package": "dever-backend-bridge",
"target": "native_runtime_link"
@@ -34,7 +18,7 @@
{
"source": "test/native_runtime_link.rs",
"name": "protocol_poll_rejects_invalid_output_before_advancing_the_operation",
"reason": "build runtime-abi-only staticlib in target/native-runtime-abi before running",
"reason": "requires an explicit runtime-only DEVER_RUNTIME_ABI_ARCHIVE",
"stage": "native",
"package": "dever-backend-bridge",
"target": "native_runtime_link"
@@ -42,7 +26,7 @@
{
"source": "test/native_runtime_link.rs",
"name": "logger_flush_is_a_stable_native_allocation_barrier",
"reason": "build runtime-abi-only staticlib in target/native-runtime-abi before running",
"reason": "requires an explicit runtime-only DEVER_RUNTIME_ABI_ARCHIVE",
"stage": "native",
"package": "dever-backend-bridge",
"target": "native_runtime_link"
@@ -50,7 +34,7 @@
{
"source": "test/native_runtime_link.rs",
"name": "wire_codecs_validate_slots_before_callback_and_release_partial_values",
"reason": "build runtime-abi-only staticlib in target/native-runtime-abi before running",
"reason": "requires an explicit runtime-only DEVER_RUNTIME_ABI_ARCHIVE",
"stage": "native",
"package": "dever-backend-bridge",
"target": "native_runtime_link"
@@ -58,7 +42,7 @@
{
"source": "test/native_runtime_link.rs",
"name": "database_abi_preserves_typed_owners_and_rejects_invalid_slots_without_io",
"reason": "build a runtime-sqlite or runtime-postgres staticlib in target/native-runtime-abi before running",
"reason": "requires a runtime-sqlite or runtime-postgres DEVER_RUNTIME_ABI_ARCHIVE",
"stage": "native",
"package": "dever-backend-bridge",
"target": "native_runtime_link"
@@ -641,8 +625,8 @@
},
{
"source": "test/llvm_job_test_source.rs",
"name": "timed_out_job_drains_children_and_rolls_back_before_retry",
"reason": "requires explicit runtime archive; bounded task cancellation inside owned SQLite transactions",
"name": "timed_out_job_drains_children_and_preserves_committed_stages",
"reason": "requires explicit runtime archive; bounded cancellation between committed SQLite stages",
"stage": "native",
"package": "dever-backend-bridge",
"target": "llvm_job_test_source"
@@ -1287,110 +1271,6 @@
"package": "dever-backend-bridge",
"target": "llvm_application_source"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "enforces_files_network_process_and_threads",
"reason": "requires explicitly prepared target/sandbox-inputs assets; owns only test processes",
"stage": "sandbox",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "pins_grants_against_directory_replacement",
"reason": "requires explicitly prepared sandbox assets; mutates only owned fixture paths",
"stage": "sandbox",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "pinned_grant_probe",
"reason": "internal sandbox descriptor-pinning probe",
"stage": "probe",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "rejects_writable_aliases_of_executable_inputs",
"reason": "requires explicitly prepared sandbox assets",
"stage": "sandbox",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "prepared_python_and_node_run_without_host_files_or_process_creation",
"reason": "requires explicitly prepared signed-release Python/Node inputs and sandbox assets",
"stage": "sandbox",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "enforces_unprivileged_namespace",
"reason": "requires explicitly prepared assets and host permission for unprivileged user namespaces",
"stage": "sandbox",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "restricted_probe",
"reason": "internal sandbox probe, invoked only inside an owned namespace",
"stage": "probe",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "terminates_detached_descendants_on_kill",
"reason": "requires explicitly prepared assets; kills only this test's namespace",
"stage": "sandbox",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "tree_probe",
"reason": "internal sandbox process-tree probe",
"stage": "probe",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "descendant_probe",
"reason": "internal sandbox descendant probe",
"stage": "probe",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "granted_probe",
"reason": "internal sandbox probe, invoked only inside an owned namespace",
"stage": "probe",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "rejects_missing_dependency_before_host_loader_can_supply_it",
"reason": "requires explicitly prepared target/sandbox-inputs assets",
"stage": "sandbox",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "kernel_rejects_namespace_ptrace_and_truncated_ioctl_requests",
"reason": "requires explicit assets and the prepared C syscall probe",
"stage": "sandbox",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-cli-tests/llvm_cli.rs",
"name": "default_llvm_runs_builds_moves_and_reuses_cache_without_host_toolchain",
@@ -1442,14 +1322,14 @@
{
"source": "test/dever-cli-tests/external_workers.rs",
"name": "compiles_go_worker_with_locked_tools_and_runs_without_source_or_go",
"reason": "requires explicit Go runtime pack, official UUID proof/archive and sandbox assets",
"reason": "requires explicit Go runtime pack, official UUID proof/archive and process_runtime assets",
"stage": "native",
"package": "dever-cli",
"target": "external_workers"
},
{
"source": "test/dever-cli-tests/external_workers.rs",
"name": "runs_official_native_python_wheel_in_sandbox",
"name": "runs_official_native_python_wheel_as_managed_worker",
"reason": "requires the explicit official native-wheel and complete Python runtime fixture at target/python-managed",
"stage": "native",
"package": "dever-cli",
@@ -1457,7 +1337,7 @@
},
{
"source": "test/dever-cli-tests/external_workers.rs",
"name": "runs_official_python_source_build_in_sandbox",
"name": "runs_official_python_source_build_as_managed_worker",
"reason": "requires explicitly prepared official PEP517 source build at target/source-build",
"stage": "native",
"package": "dever-cli",
@@ -1538,7 +1418,7 @@
{
"source": "test/dever-tests/tests/external_resources.rs",
"name": "packaged_exec_runs_without_source_worker_or_inherited_environment",
"reason": "requires explicitly prepared sandbox assets and the private Rust bootstrap compiler",
"reason": "requires explicitly prepared process_runtime assets and the private Rust bootstrap compiler",
"stage": "native",
"package": "dever-tests",
"target": "external_resources"
@@ -1623,6 +1503,14 @@
"package": "dever-cli",
"target": "native_release"
},
{
"source": "test/dever-cli-tests/native_release/ecosystem_acceptance.rs",
"name": "signed_zero_lib_ecosystems_run_and_build_in_the_current_container",
"reason": "requires explicit native and zero-Lib ecosystem packs; uses an owned root installation and daemon",
"stage": "native",
"package": "dever-cli",
"target": "native_release"
},
{
"source": "test/dever-cli-tests/native_release/ecosystem_acceptance.rs",
"name": "signed_ecosystems_build_and_run_without_system_language_installations",
@@ -1698,7 +1586,7 @@
{
"source": "test/dever-cli-tests/external_libs/npm_build.rs",
"name": "builds_official_npm_addon_with_locked_lifecycle",
"reason": "explicit official npm native source build; network only in registry resolver, never in lifecycle sandbox",
"reason": "explicit official npm native source build; network only in registry resolver, never in lifecycle process_runtime",
"stage": "author",
"package": "dever-cli",
"target": "external_libs"
@@ -1734,5 +1622,93 @@
"stage": "author",
"package": "dever-cli",
"target": "external_libs"
},
{
"source": "test/dever-process-tests/execution.rs",
"name": "direct_child_probe",
"reason": "internal direct-process probe",
"stage": "probe",
"package": "dever-process",
"target": "execution"
},
{
"source": "test/dever-cli-tests/external_workers/ffi.rs",
"name": "real_c_abi_packages_compiles_and_checks_the_generated_protocol",
"reason": "requires explicit DEVER_FFI_CC, DEVER_PROCESS_RUNTIME inputs",
"stage": "native",
"package": "dever-cli",
"target": "external_workers"
},
{
"source": "test/dever-cli-tests/external_workers/ffi.rs",
"name": "real_c_abi_direct_worker_errors_deadlines_and_cancellation",
"reason": "requires explicit DEVER_FFI_CC and DEVER_PROCESS_RUNTIME inputs",
"stage": "native",
"package": "dever-cli",
"target": "external_workers"
},
{
"source": "test/dever-cli-tests/llvm_cli.rs",
"name": "llvm_test_controls_preserve_indices_prefix_boundaries_and_timeout_continuation",
"reason": "requires an explicitly prepared runtime pack adjacent to CARGO_BIN_EXE_dever; owned short-lived cases only",
"stage": "native",
"package": "dever-cli",
"target": "llvm_cli"
},
{
"source": "test/dever-cli-tests/native_release/command_acceptance.rs",
"name": "signed_cli_runs_and_packages_ordinary_command_with_private_library",
"reason": "requires explicit current runtime archive, author C/LLVM inputs and owned daemon/process_runtime",
"stage": "native",
"package": "dever-cli",
"target": "native_release"
},
{
"source": "test/dever-cli-tests/native_release/cross_acceptance.rs",
"name": "signed_host_compiler_builds_arm_profiles_and_runs_standalone_under_qemu",
"reason": "requires prepared x86/ARM four-profile runtime archives and private QEMU; starts only an owned daemon",
"stage": "native",
"package": "dever-cli",
"target": "native_release"
},
{
"source": "test/dever-cli-tests/native_release/cross_ecosystems.rs",
"name": "signed_arm_ecosystems_build_offline_standalone_apps_for_system_guest",
"reason": "requires explicit ARM runtimes/dependencies/process_runtime and native archives; builds with an owned host daemon and exports apps for ARM system QEMU",
"stage": "native",
"package": "dever-cli",
"target": "native_release"
},
{
"source": "test/dever-cli-tests/native_release/cross_ecosystems.rs",
"name": "signed_arm_python_worker_rebuilds_with_current_sdk",
"reason": "requires explicit ARM inputs; rebuilds only Python for SDK shutdown regression under the ARM guest",
"stage": "native",
"package": "dever-cli",
"target": "native_release"
},
{
"source": "test/dever-cli-tests/shared_toolchain.rs",
"name": "active_skill_is_readable_by_other_machine_users_without_install_lock_access",
"reason": "requires root; drops only an owned skill client to an unprivileged uid",
"stage": "native",
"package": "dever-cli",
"target": "shared_toolchain"
},
{
"source": "test/llvm_database_source.rs",
"name": "sqlite_long_command_preserves_commit_on_encoding_failure",
"reason": "requires explicit runtime archive; committed long-entry SQLite stages",
"stage": "native",
"package": "dever-backend-bridge",
"target": "llvm_database_source"
},
{
"source": "test/llvm_external_source.rs",
"name": "command_binary_io_and_exit_status_use_real_typed_llvm_calls",
"reason": "requires fresh runtime-external archive, command-fixture and process_runtime assets",
"stage": "native",
"package": "dever-backend-bridge",
"target": "llvm_external_source"
}
]
+1 -1
View File
@@ -17,7 +17,7 @@ sys.modules["linux_gate"] = sys.modules[__name__]
from settings import configured_peer, write_private_json
STAGES = ("static", "default", "all-features", "features", "python", "sdk",
"inventory", "author", "native", "postgres", "sandbox", "bounded-perf",
"inventory", "author", "native", "postgres", "process-runtime", "bounded-perf",
"cms-64", "cms-128", "http2-64", "http2-128", "recovery-64", "recovery-128",
"live-64", "live-128")
SOURCE_ROOTS = ("crates", "test", "sdk", "library", "examples", "skills", ".cargo")
+5 -5
View File
@@ -82,8 +82,8 @@ class GateTests(unittest.TestCase):
gate.check_output("python", {"python_tests": 0}, "OK")
def test_classification_rejects_unknown_duplicate_and_wrong_target(self):
row = {"package": "sandbox", "target": "isolation", "name": "restricted_probe", "stage": "probe"}
target = {"package": "sandbox", "target": "isolation"}
row = {"package": "process-runtime", "target": "isolation", "name": "restricted_probe", "stage": "probe"}
target = {"package": "process-runtime", "target": "isolation"}
self.assertEqual(classify_cases(["restricted_probe"], [row], target)[0]["stage"], "probe")
for names, rows, owner in ((["unknown"], [row], target), (["restricted_probe"], [row, row], target),
(["restricted_probe"], [row], {**target, "target": "other"}),
@@ -135,8 +135,8 @@ class GateTests(unittest.TestCase):
"failed": parent_failed, "ignored": parent_ignored})
def test_exact_ignored_acceptance_requires_one_successful_parent(self):
target = {"package": "sandbox", "target": "isolation"}
case = {"name": "parent", "exact": "parent", "stage": "sandbox",
target = {"package": "process-runtime", "target": "isolation"}
case = {"name": "parent", "exact": "parent", "stage": "process-runtime",
"target": target, "target_directory": "/unused/target"}
for counts in ({"passed": 1, "failed": 0, "ignored": 0},
{"passed": 0, "failed": 0, "ignored": 0},
@@ -154,7 +154,7 @@ class GateTests(unittest.TestCase):
result = {"commands": []}
expected = counts == {"passed": 1, "failed": 0, "ignored": 0}
with contextlib.nullcontext() if expected else self.assertRaisesRegex(RuntimeError, "one case"):
acceptance.ignored_stage("sandbox", {}, {}, Path("/unused"), result)
acceptance.ignored_stage("process-runtime", {}, {}, Path("/unused"), result)
self.assertTrue(execute.call_args.kwargs["single_harness"])
self.assertEqual(len(result.get("cases", [])), int(expected))
+6 -1
View File
@@ -144,7 +144,7 @@ fn exec_preparation_embeds_every_candidate_with_its_current_bytes() {
dever_cli::workers::prepare(root.path(), &program, &input).unwrap()
};
let resources = prepare();
assert_eq!(resources.len(), entries.len() * 2);
assert_eq!(resources.len(), entries.len() * 3);
for entry in &entries {
let manifest = resources
.iter()
@@ -157,6 +157,11 @@ fn exec_preparation_embeds_every_candidate_with_its_current_bytes() {
.unwrap();
assert_eq!(worker.bytes, fs::read(root.path().join(entry)).unwrap());
assert!(worker.executable);
assert!(resources.iter().any(|resource| resource.path
== format!(
"{}/contract.json",
manifest["working_directory"].as_str().unwrap()
)));
}
let mut changed_binary = native_elf::static_executable();
changed_binary.extend_from_slice(b"changed");
+6 -6
View File
@@ -2,8 +2,8 @@ use super::*;
use dever_cli::libs::build::{Inputs, Session, pack};
use serde::{Deserialize, Serialize};
#[path = "../../dever-tests/tests/support/sandbox.rs"]
pub(super) mod sandbox;
#[path = "../../dever-tests/tests/support/process_runtime.rs"]
pub(super) mod process_runtime;
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
@@ -111,12 +111,12 @@ impl Inputs for AuthorInputs {
}
fn assets(&self) -> Result<Vec<(String, Vec<u8>)>, String> {
let workspace = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../..");
Ok(sandbox::assets(&workspace)
Ok(process_runtime::assets(&workspace)
.into_iter()
.map(|asset| {
let _mode = asset.executable;
(
asset.path.trim_start_matches("sandbox/").into(),
asset.path.trim_start_matches("process-runtime/").into(),
asset.bytes,
)
})
@@ -195,11 +195,11 @@ pub(super) fn prepare_tool_pack(
}
files.sort_by(|left, right| left.0.cmp(&right.0));
manifest.validate(&files, &descriptor).unwrap();
let assets = sandbox::assets(&workspace)
let assets = process_runtime::assets(&workspace)
.into_iter()
.map(|asset| {
(
asset.path.trim_start_matches("sandbox/").into(),
asset.path.trim_start_matches("process-runtime/").into(),
asset.bytes,
)
})
Loaded 100 of 153 files, more files were not shown because too many files have changed in this diff. Show more