feat: release Dever 0.1.5 with direct workers and long transactions
This commit is contained in:
1 parent
89629c2197
commit
a900d7e8db
153 files changed
+3676
-3708
No files matched your search
@@ -88,7 +88,7 @@ Test authority comes from compiler-assigned source origin, never a user-controll
|
||||
|
||||
For `module/<component>/<domain>/api.dever`, the route is `/<component>/<domain>/<action>`; recursive `api/<scopes>/<file>.dever` inserts scopes and file before action. `get/post/put/delete` bind same-domain App functions and inherit their wire-safe input/output shapes; GET/DELETE read query, POST/PUT read a unique-key JSON object. `rest` generates Model CRUD and can use checked owner/create/replace/search field bindings. Each successful request receives JSON `{code:0,message:"ok",data:<value>}` with HTTP 200; standard API errors map to fixed 400/401/403/404/409/429 envelopes and unknown failures become a redacted 500. A top-level POST `Upload` switches the statically known route to bounded multipart parsing and affine ownership. No environment-variable config or implicit database fallback is introduced.
|
||||
|
||||
Response Header/Cookie mutations are staged in request-local pending metadata. Generated dispatch serializes the response value, commits the database transaction, and only then promotes pending metadata for the server response; serialization, business, commit or rollback failure discards it. Cookie issue/clear uses the configured name, lifetime and attributes. Credential-changing requests require Origin and Host to exactly match the site's configured external HTTP/HTTPS scheme, host and effective port; a missing or cross-origin value is rejected without trusting forwarding headers. A write API call graph containing password verification or hashing does not receive an entry-wide transaction. Every reachable write must instead cross an explicit short `transaction` function, and password crypto is forbidden inside that transaction. Other write entries retain their generated transaction.
|
||||
Response Header/Cookie mutations are staged in request-local pending metadata. Generated dispatch publishes metadata only after successful business execution and response serialization; failure discards pending metadata, rolls back ordinary entry transactions and cannot undo already committed long-entry App transactions. Cookie issue/clear uses the configured name, lifetime and attributes. Credential-changing requests require Origin and Host to exactly match the site's configured external HTTP/HTTPS scheme, host and effective port; a missing or cross-origin value is rejected without trusting forwarding headers. Ordinary writing HTTP, CMD and Job entries retain their automatic transaction through response encoding. The contextual `long` prefix on an entry explicitly disables only its entry-wide transaction and requires `transaction` functions around business writes and enqueue. Both backends obey the entry flag, never an inferred effect-based opt-out. `public long` preserves independent access control; `long` changes neither timeout nor scheduling. Generated REST keeps its own write boundary. External Worker, HTTP, file-transfer and password operations cannot occur inside a transaction. Database binding groups include ordinary automatic roots and explicit transactions, but not an entire long-entry orchestration spanning independent stages.
|
||||
|
||||
## 4. Validation & Error Matrix
|
||||
|
||||
@@ -145,7 +145,7 @@ Sensitivity belongs to the cached type graph in `types.rs`: Secret and private M
|
||||
| User Job provider/site/reference, compiled permission, component or current role fails | Block with a redacted identity/component category; do not enter the business transaction |
|
||||
| Credential-changing request has missing or non-matching Origin | JSON authentication error; no Cookie/Header is committed |
|
||||
| Response serialization, business transaction or commit fails after metadata was staged | Discard staged response metadata |
|
||||
| Password crypto API reaches a write outside an explicit transaction, or a transaction reaches password crypto | C014 before native generation |
|
||||
| A long entry reaches a Model write or enqueue outside an explicit transaction, or an automatic/explicit transaction reaches external I/O, a Worker, wait/concurrency or password crypto | C014 before native generation |
|
||||
|
||||
## 5. Good / Base / Bad Cases
|
||||
|
||||
|
||||
@@ -81,7 +81,7 @@ Model operations, row-stream pulls and transaction begin/commit propagate `dever
|
||||
|
||||
Declare a business function with `transaction name(...) (...)`. The compiler propagates its database effects through ordinary calls and specialized handler bindings, then validates the concrete call graph. The generated hidden context owns one pool object at the outer boundary; nested transaction calls reuse it without savepoints.
|
||||
|
||||
Success commits. Error/fault rolls back before propagation. A rollback failure is appended as a cause and does not replace the original error. Commit and rollback consume the transaction, immediately returning a one-connection SQLite pool object after completion. SQLite write transactions use `BEGIN IMMEDIATE`. An API call graph containing password verification or hashing receives no entry-wide transaction; every write in that graph must be owned by an explicit short transaction reached after the expensive work, and that transaction must recheck the mutable state on which the write depends.
|
||||
Success commits. Error/fault rolls back before propagation. A rollback failure is appended as a cause and does not replace the original error. Commit and rollback consume the transaction, immediately returning a one-connection SQLite pool object after completion. SQLite write transactions use `BEGIN IMMEDIATE`. Ordinary writing HTTP, CMD and Job roots own an entry-wide transaction through output encoding. Only explicitly declared `long` entries omit that transaction; every Model write and persistent enqueue in a long entry must cross an explicit transaction. Generated REST retains its own write transaction. Never infer a mode change from external work or elapsed time. External Worker, HTTP, file-transfer and password work belongs outside transactions. Recheck mutable write preconditions inside each transaction. In long entries, previously committed stages survive later errors, cancellation and response serialization failure; ordinary entries retain all-or-nothing rollback. Nested transactions reuse the entry owner when present.
|
||||
|
||||
Use one checked query planner and shared value model for both database executors. Backend-specific code is limited to SQL dialect, bind/row codecs, pool operations and catalog migration. Keep business transactions in the owning domain App; Model CRUD remains compiler-generated.
|
||||
|
||||
@@ -168,7 +168,7 @@ Checked Model/SQL/transaction lowering serves LLVM CMD, API, Job and case-local
|
||||
|
||||
Deployment settings still come only from executable-local `config/setting.json`; no connection environment variable or PATH language runtime is introduced. Ordinary resolution is explicit connection → component root → required default. Persistence ABI accepts only fixed storage scalar rows; business Model records, nullable fields, Related and DbError are concrete typed owners, not dynamic business values. Related ToMany groups at most 998 parent keys per SQL batch and retains the existing per-parent row limit. Migrations group by resolved database, create all tables before foreign keys and preserve existing history/Seed validation.
|
||||
|
||||
Writing CMD order is input decode → begin → sequential App/handler calls → output encode → commit → stdout. Nested transaction shares the outer owner, without savepoints; Task and parallel boundaries detach. Failed or cancelled unconfirmed operations discard the transaction connection. Rollback must move the original fault, retain its type/source/payload and only append cleanup cause; commit failure must release the already-initialized output. Drain children, stop every pool, await pool drivers, then destroy the runtime. Never reuse the process-global Settings/pools for repeated embedded roots.
|
||||
Ordinary writing CMD order is input decode → begin → sequential App/handler calls → output encode → commit → stdout. A `long cmd` omits entry begin/commit and instead uses explicit App transaction stages. Nested transaction shares the outer owner, without savepoints; Task and parallel boundaries detach. Failed or cancelled unconfirmed operations discard the transaction connection. Rollback must move the original fault, retain its type/source/payload and only append cleanup cause; commit failure must release the already-initialized output. Drain children, stop every pool, await pool drivers, then destroy the runtime. Never reuse the process-global Settings/pools for repeated embedded roots.
|
||||
|
||||
### Validation & Error Matrix
|
||||
|
||||
@@ -177,7 +177,8 @@ Writing CMD order is input decode → begin → sequential App/handler calls →
|
||||
| Invalid/misaligned output or typed callback slot | ABI fault before I/O, receiver advancement or fault consumption |
|
||||
| Missing/invalid settings or missing tenant identity/readiness | Explicit root failure, no default connection fallback |
|
||||
| Database error | Exact one of the ten checked DbError variants with owned Text payload |
|
||||
| Output encoding or numeric fault after a write | Original runtime fault; rollback; no stdout |
|
||||
| Ordinary entry business/output fault, or fault before a long stage commits | Original fault; rollback of the current owned transaction |
|
||||
| Long-entry output encoding or later-stage fault after commit | Original fault; no successful stdout; previously committed rows remain |
|
||||
| Captured PostgreSQL SQL constraint error | Current transaction remains aborted; no implicit savepoint/continued-query promise |
|
||||
| Partial decode, stream stop/cancel or root exit | Paired typed cleanup and connection release/discard in its known state |
|
||||
|
||||
@@ -187,7 +188,7 @@ Base: a CMD creates, reads and deletes a row in its owned SQLite database. Good:
|
||||
|
||||
### Tests Required
|
||||
|
||||
Select root `llvm_database_source`, `runtime_entry_cleanup` and the canonical C database driver in `native_runtime_link`. Static cases must pass the existing checker and emit objects for all six targets. Opt-in native cases link the explicitly prepared runtime-only archive, execute with a cleared environment, and check output/fault plus allocation balance over warmup and 64 repeats. Cover CRUD/bulk/upsert, nullable/codec values, cursor/offset, stream cancellation with one connection, ToOne/ToMany, typed SQL, nested/sequential transactions, uncaught constraint/numeric/encoding rollback and successive schema/Seed revisions. Live PostgreSQL requires only the shared `database.postgres_test` isolated fixture, not an environment URL. These checks do not prove target runtime packs, throughput/RSS, tenant/API equivalence or injected commit/close failures.
|
||||
Select root `llvm_database_source`, `runtime_entry_cleanup` and the canonical C database driver in `native_runtime_link`. Static cases must pass the existing checker and emit objects for all six targets. Opt-in native cases link the explicitly prepared runtime-only archive, execute with a cleared environment, and check output/fault plus allocation balance over warmup and 64 repeats. Cover CRUD/bulk/upsert, nullable/codec values, cursor/offset, stream cancellation with one connection, ToOne/ToMany, typed SQL, nested/sequential transactions, uncaught constraint/numeric rollback, committed state after encoding failure and successive schema/Seed revisions. Live PostgreSQL requires only the shared `database.postgres_test` isolated fixture, not an environment URL. These checks do not prove target runtime packs, throughput/RSS, tenant/API equivalence or injected commit/close failures.
|
||||
|
||||
### Wrong Versus Correct
|
||||
|
||||
|
||||
@@ -4,9 +4,9 @@
|
||||
|
||||
Application deployment targets belong to `toolchain::BuildTarget`, independently of the compiler host. Public `build --target linux-aarch64` may run on Linux x86_64; omitted targets and `run/test` stay native. Propagate the selected target through Lib preparation, CompileRequest, runtime pack selection, signature closure verification and both caches. Verify ELF machine/type for archive members and ordinary objects; a matching manifest label/hash alone does not prove architecture. A warm shared cache must still validate the selected target pack. Do not globally replace `platform_identity` or require an ARM compiler core for cross builds.
|
||||
|
||||
Target runtime/interpreter/extension/exec/sandbox assets must match the deployment target. Go build tools explicitly declare `build.host`, while Go output and stdlib headers match `build.goos/goarch` and version. Their sandbox uses host assets; emitted Workers use target assets. Product processes keep cleared environments; author-only cross linker settings or Go overlays must not become application configuration/environment knobs. Python sdist/npm hook execution requires a matching execution environment or a previously prepared complete target receipt; never relabel host native output.
|
||||
Target runtime/interpreter/extension/exec/process-runtime assets must match the deployment target. Go build tools explicitly declare `build.host`, while Go output and stdlib headers match `build.goos/goarch` and version. Their managed process uses host assets; emitted Workers use target assets. Product processes clear non-allowlisted environments; author-only cross linker settings or Go overlays must not become application configuration/environment knobs. Python sdist/npm hook execution requires a matching execution environment or a previously prepared complete target receipt; never relabel host native output.
|
||||
|
||||
Executable modes are also part of the cross-target resource contract. Sandbox packaging and application embedding share `asset_executable`; classify both supported target loaders independently of the author's host, while retaining target ELF validation. Regressions must check actual packaged modes from non-executable source inputs and execute the embedded Worker. Host-only loader assertions and successful linking cannot establish target bootstrap behavior. Full-system sandbox fixtures must switch from initramfs rootfs to a normal root mount before bubblewrap uses `pivot_root`; do not loosen the production sandbox to accommodate a fixture mount error.
|
||||
Executable modes are also part of the cross-target resource contract. Process runtime packaging and application embedding share `asset_executable`; classify both supported target loaders independently of the author's host, while retaining target ELF validation. Regressions must check actual packaged modes from non-executable source inputs and execute the embedded Worker. Host-only loader assertions and successful linking cannot establish target bootstrap behavior.
|
||||
|
||||
Applies to formatter/CLI, reference execution, native ownership and clause optimization, concurrent sequence actions and official source packages. JSON algorithms belong to `.dever`; HTTP reuses Hyper in the runtime, with source-owned public types and static business handlers. `intrinsic.rs` remains the exhaustive bridge for checker, native emission and reference evaluation.
|
||||
|
||||
@@ -24,7 +24,7 @@ Private `dever run/build/test` uses checked HIR → LLVM → object → in-proce
|
||||
|
||||
Native authoring uses the independent four-profile `runtime-pack` build; the old all-features debug archive remains a test fixture only. Signing prepared inputs and compiling source are separate operations. Do not move Cargo/tool discovery into the maker or application run/build.
|
||||
|
||||
Linux first installation reuses the release signature and machine version owner. Author `bootstrap` inputs provide launcher/daemon and their own complete private ELF library closure. The independently authenticated installer reads only its absolute root-owned `config/setting.json`; its public key must be supplied outside the downloaded release, then pinned atomically without key replacement. A bounded journal may name only the seven fixed bootstrap destinations, including the sandbox helper tree and dedicated AppArmor policy. Persist final file modes and child directories before publication; recover uncommitted changes, but never roll back a committed publication because cleanup failed. Stop a newly started service before removing its unit; restore previous service state after restoring files. Offline image installation never invokes systemctl. Incoming packages must match the supported service template; existing signed versions retain their original template and remain usable by pinned projects. Actual global activation and public release require separate operational authorization and evidence.
|
||||
Linux first installation reuses the release signature and machine version owner. Author `bootstrap` inputs provide launcher/daemon and their own complete private ELF library closure. The independently authenticated installer reads only its absolute root-owned `config/setting.json`; its public key must be supplied outside the downloaded release, then pinned atomically without key replacement. A bounded journal may name only the five fixed bootstrap destinations: launcher/daemon bin, system service, command entry, enable link and active version. Persist final file modes and child directories before publication; recover uncommitted changes, but never roll back a committed publication because cleanup failed. Stop a newly started service before removing its unit; restore previous service state after restoring files. Offline image installation never invokes systemctl. Incoming packages must match the supported service template; existing signed versions retain their original template and remain usable by pinned projects. Actual global activation and public release require separate operational authorization and evidence.
|
||||
|
||||
Process main is separate from callable roots. Initialize Unix SIGPIPE policy before application threads, preserving stdout EPIPE as the existing located runtime error. Do not change an embedding host's signals from callable roots. Reuse typed fault render, indexed Test fault size/render/release, output cleanup and logger flush; Test main accepts exactly one nonempty unsigned decimal index below case count. Verify actual env-cleared CLI execution, relocated embedded Worker, selected-pack tampering on cache hits, broken stdout, and launcher/daemon dynamic dependencies. Six-target objects do not establish cross-platform execution.
|
||||
|
||||
@@ -85,7 +85,7 @@ Process main is separate from callable roots. Initialize Unix SIGPIPE policy bef
|
||||
|
||||
Developer onboarding uses one official source repository, `ssh://git@git.dever.cc:2222/shemic/dever.git`. Its ordinary `skills/` directory is committed and versioned with the compiler, without a submodule or separate skill repository; it distinguishes this language from Go Dever. Author `skill` is a required list of `{source,path,sha256}` entries published as signed `skills/dever-language/<path>` artifacts, including SKILL.md; at most 256 files, 1 MiB each and 8 MiB total. New-project source bytes come from the same skill templates; Markdown documentation wraps them without a second semantic copy. Keep the generated configuration CMD-only and free of database/auth prerequisites.
|
||||
|
||||
Official `install`, `update` and extension preparation resolve `https://git.dever.cc/api/v1/repos/shemic/dever/releases/latest` or `/tags/v<major.minor.patch>` anonymously. Neither project setting.json nor environment variables choose that origin. Resolve once per operation; the selected stable tag must match the signed manifest even for latest. Metadata is bounded to 2 MiB and 1–512 unique attachments with positive bounded sizes. Download the exact attachment name through its validated `browser_download_url`, enforcing the declared length and HTTPS same-origin redirects. Reject draft/prerelease, malformed tags, duplicate names, missing assets, foreign URLs, truncation and tag/signature mismatch before publication. `dever-release-v2` binds base `artifacts` and optional `extensions` in one signature. Each release publishes `dever-<platform>.manifest.json`, `.manifest.sig`, `.tar.zst`, plus `.ext-<kind>-<target>.tar.zst`. Kind is runtime-pip/npm/go, build-pip/npm, or target; target holds nonhost native runtime/CRT and sandbox. Validate closed IDs, path namespaces, duplicate/overlapping paths, digests and size/count limits in the shared catalog owner. Base retains all four host database profiles, compiler closure, bootstrap, host sandbox and skill. Extensions cannot override base files. Do not fabricate an unsigned merged manifest.
|
||||
Official `install`, `update` and extension preparation resolve `https://git.dever.cc/api/v1/repos/shemic/dever/releases/latest` or `/tags/v<major.minor.patch>` anonymously. Neither project setting.json nor environment variables choose that origin. Resolve once per operation; the selected stable tag must match the signed manifest even for latest. Metadata is bounded to 2 MiB and 1–512 unique attachments with positive bounded sizes. Download the exact attachment name through its validated `browser_download_url`, enforcing the declared length and HTTPS same-origin redirects. Reject draft/prerelease, malformed tags, duplicate names, missing assets, foreign URLs, truncation and tag/signature mismatch before publication. `dever-release-v2` binds base `artifacts` and optional `extensions` in one signature. Each release publishes `dever-<platform>.manifest.json`, `.manifest.sig`, `.tar.zst`, plus `.ext-<kind>-<target>.tar.zst`. Kind is runtime-pip/npm/go, build-pip/npm, or target; target holds nonhost native runtime/CRT and process runtime libraries. Validate closed IDs, path namespaces, duplicate/overlapping paths, digests and size/count limits in the shared catalog owner. Base retains all four host database profiles, compiler closure, bootstrap, host process runtime libraries and skill. Extensions cannot override base files. Do not fabricate an unsigned merged manifest.
|
||||
|
||||
Authenticate the catalog before extraction. Zstandard is a single checksummed frame with window_log_max=27; verify complete compressed EOF/trailer and bounded expansion even after TAR EOF. USTAR contains only its signed ordinary artifact files, with no duplicate, escaping or extra entries; require complete terminator/padding and reject trailing content. Redirects remain on fixed official HTTPS hosts. Publication never replaces another writer's directory. First-install raw `.blob-<sha256>` assets include the launcher and every bootstrap/lib dependency, independently digest-checked before execution. The launcher owns extraction; Python must not implement a second TAR/Zstd parser or require system zstd.
|
||||
|
||||
@@ -127,7 +127,7 @@ The machine toolchain has one stable `dever` launcher and a signed version store
|
||||
|
||||
Linux cross-user IPC uses kernel `SO_PEERCRED`, verifies endpoint/server UID against the machine state owner and keeps the administrator token owner-only. Foreign users may transfer Lib artifacts and query metadata, never clean or trigger whole-cache hashing. Eight slots admit at most two foreign connections per UID and six total. Artifact transfer accepts at most 64 MiB and stores at most 2 GiB/4096 entries. Compiled entries have separate equal quotas and are inaccessible through ArtifactGet/Put: retrieving a program requires its complete canonical compilation request.
|
||||
|
||||
Compilation admission is two requests, with 384 MiB wire input, 4096/16 MiB sources and 4096/256 MiB resources (128 MiB each). The service verifies the installed release and complete runtime reference closure, including its native manifest and LLVM library, on every request before cache lookup. Identity binds canonical input and signed manifest; a persistent private install lock is shared through compilation/transfer and exclusive for install/uninstall, never unlinked. The whole request holds a cache operation; transmission retains its read lease. Only the signed core's hidden compile worker may produce a publishable program. It reads a service-owned 0600 request inside 0700 staging; trusted state/cache ancestors remain traversable and non-writable by other users. The worker reconstructs/checks sources and links, without loading deployment Settings, preparing Libs, executing App/Test or writing shared cache. Caller retains dependency preparation, baseline checks, program execution and per-case Test settings. Worker CPU/address-space/file/FD/core limits do not constitute an OS sandbox.
|
||||
Compilation admission is two requests, with 384 MiB wire input, 4096/16 MiB sources and 4096/256 MiB resources (128 MiB each). The service verifies the installed release and complete runtime reference closure, including its native manifest and LLVM library, on every request before cache lookup. Identity binds canonical input and signed manifest; a persistent private install lock is shared while validating/compiling and exclusive for install/uninstall, never unlinked. Release that installation lease after the verified output is restored into owned memory, before returning the response, on both cache-hit and fresh-compile paths. Temporary staging cleanup must not leave a completed caller unable to switch versions. The whole request holds a cache operation; transmission retains its read lease. Only the signed core's hidden compile worker may produce a publishable program. It reads a service-owned 0600 request inside 0700 staging; trusted state/cache ancestors remain traversable and non-writable by other users. The worker reconstructs/checks sources and links, without loading deployment Settings, preparing Libs, executing App/Test or writing shared cache. Caller retains dependency preparation, baseline checks, program execution and per-case Test settings. Worker CPU/address-space/file/FD/core limits do not constitute an OS sandbox.
|
||||
|
||||
Compilation has a 180-second absolute service deadline, including output transfer, and a 190-second client download deadline. EOF or any byte following the exact request body cancels it. Every failure kills/waits for the owned child and removes staging; diagnostics are capped at 64 KiB and output at 320 MiB. This output budget applies only to trusted compilation receipts, never opaque ArtifactGet/Put. Apply remaining time before each write/read; a per-call socket timeout alone permits slow peers to retain version and cache locks indefinitely.
|
||||
|
||||
@@ -164,7 +164,7 @@ Good: complete relocatable interpreter plus stdlib, or Go tools plus every refer
|
||||
|
||||
`native_release` verifies deterministic archive/signature bytes, computed runtime identities, bad inputs and rollback. Explicit ecosystem acceptance must lock from a signed install, execute all three managed check/run/build paths, then run programs after removing source and machine stores inside a root with no system language installations. OS baseline libraries, test key, target and untested third-party registry behavior remain explicit.
|
||||
|
||||
Linux no-host acceptance must use a genuine mount-namespace root, shared by core, Worker and bootstrap fixtures. A plain `chroot` leaves the process root different from the namespace root; Linux then rejects the Worker's nested `CLONE_NEWUSER` with EPERM, even for root. Use the explicitly prepared bwrap/loader assets to bind only the owned OS root, private proc and synthetic devices. Do not remove the product's `--unshare-all` or add host library/tool mounts to make the fixture pass. A successful managed run does not replace source-free standalone execution under this boundary.
|
||||
Linux no-host acceptance uses an explicitly prepared, test-only namespace fixture (`DEVER_TEST_ISOLATOR`) to prove standalone packaging without changing the host. This test harness is not a product dependency or Worker boundary. Source-free deployment and non-root execution remain separate checks; a managed run alone proves neither.
|
||||
|
||||
Managed Go compilation removes the invocation-owned stage prefix from recorded source paths with `compile -trimpath`. This applies to the SDK, Adapter and dependency packages through their single `Builder::compile` path. Preserve logical filenames/line numbers; do not strip diagnostics to hide path drift. The real `external_workers` Go fixture must prepare twice in fresh stages and compare every emitted resource identity, then execute after source removal. The signed three-ecosystem run/build test must publish exactly three compiled entries. A single successful Go execution does not prove reusable cache identity.
|
||||
|
||||
@@ -178,19 +178,19 @@ Application-test execution reuses checked HIR, specialization, runtime feature t
|
||||
|
||||
Port native dispatch is a direct call for one implementation or a closed match for several. `adapter.<port identity>.use` in setting.json selects an immutable startup binding; unknown choices/fields, duplicate keys and missing required settings fail before user code. A selected Adapter's contextual `setting` is a private record decoded with shared SettingInput wire helpers into a OnceLock; only that implementation requires values. Secret errors never include input values. Port-only binaries enable the wire feature without API/database runtime; a no-setting single implementation may omit setting.json. Runtime configuration uses the bounded duplicate-aware wire parser before typed config parsing.
|
||||
|
||||
External exec and managed dispatch remain behind the same selected Port binding. All executions require a verified embedded launch manifest, selected Worker tree and signed sandbox assets; no adjacent-file or bare-process fallback exists. CLI preparation reads sandbox leaves from the authenticated installed release, including for exec. Runtime clears the environment, maps only verified launch arguments, and uses length-framed stdio. Native and LLVM dispatch reuse one invocation settings snapshot for Adapter selection, business settings and deployment grants. Component tasks reuse the structured application Scope and bounded queues. `hello/ready/health` must finish before admission; request ids are sequential and exact; timeout/cancellation sends `cancel` and drains one matching `result` or `error`; application cleanup sends `shutdown`, joins, then uses a bounded kill only when the child misses the deadline.
|
||||
External exec and managed dispatch remain behind the same selected Port binding. All executions require a verified embedded launch manifest, selected Worker tree and signed process runtime assets; no adjacent-file or bare-process fallback exists. CLI preparation reads process runtime leaves from the authenticated installed release, including for exec. Runtime clears the environment, maps only verified launch arguments, and uses length-framed stdio. Native and LLVM dispatch reuse one invocation settings snapshot for Adapter selection, business settings and command limits. Component tasks reuse the structured application Scope and bounded queues. `hello/ready/health` must finish before admission; request ids are sequential and exact; timeout/cancellation sends `cancel` and drains one matching `result` or `error`; application cleanup sends `shutdown`, joins, then uses a bounded kill only when the child misses the deadline.
|
||||
|
||||
Python SDK input uses an owned asyncio pipe transport and bounded message queue. Cancel and join its reader on normal shutdown and protocol failure, then close the transport. A daemon thread blocked on `sys.stdin.buffer` can abort CPython finalization even after a successful reply. The SDK regression must wait for successful process exit after the shutdown reply while the parent's stdin writer remains open; closing it before waiting masks this lifecycle defect. SDK source changes require re-embedding the source through the host compiler before deployed application verification.
|
||||
|
||||
`dever-sandbox` owns the Linux namespace/seccomp launch boundary for Workers and Go build tools. The signed release supplies bwrap, the safe Rust guard and a bounded, complete ELF library closure; the loader cannot select libraries through environment or cache, and a host preload configuration is explicitly unsupported. Mounts are opened with no-symlink `openat2` and passed as owned descriptors to bwrap bind-fd, preserving the authorized inode across rename/replacement. Never replace this with a path check followed by a path-based bind. Input trees are read-only; output grants cannot overlap executable inputs. The runtime additionally rejects writable grants overlapping `data/cache/lib`, including its ancestors. Sandbox descendants inherit the syscall filter; namespace termination stops even children that changed process groups.
|
||||
`dever-process` owns direct Linux child execution and process groups, not an OS sandbox. Signed `process-runtime/<target>/lib` contains loader/OS ABI libraries only; no bwrap, guard, seccomp filter, mount grants or host AppArmor policy is shipped. Preserve authenticated resources, explicit paths, protocol validation, bounded queues, deadlines and cancellation. Pin the group leader with pidfd; terminate its group before reaping the leader, including normal exit and failure. Detached sessions are outside this lifecycle guarantee; trusted Workers must not daemonize. Never weaken the existing container/node security policy to run a Worker.
|
||||
|
||||
The signed bwrap must be a bounded static target ELF without an interpreter or dynamic dependencies and is executed directly. On Linux with restricted unprivileged user namespaces, a non-root launch requires the matching digest-addressed `/opt/dever/sandbox/<sha256>/bwrap`, root ownership of every ancestor, no shared writes/symlinks/setuid/setgid/file capabilities, and both dedicated AppArmor profiles loaded in enforce mode. The helper bytes must match the embedded signed asset. The runtime and real-host bootstrap require `kernel.apparmor_restrict_unprivileged_unconfined=1` before using/publishing this policy: at zero, unconfined callers can enter a capability-granting profile through `aa_change_profile`, independently of its executable attachment. Missing or invalid strict-policy state fails closed. The product only reads these flags; administrator-approved installation/loading owns host changes. Root or unrestricted-host execution uses the same embedded static helper, never an unsandboxed fallback. Test normal and immediate profile changes, with and without no-new-privileges, using the same-process negative probe; wrapper failure or unavailable setup is not a successful isolation check.
|
||||
Managed children clear ambient application environment and inherit only the allocated GPU visibility/driver variables and `LD_LIBRARY_PATH`. No process discovers language runtimes or compiler tools through host PATH. `allow` remains static effect/handshake metadata, not a file/network/device enforcement mechanism. Only trusted Workers and dependency build hooks may execute; Pod UID, mounts, network and device policies own the real permission boundary.
|
||||
|
||||
With `process=true`, POSIX spawn may reset only the effective UID/GID to the guard's captured real identity: `setresuid(-1, uid, -1)` and `setresgid(-1, gid, -1)`. Compare all three arguments as 32-bit kernel IDs; changing real/saved IDs or choosing another effective ID remains denied. Without the process capability both calls remain denied. Keep clone3 returning ENOSYS so libc falls back to the flag-checked clone syscall. The actual C probe must cover both capability states, high-bit-equivalent IDs and rejected identity changes; an unrestricted setuid allowlist is not an acceptable build compatibility fix.
|
||||
|
||||
Deployment file grants live only at `adapter.<port>.files.<name> = {"path":"data/upload", "write":true}` in setting.json. Checked `allow file` is required, names are bounded lowercase identifiers, paths stay below the application, and write defaults to false. Workers see `/data/<name>`; grant configuration is not business `setting`. Network capability permits IP sockets and supplies only bounded resolver files; host Unix sockets and raw sockets stay denied. Process capability controls fork/process clones separately from threads. GPU and unsupported OS backends fail explicitly. Linux additionally requires host permission for user/mount/PID/network namespaces and openat2; root fixture success does not establish non-root deployment support under AppArmor.
|
||||
|
||||
Go tools consume a private read-only input tree, including parent-generated import/embed configuration, and a separate writable output tree. Parent reads of generated programs must use nofollow, regular-file and single-link checks with a byte limit; never follow a tool-created output symlink on the host. Runtime/SDK/build assets and source remain explicit, without host compiler discovery or implicit networking.
|
||||
Adapter `files` and `gpu` grant fields are removed and rejected, not silently ignored. Pass real model/input/output paths as business `setting`; GPU devices and drivers come from the parent container without nested mounts. Managed C ABI remains the separate exec Worker SDK in `sdk/c`, never arbitrary dlopen in the Dever process or Dever raw pointers. A successful CPU invocation is not GPU/model validation.
|
||||
|
||||
Go tools consume a private input tree, including parent-generated import/embed configuration, and a separate writable output tree. Parent reads of generated programs must use nofollow, regular-file and single-link checks with a byte limit; never follow a tool-created output symlink on the host. Runtime/SDK/build assets and source remain explicit, without host compiler discovery or implicit networking.
|
||||
|
||||
When a build carries compiler-owned external resources, the native entry may include a deterministic resource table. `dever-runtime::external` releases it only below the executable's private `data/cache/lib/<digest>` directory, rejects symlinks/path escape and digest mismatches, and publishes a complete staging directory atomically. Existing entries are reverified; a concurrent publisher is reusable only after the same manifest verifies. This extraction path never downloads, consults `PATH`, or accepts runtime paths from environment variables.
|
||||
|
||||
@@ -252,7 +252,7 @@ Create a response-body cancellation owner only after the pending-to-committed st
|
||||
- Project has a pinned version: `skill path <project>` resolves that exact installed version. Foreign users must read matching guidance without obtaining the administrator install lock.
|
||||
- Test filter/timeout is missing, repeated, non-UTF-8 or malformed: reject before execution. No selected case: fail before compilation. Assertion, process error or timeout: retain labelled captured output, continue later selected cases and return nonzero. Cleanup only the case's process group; never reap unrelated children or leave adopted descendants to PID 1.
|
||||
|
||||
Cross build: unknown/duplicate/missing target arguments fail before execution; absent target packs, wrong ELF/archive machine, unsigned target leaves, incompatible lock/runtime/build receipt and changed target inputs fail before cache reuse or linking. Sandbox loader mode must survive both release packing and embedding; a hash-valid non-executable loader is not a usable Worker resource.
|
||||
Cross build: unknown/duplicate/missing target arguments fail before execution; absent target packs, wrong ELF/archive machine, unsigned target leaves, incompatible lock/runtime/build receipt and changed target inputs fail before cache reuse or linking. Process runtime loader mode must survive both release packing and embedding; a hash-valid non-executable loader is not a usable Worker resource.
|
||||
|
||||
Native maker: wrong format/target/version, unknown or duplicate fields, missing profile, non-regular or escaping source, SHA mismatch, exposed/invalid signing key or payload alias to key → explicit failure, no output, owned staging removed. Existing or concurrently created destination → no overwrite. Correct bytes → complete signed v2 catalog with base and extension artifacts, without private key/configuration. Unsupported host publication → explicit failure, never a weaker rename fallback.
|
||||
|
||||
@@ -260,13 +260,13 @@ Managed input with duplicate fields/noncanonical encoding, unsafe paths or forge
|
||||
|
||||
External Lib declarations live only in the root `lib` array of `config/setting.json`; reject old `config/lib.json` rather than silently combining it. Canonical `dever-lock-v6` locks bind checked Port/Adapter/schema/operations/capabilities and independent Worker roots. Registry artifacts retain exact `{ecosystem,locator,filename}` sources. Build Receipt.inputs is the complete fixed LockFile: Python build requirements or npm's pre-pruning execution graph (no self-referential build receipt). Receipt.graph remains the post-build result. Doctor rejects stale bindings, missing/cyclic closure or within-environment version conflicts; independent Workers may select different versions. Embedding compares complete normalized provider metadata, including dependencies, runtime, schema and artifacts, not just an artifact subset.
|
||||
|
||||
`dever lib install <root>` requires a current canonical lock, rejects specs, and holds the existing project mutation lock. Restore exact Package ZIPs first, then check declaration/Worker closure and selected target before restoring Lib archives. It never invokes a version resolver or writes setting/lock. A true missing archive uses its recorded source with existing registry host/path/size rules and locked Go evidence. Missing build outputs replay fixed receipt inputs through the same bounded sandbox executor used by initial resolution; validate runtime/tools/frontend, result graph/outcomes, bytes and digest before publishing. A non-reproducible build is an explicit failure, not authority to rewrite the lock. Ordinary run/build and doctor/remove remain readonly; add/update/install and Package preparation ensure runtime resources before Worker binding, including no-Lib adapters. Build resources are ensured only when the source-build path actually requests them. Cached built outputs do not need build tools.
|
||||
`dever lib install <root>` requires a current canonical lock, rejects specs, and holds the existing project mutation lock. Restore exact Package ZIPs first, then check declaration/Worker closure and selected target before restoring Lib archives. It never invokes a version resolver or writes setting/lock. A true missing archive uses its recorded source with existing registry host/path/size rules and locked Go evidence. Missing build outputs replay fixed receipt inputs through the same bounded managed executor used by initial resolution; validate runtime/tools/frontend, result graph/outcomes, bytes and digest before publishing. A non-reproducible build is an explicit failure, not authority to rewrite the lock. Ordinary run/build and doctor/remove remain readonly; add/update/install and Package preparation ensure runtime resources before Worker binding, including no-Lib adapters. Build resources are ensured only when the source-build path actually requests them. Cached built outputs do not need build tools.
|
||||
|
||||
Production registry access derives from a verified installed version core, not host tools or development fixtures. Signed assets `runtime/<ecosystem>/<target>/manifest.json` (`dever-registry-runtime-v1`) bind a RuntimePack identity and the adjacent `runtime.pack`; Python metadata additionally supplies target markers/wheel tags. Resolver owners implement PyPI PEP 508/440, npm semver/SRI and Go proxy/MVS with fixed HTTPS origins, disabled proxies and bounded work. Redirects are rejected except for Go `/@v/*.zip`: manually follow at most two hops into the exact HTTPS `storage.googleapis.com/proxy-golang-org-prod/` bucket. Validate normalized host/port/path and reject userinfo, fragments, missing/duplicate Location and loops before issuing a subsequent request; all hops share one 30-second deadline and diagnostics must not echo signed URLs. Loopback mirrors are explicit test transport, never project configuration. Unsupported extras/sdist, install hooks and Go sumdb are not silently approximated. Only explicit Lib commands resolve/download; run/build consume locked cached bytes.
|
||||
|
||||
`workers::prepare(root, Program, resources)` expands verified runtime packs (`dever-worker-runtime-v1`: ecosystem, target, executable, arguments; Go additionally has `build`) and owned Adapter sources into a per-contract `workers/<digest>` tree. Python uses exactly `-I -S -B`, installs the packaged SDK path and imports that trusted SDK before exposing business source/dependency paths, and reserves stdout for protocol; JavaScript binds ESM/CommonJS exports. Both bind each checked Port operation by name using `sdk_manifest()`. Go consumes fixed-target compiler/linker/source selector and stdlib importcfg from the same exact locked pack, compiles the SDK and checked-operation runner alongside target-selected Adapter files and locked module ZIPs, then embeds only the executable and checked contract. Go source exports one exact `func(context.Context, any, any) (any, error)` per operation; `send` maps to `Send`, dotted segments to PascalCase. Cgo, assembly and unlisted imports fail explicitly. Build tools run by absolute path with an empty environment and bounded lifetime; product compilation does not invoke host `go build`, `PATH`, `GOOS`, `GOARCH`, `GOCACHE` or `GOPATH`. `<entry>.dever-worker.json` (`dever-worker-launch-v1`: ecosystem, entry, executable, arguments, working_directory) accepts only manifest-bound resource arguments in that same tree and bounded literal flags. Runtime revalidates the entire private bundle before every launch/restart and clears the inherited environment. Discard consumed raw runtime/archive resources after expansion unless config roots or exec Workers still need them. A private fixed-target pack fixture proves the local compile/link path; signed release packs, Go sumdb and cross-platform distribution remain separate acceptance work.
|
||||
|
||||
Packaged exec Workers resolve only against the entry's embedded manifest. Bundle identity includes path, byte digest, length and executable bit. Validate all path segments, Unix owner/private permissions and file bytes on both cache reuse and every Worker restart; do not scan other bundles or fall back to source workers. Publish owner-only files/directories atomically before exposure. These protocol and integrity checks do not establish OS capability isolation; verify the sandbox launch boundary above separately.
|
||||
Packaged exec Workers resolve only against the entry's embedded manifest. Bundle identity includes path, byte digest, length and executable bit. Validate all path segments, Unix owner/private permissions and file bytes on both cache reuse and every Worker restart; do not scan other bundles or fall back to source workers. Publish owner-only files/directories atomically before exposure. These protocol and integrity checks do not establish OS capability isolation; deployment container policies own that boundary.
|
||||
|
||||
`dever.crypto` uses the existing source Result choices and exhaustive Intrinsic bridge. Secret storage lives in the base runtime with zeroize; Argon2id, SHA/HMAC and OS randomness are in the optional crypto feature. Native runtime feature selection inspects already-pruned emitted crypto calls so direct system intrinsics, API roots and Test roots are covered. Password hashing uses RustCrypto Argon2 0.5.3, Argon2id v19 and default Params; PHC verification bounds algorithm/version/memory/time/parallelism/output before computing. Failure messages contain no input. Hash/verify use the existing blocking-worker effect boundary. `dever.process` exposes arguments only; application settings never come from environment variables.
|
||||
|
||||
@@ -360,7 +360,7 @@ Native release changes select `native_release` plus affected private pack/instal
|
||||
|
||||
Managed compilation selects `packages::managed_compile`, the installed Package execution fixture, affected `shared_toolchain` cache/install/IPC cases, and private `llvm_cli`. Cover secret-free snapshots, reserved/strict source checks, cross-UID reuse, changed sources, failure recovery, fake-only Test execution, signature reference omissions, cache-hit tampering, limits, real cancellation/disconnect/deadline reaping, and shared install lock lifetime. Tests use owned daemons and temporary signing keys. Check final `dever`/`deverd` DT_NEEDED independently: empty-environment success on an LLVM-equipped host cannot establish absence of a dependency.
|
||||
|
||||
Runtime ABI changes select `native_runtime_abi` and opt-in `native_runtime_link`: exact Decimal word round-trip, checked errors/IEEE floats, borrowed UTF-8/Bytes including null-empty and invalid input, real C-header/staticlib linkage, LLVM scalar/out-pointer calls and matching buffer release. Managed changes include canonical C typed lifecycle callbacks, nested rows, COW aliases, cursors/early release, duplicate Map keys, split descriptor identity and Bytes conversion/slice. Build the offline runtime-only archive explicitly in `target/native-runtime-abi`; do not recursively build it from tests or silently use an embedded-compiler archive. Opt-in `llvm_managed_native` and `llvm_result_native` reuse the shared runtime-linked runner to verify real checked-source execution and repeated allocation balance; this is not a general memory profiler or capacity test. Typed lowering/bridge tests and target-pack limitations are specified in `compiler-contracts.md`.
|
||||
Runtime ABI changes select `native_runtime_abi` and opt-in `native_runtime_link`: exact Decimal word round-trip, checked errors/IEEE floats, borrowed UTF-8/Bytes including null-empty and invalid input, real C-header/staticlib linkage, LLVM scalar/out-pointer calls and matching buffer release. Managed changes include canonical C typed lifecycle callbacks, nested rows, COW aliases, cursors/early release, duplicate Map keys, split descriptor identity and Bytes conversion/slice. Build the offline runtime-only archive explicitly, then set `DEVER_RUNTIME_ABI_ARCHIVE` to its absolute path for `native_runtime_link`; release checks must select the actual candidate archive rather than an unrelated fixed-path cache. Do not recursively build it from tests or silently use an embedded-compiler archive. Opt-in `llvm_managed_native` and `llvm_result_native` reuse the shared runtime-linked runner to verify real checked-source execution and repeated allocation balance; this is not a general memory profiler or capacity test. Typed lowering/bridge tests and target-pack limitations are specified in `compiler-contracts.md`.
|
||||
|
||||
File/Stream/coroutine ABI changes additionally select `llvm_resource_native`, `llvm_async_native` and `llvm_async_source`. Keep hand-written LLVM/C ABI smoke separate from checked-source lowering evidence; execute both against the actual archive. Assert all initial/pending/final destroy paths, actual Pending/backpressure, typed error transfer, sync input borrowing, non-null creation consumption, nullable row/End, Scope drain and repeat allocation balance. Use an atomic allocation ledger for concurrent workers. Do not run broad database/service/performance checks or claim complete application/target-pack acceptance from these tests.
|
||||
|
||||
@@ -402,11 +402,11 @@ LLVM external validation selects `llvm_external_source`, the original `external_
|
||||
|
||||
Runtime config owns the shared Job bounds: timeout <= 3,600,000 ms and lease in 100..7,200,000 ms. Tests use the maximum lease so every legal timeout fits SQLite without bypassing production policy validation.
|
||||
|
||||
Ordinary Linux commands use `workers/command.rs` with the shared `workers/elf.rs` dependency-closure checker. Prepare the domain-owned ELF and flat adjacent `lib/` from either the project or locked Package; never scan host libraries or require a Lib lock for command-only projects. Validate target, GNU interpreter, recursive DT_NEEDED, SONAME, symbol versions and loader hooks. All command RPATH/RUNPATH expansions must remain inside the logical `program` tree, which maps to `/worker`; accepting an escape such as `$ORIGIN/../data/grant` would allow a file grant to replace a validated dependency at execution time. Private libraries cannot shadow signed sandbox libraries. Dynamic launch uses the signed target loader with `--inhibit-cache --library-path lib:/lib`; user argv is appended literally after mapping only compiler-owned resource arguments. Commands use a distinct `dever-command-launch-v1` manifest and `commands/` tree under the existing verified bundle/extraction owner.
|
||||
Ordinary Linux commands use `workers/command.rs` with the shared `workers/elf.rs` dependency-closure checker. Prepare the domain-owned ELF and flat adjacent `lib/` from the project or locked Package; never scan host libraries or require a Lib lock for command-only projects. Validate target, GNU interpreter, recursive DT_NEEDED, SONAME, symbol versions and loader hooks. RPATH/RUNPATH must remain in the packaged program tree and private libraries cannot shadow signed process runtime libraries. The launch manifest names the actual executable; `dever-process::Launch` executes it directly through the kernel, with private Worker/runtime library paths plus inherited driver library paths. The deployment container supplies the matching Linux/glibc OS ABI. Do not launch language executables through the GNU loader: Node/Rust self-executable discovery and subprocess spawning would resolve the loader instead. Signed process-runtime libraries remain build-time target dependency inputs; do not inject their old libc under a different container loader. User argv is appended literally, not interpreted as shell syntax. Commands use `dever-command-launch-v1` and the `commands/` tree under the existing verified bundle owner.
|
||||
|
||||
Command limits are `adapter.<port>.command.timeout_ms` (default 30000, 1..3600000) and `output_limit` (default 1048576, 1..8388608 combined stdout/stderr); stdin is bounded at 8388608 bytes and argv at 65536 including NUL terminators, with embedded NUL rejected. A call owns one process, closes absent stdin, drains both output pipes concurrently and never replays. Nonzero program exit is output; failure before target exec must remain a runtime fault and cannot be inferred from stderr or a reserved exit code. Invocation shutdown and caller cancellation must wait for actual sandbox completion. Queued cancellation has no child to drain and must not wait behind an unrelated active call. Ordinary application tests remain fake-only. Focused evidence belongs to `external_command`, command cases in `external_workers`/`llvm_external_source`, and the opt-in signed native-release command acceptance; physical ARM execution must not be inferred from cross-target metadata tests.
|
||||
Command limits are `adapter.<port>.command.timeout_ms` (default 30000, 1..3600000) and `output_limit` (default 1048576, 1..8388608 combined stdout/stderr); stdin is bounded at 8388608 bytes and argv at 65536 including NUL terminators, with embedded NUL rejected. A call owns one process, closes absent stdin, drains both output pipes concurrently and never replays. Invalid resources and OS spawn errors are runtime faults. A successfully started executable or loader returns its actual exit status; do not infer a special failure from stderr or a reserved exit code. Invocation shutdown and caller cancellation terminate the owned process group and await the leader; this is not containment for detached sessions. Queued cancellation has no child to drain and must not wait behind an unrelated active call. Ordinary application tests remain fake-only. Focused evidence belongs to `external_command`, command cases in `external_workers`/`llvm_external_source`, and the opt-in signed native-release command acceptance; physical ARM execution must not be inferred from cross-target metadata tests.
|
||||
|
||||
Command execution observes guard exec through a private CLOEXEC pipe. Bind the PID namespace init through bwrap `--info-fd`, keep it blocked with `--block-fd` until pidfd acquisition and PPid verification, and wait for that pinned pidfd before releasing caller-scope completion. Pipe EOF (`--sync-fd`) is not sufficient: Linux closes init descriptors before tearing down descendants. Cleanup always reaps the owned launcher even if namespace observation fails, retaining both errors. Old Worker launches keep their existing handshake path.
|
||||
Command and Worker cleanup use the same pidfd-pinned process-group owner. Kill the group before reaping its leader so a reused PID cannot target unrelated processes. Test timeout, cancellation, explicit shutdown, root-scope close and successful leader exit with a descendant still holding output pipes. Invalid executable input must fail before any invocation; static checks and ignored fixtures are not execution evidence.
|
||||
|
||||
## 7. Wrong vs Correct
|
||||
|
||||
|
||||
Reference in new issue
Block a user