feat: publish Dever core with project scaffolding and versioned AI skills
This commit is contained in:
1 parent
70fa73cdc2
commit
ac458ffa7f
960 files changed
+221319
-1163
No files matched your search
@@ -0,0 +1,194 @@
|
||||
# Compiler Contracts
|
||||
|
||||
## 1. Scope / Trigger
|
||||
|
||||
Applies to AST/HIR contract metadata, typed LLVM/resource/coroutine lowering, private fields, range/failure/effect analysis, structural warnings and public API baseline enforcement. Keep clause-only control flow and ordinary value semantics. Official sources receive the same hard checks; no blanket recovery annotation or bundled exception.
|
||||
|
||||
## 2. Signatures
|
||||
|
||||
- `llvm::emit_executable[_with_resources]` and `emit_test_executable` wrap the existing callable roots in C `main`. Process startup owns SIGPIPE initialization; callable roots preserve host policy. Success/failure paths release outputs, typed faults, rendered Text and diagnostic Buffer, then flush the existing logger. Test main validates one decimal index and uses the same index for execution and all fault/output owners. Private CLI defaults to these entries and never invokes a host compiler or Rust fallback.
|
||||
- `llvm::emit_kernel(&Program, &SourceMap, entry) -> Result<String, String>` lowers typed ordinary function kernels and inferred-suspending coroutines, not the default application backend. It consumes only checked HIR and existing specialization/Domain metadata; no Rust source, generic Value or interpreter dispatch is emitted. Keep direct typed arguments, explicit output/fault pointers, clause projections, value-copy mutation, static handlers, source evaluation order, short-circuiting and checked numeric semantics. Fault origin and each caller reference the emitted source-span table; frame capacity is reachable specialization count plus one, so acyclic calls never truncate a trace. Collect nominal types from nested reachable expressions as well as locals/signatures. Validate and collect every reachable `specialize::failures` type and its recursive owned fields, including Port-declared errors unused in implementation bodies: fault layout/release still references them. Reject unsupported reachable types/effects/application roots at their source location before returning IR; never silently substitute a value or fallback backend.
|
||||
- `llvm::emit_application(&Program, &SourceMap) -> Result<String, String>` emits callable compiler-owned CMD, HTTP/REST and Job roots. `dever_application_entry(Unit*, fault*) -> i32` uses existing output/fault release exports. CMD reads a qualified name and one JSON object, decodes checked inputs, invokes App/selected Dever or external Adapter and prints the existing success envelope; a mixed HTTP/CMD application must not bind HTTP when executing CMD. HTTP/REST uses static route/auth/permission descriptors and concrete callbacks, reusing the existing API, JWT/business verify, site, tenant, RBAC, Upload and log owners. No dynamic Value enters business code. Model/SQL/Related/migration/Seed/DbError, Job resources and hidden transactions share one invocation-owned application Session. `emit_application_with_resources` additionally consumes the packager's verified `EmbeddedResource` table; private CLI adds an executable process entry; managed Linux compilation rechecks portable sources in a trusted daemon worker. Formal target packs remain unfinished.
|
||||
- LLVM external lowering reuses checked Port names, declared failures and wire field codecs. Decode selected Setting before starting a Worker. Named result/error objects require every top-level field, including nullable fields; unknown/missing fields, invalid types and undeclared error identities remain source-located runtime faults. Move fully decoded outputs or exact nominal Choice payloads into their owner once. JSON exists only at the process boundary. Embed immutable bytes and metadata with the native backend's shared digest/dedupe helpers. Production resources/startup never enter a fake-only Test graph.
|
||||
- `llvm::emit_test_suite(&Program, &SourceMap) -> Result<String, String>` emits one module/executable with stable case indices from case-local checked `test_program` metadata. It exports `dever_test_count`, `dever_test_entry`, `dever_test_fault_size`, `dever_test_fault_release`, `dever_test_fault_render` and `dever_test_text_release`. Allocate the selected fault size with eight-byte alignment; render/release with the same index. Cases retain their own effects, fakes, faults, private symbols, database and virtual clock. Only declared internal LLVM symbol/type tokens are qualified; never rewrite strings, comments or shared ABI declarations. Production routes/auth are removed and unreachable Jobs pruned before lowering; reachable Job payloads still use the shared typed wire contract. No deployment config, real Lib or service startup is allowed in ordinary tests.
|
||||
- LLVM application Dever Port dispatch reuses checked production Adapter candidates and `Settings` selection/validation. Decode only the selected typed Setting; a single unconfigured Adapter without Model needs no settings file. Concrete selection/Setting/database binding slots are module-private. A nonblocking atomic entry gate rejects overlapping/reentrant roots that own Port, database, application or clock slots. The root Scope drains before same-runtime Session close, selected Setting drop, slot reset and gate release, including partial initialization and faults. Child Tasks read immutable selected slots but detach transaction context; they neither reload config nor own independent application roots. Writing CMD begins after input decode and commits only after output encoding, before stdout. Nested/sequential calls share one transaction without savepoints; cancelled unconfirmed operations discard the connection. Database ABI/storage/error contracts and live acceptance are owned by `database-guidelines.md` §9, not a second ORM implementation.
|
||||
- LLVM inferred-suspending functions are `ptr @f(...typed inputs, ptr out, ptr fault, ptr state) presplitcoroutine`; synchronous functions remain `i32 @f(...typed inputs, ptr out, ptr fault)`. Both share clause/binding/statement emission, TypeOwner guards and specialized typed invocation. Coroutine/runtime descriptors and fixed ABI signatures live in `toolchain-and-library.md` and the canonical C header.
|
||||
- Application source identity is `module/<component>/<domain>/<role>.dever[.md]` or a cohesive role topic file; role is app/domain/model/port/adapter/api/job. Generated API/CMD/Job roots replace application `main.dever`. Model/API root files may coexist with topic directories; other roles keep root/directory exclusivity and non-API topics are flat. Reject generic common/shared/utils/helper/base/service buckets. Application source has no package/exposes/import declarations; `public` is limited to an individual HTTP declaration.
|
||||
- App functions use logical `<component>.<domain>.<function>` names; role/topic never enter the call. Main calls App; API calls its own App; App calls App or same-domain Domain/Port; Domain calls same-domain Domain; Adapter calls only same-file helpers and standard capabilities. Model operations remain owned by their domain App. API handlers and qualified Adapter/fake implementations are not source-callable.
|
||||
- Port operations are one bodyless ordinary signature with typed value inputs, unbounded outputs and required `fails app.ErrorChoice` (or public standard error choice). The choice contains an explicit error variant and belongs to the same-domain public App contract. Port/Adapter may reference the App types used by this contract, never call App. Each production Adapter completely implements one Port identity; a test file can fake multiple Ports. Input names/types and ordered output names/types agree exactly. Every implementation failure set is a subset of the declaration.
|
||||
- An external Adapter is `external <exec|pip|npm|go> "relative/entry" { ... }` in an Adapter-role file. Body clauses are newline-separated `lib "name@version"` and `allow <capability>`; `lib` derives its ecosystem from the declaration, while exec uses explicit ecosystem-prefixed Lib specs. It may coexist with one `setting` record, but not with Dever functions. The compiler synthesizes every operation of the domain's single Port identity; App code still calls the Port and never receives Worker, process or JSON values. `ExternalWorkerContract::sdk_manifest()` exposes the same checked setting/input/output/error wire DAG and fixed identity for SDK generation, never a separately maintained type registry.
|
||||
- App parameter, output and inferred failure types are public contract. Reject an App capability that leaks a private Domain/Adapter type; define externally visible business error choices in App and let same-domain private roles refer to them through `app.<Type>`.
|
||||
- Fields: `private name: Type` for records and Models; ordinary record privacy is package-owned, Model privacy belongs only to the owning domain App. `name: Int >= low and <= high` (also Decimal) applies to record/payload/output/handler fields.
|
||||
- Choice: `error Failed(message: Text)`; markers are explicit, never inferred from variant names.
|
||||
- `fail(Error.Variant(...))` terminates a clause; ordinary calls propagate inferred errors. `result(named_call(...))` requires a choice with one success variant matching the call outputs and an exact recoverable error set.
|
||||
- Functions: `name(inputs) (outputs) pure recover("reason")`; modifiers optional, identical across all clauses of one signature. Source has no `async` modifier or `await(...)` form.
|
||||
- Transaction functions use `transaction name(inputs) (outputs)` and infer a single physical database effect through their complete call graph; source never receives a transaction handle. Sequential `blocking` is allowed only through its existing synchronous/no-database/no-concurrency contract; detached, parallel, stream and channel work remain forbidden.
|
||||
- Model files synthesize opaque `model.id`, generated `id/created_at`, a record Page type and static short CRUD symbols. Anonymous Model blocks are contextual syntax, not general Map values.
|
||||
- `Program::api_snapshot() -> String`, `check_api(&str) -> Result<(), String>`, `application_warnings(entry) -> Result<Vec<Diagnostic>, String>`.
|
||||
- Application tests use `test/<component>/<domain>/<topic>.dever[.md]`; the filename selects the one ordinary zero-input, zero-output `<topic>() ()` entry. Test helpers/types remain private to that file. Tests call same-domain App/Domain and cross-domain App only; they cannot call Model operations, Port, Adapter, API or another test, and production cannot call tests.
|
||||
- Test assertions are compiler-owned `assert(Bool)` and `assert_eq(actual, expected)`. Equality requires the same comparable type; only `null`/empty expressions that already require context receive the left type, so numeric promotion does not make unequal source types valid. Business failures continue through `result(call)` and explicit recovery.
|
||||
- CLI: `check <root>`, `test <root>`, `run <root> [--tenant <id>] [-- <cmd> <json>]`, `build <root> --output <new-file>`, `tenant migrate <root> <id>`, `tenant owner <root> <id> <site> <user-id>` and `tenant component enable|disable <root> <id> <component>`. Run/build collect checked API/CMD/Job declarations into one generated entry and reject application `main.dever`. Test checks `module/ + test/` before running stable-sorted cases; check/run/build load only module/. Deployment and database settings come only from `config/setting.json`.
|
||||
- Module API files contain declarations such as `get profile = app.profile`, `post publish = app.publish`, `cmd bootstrap = app.bootstrap` and `rest model`; they do not contain handler function bodies. HTTP routes inherit the App signature and return the fixed envelope. `public` is valid only on one HTTP declaration. Recursive `api/<site-path>/<topic>.dever` supplies configured site ownership without source `site` statements.
|
||||
- Authentication providers, sites, external HTTP origins, Cookie names/lifetimes and tenant routing come only from `config/setting.json`. Verify is one read-only App function from `Claims` to an App-owned record with exactly `id: Text`, `user_id: ModelId?`, and `tenant_id: ModelId?`; it cannot return permissions. A protected site's generated entry verifies its configured provider, enters trusted tenant scope, checks every reachable tenant component and authorizes the exact permission before reading business input or dispatching App. A `public` declaration ignores an ambient browser Cookie, but an explicitly supplied invalid `Authorization` credential remains an authentication error.
|
||||
|
||||
- Trusted `auth.user_id()` / `tenant_id()` keep their default `Int?` result. An explicit nullable ModelId context selects the same identity intrinsic with that nominal type; absence remains nullable and arbitrary Int casts remain forbidden. Configured validation walks every protected API's reachable functions and each generated REST field binding, matching the selected ID against its actual verify identity field; `owns_user(ModelId)` follows the same user-ID rule. Source-only checking defers provider matching, never configured builds. Native wraps the existing nullable i64; LLVM retains its nullable ModelId ABI. Public/CMD/Job/Test capability rules still apply. Ordinary tests must reject reachable identity effects instead of synthesizing a trusted user.
|
||||
|
||||
## 3. Contracts
|
||||
|
||||
The current LLVM kernel accepts ordinary checked functions with supported value types, including Decimal, Text/Id, Bytes standard operations, List/Map and nested Record/Choice/Nullable, File/Stream, Task/Group/Channel, AsyncStream, TCP Socket/Listener and HTTP/TLS/WebSocket resources. Time/calendar, process arguments, stdout/logging, Uuid and Secret/crypto use the same typed runtime bridge. List/Bytes each/filter/find/reduce/reduce_until/sum and Stream/AsyncStream traversal use static typed handlers, including the existing checked suspending-handler contracts, never a generic Value interpreter. `dever_entry(ptr out, ptr fault) -> i32` initializes both slots, returns 0 on success and 1 on checked runtime or business failure; outputs are read only after success. Fault is `{i32 code, i32 origin, i32 depth, [capacity x i32] caller_locations, {ptr, i64} message, i32 business_type, i32 business_variant, [words x i64] business_payload}` with separate source-span metadata. Keep fields 0..4 stable; the aligned payload holds the complete concrete Choice and is sized for the largest reachable error type. Named fault codes (overflow=1, division-by-zero=2, RuntimeAbi=3, Business=4) are not the runtime C ABI status. RuntimeAbi retains original owned diagnostic bytes; Business retains nominal identity (`type_id+1`, 0 means absent), variant and typed payload. Callers release root outputs and both fault payloads through `dever_outputs_release` and `dever_fault_release`, including zero-initialized outputs after failure. Application entries and unsupported application intrinsics remain source-located rejections.
|
||||
|
||||
File create uses create_new and never truncates an existing path; clone/close preserve shared owner semantics. File chunks yield a shared synchronous Stream cursor, with one terminal Failed before End. Source traversal uses only its existing each/reduce/reduce_until contracts; stream_pull is internal ABI, not a new source pull/first(Stream). Preserve aliases and unvisited rows on early stop, and release resources on both typed business and runtime faults.
|
||||
|
||||
AsyncStream constructors use List/Bytes/Channel/ticks or the existing TCP producer; aliases share progress and closure. Register its consumption-close guard only after source, initial state and context expressions finish evaluating. Early stop, normal return, fault and cancellation close the shared producer; unlike synchronous Stream, another alias cannot continue. Release pending operations before closing the producer. Terminal ReadEvent/ConnectResult Failed remains one actual typed row before End. TCP operations reuse runtime net deadlines, direction locks and partial-write cancellation rather than blocking wrappers. Map async statuses explicitly before decoding standard resource choices.
|
||||
|
||||
HTTP values cross the ABI as fixed named fields plus opaque resource handles; clone/register every owned field before fallible header conversion. Static sync/suspending handlers and optional context share the existing coroutine descriptors; borrowed request/context inputs become owned before initial suspend. Handler faults retain typed ownership until terminal request-boundary rendering, log and produce 500/abort as the existing runtime requires, then permit later requests. Do not turn a handler fault into a server-wide fault or serialize intermediate values through JSON. Uuid is an actual UUID owner, not Text; Secret stays non-observable. Password operations preserve checked blocking boundaries. Reachable Log/HTTP-server roots flush after scope drain on success and failure; pure kernels do not initialize a logger.
|
||||
|
||||
Timeout consumes a Task and uses an independent presence flag with the full output row: successful null and zero-output success are not timeouts. Only an actual timeout invokes its statically specialized fallback after runtime cleanup; numeric, business and RuntimeAbi faults propagate with their original kind and spans. Race uses canonical output descriptors shared by same checked signatures, consumes tasks only after non-null operation creation and drains all losers on every completion path. ParallelEach uses the existing scoped concurrent, bounded blocking and async stream owners for each checked sequence/handler combination; a plain Group loop is not equivalent to capacity-before-pull and failure observation while a producer is idle.
|
||||
|
||||
Coroutines clone borrowed inputs before initial suspend; all live guards, operation owners, values, output slots and typed faults survive real Pending. Initial/pending destroy runs reverse cleanup once; normal completion runs cleanup then complete/final suspend, and final destroy frees only the frame. Generated wrappers reuse the existing single task Scope/supervisor and bounded blocking pool, never per-function block_on. Validate cross-thread Send before returning IR against checked inputs/outputs, inferred error-variant fields and all reachable suspending frames including expression temporaries. Task/Group affine locals are frame-safe, not source-transferable. Non-suspending blocking functions may create/drop a Stream locally without transferring it. Internal Unit/Outputs ABI rows derive transferability field-wise (Unit=true); never copy their source `Type::transferable=false` property or change the language property to fix ABI metadata.
|
||||
|
||||
LLVM `fail` transfers the concrete Choice into the business tail and disarms its temporary owner; normal propagation appends the call's source frame. `result(named_call)` shares ordinary invocation and uses only `specialize::failures` plus `capture::targets`, including handler specialization. Success packs zero/one/multiple ordered outputs into the checked success variant. A matching business failure transfers directly or wraps one error-only Choice according to that mapping, then clears the entire fault; explicitly raised outer variants must not be unwrapped. Numeric and RuntimeAbi faults remain uncaptured and propagate with their original origin and caller frames. Root fault release statically dispatches nominal error types through the existing typed drop owner, including nonzero variants. Do not add runtime generic Value, parser-based error matching or a parallel failure registry.
|
||||
|
||||
BytesToText/BytesFromInts/BytesSlice use the existing runtime handle ABI: status 0 transfers a success value; only status 1 creates the standard Failed(Text) from the original diagnostic Buffer, freeing that Buffer after Text construction; status 2 stays RuntimeAbi. Standard Bytes wrappers then use the same `fail`/capture channel, not a separate error-return shortcut. Never inspect a success output on a failing status.
|
||||
|
||||
LLVM ordinary function inputs are borrowed; clause bindings, locals and temporaries own typed copies. Track ownership with entry-initialized guards, recursively retain/drop only present Nullable payloads and active Choice variants, and release on normal and fault exits. Success transfers outputs and disarms the corresponding guards. Reuse native last-use/field-update analyses rather than duplicating them; a final field move clears that field while preserving its containing record's owner. An ABI `_take` consumes its receiver even on failure, so disarm before calling it. First/Get allocate output storage for the actual List element or Map value type, not just the inner type of the nullable result. The ABI's outer presence flag only determines whether this row may be read: an already-nullable row is transferred intact, not wrapped again or overwritten with Some.
|
||||
|
||||
Reuse clause interval normalization and facts/domain containment at every constrained construction, write, output and handler boundary. Concrete handlers may establish outputs by inference; forwarded signatures require input contravariance and output covariance. Unknown operations establish no new bounds.
|
||||
|
||||
Facts preserve value snapshots and invalidate the modified path. Use shared immutable field graphs and bounded proof work, not recursive copying of a type/value DAG. `contracts::MAX_PROOF_WORK` is 16,384; exceeding proof work reports C015 and never grants a guarantee. Before analysis, nominal type depths are computed iteratively and must not exceed the shared 128-level syntax depth limit. Failure-free nominal type graphs are memoized and skipped by failure expansion.
|
||||
|
||||
Failure origins survive aliases, aggregates and projections. An unknown List may be empty and Nullable may be null, so possible error payloads do not establish guaranteed failure. A short-circuited handler or empty traversal cannot consume a pre-existing obligation. Empty reduce returns the original state's obligation unchanged. Replacing a Map entry cannot retain the replaced failure's identity in the new Map. Partial selection is conservative about the unselected remainder.
|
||||
|
||||
A capture error variant may contain one error-only choice payload, e.g. `error Database(error: dever.database.Error)`. It wraps that choice's inferred failures without renaming them or parsing messages. Explicitly raised outer variants retain their nominal identity. Success shapes, complete error sets and non-overlapping wrappers are checked before emission; captured failures still carry the ordinary handling obligation. Native, LLVM and reference execution share the same capture mapping, including specialized handler error sets.
|
||||
|
||||
Effects and effective recovery reasons propagate through the existing call/handler substitution flow. An unused handler argument adds no effective recovery. Recovery is not an external effect and is allowed in pure functions, but its reason belongs in reachable public API summaries even when declared in a private helper. Keep private helper names and private record representation out of snapshots.
|
||||
|
||||
Port remains a Function ID with compiler-owned successors. Shared dependencies, effect worklists and specialization include every production implementation; failure inference seeds the declared error set, independent of selection. Resolve the complete signature type graph before type-shape access checks, including App DTOs nested in Port-local DTOs; unrelated App types remain inaccessible to Port/Adapter. Unmediated file/network I/O propagates through functions and static handlers and is cleared only at the Port boundary. App/Domain reject it; only the compiler-generated application root may start API/Job service loops. Other effects retain existing policy. Production summaries exclude Test bodies. Checking and suite emission reuse one case-specific closed graph with that case's fakes, including failure-subset checks for unused fake operations and declaration checks for uninstantiated handler helpers. Each case gets an independent native module and sync/async entry; one rustc invocation still produces one suite binary. Never union different cases' fake effects or run a synchronous case under another case's async runtime. Production settings and fallback implementations are absent from the reachable suite.
|
||||
|
||||
An Adapter-role external declaration synthesizes exactly one complete implementation of the domain's single Port identity. Its schema hash is derived deterministically from the optional setting wire schema and every operation's ordered input/output wire shape, including top-level field names and declared business error payload fields. The synthetic functions enter the existing dependency, effect, failure and specialization graphs; do not add a parallel external-call registry. Only fixed `network`, `file`, `process` and `gpu` capabilities are accepted, and they seed conservative effects. An external declaration may coexist only with one setting record and never with Dever implementation/helper functions. Test graphs replace production implementations with case-local fakes and therefore never start an external Worker. Ecological declarations do not weaken these checks or imply their target packager is implemented.
|
||||
|
||||
Database effects carry a typed `DatabaseOwner::Model` or `DatabaseOwner::Job`, resolved centrally by Program to connection plus global/tenant scope. A transaction must reach at least one database operation (including enqueue), may resolve to only one physical database and cannot mix global/tenant scopes. It cannot start detached, parallel, streaming or channel work; a validated sequential `BlockingCall` does not escape the transaction. Ordinary database helpers accept a generated hidden transaction context so nested transaction calls and enqueue reuse the outer physical transaction. Never synthesize a Model to represent a queue.
|
||||
|
||||
Job declares an explicit database and a private `job name(payload?) () retry(n) timeout(ms)` entry. One input must be a same-domain App record; no outputs, overloading, multiple clauses, pure/recover/fails or direct non-App calls. App enqueue uses a checked static same-domain target and lowers to a dedicated HIR node. Enqueue propagates only its queue connection/failures, not handler effects, failure or synchronous recursion. Native execution reachability adds handlers as independent roots; application test closures apply the case's Port fakes to those roots too. UTC cron checking and runtime scheduling share one bounded parser.
|
||||
|
||||
Every Job enqueue requires a compiler/runtime-established execution identity. Generated CMD, cron and Test roots establish `System`; a protected API establishes `User` only after provider verification, component checks and automatic authorization. Missing identity never falls back to `System`, and a `public` API call graph that can enqueue a Job is rejected. User enqueue persists a re-verifiable provider/site/subject/session/tenant reference plus the one exact permission key of its HTTP entry; it never persists credentials, roles or a permission collection. Every attempt invokes verify again, validates the restored tenant, confirms the key still belongs to the current compiled permission catalog, checks reachable component state and performs the current role query in that order before opening the business transaction. Chained enqueue inherits the restored identity and permission key.
|
||||
|
||||
Protected custom API permissions are `component.domain.site.action`; REST reserves `read/create/replace/delete`. The HTTP method is catalog metadata, not part of the key. Two declarations cannot own the same key even if their method or URL differs. `public` is the only anonymous form and emits no permission; its complete reachable graph cannot touch a tenant component. The service atomically synchronizes compiled permissions into the global control catalog and marks removed rows inactive. Site and tenant are hidden context for role management; source never writes `auth.require`, an `access` clause or a permission list in Identity.
|
||||
|
||||
Suspension is an internal monotone effect inferred from async runtime intrinsics, ordinary calls, bound static handlers and collection handlers. HIR records `suspends`; API snapshots do not expose it. Direct source calls preserve sequential completion, while `run` is the explicit concurrency boundary and `wait` consumes Task/Group values. A fully non-suspending entry emits no Tokio runtime.
|
||||
|
||||
Facts, effects and failures share `contracts/dependencies.rs` for function/static-handler dependencies and initial order. Effects/failures enqueue reverse callers only when a summary changes; conservative dependency cycles still converge without discarding obligations. Type comparison/move/transfer properties are computed once per nominal definition after type-cycle validation, avoiding repeated expansion of shared type graphs.
|
||||
|
||||
Domain dependency edges come only from resolved type, function and handler references. Physical role files in one domain share one dependency boundary; cross-domain cycles remain invalid and function recursion remains independently invalid. Source code does not duplicate edges in a header, and dependency inference does not install or download packages.
|
||||
|
||||
API snapshots sort semantic names and contracts, not source paths/IDs. A baseline is optional, explicit and create-new; checks never update it. Protect its edits outside the language. CLI suppresses bundled-source suggestions only; Program retains diagnostics and bundled hard errors remain visible. Entry-specific export advice must not infer unknown library consumers.
|
||||
|
||||
Test authority comes from compiler-assigned source origin, never a user-controlled path string. A checked `TestCase` holds the private function ID and database-effect metadata; native compilation accepts only that checked identity and does not relax normal entry visibility. Each case runs serially in its own native process and temporary project. Successful child output stays captured; failed stdout/stderr is reported in labelled sections, later tests continue, and any failed case makes the command fail. Tests cannot reach `dever.api.serve()`.
|
||||
|
||||
For `module/<component>/<domain>/api.dever`, the route is `/<component>/<domain>/<action>`; recursive `api/<scopes>/<file>.dever` inserts scopes and file before action. `get/post/put/delete` bind same-domain App functions and inherit their wire-safe input/output shapes; GET/DELETE read query, POST/PUT read a unique-key JSON object. `rest` generates Model CRUD and can use checked owner/create/replace/search field bindings. Each successful request receives JSON `{code:0,message:"ok",data:<value>}` with HTTP 200; standard API errors map to fixed 400/401/403/404/409/429 envelopes and unknown failures become a redacted 500. A top-level POST `Upload` switches the statically known route to bounded multipart parsing and affine ownership. No environment-variable config or implicit database fallback is introduced.
|
||||
|
||||
Response Header/Cookie mutations are staged in request-local pending metadata. Generated dispatch serializes the response value, commits the database transaction, and only then promotes pending metadata for the server response; serialization, business, commit or rollback failure discards it. Cookie issue/clear uses the configured name, lifetime and attributes. Credential-changing requests require Origin and Host to exactly match the site's configured external HTTP/HTTPS scheme, host and effective port; a missing or cross-origin value is rejected without trusting forwarding headers. A write API call graph containing password verification or hashing does not receive an entry-wide transaction. Every reachable write must instead cross an explicit short `transaction` function, and password crypto is forbidden inside that transaction. Other write entries retain their generated transaction.
|
||||
|
||||
## 4. Validation & Error Matrix
|
||||
|
||||
`wire.rs` is the sole Type-to-wire eligibility owner; API checking and `native/wire.rs` consume its interned DAG. Policy (Output, ApiInput, SettingInput, Job) participates in the versioned fingerprint and specialization cache. Records reject private or bounded fields until concrete bound validation exists; Choice, Map, Related and resources are rejected. ModelId encodes as i64 while retaining nominal Model name in the fingerprint, never unstable compiler IDs. Secret is decode-only for ApiInput/SettingInput. `Program::wire_schema` exposes checked nominal schemas; no consumer may add a parallel Type allow-list. Concrete helpers retain field order and map missing nullable fields to None, while unknown/missing required fields fail. Persisted Model fingerprints never include wire schemas.
|
||||
|
||||
Sensitivity belongs to the cached type graph in `types.rs`: Secret and private Model data propagate through nominal records/choices and every container, including Related association cycles via a monotone reverse-edge worklist. Ordinary record private fields retain their existing package access contract. App inputs reject private Models; App/entry outputs and error payloads reject both sensitive categories. API JSON and assertion rendering reuse the same observable property. Generate Render only for observable nominal types; Debug for runtime Secret is always redacted. Test-only `secret(Text)` is resolved from compiler-assigned source origin and reserved against user declarations.
|
||||
|
||||
| Condition | Result |
|
||||
| --- | --- |
|
||||
| Reachable LLVM kernel type/effect/application root is not implemented | Source-located Err before IR is returned; no Rust fallback or successful stub |
|
||||
| Dynamic LLVM Int add/sub/mul/neg overflow or div/rem invalid operation | Entry status 1 with original source-span index and every caller frame; never unchecked poison/trap arithmetic |
|
||||
| Runtime value error during a managed LLVM call other than a recoverable Bytes operation | RuntimeAbi fault with the original diagnostic Buffer and source/caller spans; clean all guards and release message through the caller boundary |
|
||||
| Explicit business fail or its ordinary-call propagation | Business fault with concrete nominal type/variant/payload and original source/caller spans; release through the caller boundary unless captured |
|
||||
| Result matches a declared business failure | Transfer direct/wrapped typed payload and clear fault before any following call; do not swallow numeric/RuntimeAbi faults |
|
||||
| Bytes conversion/range value error; invalid ABI | Value error becomes the standard Failed(Text); invalid ABI remains an uncaptured RuntimeAbi fault |
|
||||
| File I/O value error; negative time/task sleep | Standard original Failed(Text); only the operation's recoverable status is captured |
|
||||
| Pending, optional receive End, or Channel element null | Pending initializes no row/fault; receive reads a row only when present; preserve the element's full Nullable state |
|
||||
| Async initial/pending/final destroy | Drop live guards/operation owners once at initial/pending; final destroy frees frame only after normal cleanup |
|
||||
| Non-Send task/blocking frame or inferred failure payload | Source-located lowering Err before IR; no unsafe Send shortcut or erased fault |
|
||||
| First/Get has no row, or finds a row whose value is null | No row: return zero Nullable without reading untouched output; nullable row: preserve its own null flag and full concrete layout |
|
||||
| Invalid role layout, generic bucket, file/directory conflict or one-file role directory | C003 |
|
||||
| `package`/`exposes`/`public` application syntax or forbidden role call | P001 / C006 |
|
||||
| App signature or propagated failure exposes a private type | C006 |
|
||||
| Duplicate API method/path | C002 |
|
||||
| API non-prefixed/unsupported input or output | C005 |
|
||||
| API missing/duplicate/extra or malformed request inputs | JSON 400, code=400 |
|
||||
| Unknown route / unsupported method on known route / handler failure | JSON 404 / 405 (Allow header) / 500 |
|
||||
| Unproved record/payload/write/output/handler bound | C011 |
|
||||
| Discarded, overwritten or merely logged failure | C012 |
|
||||
| Capture error set mismatch or overlapping nested error wrappers | C012 |
|
||||
| Effect in pure function or inconsistent clause contract | C013 |
|
||||
| Non-transferable value live across an inferred suspension point | C005 |
|
||||
| Blocking or parallel effect executed inline on a suspending path, including a concretely bound handler | C005 |
|
||||
| Invalid Model or transaction database contract | C014 |
|
||||
| Contract analysis exceeds bounded work | C015, no executable Program |
|
||||
| Proven redundant check / duplicate private implementation / repeated pure call / trivial private forwarding | W002 / W003 / W004 / W005 |
|
||||
| Unused or overwritten write | Existing W001, no duplicate W006 |
|
||||
| Unneeded exposure for a selected application | W007, not a hard error |
|
||||
| API drift or unreadable baseline | CLI failure before native compilation |
|
||||
| Existing snapshot output | CLI failure, existing bytes preserved |
|
||||
| Invalid test path, missing/overloaded/non-ordinary entry or duplicate `.dever`/`.dever.md` identity | C002/C003/C005 before any test runs |
|
||||
| Test crosses a private role, calls Model/API/another test, or production calls a test | C006 before native compilation |
|
||||
| `assert` receives non-Bool or `assert_eq` receives unequal/non-comparable types | C005 at the assertion |
|
||||
| Test assertion, uncaptured business error or runtime fault | Current case fails; later cases run; final status is nonzero |
|
||||
| `external` outside Adapter, mixed with a Dever function, or unable to select exactly one domain Port identity | P001/C005 before native generation |
|
||||
| External entry is absolute, escapes with `.`/`..`, contains empty/Windows-only segments, or names an unknown/duplicate capability | C002/C005 before native generation |
|
||||
| External input/output/error payload is not eligible for the shared wire policy | C005 before native generation |
|
||||
| `public` API can reach Job enqueue, or enqueue has no trusted execution identity | C006 / runtime rejection before persistence; never synthesize System |
|
||||
| `public` API reaches a tenant component | C014 before native generation |
|
||||
| `public` API reaches authorization administration, or GET reaches role mutation | C006 / C013 before native generation |
|
||||
| Duplicate `component.domain.site.action` across methods or paths | C002 before native generation |
|
||||
| Protected API has no current role grant | JSON 403 before business input/App dispatch |
|
||||
| User Job provider/site/reference, compiled permission, component or current role fails | Block with a redacted identity/component category; do not enter the business transaction |
|
||||
| Credential-changing request has missing or non-matching Origin | JSON authentication error; no Cookie/Header is committed |
|
||||
| Response serialization, business transaction or commit fails after metadata was staged | Discard staged response metadata |
|
||||
| Password crypto API reaches a write outside an explicit transaction, or a transaction reaches password crypto | C014 before native generation |
|
||||
|
||||
## 5. Good / Base / Bad Cases
|
||||
|
||||
LLVM base: lower checked Int 40+2, emit and link real objects, execute on the native host. Good: a nested nominal value that exists only inside a comparison still declares all required LLVM types; a short-circuited failing RHS never runs; managed aliases survive COW updates and fault cleanup. First/Get/Find on List<Text?> or Map<Text,Text?> preserve null and present values without double wrapping. List/Map Float values containing NaN remain non-reflexive under equality. Resource/async good: a second create_new fails without overwriting distinct content; a large owned multi-output survives Pending; stopping pending tasks drains descendants and releases frames; internal Unit and Outputs rows transfer through typed sync Task/blocking descriptors. Bad: consider IR substring or hand-written coroutine smoke sufficient checked-source evidence, clone parameters only after initial resume, omit temporary nominal/Send types, shrink a nullable row output slot or treat foreign-target object generation as successful execution.
|
||||
|
||||
Base: direct `Port { value = 8080 }` satisfies `Int >= 1 and <= 65535`; a project without `test/` reports zero tests. Good: `test/user/account/register.dever` calls `app.register`, asserts returned values and uses `result(call)` for expected business failures. A protected `post publish = app.publish` automatically owns `news.article.admin.publish`; an enqueued User Job stores that one key and rechecks the current catalog and role on every attempt. A login reads credentials and verifies the password outside a transaction, then calls one short transaction that rechecks mutable account/tenant/membership state and creates the session. An external Adapter declares one owned entry and the minimum fixed capability set while its tests replace the production Worker with case-local Port fakes. Bad: call `model.create` or an API handler from a test, share test helpers across files, expose test entries, configure databases through an environment variable, enqueue from `public`, put permissions in Identity, write `auth.require`, copy a token/role/permission snapshot into Job storage, let password work overlap a database transaction, guess that every public POST NotFound means Unauthorized, write response Cookie/Header before commit, or let an external declaration coexist with helper functions.
|
||||
|
||||
## 6. Required Tests
|
||||
|
||||
LLVM resource/async changes select `llvm_resource_native`, `llvm_async_native` and `llvm_async_source` plus existing managed/result/kernel regressions. Use explicitly built runtime-only archives, actual checked source -> object -> archive execution and an atomic allocator ledger (one warm-up + 64 repeated calls). Cover create_new's second creation failing within the same run and verify distinct existing content; aliases, EOF/terminal error, early shared cursor stop, each business failure and runtime unwind. Async cases assert real Pending, initial/final destroy, large typed Text/List/Map/record and multi-output transfer, async static handlers, sync Task and Group, bounded backpressure, nullable Channel row/End, wait/stop/nested descendants, typed business failure/capture/refail, division code2 with exact origin and runtime ABI code3 with original bytes. Six-target object generation is not six-OS execution; bounded allocation balance is not RSS/performance acceptance.
|
||||
|
||||
AsyncStream/TCP/composition changes additionally select `llvm_stream_network_source`: nullable rows, Channel backpressure, ticks and alias closure, async/sync handlers and all ParallelEach dispatch categories, owned multi-output/nullable/Unit timeout and race, original business/numeric/runtime faults, TCP short reads/EOF/shared close, exact creation errors and one terminal failure. An owned loopback cancellation fixture observes peer EOF after timeout returns to prove descendant Socket release. Keep the hand-written C smoke's invalid status assertion separate: async invalid input is 4, not typed-fault 2. Repeat actual native allocations through the same atomic ledger; do not claim performance or six-platform execution.
|
||||
|
||||
LLVM business/Bytes changes select `llvm_lowering` plus opt-in `llvm_result_native`, reusing the runtime-linked runner in `test/dever-tests/tests/support/llvm_managed.rs`. Assert independent expected success/failure branches (especially zero-output capture), ordered multi-output payloads, direct/wrapped/explicit-outer identity, specialized each/reduce failures, capture-success-recapture, nested owned COW, root ownership transfer and nonzero-variant root release. Root faults must retain exact original source/caller spans. Dynamic division and duplicate Map-key diagnostics must remain uncaptured. Bytes cases cover Unicode, empty buffers, aliases, invalid UTF-8, -1/256 byte values and invalid/reversed ranges with original messages. Repeat native ownership checks through the same bounded archive/allocator fixture as managed tests; no recursive archive build or default database/service checks.
|
||||
|
||||
LLVM changes select root `llvm_lowering` and embedded-bridge `llvm_native`: real checked sources to IR/object/LLD/native execution, static handlers and aligned nested Record/Choice/Nullable, multiple outputs, short-circuit faulting RHS, checked Int extrema, IEEE Float and original fault/caller spans. Managed changes also select `llvm_managed_native` with embedded/runtime-abi: Unicode and Decimal, collection COW/order/key types, nullable First/Get/Find, Unit handlers, owned reduce state/early stop, nested field replacement, original runtime diagnostic and failure/root-output cleanup. Its opt-in native runs link the explicitly prepared runtime-only archive, initialize outputs through the real entry and check allocation balance across 64 repetitions after warmup; this is ownership regression evidence, not throughput/RSS/64-or-128-MiB acceptance. Simple kernels link on all six targets; complex/managed aggregates emit on all six but only native host execution is evidence of execution. Windows CRT/stack-probe dependencies require a real target runtime pack, not fixture symbols. Runtime ABI changes select root `native_runtime_abi` plus the explicitly prepared `native_runtime_link` C/LLVM acceptance fixture. Prepare only an offline `runtime-abi` staticlib; author C/Rust tools are not product runtime dependencies or formal release assets. Tests remain under root `test/` and run no database or service by default.
|
||||
|
||||
Select `structured_concurrency`, `async_native`, `contract_syntax`, `contract_facts`, `contract_semantics`, `contract_api`, `contract_redundancy`, `contract_execution`, `error_effects` according to changes. Assert suspension propagation through ordinary calls and concrete handler substitution, non-transferable liveness, worker boundaries, API omission, alias mutation, handler variance, short-circuit/empty traversal, overwrite/projection obligations, bounded shared graphs and unchanged native results. Error capture tests cover native/reference wrapping, explicit outer identity, ambiguity and discarded captures. Parser tests must reject the removed package-header dependency clause; formatter round trips must prove it cannot be emitted. Do not run the entire suite or service tests by default.
|
||||
|
||||
For this source/API contract select `source_architecture`, `source_visibility`, `contract_execution`, `markdown_source`, `api_declarations`, `rest_model`, `native_entry`, `api_routes`, `auth_runtime`, `authorization_runtime`, `secure_contracts`, `tenant_database`, `durable_jobs` and `upload_boundary` as needed. Assert App aliases, topic hiding, role call direction, Model ownership, layout rejection, generated roots, strict inputs, JSON envelopes, Markdown parity, site/auth ownership, exact permission keys, duplicate-key rejection, public-to-tenant rejection, REST row scope and Upload cleanup. Authentication coverage includes public Cookie isolation, explicit invalid Authorization, exact Origin, staged metadata ordering, generated System roots, site-scoped multi-role union, Owner isolation and User Job current-role checks. Actual loopback HTTP verification requires an explicitly scoped service check.
|
||||
|
||||
Application-test changes select `application_testing`, `formatter` and `cms_project`. Assert stable discovery, plain/Markdown duplicates, filename entry validation, both visibility directions, assertion typing/rendered source locations, existing result-choice recovery, failure continuation/output capture, zero-test success, per-case SQLite Seed isolation, ignored deployment settings, production commands ignoring test sources and module/test formatting preflight.
|
||||
|
||||
External Adapter changes select `port_adapter`, `external_component` and `application_testing`. Assert Adapter-only contextual syntax and formatting, setting/top-level wire names in the schema hash, complete Port effect/failure propagation, reference rejection, case-local fakes, strict framed messages and request IDs, business-error decoding, bounded cancel/timeout/crash/restart/shutdown, queued-call admission closure, explicit child reap and native end-to-end dispatch.
|
||||
|
||||
`external command` reuses the external Port boundary but not the Worker protocol. Require exactly one operation with named `args: List<Text>` and optional second `stdin: Bytes?`, one named `output: dever.process.Output`, and `fails dever.process.Error`; reject setting/lib declarations. The checked `Command` wire policy alone admits Bytes as strict padded base64; API, Job and protocol Worker policies retain their previous allowed types. HIR Worker contract/entry queries exclude commands, while `external_command_contracts()` shares their contract construction. Both typed backends use the same external async ABI and Bytes wire bridge. Test plain/Markdown formatting, wrong signatures, fake isolation and six-target object emission separately from actual runtime execution.
|
||||
|
||||
## 7. Wrong vs Correct
|
||||
|
||||
LLVM application wrong: emit an unbound auth descriptor, accept a nonpublic route without its exact permission, clone a borrowed handle but return the original SSA value, or give DELETE's encoded `null` two owners. Correct: validate auth metadata at source and again at the private ABI; return the actual guard-held clone before releasing its source; register one REST response owner through encoding/commit and transfer it once. Channel/Task/Group wrapper clones can change address even though Text Arc retain does not.
|
||||
|
||||
LLVM request scopes must receive one boxed compiled-handler Future. Inline embedding the same large Future in every auth/tenant/job task-local branch amplifies debug poll stacks and caused a real nonpublic HTTP SIGSEGV. Keep one request-level indirection with the original scope/cancellation ownership; do not enlarge thread stacks, relax the allocation oracle or box every database operation. Permanent probes are `llvm_api_source` (private REST including DELETE, concurrent identities, rollback and 64 repeated roots) and `llvm_async_source::cloned_channel_fields_survive_their_source_scope`.
|
||||
|
||||
LLVM wrong: collect types only from signatures/locals or assume every object needs no platform runtime. Correct: visit all reachable nested expressions, verify actual typed IR and native behavior, and distinguish cross-target object/link evidence from formal target runtime and OS execution acceptance.
|
||||
|
||||
Managed LLVM wrong: bit-copy an owned handle, overwrite an owned field without dropping it, disarm a whole record when moving one field, or use the nullable result payload as First/Get's physical row slot. Correct: use typed lifecycle callbacks/guards, reuse final-read analyses, transfer only the selected owner, and allocate the full concrete row before interpreting outer presence. Preserve internal null rather than constructing Some(null).
|
||||
|
||||
LLVM Result wrong: stringify business payloads, remap errors by variant name/message, catch numeric faults, leave a captured fault live, or accept either success or failure in every test expectation. Correct: share checked capture targets, preserve typed nominal payload and exact source frames, transfer then clear business ownership, propagate non-business faults, and assert the expected branch for each input.
|
||||
|
||||
LLVM async wrong: use source Unit/Outputs transferability for internal ABI rows, clone arguments only on first resume, mark Send from parameters alone, drop PreparedSync input inside invoke, or use a RuntimeAbi-string oracle for division. Correct: field-wise ABI metadata, initial-frame ownership, transitive suspend-frame checks including temporaries, runtime-owned synchronous input cleanup, and exact fault-kind/source oracles. Resource tests must make erroneous overwrite observable with different contents and a mutually exclusive result.
|
||||
|
||||
LLVM protocol wrong: round-trip source records through JSON, render a sensitive failure variant, make one failed request stop the server, or call all six targets runtime-tested after generating objects. Correct: fixed field ABI, typed resource/handler ownership, checked observable terminal errors, original request-boundary behavior, independent wire peers and explicit host-execution evidence.
|
||||
|
||||
Wrong: preserve application package/import/main syntax, accept arbitrary run/build entries, call private roles across domains, make topic filenames part of App calls, turn API declarations into handler functions, trust client tenant/user fields, infer System from an absent identity, put permissions in verify Identity, write manual permission strings, persist a credential/role/permission snapshot, publish pending response metadata, or count a possibly-empty List as proof of returned failure.
|
||||
|
||||
Correct: derive one source-role identity from each path, publish only logical App capabilities, keep API declarations and private roles narrow, generate application roots and exact permissions from checked API/CMD/Job contracts, establish System only at compiler-owned roots, and re-verify persisted User references plus the one entry permission on every Job attempt. Preserve conservative HIR facts and require explicit recovery where the program intentionally degrades. Derive Test identity from a compiler-owned source root, retain private checked entries and reuse the normal HIR/native/error pipeline; expose response metadata only after serialization and transaction commit.
|
||||
|
||||
External wrong: expose JSON/process handles to App, hand-write a second operation registry, omit setting or top-level wire names from schema identity, or start the production Worker from a Test. External correct: synthesize the complete implementation from one checked Port, reuse shared wire/effect/failure/specialization owners, and bind tests only to case-local Dever fakes.
|
||||
@@ -0,0 +1,235 @@
|
||||
# Database Guidelines
|
||||
|
||||
## 1. Scope
|
||||
|
||||
Dever owns a static ORM. Application code declares persistence in `module/<component>/<domain>/model.dever` or, for multiple cohesive models, `model/<topic>.dever`; Markdown uses the same identity and schema rules. It does not add CRUD App/API wrappers or pass database/transaction handles. SQLite and PostgreSQL consume the same checked `ModelSchema`, `QueryPlan`, bind values, row shapes and error propagation.
|
||||
|
||||
## 2. Model Contract
|
||||
|
||||
- A role-file Model has one automatically visible record matching the domain in UpperCamelCase; a topic Model record matches its topic. Source does not write `public`.
|
||||
- `id` and `created_at` are synthesized, generated and cannot be assigned by source code.
|
||||
- Persistent fields use lower_snake_case and the checked types `Bool/Int/Float/Decimal/Text/Bytes/Uuid/DateTime/Date/Time/Duration/Json`, Model choices or `<model-package>.id`.
|
||||
- Persistent metadata includes `generated`, constant `default`, `index`, `unique` and `from <old_field>`. REST-only `owner`, `create = expr`, `replace = expr` and `search = expr` are checked API contracts and do not alter ordinary ORM semantics or storage fingerprints.
|
||||
- `private` is language visibility metadata: only the owning domain App accesses that field, including generated create/update blocks. It does not enter ModelSchema, SQL column order or migration fingerprints. Private Model snapshots and containers cannot cross App contracts; expose a View. Secret is never a persistent type; password hashes use private Text with enough room for PHC encoding.
|
||||
- Default and Seed literals share storage validation: integer/date values fit i64 without a fractional part, Float is finite, Decimal satisfies exact precision/scale and Text lengths count Unicode scalars. Invalid values fail checking rather than Rust emission or database startup.
|
||||
- Composite `index(...)`, `unique(...)`, reverse `relation`, one `seed` block and stable named `migrate` blocks live in the Model file.
|
||||
- Model IDs are opaque nominal values. A foreign Model ID is not interchangeable with `Int` or another Model ID.
|
||||
|
||||
## 3. Runtime And Query Contract
|
||||
|
||||
Use compiler-synthesized short Model functions: `create/create_many/get/first/list/cursor/count/exists/stream/update/delete/upsert`. Only the owning domain App may call them, as `model.<operation>` or `model.<topic>.<operation>`. Main, API and other domains call the owning App instead. Anonymous create/update/query blocks are contextual syntax and never become runtime maps or reusable dynamic query objects.
|
||||
|
||||
`where` builds a static condition tree; values are always bind parameters. Equality on nullable values is null-safe (`IS`/`IS NOT` on SQLite, `IS [NOT] DISTINCT FROM` on PostgreSQL). `order` is static: an omitted order defaults to `id DESC`, while an explicit order receives a stable `id` tie-breaker when needed. `list` defaults to page 1 and size 20, and runtime rejects a size above `database.<name>.max_page_size`. Do not add an Active Record type, chainable query object, reflection decoder or duplicated backend CRUD implementation.
|
||||
|
||||
`cursor` uses the same ordered Model fields as a static composite keyset. Its nullable `next` value is the last returned Model snapshot and is passed back as nullable `after`; the compiler binds the ordered field values in declaration order and rejects nullable cursor-order fields. SQLite uses `IS` and PostgreSQL uses `IS NOT DISTINCT FROM` for equal key prefixes. The nullable guard must cast the first PostgreSQL placeholder to the ordered field's SQL type before `IS NULL`; an untyped `$n IS NULL` leaves PostgreSQL unable to infer the parameter type. `stream` remains the existing `AsyncStream<Model>` source surface and uses a bounded runtime producer. Its database worker is registered in the current structured task scope; full consumption, `close`, drop, cancellation and decoder/driver failure all release or discard the owned connection according to its known state.
|
||||
|
||||
Connection selection order is explicit `database <name>`, root package name, then required `database.default`. `config/setting.json` is strict and external to the binary. SQLite paths remain below the executable directory; invalid or missing settings fail startup.
|
||||
|
||||
Repository PostgreSQL acceptance tests and performance tools use the repository-root `config/setting.json` entry `database.postgres_test`. They do not accept connection URLs or TLS modes from environment variables or command-line flags. The local file is ignored by Git; tests stage only the selected connection as `database.default` beside their owned executable.
|
||||
|
||||
`test/dever-tests/tests/support/postgres.rs` owns both schema-level and physical-tenant-database fixtures. The latter requires an explicitly isolated test connection with `{case}` in the database name and permission to create/drop test databases. It creates one random control database and a fresh tenant prefix, rejects a pre-existing namespace, and only drops that exact control name plus the generated prefix with canonical positive tenant IDs. Cleanup failures fail acceptance, including when the test itself panics. No configured URL means no connection; these real-service tests remain ignored by default.
|
||||
|
||||
`postgres_api` runs stock CMS from a test-owned project copy and validates Owner HTTP requests plus two physical tenant databases. Its explicit test-only member fixture also exercises real non-Owner sessions, two admin roles, a same-named front role and immediate grant/revoke effects; never change reserved Owner mutability or production CMS to manufacture this evidence. Permission catalogs are site-filtered: look up each site's keys through that site's authenticated API. GET/DELETE arguments use query, not JSON bodies. Its `cms.py postgres-case` helper stages its own temporary copy: private ports, secrets, config changes and compiled artifacts must never modify the supplied source project, and persistent reports must omit credentials. A standalone Owner-only run without `--rbac-fixture` still does not prove non-Owner authorization.
|
||||
|
||||
The PostgreSQL driver receives named `ConnectionOptions`, shared by initial and tenant connections. These are internal constructor arguments, not an additional configuration layer or environment-variable path.
|
||||
|
||||
Application tests never read or copy the project's deployment database settings. A test with reachable Model effects receives a runner-generated `config/setting.json` in its per-case temporary project, containing SQLite `database.default` plus SQLite entries for every explicit Model connection. It uses the normal settings validation, complete schema initialization, migrations, Seed, transactions and shutdown. A suite containing only tests without Model effects uses the base runtime. A mixed suite links the SQLite-capable runtime once, but a selected non-database case receives no generated setting and does not load settings or initialize Models. Neither path accepts database environment variables, PostgreSQL URLs or writes project `data/`.
|
||||
|
||||
Global Models resolve their connection once during startup into a `OnceLock<Database>`. When top-level `tenant.database` is configured, ordinary Models resolve through the hidden trusted tenant scope to a bounded, idle-evicted physical pool; there is no mode switch and they never fall back to the global connection. SQLite and PostgreSQL are independent optional runtime features; native build profiles contain only configured drivers.
|
||||
|
||||
`global type` marks platform data. With top-level tenant settings, every other persistent Model and its Job queue is tenant scoped. SQLite derives a private database file below `tenant_directory`; PostgreSQL derives a database name from `tenant_database_prefix` and the positive internal tenant id. Only explicit `dever tenant migrate <project-root> <id>` creates/migrates storage and records matching local/control readiness markers. For PostgreSQL that command connects through the configured control URL, creates the derived database when absent, treats SQLSTATE `42P04` as the idempotent already-exists case, and then migrates it; ordinary requests/workers never create or fall back to another database. Requests, Jobs and tenant CMD execution require a ready marker and current schema fingerprint. Table/field modes are not accepted until their entire query, relation, unique, migration and native-SQL contracts exist.
|
||||
|
||||
`tenant` accepts only `database`, `max_pools` and `idle_timeout_ms`; `database` selects the control connection. The control database owns `_dever_tenant_storage`, `_dever_tenant_component` and the global `_dever_permission` catalog. Tenant components are the sorted component names owning a tenant Model or tenant Job; platform/global-only and unknown names cannot be toggled. A mixed component is wholly tenant scoped: every Job it owns uses tenant storage even when that Job's own body touches only global data. After migration, `dever tenant component disable|enable <root> <id> <component>` writes only the control table. API, CMD and Job check all tenant components in their reachable execution graph before business work. A `public` API reaching any tenant component is rejected statically.
|
||||
|
||||
Authorization role storage is framework-private: `_dever_auth_role`, `_dever_auth_role_permission` and `_dever_auth_user_role`, versioned by `_dever_auth_version`. Platform identities use the control database; tenant identities use their tenant database. Role identity and both relation tables use the composite `(site, role_id)` boundary, so sites may reuse role names without collision and grants cannot cross sites. The v1 single-ID schema migrates transactionally by copying each relation's site from its role before replacing the three tables; an incomplete or unknown private schema fails startup. Disabled roles do not authorize, multiple roles form a union and an `all_permissions` role covers only its site. `_dever_owner:<site>` is reserved and can be provisioned only by `dever tenant owner` after tenant migration; ordinary save, grant, revoke and disable operations must reject it. Applications may expose thin App/API adapters over the core role operations but must not create duplicate business permission tables.
|
||||
|
||||
SQLite uses `deadpool-sqlite 0.13` with bundled SQLite, WAL, foreign keys, normal synchronous mode, a busy timeout and a bounded pool. PostgreSQL uses `deadpool-postgres 0.14.2` with `tokio-postgres 0.7.18`, cached static statements and rustls system roots; `min_connections` is prepared before Model initialization. Pool create/recycle and PostgreSQL socket connection have fixed runtime bounds. `wait_timeout_ms` bounds pool wait, while independent `io_timeout_ms` bounds SQL, transaction and migration I/O; both accept 1–300000 milliseconds.
|
||||
|
||||
PostgreSQL parameter types must match the runtime's binary bind codec. Dever `Int` binds as i64 and therefore requires a `BIGINT` context in typed native SQL; `Uuid` binds as the native PostgreSQL UUID type and generated placeholders use `$n::uuid`, never `$n::text::uuid`. Casts resolve PostgreSQL parameter types and must not change the runtime value representation.
|
||||
|
||||
Model operations, row-stream pulls and transaction begin/commit propagate `dever.database.Error` categories (Pool, PoolExhausted, Connection, Timeout, Cancelled, Database, Constraint, NotFound, InvalidData, Migration). Generated code converts runtime ErrorKind directly to source variants with message payloads; it never flattens recoverable errors into Fault or guesses their category from text. A service capture can use `type ReadResult { Found(value: User) error Failed(error: dever.database.Error) }`. The inferred set conservatively includes every database category. Startup failures remain process failures before application code begins.
|
||||
|
||||
## 4. Validation And Error Matrix
|
||||
|
||||
| Condition | Required result |
|
||||
| --- | --- |
|
||||
| Missing/invalid `setting.json`, unresolved database, conflicting tables or cross-database FK | Startup or compile failure before serving work |
|
||||
| Transaction has no database effect, spans physical connections/scopes, or starts escaping concurrent work | Compile failure; validated sequential `blocking` remains allowed |
|
||||
| Bound handler specializes to database work | Specialized effect set and hidden transaction context are retained |
|
||||
| `run`/thread/parallel/HTTP callback performs database work | Detached context; it cannot inherit an ambient transaction |
|
||||
| Operation, `BEGIN`, migration or transaction finish is cancelled before confirmed completion | Detach the pool object; never recycle an unknown connection state |
|
||||
| Actual catalog differs from recorded catalog | Migration failure before schema mutation |
|
||||
| Constraint tightening conflicts with stored rows | Database error and full transaction rollback |
|
||||
| Business body fails and rollback also fails | Preserve the business error and append rollback failure as a cause |
|
||||
| Missing get row or duplicate unique key | Recoverable NotFound / Constraint, including through task wait and transaction rollback |
|
||||
| PostgreSQL cursor nullable guard leaves `$n` untyped | Generate `CAST($n AS <field SQL type>) IS NULL` |
|
||||
| PostgreSQL reports admin/crash shutdown, cannot-connect-now or database-dropped SQLSTATE | Classify as a connection error so pool recovery semantics apply |
|
||||
| Typed native SQL uses an `Int` parameter where PostgreSQL infers int4 | Give the placeholder a `BIGINT` context; do not change the runtime codec |
|
||||
| Job claim/renew/complete waits for a database write lock | Read the injected Clock only after the lock is owned; never extend or complete an already expired lease with a cached timestamp |
|
||||
| SQLite `job.lease_ms <= Job timeout_ms` | Fail startup before listen/claim; a persisted incompatible timeout becomes `blocked/invalid_policy` before App dispatch |
|
||||
| An expired Job already consumed `max_attempts` | Move it to dead with bounded cleanup; it must not occupy an execution slot or hide a later eligible Job |
|
||||
| Concurrent enqueue repeats an active `(target, key)` | Return the same Job Id from the atomic upsert; never create a second pending/running row |
|
||||
| Job enqueue has no trusted execution identity | Reject before persistence; absence never means System |
|
||||
| User Job identity cannot be re-verified or changes tenant | Block with a redacted identity category before business dispatch |
|
||||
| User Job permission is absent from the current compiled catalog or current role grant | Block as `identity_rejected` before business dispatch |
|
||||
| A reachable tenant component is disabled | API 403, CMD failure or Job `blocked/component_disabled`; no business dispatch |
|
||||
| Component name is global-only or absent from the compiled tenant manifest | Reject the control operation; do not create an arbitrary row |
|
||||
| Legacy active Job lacks identity or crosses the v4 permission-contract upgrade | Block it as `missing_execution_identity` / `authorization_contract_changed`; never upgrade it to System |
|
||||
| PostgreSQL tenant database is absent during explicit tenant migration | Create the derived database through the configured control connection, then migrate; concurrent already-exists is success |
|
||||
|
||||
## 5. Required Patterns
|
||||
|
||||
Declare a business function with `transaction name(...) (...)`. The compiler propagates its database effects through ordinary calls and specialized handler bindings, then validates the concrete call graph. The generated hidden context owns one pool object at the outer boundary; nested transaction calls reuse it without savepoints.
|
||||
|
||||
Success commits. Error/fault rolls back before propagation. A rollback failure is appended as a cause and does not replace the original error. Commit and rollback consume the transaction, immediately returning a one-connection SQLite pool object after completion. SQLite write transactions use `BEGIN IMMEDIATE`. An API call graph containing password verification or hashing receives no entry-wide transaction; every write in that graph must be owned by an explicit short transaction reached after the expensive work, and that transaction must recheck the mutable state on which the write depends.
|
||||
|
||||
Use one checked query planner and shared value model for both database executors. Backend-specific code is limited to SQL dialect, bind/row codecs, pool operations and catalog migration. Keep business transactions in the owning domain App; Model CRUD remains compiler-generated.
|
||||
|
||||
Typed Model SQL may return its private result record or an explicit owning-domain App View (`app.ArticleView`, including App topic files), as one row, optional row or list. Reuse ordinary App alias resolution and existing scalar row decoding. App Views must contain only visible supported storage scalar fields; Secret, nested Model/record values and cross-domain or Domain-owned result records are rejected. All regular SQL result records reject field bounds: row decoding does not prove source range refinements. This supports bounded list projections without exporting private Model snapshots or duplicating View types; existing App/API sensitivity checks remain mandatory.
|
||||
|
||||
GET may reach typed SQL only when both dialects are proved to be a complete single-own-table field projection: SELECT fields FROM own_table, optional WHERE field = numbered-bind joined by AND, optional ORDER BY own fields, optional LIMIT numbered-bind. `check/sql_read.rs` owns this conservative proof. Unknown syntax, functions, CTEs, subqueries, other tables or extra statements retain the existing write effect; a SELECT prefix alone never establishes safety. Test both dialects independently and preserve POST SQL write behavior.
|
||||
|
||||
## 6. Schema, Migration, Seed And Tests
|
||||
|
||||
Every database has `_dever_history` rows for schema, named migrations and Seed revisions. Initialization takes a backend lock and updates schema, indexes, Seed and history in one transaction.
|
||||
|
||||
- A missing table is created directly at the final schema; old `from` and drop history are not replayed.
|
||||
- The normalized actual catalog must match recorded history before any update. SQLite fingerprints table DDL and managed index DDL. PostgreSQL fingerprints columns, types, nullability, identity/default metadata, constraints and managed indexes.
|
||||
- `from` maps one old field to one new field. Name similarity is never inferred.
|
||||
- Removed fields require a new named `migrate ... { drop field }`; applied migrations cannot be removed or modified.
|
||||
- Named migrations may contain repeated `before` / `after` blocks with explicit `sqlite`, `postgres` and literal `parameters`. Reuse checked placeholder scanning and existing bind codecs; accept one INSERT/UPDATE/DELETE only, reject reserved internal identifiers, DDL and transaction control. Unclosed quoted regions are diagnostics, including Unicode input. Context-free parameters support Bool/Int/Text/null and finite exponent-form Float; do not silently reinterpret Decimal as Float.
|
||||
- PostgreSQL dollar-quote recognition must respect identifier boundaries and validate both delimiter and body termination. Do not eagerly construct an EOF slice inside `then_some`; malformed SQL must produce C014, never panic. The same scanner serves typed SQL and migration SQL.
|
||||
- For an existing table, validate history/order/catalog first, run pending before blocks in source order, apply automatic DDL, then pending after blocks, Seed and history in the same Model transaction. Fresh databases skip old data transforms and record the final migrations as applied. SQL revisions cover phase, source order, both dialects and parameter types/values. Keep drop-only revision calculation stable. Applied migration order is immutable.
|
||||
- The migration boundary is one Model on one database. Data transforms may reference tables already present at that point; do not promise cross-Model atomicity or arbitrary table initialization order.
|
||||
- Text/Bytes constraint changes and nullable/unique/FK/choice tightening rely on database validation and roll back without truncating or inventing values.
|
||||
- Cross-family or lossy type changes are rejected. The currently automatic widening is `Int -> Float`; Text and Bytes bound changes stay within their storage family.
|
||||
- SQLite rebuilds the table transactionally when fields change and synchronizes managed indexes directly for index-only changes.
|
||||
- PostgreSQL uses transactional `ALTER TABLE`, named `_dever_` CHECK/FK constraints, advisory locks and managed index replacement. Fresh schemas create tables first, then add foreign keys in a second phase so source order and cyclic references do not break initialization.
|
||||
- Seed requires a covered unique key and uses parameterized `ON CONFLICT DO NOTHING`. It inserts missing initialization rows and never overwrites existing business data.
|
||||
|
||||
Date/time values use integer milliseconds: DateTime is Unix epoch milliseconds, Date is UTC midnight epoch milliseconds, Time is milliseconds since midnight in [0, 86400000), and Duration is signed milliseconds. External wire strings do not change the storage codec, logical Model type or schema fingerprint. JSON uses checked text. Decimal uses a fixed-width sortable text key on SQLite and native `NUMERIC(P,S)` on PostgreSQL. UUID uses a 16-byte SQLite BLOB and native PostgreSQL `UUID`; generated UUID fields receive UUIDv7 values from the runtime.
|
||||
|
||||
Long-term tests live under repository-root `test/`. Focused verification must cover database-effect propagation through helpers, nested transactions and handler specialization; rejection of concurrent transaction work; nullable equality with both present and absent values; typed cursor placeholder casts; Decimal/UUID round trips; cancellation isolation; migration catalog drift; settings and Model syntax. PostgreSQL catalog, migration, TLS, native codecs, disconnect classification and cancellation require an explicitly supplied isolated test database; never probe or mutate a local/development PostgreSQL service by assumption.
|
||||
|
||||
Durable Job changes select `durable_jobs`. Assert atomic business-write/enqueue commit and rollback, active-key concurrency, claim-time attempt, fresh fencing tokens, old-token CAS rejection, expired-lease recovery, bounded exhausted-row cleanup, blocked target/schema/payload/policy/identity/component, cursor rollback/restart and completed-occurrence dedupe. Cover System/User persistence, missing-identity rejection, v2 identity migration, v4 User authorization-contract blocking, physical/execution tenant dedupe isolation, chained inheritance and generated User re-verification/exact-permission checks. The SQLite regression must hold `BEGIN IMMEDIATE` across a lease renewal window and advance the injected Clock while claim/renew/complete waits, proving timestamps are sampled after lock ownership. Mode/signal tests use only test-owned processes, loopback listeners and temporary SQLite databases.
|
||||
|
||||
`model_syntax` covers invalid default/Seed values; `markdown_source` covers Model metadata and schema parity; `sqlite_orm` verifies captured error identity and zero persisted rows after rollback. `orm_data_migration` covers bound SQL, revisions, catalog checks, source order, once-only execution, fresh databases, rollback and native upgrades. `contract_execution` compares both CMS source declarations, Model/API snapshots and settings; `cms_project` checks both source formats and isolated packaged executables. Native row decoding must return its ORM Result inside its own closure; a bare generated block lets `?`/`return` escape into the outer application error type, breaking `first()`.
|
||||
|
||||
## Durable Job Store
|
||||
|
||||
`job/store.rs` owns versioned private `_dever_jobs`/cursor schema, separate from Model history. Use the existing Database/Executor/Transaction and pool. Private schema v5 stores physical `tenant_id`, execution `scope_tenant_id`, `execution_kind`, provider/site/subject/session/tenant references and one `auth_permission` key inherited from the protected HTTP entry. It never stores a token, Cookie, Header, role or permission collection. SQLite initialization/claims use the ordinary write transaction; PostgreSQL schema installation uses a transaction advisory lock and claims use bounded `FOR UPDATE SKIP LOCKED` rows.
|
||||
|
||||
An active partial unique index is authoritative for physical tenant + execution tenant + target + business key. Atomic upsert with RETURNING retains the active row lock until commit and avoids an insert/select completion race. Ambient enqueue shares the caller's physical transaction; standalone enqueue owns a short transaction. Every claim increments attempt and creates a fresh token. Renew and every completion require id/token/running/unexpired lease. Unknown target, wire fingerprint mismatch and decode failure become blocked with fixed redacted categories; retry and timeout use bounded backoff, then dead.
|
||||
|
||||
CMD, cron and compiler-owned Test entries explicitly establish System identity. Protected API and restored User Jobs establish User identity from verified Claims; nested enqueue inherits it. On every User attempt generated dispatch selects the configured provider/site verify App, reconstructs the trusted Claims reference, validates the returned subject and tenant, confirms the persisted key still belongs to the compiled site catalog, checks current tenant components and performs the latest role query, and only then begins the business transaction. Invalid identity is blocked; transient verify database failures retain normal retry behavior. A missing task-local identity is always an enqueue error.
|
||||
|
||||
Private migration v2/v3 preserves the existing fail-closed identity behavior. Migration v4 to v5 adds `auth_permission`, blocks every active User row as `authorization_contract_changed`, and drops the obsolete `auth_capabilities` column because a previous capability list cannot be guessed into one entry permission. Active System rows and terminal history remain unchanged. No migration guesses System or reconstructs authorization from payload data.
|
||||
|
||||
Scheduler identity/fingerprint/cursor are durable. Occurrence insert and cursor advance share a transaction; first enable records the current minute, each pass considers at most 256 elapsed minutes. Completed occurrences cannot be recreated. Job database effects participate in settings-aware transaction validation and per-case SQLite test isolation without manufacturing Model metadata or a second storage stack.
|
||||
|
||||
Read the injected Clock after obtaining the write lock: claims establish their deadline after row selection, and renew/completion lock the claim row before checking expiry. A timestamp captured before pool/lock wait cannot prove a lease is still valid. SQLite worker preflight requires `lease_ms > Job timeout_ms`, because a business write transaction blocks heartbeat writes even with a spare connection; PostgreSQL retains shorter renewable leases.
|
||||
|
||||
Validate persisted claim policy before dispatch too: older timeout policies incompatible with the current SQLite lease become blocked with `invalid_policy`, without running App. Clean exhausted rows in a separate bounded statement within the claim transaction; they must not occupy execution slots or make test drain mistake an eligible queue for an empty one.
|
||||
|
||||
## 7. Good / Base / Bad
|
||||
|
||||
Base: a System CMD enqueues a tenant Job and persists its controlled execution tenant. Good: protected `post publish` automatically authorizes and binds `news.article.admin.publish`; each attempt re-verifies the session/membership, compiled permission, component state and current role. Bad: derive actor/tenant from payload fields, treat a missing identity as System, persist JWT/Cookie/roles/current permissions, write manual authorization strings, allow two execution tenants to share a global-queue dedupe key, or create a tenant database during an ordinary request.
|
||||
|
||||
## 8. Wrong Versus Correct
|
||||
|
||||
- Wrong: hand-written CRUD Service/API wrappers, dynamic query maps, reflection decoders or database handles in source code. Correct: Model methods lower to a checked `QueryPlan` and thin backend executors.
|
||||
- Wrong: infer a renamed column from similarity or silently accept catalog drift. Correct: require `from <old_field>` and compare recorded actual catalog before migration.
|
||||
- Wrong: carry an ambient transaction into `run` or another concurrent boundary. Correct: concurrent callbacks use detached database context; only sequential calls inherit the transaction.
|
||||
- Wrong: release a connection whose async/blocking operation did not report completion. Correct: detach it so the pool replaces it.
|
||||
- Wrong: make PostgreSQL accept an i64/UUID bind by converting the runtime value to text. Correct: keep the native binary codec and emit the exact `BIGINT`/`UUID` placeholder context.
|
||||
- Wrong: claim cursor/stream, bulk create/upsert, relation loading or live PostgreSQL behavior from AOT compilation. Correct: claim each only after its dedicated runtime tests exist.
|
||||
- Wrong: assume an extra SQLite pool connection can renew a lease while a handler holds `BEGIN IMMEDIATE`, or capture `now` before waiting for that lock. Correct: require the SQLite lease to exceed the bounded handler timeout and read Clock after acquiring the write/claim lock.
|
||||
- Wrong: let exhausted rows consume `LIMIT` and return an empty claim batch while eligible work remains. Correct: clean exhausted rows with a separate bounded statement and query execution slots only from `attempt < max_attempts` rows.
|
||||
- Wrong: snapshot authorization into payload/queue columns or let legacy rows become System. Correct: persist only a trusted re-verifiable identity reference and the protected entry's exact permission key, then re-verify the current catalog, components and role before each attempt.
|
||||
- Wrong: let password hashing hold SQLite's writer reservation or switch the whole flow to a deferred snapshot that later upgrades to a write. Correct: perform password work outside a transaction, then enter an explicit short `BEGIN IMMEDIATE` transaction that rechecks mutable state and writes.
|
||||
- Wrong: silently connect to the control/global database when a tenant database is missing. Correct: create/migrate it only through explicit `tenant migrate`, require readiness/fingerprint markers, and fail ordinary work closed.
|
||||
|
||||
AOT compilation proves feature and generated-code compatibility, not live PostgreSQL semantics.
|
||||
|
||||
## 9. LLVM Model Application Boundary
|
||||
|
||||
### Scope / Trigger
|
||||
|
||||
Checked Model/SQL/transaction lowering serves LLVM CMD, API, Job and case-local Test roots. The API Tenant Session section below defines identity, authorization and tenant-pool ownership. Missing tenant identity/readiness fails closed; it never silently selects the platform database. External Worker and default CLI integration remain separate work.
|
||||
|
||||
### Signatures
|
||||
|
||||
- Core `llvm::emit_application(&Program, &SourceMap)` emits typed Model functions and the owned database application root. Pure SQL/DDL builders remain owned by `native/orm.rs` and shared by both backends.
|
||||
- Runtime `database::Session::load(profile, bindings, transactions)`, `prepare()`, `database(explicit, root)` and `close()` own only this invocation's Settings and database pools.
|
||||
- `task::run_entry_with_typed_cleanup(config, future, cleanup)` returns the original typed result after descendants drain and cleanup completes inside the same runtime.
|
||||
- Canonical `dever_rt_v1_db_*` declarations live only in `crates/dever-backend-bridge/include/runtime.h`; `runtime-sqlite` / `runtime-postgres` add actual drivers independently of `embedded`.
|
||||
|
||||
### Contracts
|
||||
|
||||
Deployment settings still come only from executable-local `config/setting.json`; no connection environment variable or PATH language runtime is introduced. Ordinary resolution is explicit connection → component root → required default. Persistence ABI accepts only fixed storage scalar rows; business Model records, nullable fields, Related and DbError are concrete typed owners, not dynamic business values. Related ToMany groups at most 998 parent keys per SQL batch and retains the existing per-parent row limit. Migrations group by resolved database, create all tables before foreign keys and preserve existing history/Seed validation.
|
||||
|
||||
Writing CMD order is input decode → begin → sequential App/handler calls → output encode → commit → stdout. Nested transaction shares the outer owner, without savepoints; Task and parallel boundaries detach. Failed or cancelled unconfirmed operations discard the transaction connection. Rollback must move the original fault, retain its type/source/payload and only append cleanup cause; commit failure must release the already-initialized output. Drain children, stop every pool, await pool drivers, then destroy the runtime. Never reuse the process-global Settings/pools for repeated embedded roots.
|
||||
|
||||
### Validation & Error Matrix
|
||||
|
||||
| Condition | Required result |
|
||||
|---|---|
|
||||
| Invalid/misaligned output or typed callback slot | ABI fault before I/O, receiver advancement or fault consumption |
|
||||
| Missing/invalid settings or missing tenant identity/readiness | Explicit root failure, no default connection fallback |
|
||||
| Database error | Exact one of the ten checked DbError variants with owned Text payload |
|
||||
| Output encoding or numeric fault after a write | Original runtime fault; rollback; no stdout |
|
||||
| Captured PostgreSQL SQL constraint error | Current transaction remains aborted; no implicit savepoint/continued-query promise |
|
||||
| Partial decode, stream stop/cancel or root exit | Paired typed cleanup and connection release/discard in its known state |
|
||||
|
||||
### Good / Base / Bad Cases
|
||||
|
||||
Base: a CMD creates, reads and deletes a row in its owned SQLite database. Good: real PostgreSQL round-trips storage scalars, captures NotFound without losing identity, and rolls back an uncaught unique violation before the next command inspects persisted state. Bad: use SQLite's ability to continue after a SQL constraint error as a PostgreSQL contract, close a pool after its runtime disappears or reuse a closed OnceLock pool on the next entry.
|
||||
|
||||
### Tests Required
|
||||
|
||||
Select root `llvm_database_source`, `runtime_entry_cleanup` and the canonical C database driver in `native_runtime_link`. Static cases must pass the existing checker and emit objects for all six targets. Opt-in native cases link the explicitly prepared runtime-only archive, execute with a cleared environment, and check output/fault plus allocation balance over warmup and 64 repeats. Cover CRUD/bulk/upsert, nullable/codec values, cursor/offset, stream cancellation with one connection, ToOne/ToMany, typed SQL, nested/sequential transactions, uncaught constraint/numeric/encoding rollback and successive schema/Seed revisions. Live PostgreSQL requires only the shared `database.postgres_test` isolated fixture, not an environment URL. These checks do not prove target runtime packs, throughput/RSS, tenant/API equivalence or injected commit/close failures.
|
||||
|
||||
### Wrong Versus Correct
|
||||
|
||||
Wrong: publish Session settings into global OnceLock, run pool cleanup in a second runtime, copy SQL/driver code for LLVM or add savepoints to make a SQLite test pass on PostgreSQL. Correct: invocation-owned Session, original shared SQL/drivers, same-runtime drain/close ordering and tests respecting each driver's existing transaction semantics.
|
||||
|
||||
### LLVM API Tenant Session
|
||||
|
||||
Job bindings and clock belong to the same invocation Session. Enqueue uses the caller's physical database and ambient transaction; handler transactions and timeout rollback reuse the existing store/worker and transaction owners. Tenant migrations prepare the queue in each selected tenant database before readiness. Component membership alone never enables tenant isolation: Job component checks apply only when deployment settings configure tenant storage. Without tenant configuration, ordinary SQLite/System Jobs must run without a tenant ID. Protected API authorization must bind its exact permission to the trusted Job execution context before enqueue; workers recheck the current subject/session, tenant, catalog, component and role rather than trusting a stored grant.
|
||||
|
||||
HTTP/REST and log-enabled CMD use `application::Session` as the single config,
|
||||
lifecycle, database, tenant and migration-pool owner. The fixed route descriptors
|
||||
only reference checked Models and auth contracts; SQL/DDL/role storage remain
|
||||
shared with the original backend. Configured database-only tenant storage is
|
||||
supported, not a reason to reject every root. Missing tenant identity/readiness
|
||||
still fails closed; `global` Models use control storage, other Models require the
|
||||
tenant physical database. A disabled component also denies Tenant Owner.
|
||||
|
||||
Tenant pool creation registers a manager entry with `ready=false` before awaiting
|
||||
schema preparation/marker validation. Interrupted preparation never becomes a
|
||||
usable pool, and retirement remains owned through `close_pool().await`. Migration
|
||||
pools register with the Session before their first prepare await. After child
|
||||
drain, close migration pools, tenant manager and platform pools on the same
|
||||
runtime; preserve the original fault and append cleanup causes rather than
|
||||
replacing it. Do not cache a failed or incomplete pool as ready.
|
||||
|
||||
`llvm_api_source` uses 65 SQLite roots with two tenant files and the unchanged
|
||||
allocation ledger; it asserts exact site roles, component denial, owner row
|
||||
filtering, concurrent identity isolation and POST rollback. Its opt-in PostgreSQL
|
||||
case shares `support/postgres::run_in_isolated_databases`, supplies only
|
||||
`config/setting.json.database.postgres_test`, and cleans one random control and
|
||||
two tenant databases. Neither cross-target objects nor prior Model-only PG
|
||||
success proves API/tenant execution. Commit/close failure injection and
|
||||
throughput/RSS remain separate acceptance work.
|
||||
|
||||
REST SQL snapshots may inspect HIR without deployment settings, but executable
|
||||
REST fixtures must use `check_with_settings` with a real site/provider binding
|
||||
and a checked identity App. Reuse one owned fixture for this setup, as in
|
||||
`rest_model`; missing auth metadata or generated authorization helpers must not
|
||||
be worked around by disabling permissions.
|
||||
|
||||
### Dependency Lock-Order Regression
|
||||
|
||||
The old bundled SQLite 3.51.1 had an ABBA VFS lock inversion: replacement connection creation (`findReusableFd`) acquired global → inode, while WAL connection close (`unixLock` → `unixIsSharingShmNode`) acquired inode → global. A cancelled stream correctly discarded its unknown connection; concurrent background close/replacement then hung runtime shutdown. Native stdout already contained a successful CMD response, so checking only SQL success or Scope completion missed this dependency defect. Repeated opt-in process-bounded stream/root execution and both blocking-worker stacks exposed the exact cycle; no timeout extension, reusable-unknown connection or checkpoint suppression is an acceptable workaround.
|
||||
|
||||
Keep the fixed dependency chain (`deadpool-sqlite` 0.14.0, locked `rusqlite` 0.40.2 / `libsqlite3-sys` 0.38.2, bundled SQLite 3.53.2), without editing registry cache or shipping a private SQLite fork. The author workspace requires Rust 1.95. `sqlite::value(column, ValueRef)` receives the real index in ordinary, bounded, owned-SQL and stream decoders; malformed Text retains `ErrorKind::Database` and the column diagnostic. Select the native early-close/repeated-root case and `sqlite_runtime` invalid-Text/replacement tests on dependency changes. An earlier successful run is not sufficient evidence against an observed intermittent shutdown hang.
|
||||
@@ -0,0 +1,47 @@
|
||||
# Directory Structure
|
||||
|
||||
- `crates/dever-core/src/lib.rs`: separate syntax `parse` and semantic `check` entry points.
|
||||
- `source.rs`: UTF-8 original-source ownership, sorted `.dever`/`.dever.md` loading, the single main/component/domain/role/topic classifier and byte-to-line/column conversion.
|
||||
- `markdown.rs`: CommonMark top-level literal `dever` and `typescript dever` fence extraction, explicit closure and complete-declaration validation.
|
||||
- `markdown/contract/`: `.dever.md` heading/list model, one-section-per-declaration mapping and checked package/type/function documentation contracts.
|
||||
- `diagnostic.rs`: source-located errors, related labels and rendering.
|
||||
- `lexer.rs` and `token.rs`: lexical rules and preserved numeric spellings.
|
||||
- `syntax.rs`: spanned syntax structures and operator precedence.
|
||||
- `parser.rs`: declarations, types, patterns and linear statements.
|
||||
- `parser/expression.rs`: expressions, postfix operations and collection construction, sharing the same parser cursor.
|
||||
- `check.rs` and `check/`: register all declarations before bodies; `check/layout.rs` owns application layout validation, `check/symbols.rs` owns logical App aliases and role access, and `check/api.rs` validates HTTP routes after signatures resolve. Names, types, linear flow, expressions/operators/value construction, collection handlers and symbolic clause coverage retain separate owners. Cycle checks include nested expressions, handler calls and type references, collapsing physical files in one domain for dependency-cycle purposes.
|
||||
- `types.rs` and `hir.rs`: nominal types, opaque Stream values, static handler signatures/targets, concrete typed expressions, normalized value-only clause domains and local slots. Only checking constructs executable HIR.
|
||||
- `wire.rs`: shared checked wire schema, policy identity and interned concrete type graph. `native/wire.rs` and `llvm/wire.rs` emit concrete backend-specific encoders/decoders; runtime `wire.rs` owns bounded duplicate-aware JSON parsing and exact raw JSON output. API dispatch, Adapter settings and Job codecs consume this owner, never maintain independent allowed-type registries.
|
||||
- `specialize.rs`: the single function-plus-handler specialization graph used by checking and native emission. It resolves forwarded handlers and rejects recursion after substitution.
|
||||
- `llvm.rs` and `llvm/`: typed HIR function/coroutine kernels with shared ownership/fault helpers. `network.rs`, `concurrency.rs` and `parallel.rs` own streams/TCP, task composition and worker packs. `http*.rs` lower protocol capabilities; `system.rs` owns system intrinsics and `render.rs` terminal faults. `application.rs` owns compiler-generated CMD/HTTP/Job roots; `api.rs` static route/auth/permission callbacks; `api_context.rs` auth/request/Cookie/Upload capabilities; `rest.rs` generated Model REST; `application_commands.rs` typed tenant management/CMD normalization. `jobs.rs` owns typed payload dispatch, enqueue, binding and clock adapters; `testing.rs` links case-local checked programs into one suite with indexed fault ownership. `wire.rs` owns concrete checked codecs, `ports.rs` selected Dever Adapter dispatch and Setting lifetime. `database*.rs`, `orm*.rs` and `transaction.rs` own persistence codecs, shared SQL/DDL and hidden transactions. Runtime `application::Session` owns config/lifecycle/database/tenant/migration/Job resources and closes after Scope drain within the same runtime. `external.rs` owns checked Worker calls and embedded resources; `process_entry.rs` wraps callable roots in process main. Private and managed Linux CLI compilation are connected; formal six-target delivery remains unfinished.
|
||||
- `contracts.rs` and `contracts/`: post-HIR facts/domain proofs, failure obligations, transitive effects/dependencies and structural suggestions. `api.rs` owns public snapshots and application exposure analysis; CLI `api.rs` owns explicit baseline file I/O.
|
||||
- `intrinsic.rs`: the single name-to-operation mapping for system primitives; checker and emitter exhaustively match this enum. Official result choice shapes are checked before emission and variants resolve by name, not declaration index.
|
||||
- `native.rs` and `native/`: concrete record/choice generation, direct functions and collection loops, source-located failures, generated API/CMD/Job application roots and owned temporary build artifacts. `native/build.rs` owns native compilation and the single shared Cargo runtime target; `native/build/cache.rs` owns exact immutable inputs and executable cache identity; `native/build/cleanup.rs` owns marker-protected temporary directories and bounded local cleanup. CLI code must not reproduce those filesystem rules. `native/api.rs` emits checked static route dispatch, auth scope and response conversion. No generic runtime Value or interpreter dispatch.
|
||||
- `check/jobs.rs` owns Job declarations, App payload contracts and static enqueue checking. `native/jobs.rs` owns concrete payload dispatch and queue initialization, reusing the shared wire and database owners. `specialize::execution_reachable` adds Job handlers as independent execution roots without adding synchronous call edges.
|
||||
- `crates/dever-runtime/src/`: shared Decimal128/checked Int rules, ordered copy-on-write collections, Bytes, resource handles, pull-based Stream ownership, rendering and stdout. Native programs link this same runtime library; generated Rust is not a duplicate runtime implementation.
|
||||
- `crates/dever-backend-bridge/`: private LLVM/LLD object/link owner. Its safe API uses explicit six-target selection; only private `ffi.rs` may contain authorized unsafe calls. LLD calls are serialized; failed crash recovery terminates the private compiler worker before any cleanup/re-entry. Outputs are exclusively claimed before linking. The opt-in `embedded` authoring build requires a private LLVM 18 SDK; independent `runtime-abi` exports the canonical C runtime boundary without that SDK. HIR lowering remains in core, and neither build mode is a formal released compiler/target pack.
|
||||
- Runtime `async_stream.rs` owns the shared async producer/cursor and closure; `net.rs` owns Tokio Socket/Listener, duplex I/O, deadlines and resource cancellation. `task.rs` owns the single runtime, structured tasks/Group and bounded async stream dispatch. File resources remain in `resource.rs`.
|
||||
- Runtime `job.rs` owns transactional enqueue and trusted execution/component checks, `job/store.rs` owns private queue/cursor schema and fenced state changes, and `job/worker.rs` owns bounded production/test dispatch. `job/resources.rs` owns invocation-local bindings, clock, initialization and tenant queue migrations. `auth::authorize_permission` binds the checked permission after authorization for both backends. `cron.rs` supplies the shared UTC parser. `lifecycle.rs` owns service-mode gating and signal-driven stop/drain; all work remains under the existing structured task owner.
|
||||
- Runtime `component.rs` owns external Adapter resolution, bounded admission, Worker protocol, restart and shutdown. `component/command.rs` reuses that owner for one ordinary process per call, with binary I/O and caller-scope completion; `dever.process.Output` is its checked result, never an OS process handle. `check/ports.rs` synthesizes external implementations from the existing Port owner; `native/ports.rs` emits concrete codecs and dispatch. Do not add per-language runtime supervisors or expose protocol JSON to App code. CLI `workers/command.rs` packages domain-owned ELF/resources and reuses `workers/elf.rs` with native Python library validation.
|
||||
- Runtime `external.rs` owns verified embedded external-resource extraction below the executable's private `data/cache/lib/<digest>` root. It is a packager primitive, not a registry client or shared-cache writer.
|
||||
- Runtime `http/` owns Hyper HTTP/1, bounded messages, streaming reply/session ownership and connection deadlines. Runtime `api.rs` adapts typed inputs/JSON envelopes and reuses HTTP serving. `sse.rs` owns event encoding; `websocket.rs` reuses tokio-tungstenite. TCP and WebSocket share private `endpoint.rs` for close/wakeup ownership. Native `http.rs` converts checked source records and statically bound handlers; no application business code belongs in the runtime.
|
||||
- Runtime `http/client.rs` owns fixed-origin pooled requests and leases; `client_driver.rs` owns physical/request capacity, idle expiry and the one structured pool driver; `client_body.rs` owns pull-driven upload/download. `tls.rs` owns shared rustls configurations; the private `transport.rs` enum delegates TCP/TLS I/O for HTTP and WebSocket.
|
||||
- `library/**/*.dever`: bundled official packages processed by the same parser/checker as user source. Compiler-assigned source origins, not filenames, authorize reserved namespaces; `SourceMap::files()` remains user-only while `get()` also resolves bundled diagnostic spans.
|
||||
- `format.rs` and `format/`: shared AST-driven declaration layout, preserved per-block comment attachment and syntax-only canonical formatting. Markdown replaces only program-block interiors.
|
||||
- `reference.rs` and `reference/`: opt-in HIR semantic evaluator and differential output/fault representation. Never link into production native programs.
|
||||
- `native/liveness.rs` and `native/updates.rs`: clause-local last-use analysis, safe record alias coalescing and scalar snapshots before aggregate consumption. `types.rs` caches nominal comparison/move/transfer properties after cycle validation.
|
||||
- `native/sequences.rs`: conservative total-handler proof and eligible collection fusion; collection emission shares lazy Text splitting with the runtime.
|
||||
- `contracts/dependencies.rs`: shared dependency order and reverse-caller work scheduling for contract summaries.
|
||||
- `check/cache.rs`: per-thread last-result cache with complete source identity; bundled AST caching belongs to `lib.rs`/`source.rs`.
|
||||
- `native/build/cache.rs`: native artifact identity, integrity validation and atomic publication. `native/build.rs` owns current runtime freshness, snapshots and isolated executable lifetimes.
|
||||
- `crates/dever-cli/src/main.rs` and `format.rs`: argument parsing, source loading, direct execution of the owned native program, local clean command adaptation and prepared/atomic source formatting writes. `test_runner.rs` is the single owner of application-test ordering, isolated native processes, captured output and temporary project cleanup.
|
||||
- CLI binary-private `compile.rs` owns LLVM/object/LLD orchestration; `worker.rs` is the service-owned source compiler. Shared `toolchain/runtime_pack.rs` validates packs/snapshots without bridge imports; `compilation.rs` owns canonical input and staging validation, while `service/` owns authenticated transport and child supervision. Reuse core `native::compile_artifact` for private builds and `NativeProgram` for caller-owned received executables. `toolchain/packaging.rs` owns deterministic native release assembly/signing from explicit author inputs; `sdk/native-release.rs` only adapts its private author CLI. Root `runtime-pack` profile and `sdk/native-release.md` own the four-archive build recipe. `test/dever-cli-tests/support/llvm_inputs.rs` holds explicit fixture SDK inputs, and `support/daemon.rs` holds the shared owned-daemon lifecycle; neither is a production input discovery mechanism.
|
||||
- `library/dever/json/`: ordinary Dever JSON algorithm. `library/dever/http.dever`, `sse.dever`, `websocket.dever` and `tls.dever`: public protocol types/helpers and the native bridge; no duplicate source protocol stack. `task.dever` exposes runtime configuration, sleep and periodic streams.
|
||||
- `examples/cms/dever/` and `examples/cms/md/`: maintained CMS projects with equal declarations, schema, settings, business behavior and application tests. Generated API/CMD roots replace `module/main.dever`; `api/admin/**` and `api/front/**` are assigned by `sites.<key>.path`. Both projects keep matching `test/<component>/<domain>/<topic>` cases and `config/setting.json`.
|
||||
- `examples/old/`: archived pre-role-layout language and regression material. It is not a current application root and must not be presented as a directory or visibility template. Formal tests remain under root `test/`.
|
||||
- `examples/README.md`: current CMS commands and the archive boundary. Historical rejection fixtures keep their original diagnostics only when invoked by dedicated compiler-level tests, not as current application projects.
|
||||
- `test/dever-tests/tests/`: permanent focused tests. The canonical full-language syntax example is read directly from the PRD so a copied fixture cannot drift.
|
||||
|
||||
Core foundations, formatting, optional reference evaluation, official library follow-through and the SQLite/PostgreSQL static ORM have concrete implementations. Compiler distribution and self-hosting remain unshipped; validation evidence belongs to the active task, not this directory map. Compiler phase APIs are bootstrap internals, not an alternative program source format.
|
||||
|
||||
Use one owning implementation for each language rule. Do not introduce a generic parser framework, service layer or backend registry for a single concrete implementation.
|
||||
@@ -0,0 +1,88 @@
|
||||
# Error Handling
|
||||
|
||||
Source loading returns path-specific `LoadError` values because unavailable or invalid UTF-8 files do not have a valid source span. Parsed-source errors use `Diagnostic` with a stable code, primary span and optional related labels.
|
||||
|
||||
- `Span` is a half-open UTF-8 byte range owned by one `SourceId`.
|
||||
- Display positions are one-based lines and Unicode scalar columns. LF, CRLF and CR line endings use the same source model.
|
||||
- Render the original source line with its explicit line/column location. Do not guess terminal display width with one ASCII space per Unicode scalar.
|
||||
- Preserve source literal spelling through parsing; numeric representability belongs to checking.
|
||||
- Reject invalid source with a diagnostic. Never turn an unsupported construct into a successful default AST.
|
||||
- Parser recovery synchronizes only at balanced body boundaries or top-level newlines.
|
||||
- Both parser call depth and resulting expression tree depth are bounded by `MAX_SYNTAX_DEPTH`. Long iterative operator/postfix chains must not bypass the limit and overflow during recursive AST destruction.
|
||||
- I/O and syntax errors are explicit results. `expect` is reserved for compiler-owned invariants or infallible String formatting, not malformed source.
|
||||
- Contract structural-depth rejection is fatal to that analysis pass: `contracts::check` returns `Err(C015)` before producing summaries. Both the main checker and Port fake reanalysis must stop before consumers read those summaries; ordinary contract diagnostics still aggregate after summaries are populated. The 256-layer nominal-type regression must report C015 rather than panic in a later phase.
|
||||
|
||||
Current codes: `L001` unexpected character, `L003` excluded keyword, `L004` malformed number, `L005` unterminated Text, `L006` invalid escape, `P001` expected syntax, `P002` invalid statement/assignment, `P009` excessive syntax nesting.
|
||||
|
||||
Markdown source codes: M001 no top-level program block, M002 unclosed program fence, M003 declaration spans multiple blocks, M004 invalid package documentation structure, M005 invalid declaration documentation structure, M006 documented package/API/declaration name mismatch, M007 function input mismatch, M008 function output or type-shape mismatch. Semantic documentation mismatches use the Markdown value as the primary span and add the corresponding Dever declaration as a related label. Tokens and diagnostics refer to original `.dever.md` byte positions, including Unicode columns and source newline conventions. Markdown cannot introduce a synthetic filename or bypass normal language diagnostics.
|
||||
|
||||
Semantic codes: `C002` duplicate declarations/bindings/known Map keys, `C003` package path or reserved namespace, `C004` unresolved names/signatures or uninitialized locals, `C005` type/arity/construction/primitive-contract errors, `C006` private cross-package access, `C007` function/type/package cycles, `C008` statically known numeric faults, `C009` clause overlap/coverage/reachability, `C010` naming format. `W001` reports unused/overwritten writes without rejecting the program. Diagnostics sort by source ID and byte span; check unused declarations before native reachability pruning.
|
||||
|
||||
Standard output errors propagate to the native entry as a failure containing the `.dever` call location. The process exits nonzero even if stderr itself cannot be written. CLI/toolchain failures are explicit command errors; `check` never invokes rustc. Native temporary directories have exclusive ownership and are cleaned on compilation failure or after use. Cleanup failure cannot mask the primary result; explicit build output never overwrites an existing file.
|
||||
|
||||
External Adapter business errors are decoded only when their exact identity and payload match a Port-declared error variant. Startup, schema/handshake mismatch, invalid or out-of-order protocol messages, timeout, child crash and undeclared error identities are runtime faults at the Port call site, never synthetic business variants. Protocol diagnostics contain metadata only and must not echo setting secrets or call payloads. A protocol fault terminates the Worker channel and closes pending calls. Generated entry cleanup runs after success, business failure and runtime fault; shutdown/database cleanup augments rather than replaces the primary error.
|
||||
|
||||
Embedded external resources are never executed before their relative path, regular-file type and SHA-256 digest verify. Extraction rejects symlinks, path escape, duplicate entries, truncated staging and invalid bundle identity; an existing or concurrently published directory is reusable only after a full recheck. Missing artifacts remain an explicit offline preparation error with the corresponding `dever lib add/update` command.
|
||||
|
||||
## Core Execution Contract
|
||||
|
||||
Compiler contract diagnostics add C011 (unproved range), C012 (unhandled failure), C013 (pure/clause contract), C014 (Model/transaction contract) and C015 (bounded proof work). W002–W005 and W007 are suggestions; W001 remains the owner of unused/overwritten writes. Details and counterexamples are in [Compiler Contracts](./compiler-contracts.md).
|
||||
|
||||
### 1. Scope / Trigger
|
||||
|
||||
Applies when changing checker-to-HIR-to-native contracts, bundled packages or runtime operations.
|
||||
|
||||
### 2. Signatures
|
||||
|
||||
- `check(&SourceMap) -> Result<Program, Vec<Diagnostic>>`; successful programs expose `warnings()` separately.
|
||||
- `native::compile_project(&Program, &SourceMap, rustc, profile) -> Result<NativeProgram, String>` generates the application entry from API/CMD/Job declarations; compiler-internal library fixtures may still use `native::compile` with an explicitly selected zero-input entry.
|
||||
- `dever.io.read(file, limit) -> ReadState`; `dever.net.read(socket, limit)` returns the same state. ReadResult is the capture choice.
|
||||
- `dever.io.chunks(file, limit) -> Stream<ReadEvent>`; `dever.net.chunks(socket, limit) -> AsyncStream<ReadEvent>`, and `dever.net.connections(listener) -> AsyncStream<ConnectResult>`. ReadStreamResult choices capture creation failures.
|
||||
- `dever.http.send -> Response`; ResponseResult captures recoverable errors. `serve` has zero outputs and propagates configuration/accept faults. Handler faults are logged at the request boundary and return 500 or abort an already-started response; later requests remain available. Peer protocol/I/O errors close only the affected connection; body/handler timeouts return 408/504 before closing, incomplete-header timeout closes directly.
|
||||
|
||||
### 3. Contracts
|
||||
|
||||
`ReadState` is `Read(bytes: Bytes)` or `End`; `ReadResult` is `Done(state: ReadState)` or `Failed(message: Text)`. Ordinary calls return success values; explicit result captures convert the inferred failure set into choice values. Stream reads yield source-defined `ReadEvent` values and produce at most one terminal `Failed`; normal EOF produces no synthetic item. File creation uses create-new; resources share identity and close state across aliases, unlike ordinary value-semantic records/collections. Read limits and socket timeouts are positive Ints; reads may be short. Stream creation rejects a non-positive limit or already-closed source before returning a Stream. Core `close(stream)` is idempotent.
|
||||
|
||||
Handler parameters are compile-time bindings, never runtime values or clause axes. Every value type nested in a handler signature uses the same generic arity and Map-key validation as an ordinary function boundary. Handler signature, output-name and specialization-cycle errors are source diagnostics before build. `reduce_until` applies the current item, then stops without pulling another item.
|
||||
|
||||
`result(call)` changes business-failure representation, never scheduling. Captured calls share ordinary-call suspension/liveness and inline-blocking checks. A synchronous capture must remain legal with synchronous resources; capturing a suspending call must diagnose non-transferable live locals at the capture span, and capturing a blocking call must not bypass an async caller's worker boundary. Keep both negative paths and the synchronous positive control in `structured_concurrency`.
|
||||
|
||||
Build/run use the local bootstrap toolchain (`RUSTC`, optional `CARGO`) and locked cached dependencies with `--offline`; missing tools/dependencies fail explicitly. Generated binaries link the runtime, not the checker or a generic evaluator, and execute without the build toolchain.
|
||||
|
||||
All runtime feature combinations build through one Cargo target. Only artifacts reported for the current Cargo graph enter an immutable input view; a missing, symbolic-link, changed or incomplete input fails before generated code is linked. Native temporary directories carry a versioned owner marker and are removed immediately by RAII. Recovery cleanup deletes only current-user, real directories with an exact Dever name/marker older than 24 hours; unknown, fresh, active or symbolic-link paths are skipped. Cleanup failure may be returned by explicit `clean`, but automatic cleanup never replaces the current compile or run result.
|
||||
|
||||
### 4. Validation & Error Matrix
|
||||
|
||||
| Condition | Result |
|
||||
| --- | --- |
|
||||
| Incomplete record/output, type drift, illegal key | Source diagnostic before build |
|
||||
| Invalid generic or Map key nested in a handler signature | C005 before build |
|
||||
| Clause overlap, gap or empty complement | C009; overlap labels both clauses |
|
||||
| Known Int/Decimal overflow or zero divisor | C008 before build |
|
||||
| Dynamic Int/Decimal fault or stdout failure | Source-located native failure and nonzero exit |
|
||||
| Invalid bytes, file/socket error, closed resource | Explicit Failed choice |
|
||||
| Invalid stream limit or closed source at creation | Explicit Failed choice; no Stream returned |
|
||||
| Stream producer terminal error | One Failed item, then end; producer released immediately |
|
||||
| timeout/race receives a program fault | Propagate after structured cleanup; never invoke timeout fallback for a fault |
|
||||
| timeout expires | Stop and drain the task before its signature-matched fallback runs |
|
||||
| Pool closed, request/stream deadline or TLS verification failure | Explicit Failed; failed/incomplete connections are not reused |
|
||||
| Stream(Channel/List/Bytes) contains error choices | Preserve their failure obligations; constructing or closing the stream does not handle them |
|
||||
| HTTP listener explicitly closed | Stop accepting, drain active responses; application timeout may stop remaining work |
|
||||
| Invalid/reused build output path | Command error; existing file preserved |
|
||||
| Runtime input view is incomplete or changed | Build error; never scan unrelated historical artifacts |
|
||||
| Clean candidate has no valid owner marker, is fresh, active or a symlink | Preserve it unchanged |
|
||||
|
||||
### 5. Good / Base / Bad Cases
|
||||
|
||||
Good: `Text` plus `null` clauses infer `Text?`; `first([null])` in an `Int?` output context remains `Int?`. Base: hello prints one UTF-8 line. Bad: `Bool` plus `true` clauses overlap; repeated constant or zero-payload choice Map keys are rejected.
|
||||
|
||||
### 6. Required Tests
|
||||
|
||||
Use `core_semantics` for checking/rejection, `core_native` for exact outputs/evaluation order/native failure locations, `runtime_foundations` for decimal precision/COW/resource closure, and `hello_native` for CLI-backend regressions. Stream tests must assert that mapping preserves a terminal item as terminal, releases its producer on that pull, and that official wrappers map invalid creation to `ReadStreamResult.Failed`. Run only affected targets and explicitly announce owned temporary-file/loopback checks.
|
||||
|
||||
### 7. Wrong vs Correct
|
||||
|
||||
Wrong: select the first matching source clause, append anonymous standard files to a cloned SourceMap, emit an untyped Value interpreter, reorder record expressions by field name, or implement Stream mapping through an API that erases `Pull::Last`.
|
||||
|
||||
Correct: prove disjoint exhaustive domains before emission, keep bundled source origins retrievable by the caller's SourceMap, emit concrete types/direct handlers, evaluate constructor fields in source order, and preserve `Item`/`Last`/`End` through Stream adapters.
|
||||
@@ -0,0 +1,23 @@
|
||||
# Compiler Development Guidelines
|
||||
|
||||
This repository implements the Dever language compiler in Rust. It is not a Go Dever application or a database-backed service.
|
||||
|
||||
## Pre-Development Checklist
|
||||
|
||||
1. Read the current task PRD, design and implementation progress.
|
||||
2. Read [Directory Structure](./directory-structure.md), [Error Handling](./error-handling.md) and [Quality Guidelines](./quality-guidelines.md).
|
||||
For CLI, formatter, native optimization or official packages, also read [Toolchain and Library Contracts](./toolchain-and-library.md), including signed ecosystem packs and bounded binary resource attachments.
|
||||
For language contracts and API boundaries, read [Compiler Contracts](./compiler-contracts.md).
|
||||
For LLVM typed lowering or runtime C ABI work, read both contracts; distinguish kernel/object evidence from complete application semantics and formal target runtime packs.
|
||||
For Model, ORM, database configuration or runtime connections, also read [Database Guidelines](./database-guidelines.md).
|
||||
For Dever logging or HTTP request-boundary records, read [Logging Guidelines](./logging-guidelines.md).
|
||||
3. Inspect affected compiler phases and existing tests before adding another rule or helper.
|
||||
|
||||
## Quality Check
|
||||
|
||||
- Check the implemented milestone only; syntax success must not imply semantic validity.
|
||||
- Run the focused checks listed in the quality guide.
|
||||
- Keep all tests under repository-root `test/`.
|
||||
- Preserve unrelated user changes and do not create Git commits without authorization.
|
||||
|
||||
Database behavior is a compiler/runtime contract only within the static ORM boundary documented in [Database Guidelines](./database-guidelines.md). Shared logging and native exit-flush contracts are documented in [Logging Guidelines](./logging-guidelines.md).
|
||||
@@ -0,0 +1,46 @@
|
||||
# Logging Guidelines
|
||||
|
||||
`library/dever/log.dever` provides `debug`, `info`, `warn` and `error` with a
|
||||
`Text` message and `Map<Text, Text>` fields. Native emission routes these calls
|
||||
to `crates/dever-runtime/src/log.rs`; do not add a second logger in the CLI or
|
||||
generated program.
|
||||
|
||||
The runtime writes one JSON record per line to stderr, with `level`, `message`
|
||||
and `fields`. The bounded writer may drop debug/info events under pressure and
|
||||
reports the count in `dropped_events` on a later record; warn/error events wait
|
||||
for room, and error records flush immediately. Native program exit flushes the
|
||||
writer. Keep log messages and fields bounded instead of logging unbounded
|
||||
payloads.
|
||||
|
||||
LLVM applications with reachable log calls use the same invocation-owned
|
||||
`application::Session` and load `config/setting.json` once, even for log-only CMD.
|
||||
Configured level filtering and request identity fields come from that owner;
|
||||
Scope drain precedes pool close and log flush. Pure scalar CMD/kernel entries
|
||||
do not acquire an API owner. `llvm_api_source` tests warn-level filtering and
|
||||
65 flush-before-return invocations; do not infer configured logging support
|
||||
from the standalone kernel logger alone.
|
||||
|
||||
The single writer uses one preallocated 128-command queue; debug/info overflow
|
||||
and warn/error backpressure remain distinct. Flush callers are serialized onto
|
||||
one persistent acknowledgement state. Release preceding records and flush the
|
||||
output before acknowledgement; writer exit/panic clears queued owners and wakes
|
||||
all waiters. Do not add per-flush channels or a second queue. A pure-flush native
|
||||
ledger exposed lazy mpsc receiver waiter allocations after acknowledgement, so
|
||||
ordinary first-flush success alone is not a stable exit/allocation boundary.
|
||||
Keep the original strict ledger, not sleeps, extra warmup or relaxed counts.
|
||||
Root test/runtime_logging.rs covers FIFO/drop counts, backpressure, concurrent
|
||||
flushes and writer failure; native_runtime_link covers the actual ABI barrier.
|
||||
|
||||
`crates/dever-runtime/src/http/mod.rs` records an HTTP request-boundary error
|
||||
with `boundary`, `request_id`, `protocol` and `target`. Do not include secrets,
|
||||
credentials or full request/response bodies in fields. The source-to-stderr
|
||||
contract is covered by `test/dever-tests/tests/logging.rs`; extend that focused
|
||||
test when changing the public logging surface or output format.
|
||||
|
||||
Secret and complete Models containing private fields are non-observable types,
|
||||
including nested containers and related Models. Compiler entry/error/API/assertion
|
||||
render boundaries must reject them before native emission; log arguments remain
|
||||
Text and Map<Text, Text>. Runtime Secret Debug emits one fixed redacted marker and
|
||||
never reveals bytes. Do not add a generic conversion or derive Render for a
|
||||
sensitive nominal type. Ordinary record private fields remain an access rule,
|
||||
not a promise of cryptographic secrecy.
|
||||
@@ -0,0 +1,120 @@
|
||||
# Quality Guidelines
|
||||
|
||||
Workspace-owned Rust forbids unsafe code. Decimal128 and insertion-ordered Map reuse the pinned `dec` and `indexmap` dependencies in `Cargo.lock`; do not reimplement these domains. Keep source loading, parsing, semantic checking, native emission, runtime and CLI separate. Parser success is not semantic success.
|
||||
|
||||
Root dev/test profiles keep `debug = 0` and `incremental = false` so independent focused test binaries do not recreate multi-gigabyte debuginfo and incremental object trees. Release keeps fat LTO and one codegen unit. Native runtime feature combinations share `target/native-runtime/cargo`; exact per-build inputs live under `target/native-runtime/inputs`. Do not restore one complete Cargo target per feature combination.
|
||||
|
||||
Native ABI release archives use the root `runtime-pack` profile and one shared `target/native-runtime-abi` Cargo target for all four driver combinations. Save each completed archive before the next feature build. Preserve link symbols, keep `embedded` disabled, and distinguish deterministic pack assembly from cross-host source reproduction; the author workflow and assertion points live in toolchain-and-library.md and sdk/native-release.md.
|
||||
|
||||
Tests exercise source contracts and observable AST structure, not private helper implementations. Include the PRD's complete package, precedence and associativity, Unicode and line positions, lexical failures, excluded syntax, recovery, structural limits and directory loading. Filesystem tests create and remove only their own temporary directories; source loading never follows symbolic links.
|
||||
|
||||
Explicit LLVM Worker acceptance consumes the prepared immutable `target/native-release-inputs/component-fixture`, with recorded source and binary digests. Cargo's `target/debug/component-fixture` is a mutable build output: feature unification can change its bytes while running the workspace matrix. Ordinary Worker tests use their Cargo-provided executable; never bypass the CI input-change gate to treat a mutable output as a fixed author input.
|
||||
|
||||
Explicit single-libtest acceptance uses `run_command(..., single_harness=True)` and requires the final parent summary to contain exactly one passed test, no failures and no ignored tests. Nested probe summaries must not be added to that parent count; ordinary Cargo multi-harness commands retain aggregate counting. A zero-test parent, failed parent or missing summary is not a pass even if a child probe succeeded. `test/ci/test_gate.py` owns these rejection cases.
|
||||
|
||||
The shared `test/native-runtime-abi/managed-driver.c` runs one warmup and 64 repeated entries on joined host threads, then checks strict allocation balance after each join/TLS destruction. Rust's bounded-channel producer may first allocate its 48-byte thread-local Context only after warmup when the queue fills; measuring before the host thread exits misreports that cache as a leak. Preserve process initialization before any thread, original result checks and strict zero allocation delta; do not add retries, random warmups or an allowed-leak threshold. This measures entry-plus-host-thread cleanup, not unbounded TLS growth on a long-lived embedding thread. `test/ci/test_driver.py` compiles the shared driver with the explicit existing author C tool and verifies late TLS cleanup succeeds while intentional per-entry leaks and changed results fail. Suite probes include the same driver instead of copying its ledger.
|
||||
|
||||
Markdown source uses pinned pulldown-cmark in the compiler only. Preserve the `.dever` path and existing semantic/contract checks; shared Markdown extraction/formatting rules and the focused `markdown_source` target are specified in [Toolchain And Official Library Contracts](toolchain-and-library.md).
|
||||
|
||||
Focused checks for the current milestone:
|
||||
|
||||
```bash
|
||||
cargo fmt --all --check
|
||||
cargo clippy --offline -p dever-core -p dever-tests --test syntax_and_format -- -D warnings
|
||||
cargo test --offline -p dever-tests --test syntax_and_format
|
||||
git diff --check
|
||||
```
|
||||
|
||||
For core language changes, select the affected targets and announce bounded native/resource checks before running:
|
||||
|
||||
```bash
|
||||
cargo test --offline -p dever-tests --test hello_native
|
||||
cargo test --offline -p dever-tests --test core_semantics
|
||||
cargo test --offline -p dever-tests --test core_native
|
||||
cargo test --offline -p dever-tests --test runtime_foundations
|
||||
cargo test --offline -p dever-tests --test syntax_and_format
|
||||
cargo clippy --offline -p dever-core -p dever-runtime -p dever-tests --test hello_native -- -D warnings
|
||||
cargo clippy --offline -p dever-cli --bin dever -- -D warnings
|
||||
```
|
||||
|
||||
The hello target retains stdout, value mutation, package boundaries, missing toolchain and artifact cleanup regressions. Core targets cover the full PRD sample, static rejection, clause permutation, concrete native output, COW ordering, nullable flattening and source-located faults. Foundation-extension cases additionally cover contextual handler syntax, forwarding/specialization recursion, value-only clause domains, List/Bytes/Stream traversal, early stop/resume and one terminal stream failure. Resource tests create only their own temporary files and loopback sockets on port 0 with bounded reads/timeouts. CLI changes also require the actual check/run/build command. These are focused compiler checks, not an integration environment or workspace-wide test run.
|
||||
|
||||
Parentheses must not affect context inference: use the shared Group-unwrapping helper for nulls, empty literals and named handlers. Handler signature types must reuse ordinary boundary validation, including generic arity and nested Map-key rules. Stream adapters must preserve `Pull::Last` rather than turning it into an ordinary item. Never retype an existing Decimal/Int local as Float, even after constant propagation. Float zero patterns match both signs, so substituting the pattern constant for the actual input changes division semantics. Preserve source field and Map pair evaluation order in generated code; declaration order controls layout/display only.
|
||||
|
||||
Use an explicit installed toolchain or a command-scoped PATH if Cargo is not discoverable. Do not change global shell configuration or replace the existing `dever` command.
|
||||
|
||||
Do not run workspace-wide tests, npm builds, service checks, native compilation benchmarks or integration environments by default. Focused syntax tests do not establish type correctness, formatting round trips or runtime equivalence. Record those later-phase checks as not run.
|
||||
|
||||
Explicit Linux no-host release checks share `native_release::acceptance::isolated_os_command`: a real mount-namespace root with only declared OS files, proc and synthetic devices. Plain chroot blocks the nested user namespace required by Worker sandboxing. The explicitly built static `native-acceptance-init` replaces bwrap's masked proc with a complete private proc inside the already pivoted outer namespace; otherwise the kernel rejects the nested proc mount. Preserve the outer fixture's original capabilities and the product's inner capability removal; do not substitute a host proc bind. Verify the final standalone executable after removing source and machine stores; an earlier pre-sandbox artifact or managed-only pass is insufficient.
|
||||
|
||||
Installed daemon and launcher checks must share that owned PID/user/mount namespace so kernel peer credentials remain visible. Use the owned bwrap PID report, verify its live process and root inode, then enter it explicitly for clients; never relax production peer authentication to accommodate sibling PID namespaces. Pin the namespace init with a pidfd and wait for namespace exit and daemon-lock release before starting the next daemon. Run real distinct-UID checks separately because a single-UID test user namespace cannot represent multiple host identities.
|
||||
|
||||
Toolchain/library changes select `formatter`, `standard_library`, `native_ownership`, `concurrency`, `json_library` or `http_library`. The optional `reference` feature enables only `differential`; production CLI feature resolution must exclude it. HTTP tests independently assert wire bytes and malformed framing, with bounded loopback peers. Formatter changes also exercise the actual `fmt`/`--check` CLI and failure-before-rewrite behavior.
|
||||
|
||||
Configured concurrency and reusable TLS clients select `async_composition`, `pooled_network` and `pooled_library`. Assert task cleanup even before first poll, complete-body connection reuse versus abandoned-body close, bounded physical connections including idle entries, stream deadlines, verified CA/name failures, WSS and graceful shutdown. Check real native source conversion alongside runtime peers; include compile rejection for mismatched context/output signatures and unhandled stream failures. Certificates and test private keys belong under root `test/`; peers use only owned loopback ports.
|
||||
|
||||
Runtime root entries pin the caller Future once before forwarding it through cleanup adapters and task-local wrappers. Job execution scopes also pin before constructing their async wrapper; identity initialization and task-local entry remain lazy. Preserve borrowed and non-Send roots, concrete errors, component-close/child-drain/cleanup order, and nested-entry rejection. Exercise `runtime_entry_cleanup` and all-feature `durable_jobs`, including signal/transaction shutdown: default-feature success alone misses the larger combined-driver Future layout. Do not increase thread stacks to mask wrapper-induced stack overflow.
|
||||
|
||||
Standalone C fixture drivers must acquire the existing process initialization before starting kernel/runtime threads, just like generated process main. The shared `managed-driver.c` also owns initialization for the indexed suite driver. Callable roots deliberately preserve an embedding host's signal policy; do not modify runtime network leaves or test subprocess inheritance to hide a missing driver initialization. Keep the direct SIGPIPE ABI assertion and real TLS upload/download allocation-ledger regression.
|
||||
|
||||
The ignored `performance` target runs only for an explicitly announced bounded performance check. Match Decimal initial representation as well as numeric value; prepare parsing outside the timed region. Five native computation kernels cover Int/Float/List/Decimal/Map with exact results and a 1.15 baseline ratio budget. Alternate a fixed 25 independent processes per side, discard each process's warmup, retain every process's nine raw samples, then compare the medians of all 225 samples per side. The same ELF has exhibited clustered timing modes even with CPU affinity; five process groups can accidentally favor one side. More process groups improve coverage without proving host noise is eliminated. Never selectively omit groups or add samples until a failing ratio passes. Production and baseline share `native::OPTIMIZATION_ARGS`, including one codegen unit so generic collection inlining does not depend on generated program size. Record compilation separately and report CPU/environment without inferring HTTP throughput or whole-language performance.
|
||||
|
||||
## Standalone Runtime Benchmarks
|
||||
|
||||
### 1. Scope / Trigger
|
||||
|
||||
Changes to root `test/performance/` and the `network_bench`, `async_bench`, `live_bench` Rust examples. These explicit opt-in benchmarks measure owned binaries/loopback peers, never existing services or default test execution. Operational commands and limitations live in `test/performance/README.md`.
|
||||
|
||||
### 2. Signatures
|
||||
|
||||
`python3 test/performance/run.py build --output <new-dir>` compiles fixtures and records binary fingerprints/configuration. `--selection protocols` selects runtime/HTTP/live and three explicitly prepared Rust peers; `profiles` selects four runtime profiles; `cms-profiles` also builds both CMS sources. Every Dever fixture uses the public LLVM project build and CMD dispatch, never the removed native Rust fixture builder. `run --artifacts <dir> --output <new-dir> --suite memory|async|http|live|http2|http2-recovery|all` executes only prepared binaries. The HTTP peer CLI is `network_bench load h1|h2 URL PATH RATE SECONDS PHYSICAL_CONNECTIONS STREAMS_PER_CONNECTION TIMEOUT_MS REUSE CA`; H1 requires one stream per connection, while H2 requires connection reuse. Service modes are runtime-http/runtime-https/hyper-http/hyper-https plus runtime-http2/runtime-https2/hyper-http2/hyper-https2. Async comparison uses `async_bench root|worker WORKERS TASK_CAPACITY ITERATIONS`. Long connections use `live_bench PROTOCOL URL CONNECTIONS CYCLES HOLD_MS TIMEOUT_MS`, selected by `--connection-tiers`, `--cycles` and `--duration` in the runner.
|
||||
|
||||
For `run`, `--server-cpu <id>` optionally binds the server to one CPU; `--client-cpus <id,id,...>` optionally binds HTTP/1, HTTP/2 and live clients to a CPU set, and `--client-workers <positive-int>` defaults to 1. `Process(cpus=...)` changes only its owned child before exec. An omitted client set inherits the runner's allowed CPUs. The report records `arguments.client_cpus`, `arguments.client_workers` and the effective `environment.client_cpus`/`client_workers`.
|
||||
|
||||
### 3. Contracts
|
||||
|
||||
Servers publish `READY|port` after preparing shared payloads. JSON counters satisfy `scheduled=sent+dropped`, `sent=completed=succeeded+errors`, and `dropped=dropped_capacity+dispatch_expired`. `latency_ms` measures planned-to-successful-completion; `request_latency_ms` measures dispatched-to-successful-completion; dispatch lag includes completed failures. Empty histograms are null and success histograms have exactly succeeded samples. Record raw rounds, successful throughput, load-generator CPU/lag, config and limits; never average p99 values into a global p99. HTTP/1, HTTP/2 and live clients share explicit `--client-workers`/`--client-cpus` configuration; record both in the report. Python stages each peer beside an owned config/setting.json with a benchmark object; peers read it relative to their executable and never use application environment variables. The manifest records setting-json-v1 and build checks the peer's check-config output before accepting it. Server affinity remains one CPU, distinct from the specified client CPU set.
|
||||
|
||||
HTTP/2 records `physical_connections`, `streams_per_connection`, their `request_slots` product and `successful_handshakes`. Preconnect the exact physical connection count before timing, share cloned senders across stream slots, and do not retry, replace or expand connections during measurement. H2 cleartext is prior knowledge; TLS advertises and requires only ALPN `h2`. Runtime and raw Hyper baselines use the same stream/window/header/frame/send-buffer limits. After each warmup, case or recovery cycle, close clients, wait the fixed settle interval and require service FDs to return to the READY baseline; preserve the RSS sequence without treating allocator retention alone as a leak. Any HTTP/2 request error or cgroup OOM event fails the run after completed case data is persisted; capacity drops remain saturation evidence rather than a pass/fail threshold.
|
||||
|
||||
Manifest `application_configuration=llvm-cmd-v1` is mandatory alongside `setting-json-v1`; reject older application artifacts before launch. Each runtime/http/live project contains its group's CMD declarations, so per-command aliases share the same multi-command binary and recorded identity. `fixture_command` derives the qualified CMD from the same `ENTRIES` used for building. Live settings belong beside each owned executable; ORM keeps its separate `benchmark.command.run` entry. LLVM O2/LLD and the exact prebuilt runtime archives determine Dever output. Rust peers are separately prepared with Cargo release, copied and fingerprinted. Do not claim identical optimizers or scheduling: Dever uses production defaults, while explicit workers/capacity affect Rust peers. Both retain checked Int arithmetic and structured run/wait; root/worker modes differ by an untimed outer structured task. Sampled /proc context-switch totals count currently alive threads and can decrease as threads exit; use perf for complete process/thread totals.
|
||||
|
||||
Live clients flush zero-based `PHASE|cycle|connecting/connected/closing/disconnected` in order. Connected holds perform validated interactions; disconnected holds last 300 ms before the next connecting marker. Compare progress before and after sampling; discard ambiguous transition observations from phase reports. Success requires attempted=connected=disconnected=connections*cycles, errors=0 and at least one validated message per connection (SSE also requires a heartbeat during hold). Counters alone do not prove server cleanup: preserve per-phase FD/RSS and their post-disconnect deltas. TCP/WS/SSE payload validation, finite limits and driver ownership apply on errors as well as success.
|
||||
|
||||
CMS concurrent reads close the control client before the baseline. `Process.descriptors()` records FD-to-target maps; `validate_read_descriptor_recovery(load)` is shared by the runner and CI. Its input includes `descriptors_before`, `descriptors_after`, and fixed `pooled_descriptor_limits`: each existing SQLite database/main WAL target is bounded by configured `max_connections`, SHM by one. All remaining descriptor targets and multiplicities must match (`Counter(non_pooled_before) == Counter(non_pooled_after)`). Preserve total FD change as observation. Persist each case's `report.json`, including partial completed data, raw samples, final budget and errors; aggregate failures as well as successes.
|
||||
|
||||
### 4. Validation & Error Matrix
|
||||
|
||||
Wrong status/type/body -> counted error, no successful latency sample. Exhausted load capacity -> dropped_capacity. Missed measurement window -> dispatch_expired; one late timer wake is not itself a drop. Incomplete samples/checksum mismatch -> runner failure. Missing cgroup permission/controller -> explicit failure and owned-group cleanup, never unlimited fallback. Interrupted run -> terminate/reap owned processes and remove owned empty groups, preserving completed case records.
|
||||
|
||||
An empty, duplicate, negative or non-integer `--client-cpus` set fails at argument parsing; an unavailable CPU or overlap with the specified server CPU fails before any output directory or process is created. A nonpositive `--client-workers` value fails at argument parsing. Do not claim independent client/server cores when affinity was omitted.
|
||||
|
||||
CMS extra/replaced/duplicated sockets, unexpected files or SQLite handles over the configured bounds fail even if total FD count is unchanged. Record completed publication fields and baseline/limits before starting the read load, so an early read or sampling error cannot erase existing evidence.
|
||||
|
||||
Process memory may disappear before waitpid reports exit: skip that unavailable observation and let the process exit check own failure reporting. Every owned server context rejects cgroup OOM after teardown without masking an earlier failure. HTTP warmup and measurement require at least one success and zero request errors; capacity drops remain explicit measurements. Live missing/out-of-order phases or incomplete connection cleanup fail the run. A valid partial SSE frame at the intentional measurement deadline is not a malformed peer frame; never require TCP chunk boundaries to coincide with events.
|
||||
|
||||
### 5. Good / Base / Bad Cases
|
||||
|
||||
Base: single-worker runtime, 32 server connections, 16 client slots. Good: each implementation prepares and shares the 64 KiB body once, validates exact response bytes and waits the entire measurement window. Bad: only one side reallocates the body per request, failed responses improve p99, or sub-window completion inflates QPS.
|
||||
|
||||
CPU placement good: `--server-cpu 0 --client-cpus 1,2,3,4 --client-workers 4` on a host allowing CPUs 0–4. Base: omit client flags for one worker inheriting allowed CPUs, with no isolation claim. Bad: repeat CPU 1 or place CPU 0 in the client set, or infer multi-core use solely from a requested worker count.
|
||||
|
||||
CMS FD base: one main/WAL/SHM handle per database. Good: a configured four-connection pool expands to four main and WAL handles, one SHM, and all non-database descriptors recover. Bad: a fifth database handle, an extra socket or a replacement socket masked by a closed control connection.
|
||||
|
||||
### 6. Required Tests
|
||||
|
||||
Run `python3 -m unittest discover -s test/performance -p test_runner.py -v` for counting/warmup/phase ordering/process teardown, HTTP/2 shape/summary validation, isolated JSON staging and missing-controller cleanup. Assert staged public projects declare the CMD used by every runner call and reject an obsolete application manifest. Configure test binaries in root config/setting.json at performance.network_peer and performance.live_peer. `python3 test/performance/test_network_bench.py -v` uses independent owned Python H1 peers and owned h2c/TLS services for exact-body success, one-connection multi-stream behavior and deliberate wrong-body failure. `python3 test/performance/test_live_bench.py -v` verifies TCP reconnect and rejects wrong TCP/SSE bodies or an SSE hold without heartbeats. Both also provide a no-network check-config test. Announce owned loopback checks before running them; select bounded protocol validation after driver changes, never the full suite by default. Network examples build offline with the workspace release profile; Dever archive/source provenance is recorded separately from Rust reference builds.
|
||||
|
||||
After changing CPU placement, assert the child reports the requested affinity without changing the parent, reject empty/duplicate/out-of-range/overlapping IDs, and verify serialized report workers/CPU sets. Run a bounded h2c/TLS+h2 smoke and minimal HTTP/1/live smoke against fingerprinted artifacts. To establish that the client actually exceeded one core, inspect `load.client_resources.cpu_cores_sampled` in high-rate raw cases, alongside service CPU and slot/dispatch drops.
|
||||
|
||||
CMS recovery changes require `test_runner.py` and `test/ci/test_gate.py`: bounded pool growth passes, socket replacement/duplication and excessive pool handles fail, and read/recovery failures preserve case and aggregate evidence after cleanup. A real read check longer than the default 30-second HTTP idle timeout exercises the closed-control baseline; it does not replace the 1800-second soak.
|
||||
|
||||
### 7. Wrong vs Correct
|
||||
|
||||
Wrong: call warmed cgroup current the application's cold-start RAM; subtract two p99 values to infer request latency; equate a completed report with zero errors/OOM. Correct: report RSS/PSS alongside charged cgroup current/peak and OOM/throttling counters, distinguish warmed file pages and pre-exec Python peaks, record request latency directly, and inspect all failure counters before drawing conclusions. No global cache drops. Raw Hyper is a normal-GET baseline, not an equivalent implementation of production slow-peer/fault/structured-cleanup semantics.
|
||||
|
||||
Wrong: require CMS total FD delta zero across idle control-client expiry and lazy SQLite pool expansion. Correct: close that client first, preserve exact non-pooled descriptor identity/count and independently enforce the database pool's descriptor bounds.
|
||||
|
||||
Wrong: equate a 5-second saturation platform or a complete report with a sustainable 60-second rate. Correct: pick a rate below the observed platform, require at least 99% planned-request success over 60 seconds in both transport modes and memory budgets, and preserve nonpassing calibration data separately.
|
||||
|
||||
For the 2026-09-18 HTTP/2 ceiling check, `--client-workers 4 --client-cpus 1,2,3,4` removed the former single-core client constraint while the Dever server stayed on CPU 0. A 16x16 `/plain` scan at 30k/45k/60k/90k observed roughly 36k–40k successful req/s at the upper rates, with the service near one CPU and request-slot drops dominant; this is not an isolated server-only ceiling. Short scans cannot select a reliable steady rate by themselves: 28k failed the 99% planned-request success gate over 60 seconds, while 20k passed for both h2c and TLS+h2 at 128 and 64 MiB. `test/performance/README.md` owns exact commands, counts, RSS/latency, failure and calibration reports, and the limits of the 100-cycle recovery evidence. Do not extrapolate loopback or warm-cache results to production.
|
||||
@@ -0,0 +1,417 @@
|
||||
# Toolchain And Official Library Contracts
|
||||
|
||||
## 1. Scope / Trigger
|
||||
|
||||
Application deployment targets belong to `toolchain::BuildTarget`, independently of the compiler host. Public `build --target linux-aarch64` may run on Linux x86_64; omitted targets and `run/test` stay native. Propagate the selected target through Lib preparation, CompileRequest, runtime pack selection, signature closure verification and both caches. Verify ELF machine/type for archive members and ordinary objects; a matching manifest label/hash alone does not prove architecture. A warm shared cache must still validate the selected target pack. Do not globally replace `platform_identity` or require an ARM compiler core for cross builds.
|
||||
|
||||
Target runtime/interpreter/extension/exec/sandbox assets must match the deployment target. Go build tools explicitly declare `build.host`, while Go output and stdlib headers match `build.goos/goarch` and version. Their sandbox uses host assets; emitted Workers use target assets. Product processes keep cleared environments; author-only cross linker settings or Go overlays must not become application configuration/environment knobs. Python sdist/npm hook execution requires a matching execution environment or a previously prepared complete target receipt; never relabel host native output. The canonical author maps and preparation commands are in `sdk/native-release.md`.
|
||||
|
||||
Executable modes are also part of the cross-target resource contract. Sandbox packaging and application embedding share `asset_executable`; classify both supported target loaders independently of the author's host, while retaining target ELF validation. Regressions must check actual packaged modes from non-executable source inputs and execute the embedded Worker. Host-only loader assertions and successful linking cannot establish target bootstrap behavior. Full-system sandbox fixtures must switch from initramfs rootfs to a normal root mount before bubblewrap uses `pivot_root`; do not loosen the production sandbox to accommodate a fixture mount error.
|
||||
|
||||
Applies to formatter/CLI, reference execution, native ownership and clause optimization, concurrent sequence actions and official source packages. JSON algorithms belong to `.dever`; HTTP reuses Hyper in the runtime, with source-owned public types and static business handlers. `intrinsic.rs` remains the exhaustive bridge for checker, native emission and reference evaluation.
|
||||
|
||||
The compiler author workspace requires Rust 1.95 or newer. SQLite is pinned to `deadpool-sqlite` 0.14.0 and its locked bundled SQLite dependency chain, which removes the observed WAL-close/replacement-open VFS lock inversion. Do not lower this requirement by reverting to the defective 3.51.1 engine, changing global toolchains or mutating Cargo registry sources. Initial developer dependency preparation may fetch explicitly; product run/build still consume prepared artifacts offline. See database-guidelines §9 for the root/connection lifecycle and regression evidence.
|
||||
|
||||
Compiler bridge exception (explicitly authorized 2026-09-30): workspace Rust remains `unsafe_code = forbid`. Only `dever-backend-bridge` may override this with `deny`, and only its private `ffi.rs` may allow necessary foreign calls, runtime C exports and their raw-memory operations. The rest of that crate, its build script, core, runtime, CLI and tests remain safe Rust; no public unsafe Rust API escapes the private boundary. Each foreign call and output transfer documents pointer lifetime, buffer length, allocation ownership and serialized LLD state. This is not a Dever-language FFI feature. LLVM/LLD object/link tests do not establish complete HIR semantic equivalence or a shipped runtime pack.
|
||||
|
||||
Keep `embedded` (LLVM/LLD compiler bridge) and `runtime-abi` (application staticlib) independently buildable. Runtime-only builds never link the LLVM/C++ SDK. Versioned signatures belong to `include/runtime.h`: fixed-width scalars/status, explicit output pointers, borrowed slices and matching owner releases. Decimal uses explicit little-endian low/high u64 words, not aggregate-by-value or Rust layout. Numeric success leaves Buffer null/0; errors preserve the original diagnostic and use only `dever_rt_v1_buffer_free`; compiler Reply has its separate C++ allocator. Reuse existing runtime numeric/collection/resource/protocol/system implementations. Application slices include CMD/wire/Dever/external Adapter, Model/SQL/transactions, HTTP/REST/auth/tenant/permission/Cookie/Upload/log and Job/Test. `runtime-api` adds owned application resources independently of `embedded`; `runtime-sqlite` and `runtime-postgres` select real drivers; `runtime-external` adds verified Worker resources. Private and managed Linux compilation use LLVM; formal six-target packs remain unfinished.
|
||||
|
||||
Each task root owns a component Session and its prepared bundle, propagated into child Tasks, blocking work and API requests. Copy external metadata into owned strings/vectors; never leak C name arrays or reset a process-global registry between entries. Register a Worker before awaiting startup and attach its supervisor to the root Scope. Explicit shutdown can be followed by a new start; terminal close permanently rejects admission. Retain task ownership while awaiting shutdown so cancellation can resume cleanup. Close Workers before cancelling/draining descendants, then close databases and other application owners in the same runtime. Generated typed roots use `run_entry_with_typed_resource_cleanup` to append resource failure to an existing typed fault; the original Rust backend preserves its explicit shutdown/error-cause path.
|
||||
|
||||
Private task constructors without suspension remain ordinary functions. Scope adapters should directly return the existing Tokio scoped future when no additional await is needed. An extra async wrapper can store both its input future and the child future, multiplying debug-build stack usage through nested callers; do not compensate by increasing thread stacks or boxing individual test phases.
|
||||
|
||||
Private `dever run/build/test` uses checked HIR → LLVM → object → in-process LLD, without Rust/Cargo/cc lookup or fallback. Keep LLVM calls binary-private; shared `toolchain/runtime_pack` validates compiler-relative `runtime/<platform>/manifest.json`, normalized paths, version/v1 ABI/target, four profiles and exact CRT/archive digests. Shared code must not reference bridge target methods: actual linkage pulls LLVM into launcher/daemon even when the call appears to be metadata only. Reject symlinks, scripts, thin archives and incomplete references; snapshot/recheck selected inputs before linking. Private compilation reuses `native::compile_artifact`; managed compilation requires authenticated daemon IPC and never falls back. Ephemeral signed Linux fixtures do not establish trimmed or formal six-target release assets.
|
||||
|
||||
Native authoring now uses the independent four-profile `runtime-pack` build recipe and `sdk/native-release.md`; the old all-features debug archive remains a test fixture only. Signing prepared inputs and compiling source are separate operations. Do not move Cargo/tool discovery into the maker or application run/build.
|
||||
|
||||
Linux first installation reuses the release signature and machine version owner. Author `bootstrap` inputs provide launcher/daemon and their own complete private ELF library closure. The independently authenticated installer reads only its absolute root-owned `config/setting.json`; its public key must be supplied outside the downloaded release, then pinned atomically without key replacement. A bounded journal may name only the seven fixed bootstrap destinations, including the sandbox helper tree and dedicated AppArmor policy. Persist final file modes and child directories before publication; recover uncommitted changes, but never roll back a committed publication because cleanup failed. Stop a newly started service before removing its unit; restore previous service state after restoring files. Offline image installation never invokes systemctl. Incoming packages must match the supported service template; existing signed versions retain their original template and remain usable by pinned projects. Actual global activation and public release require separate operational authorization and evidence.
|
||||
|
||||
Process main is separate from callable roots. Initialize Unix SIGPIPE policy before application threads, preserving stdout EPIPE as the existing located runtime error. Do not change an embedding host's signals from callable roots. Reuse typed fault render, indexed Test fault size/render/release, output cleanup and logger flush; Test main accepts exactly one nonempty unsigned decimal index below case count. Verify actual env-cleared CLI execution, relocated embedded Worker, selected-pack tampering on cache hits, broken stdout, and launcher/daemon dynamic dependencies. Six-target objects do not establish cross-platform execution.
|
||||
|
||||
## 2. Signatures
|
||||
|
||||
- Project creation: `dever new <new-directory> [--markdown]`; `project::create(&Path, bool)` consumes the single template source under the pinned `skills/assets/template` submodule. The machine launcher selects the active core without reading nonexistent target configuration.
|
||||
- AI guidance: `dever skill path [project-root]` resolves and verifies the active or project-pinned release's `skills/dever-language`; `dever skill install <new-directory>` atomically publishes a stable loader that invokes the absolute machine launcher. Skill resolution is a public read-only operation and must not open the root-only install lock.
|
||||
|
||||
- Application target: `dever build <root> --output <new-file> [--target linux-x86_64|linux-aarch64]`; explicit dependency preparation accepts the same target on `dever lib add/update/doctor`. Omission selects `BuildTarget::host()`. `CompileRequest.target: BuildTarget` is required and serialized canonically; test requests must use the host.
|
||||
|
||||
- Private author entry: `native-release <author-root> --output <new-dir>` calls `toolchain::packaging::create(&Path, &Path) -> Result<ReleaseManifest, String>`. It reads only `<author-root>/config/setting.json`; this is not an application configuration section or another public `dever` command.
|
||||
|
||||
- Runtime ABI examples: `uint32_t dever_rt_v1_int_add(int64_t left, int64_t right, int64_t *out, DeverRtBuffer *buffer)` and `void dever_rt_v1_buffer_free(uint8_t *ptr, uint64_t len)`. Buffer is `{uint8_t *ptr; uint64_t len;}`; Decimal output is `{uint64_t low; uint64_t high;}` and input words are separate scalar parameters. Full declarations live in the canonical C header, not independently handwritten per consumer.
|
||||
- Managed ABI: Text/Bytes/List/Map use opaque owner pointers and paired retain/release; Id shares Text representation. A process-static immutable `DeverRtType` holds size/align and clone/drop/equal/hash callbacks for the actual compiled type. `_take` consumes the non-null receiver on success, value error or invalid output; other inputs are borrowed. List/Map cursors consume their receiver and release unvisited rows on early release. Optional output uses a separate u8 presence flag, and the value slot holds the complete element/value type, including its own Nullable flag when applicable.
|
||||
- Application wire ABI uses the same process-static `DeverRtType` and synchronous typed encode/decode callbacks. Borrowed runtime wire Nodes and CMD Inputs never escape a callback or survive suspension. Validate aligned writable slots before parsing, consuming a field, selecting an Adapter or calling user callbacks. A successful decode initializes a complete typed row; if subsequent CMD `Inputs.finish()` or error-buffer validation fails, drop that row once. Partial failed rows remain the compiled callback's guarded responsibility. Keep CMD top-level `Inputs` policy distinct from nested checked wire parsing; required/unknown field wording and field-error precedence must match the Rust backend. Runtime `wire` feature supplies these owners without enabling a second protocol engine or LLVM SDK.
|
||||
- Adapter settings ABI reuses `Settings::load_adapter_settings`, name validation and selection. Runtime owns only the temporary opaque config handle; generated concrete Setting slots retain decoded fields after that handle is released. Decode errors never render Secret content. Scope drain precedes typed Setting release and the module-local entry gate reset; repeated success, partial init failure and child Task execution must preserve allocation balance.
|
||||
- API ABI: `dever_rt_v1_api_serve(session, routes, count, tenant_components, component_count, error) -> DeverRtAsyncOp*`; process-static `DeverRtApiRoute` references concrete callable/input/failure/auth/permission metadata. Claims pack borrows subject/session/optional tenant/site Text handles and initializes one concrete input row. Identity unpack borrows the returned concrete row, retains one id Text, and writes separate i64/u8 user/tenant value/presence slots. Release both unpacked id and the original output once. Runtime dispatches checked callbacks; source App is never looked up by string or called through dynamic business Value.
|
||||
- Resource ABI: File retain/release shares the runtime File; open/create/read/write/close/chunks borrow inputs, and create uses create_new. File read has an independent presence byte. `DeverRtReadEvent` supplies a concrete descriptor and Chunk/Failed callbacks, retaining borrowed Bytes/Text into the full original Choice. Stream retain/close/pull shares the runtime cursor; internal pull has typed row/presence outputs, emits a terminal Failed once and then End, and validates writable outputs before advancing the cursor or doing I/O.
|
||||
- Async owned ABI: `DeverRtOwnedType {size, align, move_init, drop, transferable}` has process-static callbacks; move initializes matching aligned storage and deinitializes the source owner. `DeverRtAsyncFunction` carries output/fault descriptors and create/resume/destroy/done/send_safe; create clones borrowed input before initial suspend. `DeverRtSyncFunction` carries input/output/fault descriptors and invoke/send_safe; invoke borrows runtime-owned input, returns 0 initializing output or 1 initializing fault, and never drops the input itself. Internal Unit rows are transferable, and internal Outputs rows derive transferability from fields; the source type properties are unchanged.
|
||||
- Async polling ABI: `dever_rt_v1_async_root(function, input, output, fault, error) -> u32` starts one task Scope; `dever_rt_v1_async_op_poll(state, operation, output, present, fault, error) -> u32` polls under only the current Context; `async_op_release` cancels/drops one operation. Statuses are Pending=0, ReadyOK=1, ReadyTypedFault=2, RuntimeError=3, InvalidInput=4, distinct from regular ABI 0/1/2 and kernel 0/1. Pending initializes no result, presence or fault. Ready moves only the appropriate typed owner; numeric faults and owned RuntimeAbi buffers remain typed faults, not business failures. Initial/pending/final coroutine destruction uses paired frame_alloc/free and does not retain poll Context.
|
||||
- Async concurrency ABI: task/group run accepts checked async descriptors or checked sync descriptors routed through the bounded blocking pool. Task/Group retain clones a wrapper owner, not the affine task/group; wait/stop consumes one wrapper and takes its inner value once. Group run and Channel operations borrow handles. Successful non-null blocking/parallel/sync-run/send operation creation consumes its typed input even if later pending, failed or cancelled; null leaves the input caller-owned. Channel receive's separate presence flag distinguishes End from Some(null) before language-level flattening. SendForeignFuture is constructed only after checked transitive frame/output/fault transferability; local root/call polling never gains Send by blanket unsafe implementation.
|
||||
- AsyncStream/TCP ABI: constructors borrow source handles and return a shared opaque producer. Rows use concrete managed clone or OwnedType move callbacks, never padding copies; separate presence distinguishes End from a nullable row. A consumption scope closes aliases on drop. Socket/Listener retain/release preserves runtime identity; async operations clone borrowed handles into their operation owner. TCP standard choices retain original value errors. Timeout returns a complete Task row plus presence, race validates matching canonical output descriptors before consuming its task wrappers, and both reuse structured runtime drain. ParallelFunction packs an element/context into one checked input row for the existing runtime dispatch owner, validating frame/input/output/fault transferability.
|
||||
- `List::into_values()` consumes unique elements and lazily clones shared elements. Emit consuming loops only for ordinary movable values; resource-containing lists retain lexical ownership.
|
||||
- `List::into_first()`, `Map::into_get()` and `Map::into_entries()` transfer uniquely owned values and clone only through a live alias. `Bytes::from_string()`/`into_text()` transfer compatible Text/Bytes storage.
|
||||
- `Render::render_to(&mut String)` recursively appends composite values into one caller-owned buffer; `render()` remains the allocating convenience boundary.
|
||||
- `concurrent::each_slice(&[T], workers, apply)` batches memory sequences; `each(iterator, workers, apply)` retains producer-owned Stream pulls. Both share cancellation/join logic.
|
||||
- `dever.http.serve(handler, Listener, Limits) -> ()` owns bounded HTTP/1 or HTTP/2 connections; `send(address, port, Request, Limits) -> Response` owns a single request and driver without a client pool. `result(send(...))` captures ResponseResult. Handler suspension is inferred. `Limits.http2: Http2Limits?` selects HTTP/2 explicitly; null selects HTTP/1.
|
||||
- `http.serve_live(handler(Request, HttpReply) -> (), Listener, Limits, LiveLimits)` owns streaming/SSE/WebSocket handlers. `respond/start/write/finish` share checked headers and a one-slot body channel. SSE Event has Text event/data and nullable Text id/Int retry_ms. WebSocket accept/connect/send/receive/messages/close reuse tokio-tungstenite; public source owns records/results/Message.
|
||||
- `http.client(origin, ClientTls, Limits, PoolLimits) -> HttpClient` creates a fixed-origin shared resource in the caller's task scope. `request -> Response` buffers; `open` and `open_stream` return StreamResponse with `AsyncStream<dever.io.ReadEvent>`. StreamRequest uses the same stream body. `close_client -> ()` closes aliases and drains its driver. Result choices describe explicit captures, not ordinary output signatures.
|
||||
- `tls.system/client/server` own public roots, explicit custom CA and server PEM configuration. `http.serve_tls/serve_live_tls` and `websocket.open(ws/wss URL, ClientTls, Limits)` reuse the same verified TCP/TLS transport. All four system http_serve intrinsics accept an optional final transferable Context, appended to the static handler signature.
|
||||
- `native::compile_with_cache(..., cache_directory) -> NativeProgram`; `cache_hit()` reports reuse. This explicit development/test path uses the workspace `target/native-artifacts`; a released `dever` must request the protected machine `deverd` service and must not expose that directory as a fallback.
|
||||
- `native::compile_test_suite(&Program, &SourceMap, rustc) -> NativeProgram`; every suite entry must come from successful checking and remains private. The suite selects one stable checked case index per process.
|
||||
- `toolchain::artifact_put(&Layout, &[u8]) -> ArtifactReceipt` and `artifact_get(&Layout, sha256, bytes) -> Vec<u8>` use authenticated daemon IPC. Receipts carry lowercase 64-hex SHA-256 and exact length, not a client-selected path. `CacheStore::summary()` returns metadata only with `verified:false`; administrator `status()` verifies bytes and returns `verified:true`.
|
||||
- `toolchain::compile(&Layout, &CompileRequest) -> Result<Vec<u8>, String>` submits canonical JSON `{version, kind, target, sources, bindings, resources}`. Sources are logical path/test origin/text; bindings contain only database names/drivers, provider verify names and site paths/auth keys. Resources are normalized paths, base64 bytes, digests and executable flags. No project root, secret Settings, IR, output path or client cache key is accepted. `SourceMap::from_compilation_sources` forces strict layout and compiler-owned standard sources; `check_with_bindings` and `validate_database_bindings` share the ordinary configuration rules.
|
||||
|
||||
- `format::format(&SourceFile) -> Result<String, Vec<Diagnostic>>`; `dever fmt <project-root> [--check]` prepares both `module/` and optional `test/` before committing either source set.
|
||||
- `SourceMap::load(root)` accepts `.dever` and `.dever.md`; plain `.md` is not program source. It derives package identity from the relative filename, and both source formats use the same `parse/check/api/run/build` contracts.
|
||||
- `SourceMap::load_project(module_root, test_root)` preserves separate display/logical paths and assigns Test origin internally. A missing test root is empty; an existing non-directory or unreadable root is an explicit load error.
|
||||
- Static API declarations are generated into the async application entry; application source does not call `dever.api.serve()`. Runtime `api::Inputs` binds GET/DELETE query parameters, POST/PUT JSON objects or a statically selected POST multipart shape, and runtime response helpers build the fixed JSON envelope. `config/setting.json` alone supplies HTTP, site/auth and upload limits; application config never uses environment variables.
|
||||
- Optional feature `reference`: `reference::evaluate(&Program, entry) -> Result<Evaluation, String>`, with captured stdout and structured source-span fault. No interpreter production CLI path.
|
||||
- `parallel_each(handler, List|Bytes|Stream, workers[, context]) -> ()`; zero-output synchronous handler, 1–256 workers, no recursively nested Stream in worker input/context. In async code the handler must be a named function with no transitive concurrency effect.
|
||||
- Synchronous and asynchronous parallel dispatch use typed failure paths. Generated handlers keep `AppError` and business-error identity across worker boundaries; String-only runtime helpers delegate to the same generic implementation and are not a compiler fallback.
|
||||
- List/Bytes accept both non-suspending and inferred-suspending handlers. Sequential collection operations await internally when required; suspending `parallel_each` reuses the AsyncStream Group path with capacity 1–65536 and the global task cap. Synchronous Stream remains synchronous.
|
||||
- `AsyncStream<T>` shares one Send producer/cursor and close signal. Traversal accepts effect-checked handlers and infers suspension; stream `parallel_each` requires a zero-output handler and a Group limit of 1–65536. Both `AsyncStream<T>` and `Channel<T>` require transferable T in every declaration position.
|
||||
- TCP connect/listen/accept/read/write infer suspension; metadata, close and stream construction do not suspend. `net.chunks` returns AsyncStream; file chunks return Stream. Explicit captures use their respective ReadStreamResult choices. No synchronous TCP backend or compatibility aliases.
|
||||
- `run`/`wait`/`stop`, bounded Group and bounded Channel are structured under one task-local Scope. Direct calls complete sequentially; `run` is the only call-level explicit concurrency operation and accepts both suspending and non-suspending static calls. `receive(Channel<T>) -> T?`.
|
||||
- Source code cannot start a runtime. A fully non-suspending entry executes directly; an inferred-suspending entry starts the single bounded runtime. `timeout(Task, millis, zero-input fallback)` and `race(Task, Task, ...)` consume affine tasks with identical output signatures. `stream(Channel|List|Bytes)` and `task.ticks(millis)` are pull-driven AsyncStreams.
|
||||
- `text.at(Text, Int) -> Text?`, `slice(Text, start, end) -> Text?`, `codepoint(Text) -> Int?`, `from_codepoint(Int) -> Text?`; positions count Unicode scalars, slice is half-open. Empty-separator split yields scalars without synthetic empty edges.
|
||||
- `int.parse`, `decimal.parse`, `float.parse` return nullable values. Decimal remains finite/exact at parse; Float supports its rendered IEEE special spellings. `to_text` uses runtime rendering.
|
||||
- `dever.process.arguments -> List<Text>` excludes argv[0] and rejects non-UTF-8 arguments. There is no application environment-reading capability. Application and benchmark configuration uses config/setting.json only.
|
||||
- `dever.time` owns UTC now, strict DateTime/Date/Time parse/format, millisecond Duration construction and checked DateTime add/subtract/difference. Deterministic operations are pure; now has a time effect. Result choices expose Read(value)/Failed(message). Existing Int clocks and zero-output sleep remain. Chrono 0.4.45 (std only) validates calendars without a local timezone dependency. `dever.net.connect_timeout(address, port, milliseconds)` requires a numeric IP and positive timeout.
|
||||
- JSON: `dever.json.parse(Text) -> dever.json.value.Document`, `stringify(Document) -> Text`; ParseResult/WriteResult apply to explicit result captures. node/property/element access returns nullable values or node IDs.
|
||||
- HTTP: source-defined Header/Request/Response/Limits. Headers are a List of Text names and Bytes values; preserve duplicate values. Bodies are bounded Bytes. Incoming names normalize to lowercase; no promise of ordering across distinct names.
|
||||
|
||||
## 3. Contracts
|
||||
|
||||
Developer onboarding has two official repositories: `shemic/dever-main` and `shemic/dever-main-skills`. The skill directory is an independent Git repository, pinned by the compiler's submodule; it distinguishes this language from Go Dever. Author `skill` is a required list of `{source,path,sha256}` entries published as signed `skills/dever-language/<path>` artifacts, including SKILL.md; at most 256 files, 1 MiB each and 8 MiB total. New-project source bytes come from the same skill templates; Markdown documentation wraps them without a second semantic copy. Keep the generated configuration CMD-only and free of database/auth prerequisites.
|
||||
|
||||
Official `install` and `update` explicitly download from the compiled GitHub Releases origin; neither project setting.json nor environment variables choose that origin. Each platform publishes `dever-<platform>.manifest.json`, `.manifest.sig`, `.tar.gz`, with immutable version tags `v<major.minor.patch>`. Verify the pinned signature before unpacking, require a matching skill, reject unsigned/duplicate/nonregular/escaping archive entries, and bound both metadata and payload. USTAR contains only signed artifact files; the two metadata files are separate assets. Redirects remain on the enumerated official HTTPS asset hosts. Privately staged publication never overwrites an existing release; malformed downloads leave the active version unchanged. The existing active-version journal is the sole commit point for both core and skill. The stable skill entry resolves it on each task; do not maintain a second independently switched skill pointer or rewrite every user's home directory.
|
||||
|
||||
The skill's first-install helper uses its independently distributed public key, verifies the full package before executing bootstrap, and generates only an owned temporary installer config. It never reads business configuration. Real-root installation explicitly starts the machine service unless --no-start is selected; image-root verification must not touch host services. Python/OpenSSL are installer prerequisites, not application runtime dependencies. `sdk/release-assets.py` publishes the three deterministic assets with Linux NOREPLACE directory rename and cannot clean another writer's public output.
|
||||
|
||||
Native author settings use `dever-native-release-input-v1`: version/target/signing_key, core and `core_libraries` inputs, required base/sqlite/postgres/both archive inputs, and ordered start/libraries/end. Input `source` is a normalized author-relative path and `sha256` is the exact lowercase digest; link inputs also specify the destination `path`. Core libraries use plain SONAME filenames and publish below `lib/`; runtime link inputs remain runtime-relative. Reuse the runtime v1 manifest types and validator. Copy into exclusively owned private staging, verify both source and copied digest, sort manifest artifacts, and sign those exact bytes with the supplied owner-private Ed25519 PKCS#8 key. Reject key content as payload, symlinks, path escape and reserved/duplicate destinations. Keep source paths, key material and timestamps out of release metadata. Linux publication uses atomic NOREPLACE; failures clean only owned staging. No tool invocation, environment configuration, downloads, input hard links or implicit trust-key installation belongs to this owner.
|
||||
|
||||
`toolchain::core_libraries` validates the same complete ELF closure during authoring and verification of signed native releases (identified by the signed native runtime manifest). Require matching GNU target/class/endian, exact SONAMEs and core `DT_RPATH=$ORIGIN/lib`, with no core RUNPATH. Libraries may resolve only their own private directory; every non-OS dependency must be declared and every supplied library used. Reject loader audit/filter hooks, host `/etc/ld.so.preload`, unsigned flat leaves, symlinks and subdirectories including glibc-hwcaps. The installed directory must exactly match the signed regular files. GNU libc-family/loader are the explicitly declared OS ABI exception; LLVM and its transitive non-OS dependencies are not host prerequisites. Generic signed payload fixtures without a native manifest do not become compilable releases. Verify a real signed compiler can check/build in an owned root containing only declared OS ABI libraries, then run the output after deleting its source.
|
||||
|
||||
Build archives with the root `runtime-pack` profile, one shared Cargo target, and independent features: runtime-api/runtime-external for every profile, then runtime-sqlite/runtime-postgres for the selected drivers. Never enable embedded or strip linkable symbols. Deterministic packaging of fixed inputs does not establish cross-host reproducible source builds. Assert actual driver separation and run the produced programs; a file named base.a is not evidence of trimming.
|
||||
|
||||
Runtime ABI status is u32: 0 success, 1 original runtime value error, 2 invalid ABI input. Output slots must be live, writable, aligned and disjoint; borrowed inputs stay live and immutable for the whole call. Null input is valid only at zero length; reject lengths outside usize/isize range before constructing a slice. Caller releases any previous Buffer allocation before reuse, and releases each returned non-null allocation once through the matching runtime ABI. Error paths do not initialize numeric output. These are compiler-owned internal interfaces, not a language-level unsafe capability; application configuration gains no environment variables.
|
||||
|
||||
Managed element storage is checked size/alignment with 8-byte-aligned `MaybeUninit<u64>` backing; descriptors and callbacks stay valid for the entire collection lifetime. Clone/drop/equality/hash act only through the concrete typed callbacks, never by reading/copying raw bytes or padding. Reuse runtime `List<Element>` and `Map<MapKey, Element>` COW/cursors. Element implements PartialEq, preserving NaN semantics in List and Map values; only MapKey wraps an eligible key with Eq/Hash. A non-null receiver passed to `_take` is owned by the call even if output validation fails. Successful optional output with presence=0 leaves its row untouched: never read it. Presence=1 initializes the entire row; its own Nullable flag remains authoritative. Text split receives the same Text descriptor used by ordinary List<Text> construction, not an independent runtime type identity.
|
||||
|
||||
The runtime C ABI's value-error status does not by itself define a recoverable language failure. LLVM BytesToText/BytesFromInts/BytesSlice explicitly map status 1 to their checked standard Failed(Text), then use ordinary library fail/result propagation; other value faults (such as duplicate Map keys) remain RuntimeAbi. Status 2 always propagates as an ABI fault. Convert the original error Buffer to an owned Text handle, free the Buffer exactly once, and read value output only on status 0. LLVM business payloads belong to its typed fault tail and reuse compiler capture/drop owners, not a new C runtime error registry; details live in compiler-contracts.md.
|
||||
|
||||
Protocol async operations validate output shape/alignment and required presence before polling, including operations whose successful output is Unit. Invalid metadata returns async ABI status 4 without changing the operation or output/fault sentinels; a corrected call may retry. Do not perform network I/O and only then reject the result destination. Protocol getters validate destination slots before allocating owned Text/Bytes/stream fields. Their fixed records and handles live in runtime.h; this is not a generic serializer ABI.
|
||||
|
||||
Nominal type properties are computed once after cycle validation. Contract analyses reuse a shared dependency graph; monotone summaries enqueue only affected callers. Bundled AST is cached in process; each thread retains only its preceding exact path/text checked Program or diagnostics. Separate CLI processes still check all source and the API baseline.
|
||||
|
||||
Native optimization works on a cloned checked clause: coalesce record aliases only while old fields remain equivalent; snapshot only ordinary scalar future reads with no intervening host writes; move only the final overlapping read of a replaced field. Direct split consumers reuse runtime text::pieces Unicode/empty-separator semantics. Fuse only non-resource stages with structurally proven total, synchronous, effect-free handlers; pure alone never proves totality. Preserve argument evaluation, numerical faults and left-to-right Float sums.
|
||||
|
||||
Native argument bindings are disposable owned temporaries: a final movable read transfers the source, while a still-live or resource-containing source first clones according to value/resource semantics. Collection lookup/extraction and compatible Text/Bytes conversions consume that temporary so uniquely owned payloads are not copied. Composite rendering recursively writes into one String, including generated records/choices and named entry outputs; numeric `to_text` keeps its direct `to_string()` hot path.
|
||||
|
||||
Memory parallel batches are capped at 256 elements and check the shared cancellation state between elements. Stream dispatch remains item-by-item. HTTP uses Hyper's incremental protocol state. Bytes shares engine buffers and slices; multi-frame body collection merges incrementally instead of retaining metadata per tiny frame. Empty concat reuses storage without altering aliases.
|
||||
|
||||
Native cache keys use SHA-256 and include emitted input, compiler/toolchain identity, exact runtime/dependency bytes, options/platform/environment. A compiler-owned path/size/mtime memo avoids rehashing unchanged runtime files on every cache hit; any metadata change recomputes the exact content identity, and a cache miss verifies copied runtime bytes against that identity before compilation. Publish complete private entries atomically; restore verifies identity and executable integrity, copies into per-call temp ownership, and never skips CLI API checking. Generated Rust is not persisted. Cache digests are content identities, not publisher signatures.
|
||||
|
||||
The machine toolchain has one stable `dever` launcher and a signed version store. A project may select one exact `major.minor.patch` only through `config/setting.json` at `dever.version`; otherwise the machine active version is used. The launcher verifies the signed manifest and selected core before reading project source, then preserves arguments and exit status. Version publication uses staging and an atomic directory rename. Active-version state is a checksum-valid append journal so an interrupted write leaves the preceding complete record selectable on Unix and Windows. Cargo's default `dever` binary is the development compiler and becomes `dever-core` in a release. Cargo's `dever-launcher` binary is the machine entry installed as `dever`; it must stay independent of LLVM. This is one public command name, with no legacy command alias. Builds do not replace global commands.
|
||||
|
||||
`dever clean <project-root>` is a version-core project command. It removes only current-user Dever temporary build directories and legacy `.dever-run-*` files that pass exact name/type/owner/age checks and whose recorded process is no longer active, then reports directory, file and byte counts. It never touches explicit build outputs, project data, Lib artifacts, Cargo caches or the machine store. `dever cache clean` remains an authenticated `deverd` machine-cache operation with no local fallback; compiler contributors use `cargo clean` for the repository `target/`.
|
||||
|
||||
`deverd` is the only writer of released cross-user cache entries. Its storage engine uses complete length-delimited build inputs, SHA-256 identities, immutable atomic entries, integrity checks, active-operation/read leases and maintenance locking. Released clients require platform-authenticated IPC; service unavailability fails without a private or uncached fallback. OS service installation and remaining platform peer adapters belong to distribution, not a caller-writable cache directory.
|
||||
|
||||
Linux cross-user IPC uses kernel `SO_PEERCRED`, verifies endpoint/server UID against the machine state owner and keeps the administrator token owner-only. Foreign users may transfer Lib artifacts and query metadata, never clean or trigger whole-cache hashing. Eight slots admit at most two foreign connections per UID and six total. Artifact transfer accepts at most 64 MiB and stores at most 2 GiB/4096 entries. Compiled entries have separate equal quotas and are inaccessible through ArtifactGet/Put: retrieving a program requires its complete canonical compilation request.
|
||||
|
||||
Compilation admission is two requests, with 384 MiB wire input, 4096/16 MiB sources and 4096/256 MiB resources (128 MiB each). The service verifies the installed release and complete runtime reference closure, including its native manifest and LLVM library, on every request before cache lookup. Identity binds canonical input and signed manifest; a persistent private install lock is shared through compilation/transfer and exclusive for install/uninstall, never unlinked. The whole request holds a cache operation; transmission retains its read lease. Only the signed core's hidden compile worker may produce a publishable program. It reads a service-owned 0600 request inside 0700 staging; trusted state/cache ancestors remain traversable and non-writable by other users. The worker reconstructs/checks sources and links, without loading deployment Settings, preparing Libs, executing App/Test or writing shared cache. Caller retains dependency preparation, baseline checks, program execution and per-case Test settings. Worker CPU/address-space/file/FD/core limits do not constitute an OS sandbox.
|
||||
|
||||
Compilation has a 180-second absolute service deadline, including output transfer, and a 190-second client download deadline. EOF or any byte following the exact request body cancels it. Every failure kills/waits for the owned child and removes staging; diagnostics are capped at 64 KiB and output at 320 MiB. This output budget applies only to trusted compilation receipts, never opaque ArtifactGet/Put. Apply remaining time before each write/read; a per-call socket timeout alone permits slow peers to retain version and cache locks indefinitely.
|
||||
|
||||
Binary resources cross the compiler bridge separately from bounded textual LLVM IR (8 MiB). The shared resource index verifies digests and binds path, length and executable status into IR/cache identity; declarations are initialized from borrowed bytes before LLVM verification and optimization. Reject duplicate, unused, defined or incorrectly typed/sized attachment symbols; six targets preserve the same byte order. Do not hex-escape interpreter payloads into IR or duplicate large byte buffers merely to sort canonical requests. Attachment limits match the managed resource count/byte budgets, and bridge object output is bounded at 320 MiB.
|
||||
|
||||
### Signed ecosystem pack contract
|
||||
|
||||
#### Scope / trigger
|
||||
|
||||
Native release authoring also packages Python/Node interpreters and Go build tools; Worker preparation consumes the same runtime descriptor validator. The author recipe lives in [sdk/native-release.md](../../../sdk/native-release.md).
|
||||
|
||||
#### Signatures
|
||||
|
||||
`packaging::create(author_root: &Path, output: &Path) -> Result<ReleaseManifest, String>` and the private `native-release <author-root> --output <new-directory>` command retain their signatures. `workers::runtime::validate(files, ecosystem, target)` owns descriptor/file-closure validation for both maker and consumer.
|
||||
|
||||
#### Contracts
|
||||
|
||||
Author `config/setting.json.runtimes` optionally contains `pip`, `npm`, `go`; each has `name`, `version`, `files[{source,path,sha256}]`, with Python `python_markers` and `python_wheel_tags`. Configuration is bounded at 1 MiB. Sources are explicit regular files below the author root, never env/PATH lookup. The deterministic archive and registry manifest are both signed artifacts; neither an upload nor a filename can substitute for their verified identity. Prepared trees have their own 256 MiB/65,536 file bound; final embedded applications must also fit the stricter compilation limits above.
|
||||
|
||||
#### Validation and error matrix
|
||||
|
||||
| Input | Result |
|
||||
| --- | --- |
|
||||
| Missing interpreter, Go tool or importcfg archive | Reject incomplete pack before publication |
|
||||
| Wrong ecosystem/target, Python flags or registry metadata | Reject runtime identity/descriptor |
|
||||
| Duplicate/escaping paths, file/directory conflict, symlink, wrong digest or signing-key alias | Reject author input; remove owned staging |
|
||||
| Existing output or concurrent second publisher | Preserve published output; reject replacement |
|
||||
|
||||
#### Good / base / bad cases
|
||||
|
||||
Good: complete relocatable interpreter plus stdlib, or Go tools plus every referenced stdlib archive. Base: omit `runtimes` for an otherwise complete native-only release. Bad: copy only host Python and assume its host stdlib will exist on the destination; a descriptor alone cannot prove portability.
|
||||
|
||||
#### Tests required
|
||||
|
||||
`native_release` verifies deterministic archive/signature bytes, computed runtime identities, bad inputs and rollback. Explicit ecosystem acceptance must lock from a signed install, execute all three managed check/run/build paths, then run programs after removing source and machine stores inside a root with no system language installations. OS baseline libraries, test key, target and untested third-party registry behavior remain explicit.
|
||||
|
||||
Linux no-host acceptance must use a genuine mount-namespace root, shared by core, Worker and bootstrap fixtures. A plain `chroot` leaves the process root different from the namespace root; Linux then rejects the Worker's nested `CLONE_NEWUSER` with EPERM, even for root. Use the explicitly prepared bwrap/loader assets to bind only the owned OS root, private proc and synthetic devices. Do not remove the product's `--unshare-all` or add host library/tool mounts to make the fixture pass. A successful managed run does not replace source-free standalone execution under this boundary.
|
||||
|
||||
Managed Go compilation removes the invocation-owned stage prefix from recorded source paths with `compile -trimpath`. This applies to the SDK, Adapter and dependency packages through their single `Builder::compile` path. Preserve logical filenames/line numbers; do not strip diagnostics to hide path drift. The real `external_workers` Go fixture must prepare twice in fresh stages and compare every emitted resource identity, then execute after source removal. The signed three-ecosystem run/build test must publish exactly three compiled entries. A single successful Go execution does not prove reusable cache identity.
|
||||
|
||||
#### Wrong versus correct
|
||||
|
||||
Wrong: encode a 118 MiB Node binary as LLVM text, raise the IR cap, or relax opaque artifact upload limits. Correct: pass checked bytes through `emit_object_with_resources(ir, target, resources)`, keep the IR cap, and apply the separate trusted compile/output budgets. Test author compression deadlines do not change the service's compile deadline.
|
||||
|
||||
Formatting is syntax-only, stable and idempotent. It preserves comments, literal values, explicit groups and source record-expression order. Prepare every source before rewriting any; snapshot changes, syntax errors, symlinks and staging failures do not modify originals. Each final replacement is atomic, not a multi-file OS transaction.
|
||||
|
||||
Application-test execution reuses checked HIR, specialization, runtime feature trimming and native cache; it does not link the reference evaluator or add an interpreter. Each case owns its temporary executable/project directory. Production check/api/run/build source sets and cache inputs exclude `test/`; fmt intentionally includes it.
|
||||
|
||||
Port native dispatch is a direct call for one implementation or a closed match for several. `adapter.<port identity>.use` in setting.json selects an immutable startup binding; unknown choices/fields, duplicate keys and missing required settings fail before user code. A selected Adapter's contextual `setting` is a private record decoded with shared SettingInput wire helpers into a OnceLock; only that implementation requires values. Secret errors never include input values. Port-only binaries enable the wire feature without API/database runtime; a no-setting single implementation may omit setting.json. Runtime configuration uses the bounded duplicate-aware wire parser before typed config parsing.
|
||||
|
||||
External exec and managed dispatch remain behind the same selected Port binding. All executions require a verified embedded launch manifest, selected Worker tree and signed sandbox assets; no adjacent-file or bare-process fallback exists. CLI preparation reads sandbox leaves from the authenticated installed release, including for exec. Runtime clears the environment, maps only verified launch arguments, and uses length-framed stdio. Native and LLVM dispatch reuse one invocation settings snapshot for Adapter selection, business settings and deployment grants. Component tasks reuse the structured application Scope and bounded queues. `hello/ready/health` must finish before admission; request ids are sequential and exact; timeout/cancellation sends `cancel` and drains one matching `result` or `error`; application cleanup sends `shutdown`, joins, then uses a bounded kill only when the child misses the deadline.
|
||||
|
||||
Python SDK input uses an owned asyncio pipe transport and bounded message queue. Cancel and join its reader on normal shutdown and protocol failure, then close the transport. A daemon thread blocked on `sys.stdin.buffer` can abort CPython finalization even after a successful reply. The SDK regression must wait for successful process exit after the shutdown reply while the parent's stdin writer remains open; closing it before waiting masks this lifecycle defect. SDK source changes require re-embedding the source through the host compiler before deployed application verification.
|
||||
|
||||
`dever-sandbox` owns the Linux namespace/seccomp launch boundary for Workers and Go build tools. The signed release supplies bwrap, the safe Rust guard and a bounded, complete ELF library closure; the loader cannot select libraries through environment or cache, and a host preload configuration is explicitly unsupported. Mounts are opened with no-symlink `openat2` and passed as owned descriptors to bwrap bind-fd, preserving the authorized inode across rename/replacement. Never replace this with a path check followed by a path-based bind. Input trees are read-only; output grants cannot overlap executable inputs. The runtime additionally rejects writable grants overlapping `data/cache/lib`, including its ancestors. Sandbox descendants inherit the syscall filter; namespace termination stops even children that changed process groups.
|
||||
|
||||
The signed bwrap must be a bounded static target ELF without an interpreter or dynamic dependencies and is executed directly. On Linux with restricted unprivileged user namespaces, a non-root launch requires the matching digest-addressed `/opt/dever/sandbox/<sha256>/bwrap`, root ownership of every ancestor, no shared writes/symlinks/setuid/setgid/file capabilities, and both dedicated AppArmor profiles loaded in enforce mode. The helper bytes must match the embedded signed asset. The runtime and real-host bootstrap require `kernel.apparmor_restrict_unprivileged_unconfined=1` before using/publishing this policy: at zero, unconfined callers can enter a capability-granting profile through `aa_change_profile`, independently of its executable attachment. Missing or invalid strict-policy state fails closed. The product only reads these flags; administrator-approved installation/loading owns host changes. Root or unrestricted-host execution uses the same embedded static helper, never an unsandboxed fallback. Test normal and immediate profile changes, with and without no-new-privileges, using the same-process negative probe; wrapper failure or unavailable setup is not a successful isolation check.
|
||||
|
||||
With `process=true`, POSIX spawn may reset only the effective UID/GID to the guard's captured real identity: `setresuid(-1, uid, -1)` and `setresgid(-1, gid, -1)`. Compare all three arguments as 32-bit kernel IDs; changing real/saved IDs or choosing another effective ID remains denied. Without the process capability both calls remain denied. Keep clone3 returning ENOSYS so libc falls back to the flag-checked clone syscall. The actual C probe must cover both capability states, high-bit-equivalent IDs and rejected identity changes; an unrestricted setuid allowlist is not an acceptable build compatibility fix.
|
||||
|
||||
Deployment file grants live only at `adapter.<port>.files.<name> = {"path":"data/upload", "write":true}` in setting.json. Checked `allow file` is required, names are bounded lowercase identifiers, paths stay below the application, and write defaults to false. Workers see `/data/<name>`; grant configuration is not business `setting`. Network capability permits IP sockets and supplies only bounded resolver files; host Unix sockets and raw sockets stay denied. Process capability controls fork/process clones separately from threads. GPU and unsupported OS backends fail explicitly. Linux additionally requires host permission for user/mount/PID/network namespaces and openat2; root fixture success does not establish non-root deployment support under AppArmor.
|
||||
|
||||
Go tools consume a private read-only input tree, including parent-generated import/embed configuration, and a separate writable output tree. Parent reads of generated programs must use nofollow, regular-file and single-link checks with a byte limit; never follow a tool-created output symlink on the host. Runtime/SDK/build assets and source remain explicit, without host compiler discovery or implicit networking.
|
||||
|
||||
When a build carries compiler-owned external resources, the native entry may include a deterministic resource table. `dever-runtime::external` releases it only below the executable's private `data/cache/lib/<digest>` directory, rejects symlinks/path escape and digest mismatches, and publishes a complete staging directory atomically. Existing entries are reverified; a concurrent publisher is reusable only after the same manifest verifies. This extraction path never downloads, consults `PATH`, or accepts runtime paths from environment variables.
|
||||
|
||||
Resource SHA-256 must match its bytes before native cache lookup. The bootstrap emitter references private, fixed-name binary compiler inputs with `include_bytes!`, not decimal arrays; identical checked digests share one static payload/input while each resource retains its own path and executable bit. Emitter and build staging reuse one input-index owner so they cannot diverge. The build owner stages each unique input exclusively and removes it with the owned build directory. Runtime integrity checks compare exact length, every 64 KiB chunk and digest against embedded bytes without allocating another whole interpreter image.
|
||||
|
||||
Markdown Port function metadata adds `- 允许失败:` after outputs, exactly matching the fails type. Qualified Adapter/fake names remain complete in their function sections and private in public-method lists. The setting declaration uses a Type section named Setting with its ordered field metadata.
|
||||
|
||||
Markdown selects only top-level fenced blocks with literal info `dever` or `typescript dever`, using pulldown-cmark offset events rather than scanning fence-like lines. Quotes/lists/raw HTML/outer showcase blocks are presentation. Fences use backticks or tildes with CommonMark 0–3-space indentation and require explicit closure. Both formats own one package each; duplicates are C002, never merged or selected by priority.
|
||||
|
||||
Every `.dever.md` has exactly one natural-language H1 source section. Its ordered metadata is path-derived source identity, role-derived public types/functions and logical App call examples; H1 has no executable program block. Every type or logical `(function name, arity)` group has one natural-language H2 section and one program block. All clauses in a logical function share that block and input names. Type fields/variants and checked function parameter/output names, types, counts and order must equal their documented inline-code values. Empty collections use explicit `无`; per-field/variant/input/output descriptions are nonempty. H3 and deeper supplemental content may follow the section's program block.
|
||||
|
||||
Closing-fence trailing whitespace is spaces only; a tab is rejected as an unclosed program fence (M002). The Markdown parser receives a same-byte-length view with bare CR changed to LF because it does not recognize bare CR itself; source text, code lexing and diagnostic positions still use original bytes. CRLF is kept intact. Do not normalize entire files or introduce an offset-changing source transformation.
|
||||
|
||||
SourceFile text, cache keys and CLI snapshots retain original Markdown. Lex each block in its original byte range so Text/comment scanning cannot enter prose. Parsing and formatting share extraction, structural documentation parsing and block validation. After function signature resolution, semantic checking compares documentation with syntax/HIR before checking bodies. Formatting reuses the AST emitter per block, keeps comments in their block, preserves external/fence bytes and the opening fence's newline style. Runtime faults keep original positions through HIR. The Markdown dependency belongs only to the compiler.
|
||||
|
||||
Native last-use analysis walks actual HIR read order backwards, kills full assignments, retains field hosts and outputs, and identifies concrete expression nodes. It may move only ordinary values: resource aliases have observable lexical scope. Share recursive type-property traversal for comparability, transferability and movability. Full-domain Int/finite Decimal guards need no runtime comparison; actual clause bounds must remain.
|
||||
|
||||
`native::OPTIMIZATION_ARGS` owns production and baseline optimization flags: level 3, one codegen unit, fat LTO and stripped symbols. The workspace release profile applies matching fat LTO/one-unit settings so the runtime rlib retains bitcode for the final native link. Multiple units caused larger generated programs to miss the same generic hash-table inlining performed in a small equivalent baseline. These settings preserve that optimization without weakening overflow, hashing or panic semantics, at the cost of compilation parallelism and higher compile time/memory. Stripping removes release-only debug/symbol payload without removing source-location strings used by Dever failures.
|
||||
|
||||
Application test suites use level 1, 16 codegen units and stripped symbols without LTO. Tests validate observable semantics and source locations rather than production optimization quality; `run`, `build`, performance fixtures and baselines continue using `OPTIMIZATION_ARGS`.
|
||||
|
||||
Concurrent actions use a caller-owned producer, bounded queue and joined workers. Exact-size finite iterators create at most one worker per item/batch; unknown-length Stream iterators retain the explicitly requested limit. Faults stop further dispatch at cooperative pull boundaries and already-running work drains. Pending blocking operations require explicit OS timeouts; there is no forced cancellation or detached task. The reference evaluator chooses a documented sequential valid schedule; concurrent stdout ordering is not a differential invariant.
|
||||
|
||||
Inferred-suspending entry points own one bounded Tokio runtime. Child Scopes share task and blocking semaphores but keep independent work registries; body cancellation cannot cancel the supervisor responsible for recursively draining descendants and started blocking jobs. Transitive effect checks, including concrete handler substitution at wrapper calls, keep blocking system operations, stdout and hidden collection-handler concurrency off Tokio scheduling workers unless they use an explicit bounded boundary.
|
||||
|
||||
Async stream consumers reserve capacity before pulling and observe child failures while the producer is idle. Close releases the producer's resource alias, not independently held Socket/Listener aliases. TCP has independent direction locks and a shared close signal. Read buffers allocate after readiness. Operation deadlines include direction-lock wait; partial writes close the socket on abandonment before releasing the write lock, with no whole-message retry. DNS uses the shared bounded blocking path.
|
||||
|
||||
Single `read(resource, limit)` uses the caller-selected maximum buffer and must not silently clamp a valid limit: regular files may satisfy the whole request in one operation, and existing programs rely on that result. Reads may still be short according to the OS. Memory-constrained programs choose a smaller limit or use the pull-based `chunks` interface; the runtime cannot both return a `limit`-sized value and avoid storing that value.
|
||||
|
||||
JSON uses an acyclic flat document: consecutive postorder IDs, child IDs smaller than parents, Map insertion order independent of ID order. Numbers retain raw JSON spelling, object names are unique, Unicode surrogate escapes must pair. Maximum depth is defined once in `dever.json.value`; stringifier checks references and output expansion before concatenating child graphs.
|
||||
|
||||
Typed wire boundaries are separate from the ordinary source Document API. Runtime `wire` (enabled by api) owns a borrowed RawValue parser with duplicate checks at every object, total byte/value/depth budgets (16 MiB / 65536 / 64 containers), exact raw Json and bounded output writing. A missing object value must retain its incomplete frame; a byte-limit failure after a partial write permanently prevents `finish` from publishing Encoded JSON. Generated concrete decoders reject unknown/missing fields and never pass Node/Value to business signatures. API envelopes embed only validated Encoded JSON, preserving raw numeric spelling. Decimal/Id/Uuid use strings; DateTime normalizes offsets to UTC Z with optional three-digit milliseconds, Date is YYYY-MM-DD and Time is HH:MM:SS[.sss]. Reject leap seconds, submillisecond precision and calendar years outside 0000–9999. Duration remains a JSON i64. Select root `time_codec` with api/reference for the concrete native decoder, policy, parser budgets, native/reference time, Markdown parity and in-process API dispatch checks; it opens no listener.
|
||||
|
||||
HTTP/1 server connections reuse AsyncStream/Group and the same task cap, with capacity reserved before accept. Hyper owns framing/keep-alive/pipelining; source handlers specialize directly. Preserve exact record-field contracts at the intrinsic boundary and include callbacks in effects, dependencies and specialization. Header/body/handler/stalled-write deadlines release peers. Configuration, accept and handler faults propagate through serve; ordinary peer failures end only that connection. send errors return Failed, with driver and request polled together. No old wire/transport API, detached driver, fallback parser, TLS/HTTP2, nonempty trailers, upgrades or streaming response in N2. Tests compare with independent wire peers.
|
||||
|
||||
N3 live HTTP adds one structured handler per connection, using a Group of one under the same global cap. Response body Drop cancels even HEAD/buffered handlers; a Session guard ends the stream or upgraded socket and invalidates escaped aliases. Ordinary stream idleness is a peer failure; handler faults retain their original error and cannot become a successful chunked EOF. An upgraded Hyper driver finishes before its WebSocket handler: the connection owner must wait for that handler and keep its capacity reservation. Take the concrete TCP/TLS Transport and pre-read bytes out of Hyper on upgrade so the HTTP write timeout does not leak into WebSocket policy.
|
||||
|
||||
LiveLimits owns maximum chunk bytes, stream idle deadline and shorter SSE heartbeat interval. A one-slot channel bounds pending chunks; business values, current writes, engine and OS buffers are additional memory. SSE normalizes CR/LF/CRLF without discarding trailing data lines, rejects event/id field injection and checks encoded length before allocation. Heartbeats are three-byte comments generated by polling the body; replay/history is application-owned through Last-Event-ID.
|
||||
|
||||
WebSocket data message and write-buffer sizes are bounded, including fragmented messages; read buffer is 4 KiB. Independent direction locks permit simultaneous receive/send. Acquire the writer lock before arming the partial-frame cancellation guard: cancelling a lock waiter cannot close another task's active frame. Endpoint.take wakes ordinary operations while transferring the final transport owner to close-handshake cleanup. Control frames, masking and UTF-8 belong to tungstenite. receive/messages must be polled to process Ping/Close; no detached reader or automatic Ping task. Limits are per operation. N4 adds WSS; subprotocol negotiation, compression and close metadata remain deferred.
|
||||
|
||||
N4 pool ownership uses separate request and physical-connection permits, each bounded by Limits.connections; idle and queued drivers count as physical connections. One structured task polls bounded queued/running Hyper drivers and idle expiry. Construct its owner guard before task creation so dropping an unpolled child closes escaped pool aliases. Never detach a connector/driver or retry partially sent requests. Return a connection only after complete EOF; body drop/fault/cancellation aborts that entry. Streamed transfer bounds chunks/read waits, buffered request additionally bounds total body/time. TLS shares trust/session caches with full certificate/name verification and small explicit cache capacities; public roots are bundled, custom roots replace them.
|
||||
|
||||
N5 HTTP/2 reuses Hyper/h2 with fixed receive windows, 16 KiB frame/send-buffer settings and decompressed header budgets. TLS must negotiate h2 through HTTP-specific derived configs; do not mutate shared WSS trust/config or downgrade. Cleartext uses prior knowledge. Source targets remain path/query; normalize authority to Host when Host is absent and reject disagreement. Connection-specific fields are rejected rather than silently rewritten. HTTP/1 upgrades remain the WebSocket path; no extended CONNECT, h2c Upgrade, push or nonempty trailers.
|
||||
|
||||
Hyper's synchronous Executor must spawn registered Dever child Scopes. A scoped connection owner drains these tasks and their started blocking descendants before releasing its physical guard. Serialize closing admission and child registration; task completion must include fault reporting and registry removal. Cleanup belongs to an uncancellable registered supervisor, even when the caller awaiting stop/finish is cancelled. Never take a Scope work registry in a cancellable caller and lose the captured completions on Drop. Each live HTTP/2 body owns its handler Group under that stream's Scope, so reset before/after response head cancels only that handler; report handler errors explicitly because Hyper consumes HTTP/2 stream errors internally.
|
||||
|
||||
HTTP/2 clients lease streams independently from physical connections, with total requests bounded by connections × streams (at most 65536). Upload and download share the stream lease; neither idle expiry nor another stream's cancellation may abort an active neighbor. Create connection scopes when handshake jobs are polled by the persistent pool driver, never under a temporary request Scope. Queued, idle and draining connections share the physical budget. GOAWAY blocks new dispatch while accepted streams drain. Only Hyper TrySendError returning the unsent request permits requeueing within the original deadline; release the old upload lease first. Never replay a sent request or infer dispatch safety from an error string. Single-use send has one deadline spanning TCP, handshake and exchange, followed by mandatory cleanup. Server first-request and protocol liveness deadlines prevent silent peers from holding connections indefinitely.
|
||||
|
||||
Listener close ends HTTP accept normally and triggers Hyper graceful shutdown while active response/upgrade owners drain. Public net.connections retains a terminal Failed on explicit listener close. Applications use timeout(server, grace_ms, zero-output fallback) for bounded drain. timeout/race borrow supervisor handles while waiting, cancel and drain losers/expired work before returning, and preserve runtime faults. Source stream adapters retain failure obligations and nullable Channel items; closing the adapter does not close independent Channel aliases.
|
||||
|
||||
Create a response-body cancellation owner only after the pending-to-committed state transition is accepted. Dropping an uncommitted replacement body must not close a previously committed response. Rejected duplicate start/respond/SSE/upgrade calls leave the existing stream usable; head delivery failure still cancels the committed body.
|
||||
|
||||
## 4. Validation & Error Matrix
|
||||
|
||||
- New target already exists (including empty directory or dangling symlink), parent is missing, or two creators race: explicit error for the loser; preserve the existing target and remove only owned staging.
|
||||
- Signature/payload/skill corruption, incomplete archive or absent official asset: explicit failure, no active-version switch or fallback. Publishing source code is not evidence that a downloadable binary release exists.
|
||||
- Project has a pinned version: `skill path <project>` resolves that exact installed version. Foreign users must read matching guidance without obtaining the administrator install lock.
|
||||
|
||||
Cross build: unknown/duplicate/missing target arguments fail before execution; absent target packs, wrong ELF/archive machine, unsigned target leaves, incompatible lock/runtime/build receipt and changed target inputs fail before cache reuse or linking. Sandbox loader mode must survive both release packing and embedding; a hash-valid non-executable loader is not a usable Worker resource.
|
||||
|
||||
Native maker: wrong format/target/version, unknown or duplicate fields, missing profile, non-regular or escaping source, SHA mismatch, exposed/invalid signing key or payload alias to key → explicit failure, no output, owned staging removed. Existing or concurrently created destination → no overwrite. Correct bytes → complete signed v1 directory, without private key/configuration. Unsupported host publication → explicit failure, never a weaker rename fallback.
|
||||
|
||||
Managed input with duplicate fields/noncanonical encoding, unsafe paths or forged resource digest -> reject before publication; invalid source -> original checker diagnostic from worker; unavailable service -> explicit failure; signature/runtime omission or tampering even on a cache hit -> reject; full admission/quota -> bounded error; cancellation/expiry -> reap child and release staging/locks without publishing a partial executable.
|
||||
|
||||
External Lib declarations live only in the root `lib` array of `config/setting.json`; reject old `config/lib.json` rather than silently combining it. Canonical v2 locks bind checked Port/Adapter/schema/operations/capabilities and independent Worker roots. Doctor rejects a stale binding, missing/cyclic closure or within-environment version conflict; independent Workers may select different versions. Embedding compares complete normalized provider metadata, including dependencies, runtime, schema and artifacts, not just an artifact subset.
|
||||
|
||||
Production registry access derives from a verified installed version core, not host tools or development fixtures. Signed assets `runtime/<ecosystem>/<target>/manifest.json` (`dever-registry-runtime-v1`) bind a RuntimePack identity and the adjacent `runtime.pack`; Python metadata additionally supplies target markers/wheel tags. Resolver owners implement PyPI PEP 508/440, npm semver/SRI and Go proxy/MVS with fixed HTTPS origins, disabled proxies and bounded work. Redirects are rejected except for Go `/@v/*.zip`: manually follow at most two hops into the exact HTTPS `storage.googleapis.com/proxy-golang-org-prod/` bucket. Validate normalized host/port/path and reject userinfo, fragments, missing/duplicate Location and loops before issuing a subsequent request; all hops share one 30-second deadline and diagnostics must not echo signed URLs. Loopback mirrors are explicit test transport, never project configuration. Unsupported extras/sdist, install hooks and Go sumdb are not silently approximated. Only explicit Lib commands resolve/download; run/build consume locked cached bytes.
|
||||
|
||||
`workers::prepare(root, Program, resources)` expands verified runtime packs (`dever-worker-runtime-v1`: ecosystem, target, executable, arguments; Go additionally has `build`) and owned Adapter sources into a per-contract `workers/<digest>` tree. Python uses exactly `-I -S -B`, installs the packaged SDK path and imports that trusted SDK before exposing business source/dependency paths, and reserves stdout for protocol; JavaScript binds ESM/CommonJS exports. Both bind each checked Port operation by name using `sdk_manifest()`. Go consumes fixed-target compiler/linker/source selector and stdlib importcfg from the same exact locked pack, compiles the SDK and checked-operation runner alongside target-selected Adapter files and locked module ZIPs, then embeds only the executable and checked contract. Go source exports one exact `func(context.Context, any, any) (any, error)` per operation; `send` maps to `Send`, dotted segments to PascalCase. Cgo, assembly and unlisted imports fail explicitly. Build tools run by absolute path with an empty environment and bounded lifetime; product compilation does not invoke host `go build`, `PATH`, `GOOS`, `GOARCH`, `GOCACHE` or `GOPATH`. `<entry>.dever-worker.json` (`dever-worker-launch-v1`: ecosystem, entry, executable, arguments, working_directory) accepts only manifest-bound resource arguments in that same tree and bounded literal flags. Runtime revalidates the entire private bundle before every launch/restart and clears the inherited environment. Discard consumed raw runtime/archive resources after expansion unless config roots or exec Workers still need them. A private fixed-target pack fixture proves the local compile/link path; signed release packs, Go sumdb and cross-platform distribution remain separate acceptance work.
|
||||
|
||||
Packaged exec Workers resolve only against the entry's embedded manifest. Bundle identity includes path, byte digest, length and executable bit. Validate all path segments, Unix owner/private permissions and file bytes on both cache reuse and every Worker restart; do not scan other bundles or fall back to source workers. Publish owner-only files/directories atomically before exposure. These protocol and integrity checks do not establish OS capability isolation; verify the sandbox launch boundary above separately.
|
||||
|
||||
`dever.crypto` uses the existing source Result choices and exhaustive Intrinsic bridge. Secret storage lives in the base runtime with zeroize; Argon2id, SHA/HMAC and OS randomness are in the optional crypto feature. Native runtime feature selection inspects already-pruned emitted crypto calls so direct system intrinsics, API roots and Test roots are covered. Password hashing uses RustCrypto Argon2 0.5.3, Argon2id v19 and default Params; PHC verification bounds algorithm/version/memory/time/parallelism/output before computing. Failure messages contain no input. Hash/verify use the existing blocking-worker effect boundary. `dever.process` exposes arguments only; application settings never come from environment variables.
|
||||
|
||||
Cache miss -> compile and publish; changed inputs -> separate key; incomplete/modified published artifact -> explicit error; unchanged sources with modified API baseline -> reject before native reuse. Concurrent writers may reuse only a complete same-identity entry.
|
||||
|
||||
| Condition | Result |
|
||||
| --- | --- |
|
||||
| Async op Pending; ready typed fault; runtime error | No output initialization; move checked fault with original kind/source; move original error bytes into RuntimeAbi fault respectively |
|
||||
| Consuming async operation creation returns null | Keep caller-owned input live for cleanup; a non-null operation owns the input through completion/cancel |
|
||||
| Invalid File/Stream writable slot | ABI invalid-input status before I/O or shared cursor progress |
|
||||
| Null/misaligned runtime ABI output, null nonempty input or unrepresentable length | ABI status 2; never dereference that pointer or read an uninitialized value output |
|
||||
| Int overflow, numeric division by zero, nonfinite Decimal bits or invalid UTF-8 | ABI status 1 with owned original runtime diagnostic; caller frees Buffer exactly once |
|
||||
| Invalid element descriptor/layout, or missing Map-key hash callback | ABI status 2 before constructing element storage; callback pointers and concrete row layouts remain compiler-owned lifetime/type obligations |
|
||||
| Optional extraction succeeds with presence=0 | Do not read the uninitialized row slot; return zero Nullable and release consumed receiver/unvisited cursor rows |
|
||||
| Recoverable Bytes operation returns status 1, or any returns status 2 | Bytes status 1 becomes its original Failed(Text); status 2 remains RuntimeAbi and is not caught by result |
|
||||
| LLVM-generated Windows aggregate object linked without target runtime support | Explicit missing `__chkstk`/`_fltused` diagnostic; no successful output or fixture replacement symbols |
|
||||
| Invalid format input anywhere in source root | Diagnostics, zero source rewrites |
|
||||
| Markdown source has no program blocks | M001 at original source |
|
||||
| Program fence is not explicitly closed | M002 at opening fence |
|
||||
| Package header/type/function crosses program blocks | M003; no format writes |
|
||||
| Package or declaration section is absent, duplicated or contains the wrong declarations | M004/M005; no format writes |
|
||||
| Documented package/API/declaration name differs | M006 with documentation and code locations |
|
||||
| Documented function input differs | M007 with documentation and code locations |
|
||||
| Documented output/type field/variant differs | M008 with documentation and code locations |
|
||||
| Same package in `.dever` and `.dever.md` | Existing C002 duplicate-package check |
|
||||
| `fmt --check` finds changes | Exit 1, no writes |
|
||||
| Ordinary alias remains live | Preserve independent value semantics |
|
||||
| Resource-containing local reaches last read | Retain lexical lifetime; no earlier socket EOF |
|
||||
| Invalid parallel count | Source-located failure before pulling or invoking handlers |
|
||||
| Stream-containing worker input | C005 before native compilation |
|
||||
| Invalid scalar index/number input | Null, never a default zero or lossy truncation |
|
||||
| Exact finite input shorter than requested workers | Same visits/results with no more workers than items/batches |
|
||||
| Valid one-shot read limit | Caller-selected maximum is passed to the OS read; no runtime clamp |
|
||||
| Composite/named-output rendering | Exact existing bytes, appended through one result buffer |
|
||||
| JSON grammar/reference/depth/output violation | Explicit Failed with offset/node and message |
|
||||
| Component hello/ready/schema/operation/capability mismatch | Startup fault before request admission; child is killed and reaped |
|
||||
| Component duplicate/unknown/out-of-order/truncated/oversized message | Fail the current and queued calls, close the generation and reap the child |
|
||||
| Component call timeout or caller cancellation | Send one matching cancel, drain one matching result/error within a deadline, otherwise terminate the Worker |
|
||||
| Component shutdown with active or queued calls | Close admission permanently, cancel the active call, drop queued calls, then bounded shutdown/kill/wait |
|
||||
| Managed launch identity/path/argv/runtime drift | Reject before executing any interpreter; no host fallback |
|
||||
| Shared artifact bad hash/length, overflow, truncation or quota exhaustion | Fail without publication; clean the owned staging and release operation/admission guards |
|
||||
| Foreign UID requests cache clean | Reject by kernel caller identity, regardless of supplied token |
|
||||
| HTTP peer framing/body violation | Hyper rejection or explicit error response/connection close, no handler invocation |
|
||||
| HTTP invalid outgoing message | send Failed or serve task fault; never silently rewrite framing |
|
||||
| HTTP/2 TLS peer cannot negotiate h2 | Explicit TLS/ALPN failure; no HTTP/1 downgrade |
|
||||
| HTTP/2 connection-specific header, or response TE header | Explicit error; request TE allows only trailers |
|
||||
| HTTP/2 reset before/after response head, including a blocked write | Cancel that handler and drain its descendants; neighboring streams survive |
|
||||
| HTTP/2 handler program fault | Original error propagates to serve; never treat it as normal EOF |
|
||||
| HTTP/2 GOAWAY while old streams remain | Stop admission and keep the physical connection permit until cleanup |
|
||||
| HTTP/2 stream/connection budget or window outside its bounds | Explicit failure before accepting or connecting |
|
||||
|
||||
## 5. Good / Base / Bad Cases
|
||||
|
||||
Onboarding base: `dever new app` followed by `run app -- hello.greeting.greet '{"name":"Dever"}'` returns the standard envelope with `data:"Hello, Dever"`. Good: --markdown generates equivalent checked declarations; two projects/users share core and skill. Bad: manual main.dever scaffolding, Go service/Page JSON templates, update URLs in project setting.json, or a global skill copied once and allowed to drift from the installed compiler.
|
||||
|
||||
Cross build good: an x86 core uses ARM profiles, ARM Worker assets and x86 Go tools emitting ARM objects. Base: omitted target retains native behavior. Bad: replacing global host identity, executing an ARM build tool on x86, labeling x86 native extensions ARM, or deriving target loader permissions from the host.
|
||||
|
||||
Native author base: fixed inputs and temporary key produce a verifiable manifest. Good: two different output names have identical manifests/signatures and payload digests, source mutation does not affect either published copy, and optimized profile programs run through the managed compiler. Bad: hard-link output to build caches, accept a debug all-features archive as four trimmed profiles, or call an unused PostgreSQL configuration a real database test.
|
||||
|
||||
Managed base: a signed Package project completes check/test/run/build and its built program runs without daemon. Good: different UIDs and project directories submit the same logical inputs and reuse one entry, while wrong source, unsigned CRT or modified pack fails. Bad: accept client IR, fetch compiled output through artifact digest, upload full Settings, run the application under the daemon UID, or make state/socket directories private to hide a worker permission bug.
|
||||
|
||||
Resource/async base: create/read a private fixture file and lower sleep to a real switched-resume frame. Good: aliases share Stream progress; stop releases a pending child and Scope drains descendants; large typed records and errors survive Pending; sync Task inputs/outputs transfer through the same bounded blocking owner. Bad: declare async metadata safe from parameters alone, clone borrowed arguments after initial suspend, reinterpret a typed error as String, or let a same-content overwrite pass the test.
|
||||
|
||||
Runtime ABI base: Int 40+2 yields status 0/value 42/null Buffer. Good: explicitly link a runtime-abi-only archive from a C or LLVM caller, read output only on status 0 and free an error/text buffer via the matching symbol. Bad: cast Rust DecimalValue into a C layout, pass aggregates by value, reuse a live owned Buffer slot, free compiler Reply via runtime buffer_free or treat author fixtures as signed release packs.
|
||||
|
||||
Managed ABI good: typed C callbacks retain/drop a nested Text record; aliased List/Map updates preserve the original; early cursor release drops remaining rows; duplicate Map keys retain the exact original diagnostic. Bad: byte-copy a managed payload or padding, make ordinary Float elements implement Eq, or confuse outer row existence with a nullable element's null state.
|
||||
|
||||
Base: format the hello sample twice; bytes are unchanged on the second pass. Good: move an unaliased Map payload into `get`, render a nested value into one buffer, or parse and stringify nested JSON with an exact large number. Bad: clamp a valid one-shot read behind the caller's back, move the last Socket alias into a short-lived function, accept conflicting Content-Length values, or compare concurrent stdout as an ordered trace.
|
||||
|
||||
Markdown base: an application file documents its path-derived component/domain/role identity in H1 metadata; API sections document declarations and contain no handler bodies. Good: keep every type/function/declaration description synchronized with its checked signature and format without changing prose. Bad: add application `main.dever.md`, put code under H1, group two functions under one H2, split clauses across H2 sections, execute a quoted example or move a comment across prose during formatting.
|
||||
|
||||
HTTP/2 base: null Limits.http2 retains HTTP/1; default_http2_limits() selects 16 streams and fixed 65535/262144-byte receive windows. Good: a TLS proxy makes a nested request through the same one-connection pool while its outer response remains open. Bad: expire that active connection using the idle timer, or reuse a cancelled stream's lease on a replacement connection.
|
||||
|
||||
Component base: one selected external Adapter completes hello/ready/health, serves typed Port calls and acknowledges shutdown. Good: a protocol fault closes the old receiver before a later call creates a rate-limited new generation; shutdown rejects every racing or queued call and explicitly reaps the child. Bad: drain a full request queue during shutdown, restart after admission closed, rely only on `kill_on_drop`, or report handshake capability equality as an OS sandbox.
|
||||
|
||||
Managed base: a zero-Lib Python/JavaScript Adapter still locks its signed runtime and generates a typed runner. Good: preserve raw archives shared with exec while removing managed-only copies. Bad: import the Python SDK before restoring the packaged path under `-I`, use a peer-supplied cache path, or treat `verified:false` counts as byte-integrity evidence.
|
||||
|
||||
## 6. Required Tests
|
||||
|
||||
For onboarding select CLI `new_project`, lib `release_source`, shared-toolchain version/skill tests and native-release skill authoring tests, plus `python3 -B -m unittest discover -s test/onboarding -v`. Assert both formats check/format, no-overwrite concurrency, real CMD/Test/build output, exact signed extraction, corrupt-download retention, project-pinned guidance and foreign-user reads. The foreign-user test lowers only an owned client UID. Bootstrap script tests mock final installation; report that separately from real native-template execution. Do not activate host services or run full release/CI suites as an onboarding check.
|
||||
|
||||
Cross-target changes cover `build_arguments`, native-release target/ELF/mode cases, CompileRequest validation, cache isolation and Lib/Worker target selection. The opt-in `signed_host_compiler_builds_arm_profiles_and_runs_standalone_under_qemu` and `signed_arm_ecosystems_build_offline_standalone_apps_for_system_guest` require explicit author inputs. Run exported Workers/HTTP with `test/ecosystem-release/arm64/simulate.py` under a complete ARM kernel; assert each successful output, source-located expected failure and child cleanup. Do not substitute QEMU user emulation for namespace/seccomp execution or infer physical ARM performance.
|
||||
|
||||
Native release changes select `native_release` plus affected private pack/installation checks. Default cases cover deterministic bytes/signature, private-copy ownership, malformed configuration/path/digest/key rejection, cleanup, and concurrent no-overwrite publication. Explicit `native_release::acceptance` consumes prepared optimized archives and author executable; inspect SQLite/PostgreSQL symbols, then repeat packaging, verify signed installation, compile/run all four profiles and run independent outputs after source removal/daemon stop. Report PG-server and cross-host source rebuild evidence separately. Keep native checks and Cargo serial under constrained disk space.
|
||||
|
||||
Managed compilation selects `packages::managed_compile`, the installed Package execution fixture, affected `shared_toolchain` cache/install/IPC cases, and private `llvm_cli`. Cover secret-free snapshots, reserved/strict source checks, cross-UID reuse, changed sources, failure recovery, fake-only Test execution, signature reference omissions, cache-hit tampering, limits, real cancellation/disconnect/deadline reaping, and shared install lock lifetime. Tests use owned daemons and temporary signing keys. Check final `dever`/`deverd` DT_NEEDED independently: empty-environment success on an LLVM-equipped host cannot establish absence of a dependency.
|
||||
|
||||
Runtime ABI changes select `native_runtime_abi` and opt-in `native_runtime_link`: exact Decimal word round-trip, checked errors/IEEE floats, borrowed UTF-8/Bytes including null-empty and invalid input, real C-header/staticlib linkage, LLVM scalar/out-pointer calls and matching buffer release. Managed changes include canonical C typed lifecycle callbacks, nested rows, COW aliases, cursors/early release, duplicate Map keys, split descriptor identity and Bytes conversion/slice. Build the offline runtime-only archive explicitly in `target/native-runtime-abi`; do not recursively build it from tests or silently use an embedded-compiler archive. Opt-in `llvm_managed_native` and `llvm_result_native` reuse the shared runtime-linked runner to verify real checked-source execution and repeated allocation balance; this is not a general memory profiler or capacity test. Typed lowering/bridge tests and target-pack limitations are specified in `compiler-contracts.md`.
|
||||
|
||||
File/Stream/coroutine ABI changes additionally select `llvm_resource_native`, `llvm_async_native` and `llvm_async_source`. Keep hand-written LLVM/C ABI smoke separate from checked-source lowering evidence; execute both against the actual archive. Assert all initial/pending/final destroy paths, actual Pending/backpressure, typed error transfer, sync input borrowing, non-null creation consumption, nullable row/End, Scope drain and repeat allocation balance. Use an atomic allocation ledger for concurrent workers. Do not run broad database/service/performance checks or claim complete application/target-pack acceptance from these tests.
|
||||
|
||||
HTTP/system lowering selects `llvm_http_system_source` plus the above affected ABI/async targets: repeated headers, independent chunked wire, pooled H1/H2 uploads/downloads, verified TLS, live/SSE, WebSocket typed variants/end, context ownership, failure-500-next-request, original diagnostics, calendar/UUID/Secret/crypto/argv and exact stdout/log JSON. The canonical C protocol fixture must reject null/misaligned destinations before poll, preserve sentinels, allow corrected retry, and reject completed repoll. The pure-log C fixture uses the same strict allocation ledger; logger concurrency/backpressure/panic tests stay under root test/runtime_logging.rs. Six-target object generation is separate from the actual Linux archive execution.
|
||||
|
||||
AsyncStream/TCP/timeout/race/ParallelEach changes also select `llvm_stream_network_source` and the existing async composition/network reference fixtures as appropriate. Cover successful null/Unit versus genuine timeout, exact typed faults without fallback, shared stream close, terminal rows, all dispatch categories and descendant peer EOF after cleanup. C polling invalid input returns `DEVER_RT_ASYNC_INVALID_INPUT=4`; it must never look like a typed fault. Canonical signatures remain only in runtime.h.
|
||||
|
||||
Performance changes select the affected native_ownership, runtime_foundations, concurrency or native_cache cases. HTTP changes select http_engine (independent wire, repeated headers, framing, body limits, deadlines, connection backpressure and task cleanup) and http_library (static handlers/effects, native bridging, examples). Bounded ignored performance tests require explicit scope; correctness checks do not establish throughput or RSS improvements.
|
||||
|
||||
Choose relevant root targets: `formatter`, `standard_library`, `native_ownership`, `concurrency`, `json_library`, `http_library`, and opt-in `differential`. Assert independent expected results, diagnostics, source field/Map order, scope lifetime, actual worker overlap, limits, malformed protocols and wire bytes. Native/resource tests own all temporary files and loopback sockets.
|
||||
|
||||
Markdown changes select `markdown_source` and affected `formatter`/`syntax_and_format` regressions. Cover mixed formats/duplicates and unchanged language rejection; H1 metadata without a block and H2 one-block structure; public names/use examples; fields/variants; resolved clause inputs and outputs; explicit empty lists and natural descriptions; original UTF-8/LF/CRLF/CR positions; hidden/nested/nonprogram content; closure and cross-block rejection; comments, prose preservation and idempotence. Verify actual CLI check/fmt/api/run/build with isolated small programs and failure-before-rewrite cases, without a full or service integration suite.
|
||||
|
||||
`performance` is explicitly ignored by default. It measures bounded hot paths after setup against the same representation and runtime, with exact output checks and repeated median measurements. Keep the 1.15 ratio gate meaningful; never replace a baseline by a slower different representation to conceal a compiler regression. Report environment and actual scope, not generic speed claims.
|
||||
|
||||
N3 selects `live_network` and `live_library`: independent HTTP chunks and WebSocket frame bytes, preread upgrades, masking/fragmentation/control frames/UTF-8, exact SSE field/line/size rules, backpressure, phase vs idle deadlines, task/connection limits, close/cancellation and native static bridging. Existing `http_engine`, `http_library` and affected `async_network` cases cover shared owners. Service examples are checked/formatted without starting their persistent entry point.
|
||||
|
||||
N4 selects `async_composition`, `pooled_network` and `pooled_library`: source configuration, affine timeout/race, async memory fusion boundaries, Channel/ticks, exact typed context/effects/failure obligations; independent connection reuse, upload framing, stream backpressure, queue/body/idle deadlines, incomplete-body cancellation, unpolled pool cleanup, CA/name rejection, WSS and graceful draining. Native programs must exercise the complete checked source bridge. The self-contained backend example may run on its owned random loopback port; persistent examples remain check-only.
|
||||
|
||||
N5 selects `http2_client`, `http2_server`, affected `http_library`/`pooled_library`, HTTP/1 pool/live regressions and structured task lifecycle cases. Independent h2 peers must prove actual multiplexing, stream reset isolation, flow-control recovery, stream/physical budgets, active-body idle behavior, ALPN success/rejection, GOAWAY draining and handler error propagation. Native source must cover HTTP/2 single-use send and a TLS proxy that issues a nested request through the same one-connection client, plus streamed upload, SSE and run/wait inside handlers. Functional checks do not establish HTTP/2 throughput or memory improvements.
|
||||
|
||||
External Component Protocol changes select `external_component`, `port_adapter` and `application_testing`. The fixture must be an owned test executable, not Python/Node/Go or a service dependency. Assert two calls, declared business error, health, caller cancel, timeout cancel, crash/restart, malformed frames, queued shutdown, schema drift and native Worker execution. These tests establish protocol/lifecycle correctness only; they do not establish ecosystem resolver, packaging, OS sandbox, throughput or RSS.
|
||||
|
||||
Resource and Lib changes also select `external_resources`, CLI `external_libs`, `application_config` and `shared_toolchain`. Include concurrent extraction, execute-bit identity, symlink/permission rejection, no-source/env-cleared packaged exec with repeat runs and tamper rejection, full provider metadata drift and independent Worker lock bindings. The opt-in `test/external-sdk/check.py` uses explicitly supplied host interpreters and checks all three SDKs, i64 extremes, JSON budgets, health/errors and cooperative/noncooperative cancellation; it does not prove managed runtimes or no-host distribution.
|
||||
|
||||
Registry tests exercise real provider metadata through an injected bounded transport, candidate/dependency/version conflicts and runtime identity; owned loopback HTTP tests also cover the Go ZIP redirect and rejection-before-request boundary. They are not public-network or signed-release acceptance. `external_workers` tests cover Python isolation, ESM/CommonJS binding, wheel/npm rules, path/file conflicts and no-host lookup. Explicit host-copy subprocess cases remain ignored by default. Shared service tests cover real UID separation (opt-in root-owned fixture), bad/truncated concurrent uploads, entry/byte quotas, summary-vs-integrity semantics and owner maintenance. Native tests assert resource-independent generated source size, forged-digest rejection and actual packaged execution; tests derived solely from emit strings do not prove launch.
|
||||
|
||||
## Durable Worker Lifecycle
|
||||
|
||||
The compiler owns Job service startup. Both application backends validate settings and selected `runtime.mode` before preparing service databases or listening/claiming. API and worker roots stay in one binary; api/worker/all gate only the services, not source reachability. Starting a Job service requires runtime settings; CMD and management commands do not validate or start unrelated services. An enqueue-only CMD may use one database connection without HTTP/worker configuration. API-only applications use api/30000 ms defaults when omitted. Worker pools reserve at least one connection beyond worker slots for queue control and lease renewals. SQLite additionally requires `lease_ms > Job timeout_ms` before database preparation, because a spare connection does not bypass its single-writer lock.
|
||||
|
||||
Tokio's existing signal feature owns SIGTERM/Ctrl-C registration; do not install handlers with unsafe code or detach signal tasks. Lifecycle supervises the application root with the existing task owner. Shutdown stops API accept and Job claim, drains within one deadline, then cancellation joins descendants before database shutdown and log flush. Generated normal/test entries close databases on success and application/startup failure. Worker work and heartbeat use registered tasks, bounded active slots and fencing tokens; cancellation leaves the lease for recovery.
|
||||
|
||||
Test-only `dever.test.advance_clock` and bounded `drain_jobs` share production clock/store/dispatch logic. Each case has an independent clock/database/fake closure; empty drain needs no database. No test may start API/worker services, select a production Adapter or use deployment configuration. Job test roots must include enqueued handlers even though enqueue is not a synchronous dependency edge.
|
||||
|
||||
LLVM Job/Test validation selects `llvm_job_test_source` and the Job cases in `llvm_api_source`, plus affected original `durable_jobs` contracts. Check six-target object emission separately from actual archive execution. Exercise single-suite case isolation, structured failure locations, fake-only external calls, virtual delay/retry/cron, atomic enqueue/handler rollback, timeout child drain and repeated-root allocation balance. Real owned API/worker cases verify tenant queue placement and current permissions after enqueue. Revocation fixtures keep stored references structurally valid; malformed storage ownership is an earlier fail-closed store error, not a successful dispatch-time authorization test.
|
||||
|
||||
LLVM external validation selects `llvm_external_source`, the original `external_component`/`external_resources` tests and the existing fake-only Job/Test case. Verify six-target metadata/object layouts separately from actual runtime execution. Run real fixture processes for typed values/errors, malformed payloads, bad Setting/handshake, cancellation, and strict warmup + 64 repeated roots. Relocate only the linked executable and settings before checking embedded exec and managed launch manifests with an empty environment; synthetic managed launch fixtures are not real Python/npm/Go SDK or formal release acceptance. Retain resource SHA verification on every root/restart. Linux allocation ledgers must wrap `realpath(NULL)` as an owned allocation: glibc's internal malloc bypasses executable linker wrapping, while Rust's later free is wrapped. Account for that owner without relaxing balance or increasing warmups.
|
||||
|
||||
Runtime config owns the shared Job bounds: timeout <= 3,600,000 ms and lease in 100..7,200,000 ms. Tests use the maximum lease so every legal timeout fits SQLite without bypassing production policy validation.
|
||||
|
||||
Ordinary Linux commands use `workers/command.rs` with the shared `workers/elf.rs` dependency-closure checker. Prepare the domain-owned ELF and flat adjacent `lib/` from either the project or locked Package; never scan host libraries or require a Lib lock for command-only projects. Validate target, GNU interpreter, recursive DT_NEEDED, SONAME, symbol versions and loader hooks. All command RPATH/RUNPATH expansions must remain inside the logical `program` tree, which maps to `/worker`; accepting an escape such as `$ORIGIN/../data/grant` would allow a file grant to replace a validated dependency at execution time. Private libraries cannot shadow signed sandbox libraries. Dynamic launch uses the signed target loader with `--inhibit-cache --library-path lib:/lib`; user argv is appended literally after mapping only compiler-owned resource arguments. Commands use a distinct `dever-command-launch-v1` manifest and `commands/` tree under the existing verified bundle/extraction owner.
|
||||
|
||||
Command limits are `adapter.<port>.command.timeout_ms` (default 30000, 1..3600000) and `output_limit` (default 1048576, 1..8388608 combined stdout/stderr); stdin is bounded at 8388608 bytes and argv at 65536 including NUL terminators, with embedded NUL rejected. A call owns one process, closes absent stdin, drains both output pipes concurrently and never replays. Nonzero program exit is output; failure before target exec must remain a runtime fault and cannot be inferred from stderr or a reserved exit code. Invocation shutdown and caller cancellation must wait for actual sandbox completion. Queued cancellation has no child to drain and must not wait behind an unrelated active call. Ordinary application tests remain fake-only. Focused evidence belongs to `external_command`, command cases in `external_workers`/`llvm_external_source`, and the opt-in signed native-release command acceptance; physical ARM execution must not be inferred from cross-target metadata tests.
|
||||
|
||||
Command execution observes guard exec through a private CLOEXEC pipe. Bind the PID namespace init through bwrap `--info-fd`, keep it blocked with `--block-fd` until pidfd acquisition and PPid verification, and wait for that pinned pidfd before releasing caller-scope completion. Pipe EOF (`--sync-fd`) is not sufficient: Linux closes init descriptors before tearing down descendants. Cleanup always reaps the owned launcher even if namespace observation fails, retaining both errors. Old Worker launches keep their existing handshake path.
|
||||
|
||||
## 7. Wrong vs Correct
|
||||
|
||||
Wrong: update a skill symlink separately after changing the active core, treat a cached older version as a successful failed update, or delete a public output tree after a publication race. Correct: resolve core and signed skill from one active journal, return the original download failure, and publish exclusively owned staging with NOREPLACE.
|
||||
|
||||
Cross-target wrong: check ELF headers but use the host loader name to assign executable mode. Correct: validate target architecture and independently preserve each declared executable's mode across signed packaging, embedding and extraction; exercise the resulting Worker bootstrap under the target kernel.
|
||||
|
||||
Native authoring wrong: discover a compiler in PATH while signing, overwrite a completed release, or strip a hard-linked archive in place. Correct: separate the recorded source-build recipe from deterministic copy/verify/sign, preserve link symbols, and publish a wholly new directory with the existing signed installation contract.
|
||||
|
||||
Wrong: reuse shared-file validation for an owner-only lock, require 0700 on public state ancestors, unlink a held advisory lock, or trust a successful test on a host with LLVM installed. Correct: validate each owner/permission contract separately, retain the lock inode, keep source staging private, and inspect final dynamic dependencies.
|
||||
|
||||
Wrong: export a Rust aggregate directly, use C++ free on Rust ABI buffers or ship the LLVM author SDK inside the application runtime. Correct: fixed scalar arguments and explicit output pointers, one canonical header, private allocator-matched exports and separate runtime-abi/embedded features; validate formal target packs separately.
|
||||
|
||||
Managed ABI wrong: trust a copied byte buffer as an owned typed value, create per-caller descriptor identities or read an optional out row regardless of presence. Correct: one process-static descriptor per concrete type, typed lifecycle callbacks over aligned uninitialized storage, explicit receiver consumption and full-row nullable extraction.
|
||||
|
||||
Async ABI wrong: use regular 0/1/2 status checks for Pending/Ready, raw-copy fault payload padding, free normal guards twice during final destroy, or drop the runtime's input inside SyncFunction.invoke. Correct: explicit async status mapping, typed nominal move/drop with source zeroing, initial/pending reverse cleanup versus final frame-only free, and runtime-owned input release after invoke or cancellation.
|
||||
|
||||
HTTP/2 wrong: spawn Hyper futures outside CURRENT_SCOPE, let a body-reset write error stop the whole server, or retry every GOAWAY error. Correct: register protocol tasks under a connection owner, distinguish transport cancellation before propagating handler faults, and only requeue a structurally returned unsent request within its original deadline.
|
||||
|
||||
Wrong: formatter reorders effectful constructors; code generation moves resource aliases early; composite rendering first collects every child String; the runtime caps one-shot reads to an arbitrary internal chunk; a benchmark times unequal Decimal initial representations.
|
||||
|
||||
Correct: reuse AST/HIR/runtime owners, preserve source ordering and lifetime, consume only disposable native bindings, stream rendering into one output String, keep read size under caller control, and match both semantics and representation before measuring.
|
||||
|
||||
Markdown wrong: regex-extract every fence or format all extracted code as one document. Correct: identify program ranges through Markdown parsing, preserve original positions, and reuse the existing Dever pipeline and formatter within those ranges.
|
||||
|
||||
Component wrong: use shell command strings, `PATH`, application environment variables, detached tasks, automatic call replay, a second JSON parser, or a per-ecosystem supervisor. Component correct: resolve one compiler-owned relative entry, clear inherited environment, reuse bounded wire and Scope owners, preserve the first runtime fault, and explicitly kill/wait on every abnormal exit.
|
||||
Reference in new issue
Block a user