feat: release Dever 0.1.5 with direct workers and long transactions

This commit is contained in:
Codex committed 2026-10-11 11:17:33 +00:00
1 parent 89629c2197
commit a900d7e8db
153 files changed
+3676 -3708

No files matched your search

+19 -2
View File
@@ -44,10 +44,27 @@
替换路径,勿直接复制示例运行。下文 CI 字段均位于 `performance.linux_ci`。`inputs` 列出固定原始作者输入及准备好的资产文件/目录,逐文件计 SHA-256;工具、协议产物、CMS manifest/可执行文件/配置和 `performance.network_peer/live_peer` 也自动计摘要,`linux_ci` 对象不作为 peer 路径解析。输出目录不得包含在 inputs 中。报告只存摘要,不复制数据库密码。
`required_inputs` 为 `author`、`native`、`postgres`、`sandbox`、`bounded-perf`、`soak` 各提供非空绝对路径列表,必须落在 `inputs` 声明范围内。缺路径记录 blocked,不算 pass。作者 fixture 的具体固定路径/准备命令仍以 `test/ecosystem-release/prepare.py`、`test/ecosystem-release/build_inputs.py` 和对应 ignored 原因文本为准;gate 不把现存文件冒充由当前源码重建的 provenance。
`required_inputs` 为 `author`、`native`、`postgres`、`process-runtime`、`bounded-perf`、`soak` 各提供非空绝对路径列表,必须落在 `inputs` 声明范围内。缺路径记录 blocked,不算 pass。作者 fixture 的具体固定路径/准备命令仍以 `test/ecosystem-release/prepare.py`、`test/ecosystem-release/build_inputs.py` 和对应 ignored 原因文本为准;gate 不把现存文件冒充由当前源码重建的 provenance。
显式 LLVM Worker 验收读取 `target/native-release-inputs/component-fixture`:先用当前源码构建 `dever-tests` 的 `component-fixture` bin,再将程序复制到该准备目录,记录源码/程序摘要,并把固定副本列入 `inputs` 和 `required_inputs.native`。不能将 `target/debug/component-fixture` 当作不可变输入:Cargo 在 default/all-feature 组合间会重新链接这个输出。普通 Worker 测试仍使用 Cargo 为各自组合生成的 `CARGO_BIN_EXE_component-fixture`,不改质量门的输入变更拒绝规则。
原生发行验收还需在同一准备目录提供四个独立构建的 `base.a`、`sqlite.a`、`postgres.a`、`both.a`,`core-libraries/` 放与候选编译器对应的完整非系统动态库闭包,`native/` 放 CRT、静态链接库和匹配的 `runtime.a`。这些输入必须为真实文件;不再假定作者容器是某个 Ubuntu/GCC/LLVM 安装布局。`DEVER_PROCESS_RUNTIME` 显式指向已准备的进程 ABI 目录。测试使用自有签名、安装根和守护进程,不改正式信任密钥或现有安装。
同一目录的 `bin/` 准备当前源码构建的 `dever`、`deverd` 和 `native-release`,验收使用这些固定副本而非猜测 Cargo 输出路径。需要在另一容器的自有临时目录执行时,可用 `DEVER_NATIVE_ACCEPTANCE_WORKSPACE` 指向包含相同准备目录及所需 C 夹具的根。显式设置 `DEVER_NATIVE_ACCEPTANCE_OUTPUT` 为不存在的绝对目录,对应验收全部通过后才导出未压缩的签名输入目录、测试公钥与验证结果;测试私钥不导出,不能当作正式发行签名。
保留候选发行附件也必须复用 `sdk/release-assets.py`,不把基础包、语言扩展和验证记录重新合成总安装包。基础 `dever-<平台>.tar.zst` 仅含签名清单的 `artifacts`;Python、Node.js、Go 分别生成 `ext-runtime-pip/npm/go-<目标>.tar.zst`,按需下载。清单和签名为独立附件;测试公钥及验证记录不放进安装包。示例:
```sh
python3 -B sdk/release-assets.py /absolute/candidate/ecosystems/release \
--output /absolute/candidate/assets --zstd /absolute/author/bin/zstd
```
拆包测试复用 `test/onboarding/test_installer.py`,默认作者工具为 `/usr/bin/zstd`;不同工具位置通过测试专用的 `DEVER_TEST_ZSTD` 显式指定。该测试同时核验三种语言扩展互不混入、基础包内容不变且压缩字节不因新增扩展变化,不安装全局编译器。
完整版本验收使用 `test/onboarding/verify_release_update.py`,显式传入 `--previous-release`、`--release`、`--assets`、`--trusted-key`、`--extension-helper`,以及不存在的绝对 `--work`、`--report` 路径。在独立验证容器中以 root 运行,不挂载业务数据、不启动系统服务;使用容器自有临时空间,不占用正在服务的 dev/deverd 临时卷。它先验证基础包首装、正式项目模板、四档运行库、`long` 和按需语言扩展,再验证真实旧版本升级与回退。旧、新版本必须具有各自实际构建的二进制,不能只改清单版本模拟升级;临时测试密钥的结果也不能算作正式签名验收。
升级核验按签名版本读取旧版原始资源路径和 bootstrap 元数据,不恢复旧沙箱执行;0.1.5 起的新包不得携带已移除的 bootstrap 沙箱配置。旧文件仍须通过原签名中的大小和摘要校验。
`author_outputs` 精确声明 author 会制作的 pack、下载结果和 receipt 路径。author 阶段只将其余原始输入当作不可变输入,另外记录完整输出摘要;后续验收按准备后完整输入摘要绑定。不要把整个作者输入根排除掉。显式 `author_network: true` 才能执行已分类的固定官方下载/源码构建。新 ignored 必须人工更新 `ignored.json`;inventory 核对源码声明及实际 all-feature libtest 名单,执行时一律使用完整 `--exact` 名称。内部 probe 记录 parent-only,只由所属父用例运行。
资源准备归外部托管者:自有空父启用 memory/cpu,限制总 CPU<=2、memory<=4GiB、swap=0;runner/Cargo/负载客户端放 `control` 子组,自有 PostgreSQL 可放兄弟子组,64/128MiB 服务子组由现有 performance owner 创建。gate 只读检查现有父限制与自身归属,不创建/修改主机策略。作者工具所需固定 PATH/动态库路径可由外部托管者显式准备,不写入产品配置。
@@ -64,7 +81,7 @@ python3 -B test/ci/run.py run --stage python
python3 -B test/ci/run.py run --stage sdk
python3 -B test/ci/run.py run --stage native
python3 -B test/ci/run.py run --stage postgres
python3 -B test/ci/run.py run --stage sandbox
python3 -B test/ci/run.py run --stage process-runtime
python3 -B test/ci/run.py run --stage bounded-perf
```
+2 -2
View File
@@ -41,7 +41,7 @@ def classify_cases(names, manifest, target):
def audit_classification():
manifest = json.loads((ROOT / "test/ci/ignored.json").read_text())
stages = {"probe", "author", "native", "postgres", "sandbox", "bounded-perf"}
stages = {"probe", "author", "native", "postgres", "process-runtime", "bounded-perf"}
if any(row["stage"] not in stages for row in manifest):
raise RuntimeError("ignored classification contains an unknown execution stage")
expected = {(row["source"], row["name"], row["reason"]) for row in manifest}
@@ -292,7 +292,7 @@ def soak_stage(stage, config, directory, result):
def run_stage(stage, document, config, directory, result):
if stage == "inventory":
inventory(config, directory, result)
elif stage in ("author", "native", "postgres", "sandbox", "bounded-perf"):
elif stage in ("author", "native", "postgres", "process-runtime", "bounded-perf"):
ignored_stage(stage, document, config, directory, result)
else:
soak_stage(stage, config, directory, result)
+109 -133
View File
@@ -1,24 +1,8 @@
[
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "unprivileged_launcher_probe",
"reason": "internal non-root launcher probe, invoked by enforces_unprivileged_namespace",
"stage": "probe",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/trusted_helper.rs",
"name": "rejects_untrusted_helper_paths_bytes_modes_owners_and_capabilities",
"reason": "requires root and the prepared static author bwrap; touches only owned target files",
"stage": "sandbox",
"package": "dever-sandbox",
"target": "dever_sandbox"
},
{
"source": "test/native_runtime_link.rs",
"name": "canonical_c_header_links_and_executes_the_real_runtime_archive",
"reason": "build runtime-abi-only staticlib in target/native-runtime-abi before running",
"reason": "requires an explicit runtime-only DEVER_RUNTIME_ABI_ARCHIVE",
"stage": "native",
"package": "dever-backend-bridge",
"target": "native_runtime_link"
@@ -26,7 +10,7 @@
{
"source": "test/native_runtime_link.rs",
"name": "llvm_object_calls_the_versioned_runtime_abi_without_rust_layouts",
"reason": "build runtime-abi-only staticlib in target/native-runtime-abi before running",
"reason": "requires an explicit runtime-only DEVER_RUNTIME_ABI_ARCHIVE",
"stage": "native",
"package": "dever-backend-bridge",
"target": "native_runtime_link"
@@ -34,7 +18,7 @@
{
"source": "test/native_runtime_link.rs",
"name": "protocol_poll_rejects_invalid_output_before_advancing_the_operation",
"reason": "build runtime-abi-only staticlib in target/native-runtime-abi before running",
"reason": "requires an explicit runtime-only DEVER_RUNTIME_ABI_ARCHIVE",
"stage": "native",
"package": "dever-backend-bridge",
"target": "native_runtime_link"
@@ -42,7 +26,7 @@
{
"source": "test/native_runtime_link.rs",
"name": "logger_flush_is_a_stable_native_allocation_barrier",
"reason": "build runtime-abi-only staticlib in target/native-runtime-abi before running",
"reason": "requires an explicit runtime-only DEVER_RUNTIME_ABI_ARCHIVE",
"stage": "native",
"package": "dever-backend-bridge",
"target": "native_runtime_link"
@@ -50,7 +34,7 @@
{
"source": "test/native_runtime_link.rs",
"name": "wire_codecs_validate_slots_before_callback_and_release_partial_values",
"reason": "build runtime-abi-only staticlib in target/native-runtime-abi before running",
"reason": "requires an explicit runtime-only DEVER_RUNTIME_ABI_ARCHIVE",
"stage": "native",
"package": "dever-backend-bridge",
"target": "native_runtime_link"
@@ -58,7 +42,7 @@
{
"source": "test/native_runtime_link.rs",
"name": "database_abi_preserves_typed_owners_and_rejects_invalid_slots_without_io",
"reason": "build a runtime-sqlite or runtime-postgres staticlib in target/native-runtime-abi before running",
"reason": "requires a runtime-sqlite or runtime-postgres DEVER_RUNTIME_ABI_ARCHIVE",
"stage": "native",
"package": "dever-backend-bridge",
"target": "native_runtime_link"
@@ -641,8 +625,8 @@
},
{
"source": "test/llvm_job_test_source.rs",
"name": "timed_out_job_drains_children_and_rolls_back_before_retry",
"reason": "requires explicit runtime archive; bounded task cancellation inside owned SQLite transactions",
"name": "timed_out_job_drains_children_and_preserves_committed_stages",
"reason": "requires explicit runtime archive; bounded cancellation between committed SQLite stages",
"stage": "native",
"package": "dever-backend-bridge",
"target": "llvm_job_test_source"
@@ -1287,110 +1271,6 @@
"package": "dever-backend-bridge",
"target": "llvm_application_source"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "enforces_files_network_process_and_threads",
"reason": "requires explicitly prepared target/sandbox-inputs assets; owns only test processes",
"stage": "sandbox",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "pins_grants_against_directory_replacement",
"reason": "requires explicitly prepared sandbox assets; mutates only owned fixture paths",
"stage": "sandbox",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "pinned_grant_probe",
"reason": "internal sandbox descriptor-pinning probe",
"stage": "probe",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "rejects_writable_aliases_of_executable_inputs",
"reason": "requires explicitly prepared sandbox assets",
"stage": "sandbox",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "prepared_python_and_node_run_without_host_files_or_process_creation",
"reason": "requires explicitly prepared signed-release Python/Node inputs and sandbox assets",
"stage": "sandbox",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "enforces_unprivileged_namespace",
"reason": "requires explicitly prepared assets and host permission for unprivileged user namespaces",
"stage": "sandbox",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "restricted_probe",
"reason": "internal sandbox probe, invoked only inside an owned namespace",
"stage": "probe",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "terminates_detached_descendants_on_kill",
"reason": "requires explicitly prepared assets; kills only this test's namespace",
"stage": "sandbox",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "tree_probe",
"reason": "internal sandbox process-tree probe",
"stage": "probe",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "descendant_probe",
"reason": "internal sandbox descendant probe",
"stage": "probe",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "granted_probe",
"reason": "internal sandbox probe, invoked only inside an owned namespace",
"stage": "probe",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "rejects_missing_dependency_before_host_loader_can_supply_it",
"reason": "requires explicitly prepared target/sandbox-inputs assets",
"stage": "sandbox",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-sandbox-tests/isolation.rs",
"name": "kernel_rejects_namespace_ptrace_and_truncated_ioctl_requests",
"reason": "requires explicit assets and the prepared C syscall probe",
"stage": "sandbox",
"package": "dever-sandbox",
"target": "isolation"
},
{
"source": "test/dever-cli-tests/llvm_cli.rs",
"name": "default_llvm_runs_builds_moves_and_reuses_cache_without_host_toolchain",
@@ -1442,14 +1322,14 @@
{
"source": "test/dever-cli-tests/external_workers.rs",
"name": "compiles_go_worker_with_locked_tools_and_runs_without_source_or_go",
"reason": "requires explicit Go runtime pack, official UUID proof/archive and sandbox assets",
"reason": "requires explicit Go runtime pack, official UUID proof/archive and process_runtime assets",
"stage": "native",
"package": "dever-cli",
"target": "external_workers"
},
{
"source": "test/dever-cli-tests/external_workers.rs",
"name": "runs_official_native_python_wheel_in_sandbox",
"name": "runs_official_native_python_wheel_as_managed_worker",
"reason": "requires the explicit official native-wheel and complete Python runtime fixture at target/python-managed",
"stage": "native",
"package": "dever-cli",
@@ -1457,7 +1337,7 @@
},
{
"source": "test/dever-cli-tests/external_workers.rs",
"name": "runs_official_python_source_build_in_sandbox",
"name": "runs_official_python_source_build_as_managed_worker",
"reason": "requires explicitly prepared official PEP517 source build at target/source-build",
"stage": "native",
"package": "dever-cli",
@@ -1538,7 +1418,7 @@
{
"source": "test/dever-tests/tests/external_resources.rs",
"name": "packaged_exec_runs_without_source_worker_or_inherited_environment",
"reason": "requires explicitly prepared sandbox assets and the private Rust bootstrap compiler",
"reason": "requires explicitly prepared process_runtime assets and the private Rust bootstrap compiler",
"stage": "native",
"package": "dever-tests",
"target": "external_resources"
@@ -1623,6 +1503,14 @@
"package": "dever-cli",
"target": "native_release"
},
{
"source": "test/dever-cli-tests/native_release/ecosystem_acceptance.rs",
"name": "signed_zero_lib_ecosystems_run_and_build_in_the_current_container",
"reason": "requires explicit native and zero-Lib ecosystem packs; uses an owned root installation and daemon",
"stage": "native",
"package": "dever-cli",
"target": "native_release"
},
{
"source": "test/dever-cli-tests/native_release/ecosystem_acceptance.rs",
"name": "signed_ecosystems_build_and_run_without_system_language_installations",
@@ -1698,7 +1586,7 @@
{
"source": "test/dever-cli-tests/external_libs/npm_build.rs",
"name": "builds_official_npm_addon_with_locked_lifecycle",
"reason": "explicit official npm native source build; network only in registry resolver, never in lifecycle sandbox",
"reason": "explicit official npm native source build; network only in registry resolver, never in lifecycle process_runtime",
"stage": "author",
"package": "dever-cli",
"target": "external_libs"
@@ -1734,5 +1622,93 @@
"stage": "author",
"package": "dever-cli",
"target": "external_libs"
},
{
"source": "test/dever-process-tests/execution.rs",
"name": "direct_child_probe",
"reason": "internal direct-process probe",
"stage": "probe",
"package": "dever-process",
"target": "execution"
},
{
"source": "test/dever-cli-tests/external_workers/ffi.rs",
"name": "real_c_abi_packages_compiles_and_checks_the_generated_protocol",
"reason": "requires explicit DEVER_FFI_CC, DEVER_PROCESS_RUNTIME inputs",
"stage": "native",
"package": "dever-cli",
"target": "external_workers"
},
{
"source": "test/dever-cli-tests/external_workers/ffi.rs",
"name": "real_c_abi_direct_worker_errors_deadlines_and_cancellation",
"reason": "requires explicit DEVER_FFI_CC and DEVER_PROCESS_RUNTIME inputs",
"stage": "native",
"package": "dever-cli",
"target": "external_workers"
},
{
"source": "test/dever-cli-tests/llvm_cli.rs",
"name": "llvm_test_controls_preserve_indices_prefix_boundaries_and_timeout_continuation",
"reason": "requires an explicitly prepared runtime pack adjacent to CARGO_BIN_EXE_dever; owned short-lived cases only",
"stage": "native",
"package": "dever-cli",
"target": "llvm_cli"
},
{
"source": "test/dever-cli-tests/native_release/command_acceptance.rs",
"name": "signed_cli_runs_and_packages_ordinary_command_with_private_library",
"reason": "requires explicit current runtime archive, author C/LLVM inputs and owned daemon/process_runtime",
"stage": "native",
"package": "dever-cli",
"target": "native_release"
},
{
"source": "test/dever-cli-tests/native_release/cross_acceptance.rs",
"name": "signed_host_compiler_builds_arm_profiles_and_runs_standalone_under_qemu",
"reason": "requires prepared x86/ARM four-profile runtime archives and private QEMU; starts only an owned daemon",
"stage": "native",
"package": "dever-cli",
"target": "native_release"
},
{
"source": "test/dever-cli-tests/native_release/cross_ecosystems.rs",
"name": "signed_arm_ecosystems_build_offline_standalone_apps_for_system_guest",
"reason": "requires explicit ARM runtimes/dependencies/process_runtime and native archives; builds with an owned host daemon and exports apps for ARM system QEMU",
"stage": "native",
"package": "dever-cli",
"target": "native_release"
},
{
"source": "test/dever-cli-tests/native_release/cross_ecosystems.rs",
"name": "signed_arm_python_worker_rebuilds_with_current_sdk",
"reason": "requires explicit ARM inputs; rebuilds only Python for SDK shutdown regression under the ARM guest",
"stage": "native",
"package": "dever-cli",
"target": "native_release"
},
{
"source": "test/dever-cli-tests/shared_toolchain.rs",
"name": "active_skill_is_readable_by_other_machine_users_without_install_lock_access",
"reason": "requires root; drops only an owned skill client to an unprivileged uid",
"stage": "native",
"package": "dever-cli",
"target": "shared_toolchain"
},
{
"source": "test/llvm_database_source.rs",
"name": "sqlite_long_command_preserves_commit_on_encoding_failure",
"reason": "requires explicit runtime archive; committed long-entry SQLite stages",
"stage": "native",
"package": "dever-backend-bridge",
"target": "llvm_database_source"
},
{
"source": "test/llvm_external_source.rs",
"name": "command_binary_io_and_exit_status_use_real_typed_llvm_calls",
"reason": "requires fresh runtime-external archive, command-fixture and process_runtime assets",
"stage": "native",
"package": "dever-backend-bridge",
"target": "llvm_external_source"
}
]
+1 -1
View File
@@ -17,7 +17,7 @@ sys.modules["linux_gate"] = sys.modules[__name__]
from settings import configured_peer, write_private_json
STAGES = ("static", "default", "all-features", "features", "python", "sdk",
"inventory", "author", "native", "postgres", "sandbox", "bounded-perf",
"inventory", "author", "native", "postgres", "process-runtime", "bounded-perf",
"cms-64", "cms-128", "http2-64", "http2-128", "recovery-64", "recovery-128",
"live-64", "live-128")
SOURCE_ROOTS = ("crates", "test", "sdk", "library", "examples", "skills", ".cargo")
+5 -5
View File
@@ -82,8 +82,8 @@ class GateTests(unittest.TestCase):
gate.check_output("python", {"python_tests": 0}, "OK")
def test_classification_rejects_unknown_duplicate_and_wrong_target(self):
row = {"package": "sandbox", "target": "isolation", "name": "restricted_probe", "stage": "probe"}
target = {"package": "sandbox", "target": "isolation"}
row = {"package": "process-runtime", "target": "isolation", "name": "restricted_probe", "stage": "probe"}
target = {"package": "process-runtime", "target": "isolation"}
self.assertEqual(classify_cases(["restricted_probe"], [row], target)[0]["stage"], "probe")
for names, rows, owner in ((["unknown"], [row], target), (["restricted_probe"], [row, row], target),
(["restricted_probe"], [row], {**target, "target": "other"}),
@@ -135,8 +135,8 @@ class GateTests(unittest.TestCase):
"failed": parent_failed, "ignored": parent_ignored})
def test_exact_ignored_acceptance_requires_one_successful_parent(self):
target = {"package": "sandbox", "target": "isolation"}
case = {"name": "parent", "exact": "parent", "stage": "sandbox",
target = {"package": "process-runtime", "target": "isolation"}
case = {"name": "parent", "exact": "parent", "stage": "process-runtime",
"target": target, "target_directory": "/unused/target"}
for counts in ({"passed": 1, "failed": 0, "ignored": 0},
{"passed": 0, "failed": 0, "ignored": 0},
@@ -154,7 +154,7 @@ class GateTests(unittest.TestCase):
result = {"commands": []}
expected = counts == {"passed": 1, "failed": 0, "ignored": 0}
with contextlib.nullcontext() if expected else self.assertRaisesRegex(RuntimeError, "one case"):
acceptance.ignored_stage("sandbox", {}, {}, Path("/unused"), result)
acceptance.ignored_stage("process-runtime", {}, {}, Path("/unused"), result)
self.assertTrue(execute.call_args.kwargs["single_harness"])
self.assertEqual(len(result.get("cases", [])), int(expected))
+6 -1
View File
@@ -144,7 +144,7 @@ fn exec_preparation_embeds_every_candidate_with_its_current_bytes() {
dever_cli::workers::prepare(root.path(), &program, &input).unwrap()
};
let resources = prepare();
assert_eq!(resources.len(), entries.len() * 2);
assert_eq!(resources.len(), entries.len() * 3);
for entry in &entries {
let manifest = resources
.iter()
@@ -157,6 +157,11 @@ fn exec_preparation_embeds_every_candidate_with_its_current_bytes() {
.unwrap();
assert_eq!(worker.bytes, fs::read(root.path().join(entry)).unwrap());
assert!(worker.executable);
assert!(resources.iter().any(|resource| resource.path
== format!(
"{}/contract.json",
manifest["working_directory"].as_str().unwrap()
)));
}
let mut changed_binary = native_elf::static_executable();
changed_binary.extend_from_slice(b"changed");
+6 -6
View File
@@ -2,8 +2,8 @@ use super::*;
use dever_cli::libs::build::{Inputs, Session, pack};
use serde::{Deserialize, Serialize};
#[path = "../../dever-tests/tests/support/sandbox.rs"]
pub(super) mod sandbox;
#[path = "../../dever-tests/tests/support/process_runtime.rs"]
pub(super) mod process_runtime;
#[derive(Deserialize, Serialize)]
#[serde(deny_unknown_fields)]
@@ -111,12 +111,12 @@ impl Inputs for AuthorInputs {
}
fn assets(&self) -> Result<Vec<(String, Vec<u8>)>, String> {
let workspace = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../..");
Ok(sandbox::assets(&workspace)
Ok(process_runtime::assets(&workspace)
.into_iter()
.map(|asset| {
let _mode = asset.executable;
(
asset.path.trim_start_matches("sandbox/").into(),
asset.path.trim_start_matches("process-runtime/").into(),
asset.bytes,
)
})
@@ -195,11 +195,11 @@ pub(super) fn prepare_tool_pack(
}
files.sort_by(|left, right| left.0.cmp(&right.0));
manifest.validate(&files, &descriptor).unwrap();
let assets = sandbox::assets(&workspace)
let assets = process_runtime::assets(&workspace)
.into_iter()
.map(|asset| {
(
asset.path.trim_start_matches("sandbox/").into(),
asset.path.trim_start_matches("process-runtime/").into(),
asset.bytes,
)
})
@@ -338,11 +338,11 @@ impl Inputs for AuthorInputs {
}
fn assets(&self) -> Result<Vec<(String, Vec<u8>)>, String> {
let workspace = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../..");
Ok(build_tests::sandbox::assets(&workspace)
Ok(build_tests::process_runtime::assets(&workspace)
.into_iter()
.map(|asset| {
(
asset.path.trim_start_matches("sandbox/").into(),
asset.path.trim_start_matches("process-runtime/").into(),
asset.bytes,
)
})
@@ -403,7 +403,7 @@ fn resolver<'a>(
}
#[test]
#[ignore = "explicit official npm native source build; network only in registry resolver, never in lifecycle sandbox"]
#[ignore = "explicit official npm native source build; network only in registry resolver, never in lifecycle process_runtime"]
fn builds_official_npm_addon_with_locked_lifecycle() {
let inputs = AuthorInputs::load();
let session = Session::new(&inputs);
@@ -596,7 +596,7 @@ fn npm_build_output_hard_links_stay_inside_the_exported_tree() {
),
(
"outsidealias",
"const fs=require('fs');fs.writeFileSync('/data/build/outside.txt','outside');fs.linkSync('/data/build/outside.txt','alias.txt');",
"const fs=require('fs');fs.writeFileSync('../outside.txt','outside');fs.linkSync('../outside.txt','alias.txt');",
false,
),
] {
@@ -37,7 +37,7 @@ impl Inputs for RuntimeOnly {
panic!("archive restoration must not load build extensions")
}
fn assets(&self) -> Result<Vec<(String, Vec<u8>)>, String> {
panic!("archive restoration must not execute a sandbox")
panic!("archive restoration must not execute a process_runtime")
}
}
+7 -12
View File
@@ -9,7 +9,7 @@ fn validate_native(
) -> Result<(), String> {
let wheel_files = files
.keys()
.filter(|path| !path.starts_with("runtime/") && !path.starts_with("sandbox/"))
.filter(|path| !path.starts_with("runtime/") && !path.starts_with("process-runtime/"))
.cloned()
.collect();
dever_cli::workers::python_wheel::validate_native(
@@ -38,11 +38,7 @@ fn wheel_console_scripts_use_the_same_private_prefix_and_record_generated_files(
.find(|file| file.path == "runtime/bin/sample-tool")
.unwrap();
assert!(script.executable);
assert!(
script
.bytes
.starts_with(b"#!/worker/runtime/bin/python3 -I\n")
);
assert!(script.bytes.starts_with(b"#!/bin/sh\n"));
assert!(String::from_utf8_lossy(&script.bytes).contains("sample.cli"));
let record = installed
.iter()
@@ -280,11 +276,7 @@ fn wheel_installs_all_standard_schemes_and_rewrites_installed_record() {
.find(|file| file.path == "runtime/bin/tool")
.unwrap();
assert!(script.executable);
assert!(
script
.bytes
.starts_with(b"#!/worker/runtime/bin/python3 -I\n")
);
assert!(script.bytes.starts_with(b"#!/bin/sh\n"));
let record = files
.iter()
.find(|file| file.path.ends_with("/RECORD"))
@@ -449,7 +441,10 @@ fn native_wheel_checks_target_suffix_and_closed_library_search() {
.unwrap_err()
.contains("packaged dependency closure")
);
files.insert("sandbox/lib/libsample.so.1".into(), library.as_slice());
files.insert(
"process-runtime/lib/libsample.so.1".into(),
library.as_slice(),
);
validate_native("linux-x86_64", &suffixes, "runtime/bin/python3", &files).unwrap();
let old = files
.remove("site-packages/sample/_native.cpython-312-x86_64-linux-gnu.so")
+124 -31
View File
@@ -4,8 +4,8 @@
reason = "This test crate uses only the shared static ELF fixture"
)]
mod native_elf;
#[path = "../dever-tests/tests/support/sandbox.rs"]
mod sandbox;
#[path = "../dever-tests/tests/support/process_runtime.rs"]
mod process_runtime;
#[path = "../dever-tests/tests/support/temp.rs"]
mod temp;
#[path = "support/wheel.rs"]
@@ -32,6 +32,80 @@ use temp::TemporaryDirectory;
#[path = "external_workers/command.rs"]
mod command;
#[cfg(target_os = "linux")]
#[path = "external_workers/ffi.rs"]
mod ffi;
#[test]
fn native_ffi_workers_package_the_checked_contract_and_recursive_libraries() {
let root = TemporaryDirectory::new();
let mut sources = SourceMap::default();
sources.add(
"media/native/port.dever",
"triple(value: Int) (output: Int) fails app.NativeError",
);
sources.add(
"media/native/adapter.dever",
"external exec \"bin/worker\" {}",
);
sources.add(
"media/native/app.dever",
"type NativeError { error Unavailable }\ntriple(value: Int) (output: Int) { output = port.triple(value) }",
);
sources.add("media/native/api.dever", "cmd triple = app.triple");
let program = dever_core::check(&sources).unwrap_or_else(|errors| panic!("{errors:?}"));
let contract = &program.external_worker_contracts()[0];
let mut entry = resource(
contract.entry.clone(),
native_elf::compiler(&["libfixture.so"]),
);
entry.executable = true;
assert!(
workers::prepare(root.path(), &program, &[entry.clone()])
.unwrap_err()
.contains("libfixture.so")
);
let directory = root
.path()
.join(&contract.entry)
.parent()
.unwrap()
.join("lib");
fs::create_dir_all(&directory).unwrap();
fs::write(
directory.join("libfixture.so"),
native_elf::library("libfixture.so", &[]),
)
.unwrap();
let mut loader = resource(
format!("process-runtime/lib/{}", dever_process::loader_name()),
native_elf::static_executable(),
);
loader.executable = true;
let resources = workers::prepare(root.path(), &program, &[entry, loader]).unwrap();
let manifest = resources
.iter()
.find(|file| file.path.ends_with("/contract.json"))
.unwrap();
assert_eq!(
serde_json::from_slice::<serde_json::Value>(&manifest.bytes).unwrap(),
contract.sdk_manifest()
);
assert!(
resources
.iter()
.any(|file| file.path.ends_with("/lib/libfixture.so") && !file.executable)
);
let launch = resources
.iter()
.find(|file| file.path.ends_with(".dever-worker.json"))
.unwrap();
let launch: serde_json::Value = serde_json::from_slice(&launch.bytes).unwrap();
assert_eq!(launch["ecosystem"], "exec");
assert!(launch["executable"].as_str().unwrap().ends_with("/worker"));
assert_eq!(launch["arguments"], json!([]));
}
#[cfg(unix)]
struct OwnedProcessGroup(std::process::Child);
@@ -93,13 +167,13 @@ fn resource(path: String, bytes: Vec<u8>) -> EmbeddedResource {
}
}
fn prepare_sandboxed_worker(
fn prepare_managed_worker(
root: &Path,
program: &dever_core::hir::Program,
mut resources: Vec<EmbeddedResource>,
) -> Vec<EmbeddedResource> {
let workspace = Path::new(env!("CARGO_MANIFEST_DIR")).join("../..");
for asset in sandbox::assets(&workspace) {
for asset in process_runtime::assets(&workspace) {
let mut file = resource(asset.path, asset.bytes);
file.executable = asset.executable;
resources.push(file);
@@ -108,7 +182,7 @@ fn prepare_sandboxed_worker(
generated.extend(
resources
.into_iter()
.filter(|resource| resource.path.starts_with("sandbox/")),
.filter(|resource| resource.path.starts_with("process-runtime/")),
);
generated
}
@@ -450,7 +524,7 @@ fn cross_target_runtime_rejects_host_binary_even_with_matching_pack_hash() {
#[cfg(target_os = "linux")]
#[test]
#[ignore = "requires explicit Go runtime pack, official UUID proof/archive and sandbox assets"]
#[ignore = "requires explicit Go runtime pack, official UUID proof/archive and process_runtime assets"]
fn compiles_go_worker_with_locked_tools_and_runs_without_source_or_go() {
use std::os::unix::fs::PermissionsExt;
@@ -521,7 +595,7 @@ func Send(_ context.Context, payload any, _ any) (any, error) {
assert_eq!(sha(&bytes), artifact.sha256);
resources.push(resource(artifact.path.clone(), bytes));
}
for asset in sandbox::assets(&workspace) {
for asset in process_runtime::assets(&workspace) {
let mut file = resource(asset.path, asset.bytes);
file.executable = asset.executable;
resources.push(file);
@@ -567,7 +641,7 @@ func Send(_ context.Context, payload any, _ any) (any, error) {
for file in generated.iter().chain(
resources
.iter()
.filter(|file| file.path.starts_with("sandbox/")),
.filter(|file| file.path.starts_with("process-runtime/")),
) {
let path = temp.path().join(&file.path);
fs::create_dir_all(path.parent().unwrap()).unwrap();
@@ -577,14 +651,12 @@ func Send(_ context.Context, payload any, _ any) (any, error) {
}
}
fs::remove_dir_all(entry.parent().unwrap()).unwrap();
let child = dever_sandbox::Launch {
assets: &temp.path().join("sandbox"),
let child = dever_process::Launch {
assets: &temp.path().join("process-runtime"),
worker: &working_directory,
executable: &temp.path().join(&binary.path),
arguments: &[],
working_directory: &working_directory,
capabilities: dever_sandbox::Capabilities::default(),
grants: &[],
}
.command()
.unwrap()
@@ -729,13 +801,13 @@ fn installs_purelib_and_platlib_python_wheels() {
#[test]
#[ignore = "requires the explicit official native-wheel and complete Python runtime fixture at target/python-managed"]
fn runs_official_native_python_wheel_in_sandbox() {
fn runs_official_native_python_wheel_as_managed_worker() {
run_native_python_fixture("python-managed");
}
#[test]
#[ignore = "requires explicitly prepared official PEP517 source build at target/source-build"]
fn runs_official_python_source_build_in_sandbox() {
fn runs_official_python_source_build_as_managed_worker() {
run_native_python_fixture("source-build");
}
@@ -820,7 +892,7 @@ fn run_native_python_fixture(fixture: &str) {
.collect(),
});
lock.write_atomic(temp.path()).unwrap();
let generated = prepare_sandboxed_worker(temp.path(), &program, resources);
let generated = prepare_managed_worker(temp.path(), &program, resources);
assert!(
generated
.iter()
@@ -828,14 +900,12 @@ fn run_native_python_fixture(fixture: &str) {
&& resource.bytes.starts_with(b"\x7fELF"))
);
let (deployment, launch) = run_packaged_protocol(&temp, &contract, "pip", &generated);
let output = dever_sandbox::Launch {
assets: &deployment.join("sandbox"),
let output = dever_process::Launch {
assets: &deployment.join("process-runtime"),
worker: &launch.working_directory,
executable: &launch.working_directory.join("runtime/bin/prefix-probe"),
arguments: &[],
working_directory: &launch.working_directory,
capabilities: dever_sandbox::Capabilities::default(),
grants: &[],
}
.command()
.unwrap()
@@ -877,14 +947,12 @@ fn run_packaged_protocol(
ecosystem,
)
.unwrap();
let child = dever_sandbox::Launch {
assets: &deployment.join("sandbox"),
let child = dever_process::Launch {
assets: &deployment.join("process-runtime"),
worker: &launch.working_directory,
executable: &launch.executable,
arguments: &launch.arguments,
working_directory: &launch.working_directory,
capabilities: dever_sandbox::Capabilities::default(),
grants: &[],
}
.command()
.unwrap()
@@ -1015,16 +1083,23 @@ fn rejects_python_runtime_without_isolation_flags() {
);
}
fn fixture_runtime(name: &str) -> String {
let path =
std::env::var(name).unwrap_or_else(|_| panic!("set {name} to an absolute runtime path"));
assert!(Path::new(&path).is_absolute());
path
}
#[test]
#[ignore = "requires the explicit host Node fixture path"]
fn runs_javascript_runner_from_explicit_fixture_runtime_path() {
run_fixture_runtime("npm", "/root/.nvm/versions/node/v24.15.0/bin/node");
run_fixture_runtime("npm", &fixture_runtime("DEVER_TEST_NODE"));
}
#[test]
#[ignore = "requires the explicit host Python fixture path"]
fn runs_python_runner_from_explicit_fixture_runtime_path() {
run_fixture_runtime("pip", "/usr/bin/python3.12");
run_fixture_runtime("pip", &fixture_runtime("DEVER_TEST_PYTHON"));
}
#[test]
@@ -1042,7 +1117,7 @@ fn binds_commonjs_exports_without_a_registration_table() {
&temp,
&program,
&resources,
Some("/root/.nvm/versions/node/v24.15.0/bin/node"),
Some(&fixture_runtime("DEVER_TEST_NODE")),
);
}
@@ -1055,7 +1130,7 @@ fn binds_esm_exports_without_a_registration_table() {
&temp,
&program,
&resources,
Some("/root/.nvm/versions/node/v24.15.0/bin/node"),
Some(&fixture_runtime("DEVER_TEST_NODE")),
);
}
@@ -1070,7 +1145,12 @@ fn imports_packaged_sdk_before_adapter_modules() {
"raise RuntimeError('shadowed SDK')\n",
)
.unwrap();
execute_generated_fixture(&temp, &program, &resources, Some("/usr/bin/python3.12"));
execute_generated_fixture(
&temp,
&program,
&resources,
Some(&fixture_runtime("DEVER_TEST_PYTHON")),
);
}
#[cfg(unix)]
@@ -1080,18 +1160,31 @@ fn compiled_managed_worker_runs_after_source_and_pack_are_removed() {
use std::os::unix::process::CommandExt;
let temp = TemporaryDirectory::new();
let binary = fs::read("/root/.nvm/versions/node/v24.15.0/bin/node").unwrap();
let binary = fs::read(&fixture_runtime("DEVER_TEST_NODE")).unwrap();
let (program, resources, sources) =
fixture_entry_with_sources(temp.path(), "npm", &binary, None, "worker/main.mjs");
let source_entry = program.external_worker_contracts()[0].entry.clone();
fs::write(
temp.path().join(&source_entry),
r#"
import { execFileSync } from 'node:child_process';
export const send = ({ value }) => {
const receipt = Number(execFileSync(process.execPath,
['-e', 'process.stdout.write(String(Number(process.argv[1]) + 1))', String(value)],
{ encoding: 'utf8', timeout: 3000, maxBuffer: 1024 }));
return { receipt };
};
"#,
)
.unwrap();
let raw_pack_path = temp.path().join("runtime.pack");
fs::write(&raw_pack_path, &resources[0].bytes).unwrap();
let raw = resource(resources[0].path.clone(), fs::read(&raw_pack_path).unwrap());
let packaged = prepare_sandboxed_worker(temp.path(), &program, vec![raw]);
let packaged = prepare_managed_worker(temp.path(), &program, vec![raw]);
let native = dever_core::native::compile_project_with_resources(
&program,
&sources,
std::ffi::OsStr::new("/root/.rustup/toolchains/stable-x86_64-unknown-linux-gnu/bin/rustc"),
std::ffi::OsStr::new("rustc"),
dever_runtime::config::RuntimeProfile::default(),
&packaged,
)
@@ -111,21 +111,16 @@ fn command_preparation_checks_recursive_library_closure_and_loader() {
.unwrap_err()
.contains("signed target GNU loader")
);
let loader_name = dever_sandbox::loader_name();
let loader_name = dever_process::loader_name();
let mut loader = resource(
format!("sandbox/lib/{loader_name}"),
format!("process-runtime/lib/{loader_name}"),
native_elf::static_executable(),
);
loader.executable = true;
let resources = workers::prepare(temp.path(), &program, &[loader]).unwrap();
let manifest = command_manifest(&resources);
assert!(
manifest["executable"]
.as_str()
.unwrap()
.ends_with("/loader")
);
assert_eq!(manifest["arguments"][0]["literal"], "--inhibit-cache");
assert!(manifest["executable"].as_str().unwrap().ends_with("/tool"));
assert_eq!(manifest["arguments"], json!([]));
for name in ["libfirst.so", "libsecond.so"] {
assert!(resources.iter().any(
|resource| resource.path.ends_with(&format!("/lib/{name}")) && !resource.executable
@@ -171,13 +166,13 @@ fn command_rejects_host_library_paths_and_signed_library_shadowing() {
)
.unwrap();
let signed = resource(
"sandbox/lib/libfirst.so".into(),
"process-runtime/lib/libfirst.so".into(),
native_elf::library("libfirst.so", &[]),
);
assert!(
workers::prepare(temp.path(), &program, &[signed])
.unwrap_err()
.contains("overrides a signed sandbox library")
.contains("overrides a signed process_runtime library")
);
}
@@ -0,0 +1,410 @@
use super::*;
use std::collections::BTreeMap;
use std::os::unix::process::CommandExt;
fn sources() -> SourceMap {
let mut sources = SourceMap::default();
sources.add(
"media/native/port.dever",
"call(value: Int) (output: Int) fails app.NativeError",
);
sources.add(
"media/native/adapter.dever",
"external exec \"bin/worker\" {}",
);
sources.add(
"media/native/app.dever",
r#"type NativeError { error Rejected }
call(value: Int) (output: Int) { output = port.call(value) }
stall() () { output = port.call(-4) }
cancel() (output: Int) {
output = port.call(7)
pending = run(stall())
dever.task.sleep(100)
stop(pending)
}
expired() (output: Int) { output = 99 }
deadline() (output: Int) {
pending = run(call(-4))
output = timeout(pending, 100, expired)
}"#,
);
sources.add(
"media/native/api.dever",
"cmd call = app.call\ncmd cancel = app.cancel\ncmd deadline = app.deadline",
);
sources
}
#[test]
fn real_c_abi_fixture_checks_in_both_backends() {
let sources = sources();
let program = dever_core::check(&sources).unwrap_or_else(|errors| panic!("{errors:?}"));
dever_core::native::emit_project(
&program,
&sources,
dever_runtime::config::RuntimeProfile::default(),
)
.unwrap();
dever_core::llvm::emit_application(&program, &sources).unwrap();
}
struct Fixture {
directory: TemporaryDirectory,
worker: PathBuf,
executable: PathBuf,
hello: serde_json::Value,
rejected: String,
}
impl Fixture {
fn prepare() -> Self {
let compiler = PathBuf::from(std::env::var_os("DEVER_FFI_CC").expect("set DEVER_FFI_CC"));
let assets = PathBuf::from(
std::env::var_os("DEVER_PROCESS_RUNTIME").expect("set DEVER_PROCESS_RUNTIME"),
);
for path in [&compiler, &assets] {
assert!(
path.is_absolute(),
"fixture inputs must be explicit absolute paths"
);
}
let directory = TemporaryDirectory::new();
let sources = sources();
let program = dever_core::check(&sources).unwrap_or_else(|errors| panic!("{errors:?}"));
let contract = &program.external_worker_contracts()[0];
let rejected = contract.errors.keys().next().unwrap().clone();
let worker = directory.path().join(&contract.entry);
let libraries = worker.parent().unwrap().join("lib");
fs::create_dir_all(&libraries).unwrap();
let library_source = directory.path().join("library.c");
fs::write(
&library_source,
r#"#include <stdio.h>
#include <unistd.h>
long long fixture_call(long long value) {
if (value == -3) _exit(73);
if (value == -4) {
fputs("native-wait\n", stderr);
fflush(stderr);
sleep(60);
}
return value * 3;
}
"#,
)
.unwrap();
let output = Command::new(&compiler)
.args([
"-std=c11",
"-Wall",
"-Wextra",
"-Werror",
"-shared",
"-fPIC",
"-Wl,-soname,libfixture.so",
])
.arg(&library_source)
.arg("-o")
.arg(libraries.join("libfixture.so"))
.output()
.unwrap();
assert!(
output.status.success(),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let adapter_source = directory.path().join("adapter.c");
fs::write(&adapter_source, format!(r#"#include "dever_ffi.h"
#include <stdio.h>
#include <string.h>
extern long long fixture_call(long long value);
static int invoke(void *context, dever_ffi_slice operation, dever_ffi_slice payload, dever_ffi_output *output) {{
(void)context;
if (operation.size != 4 || memcmp(operation.data, "call", 4) != 0 || payload.size >= 64) return DEVER_FFI_FAULT;
char input[64];
memcpy(input, payload.data, (size_t)payload.size);
input[payload.size] = 0;
long long value = 0;
if (sscanf(input, "{{\"value\":%lld}}", &value) != 1 || value > 1000000 || value < -1000000) return DEVER_FFI_FAULT;
if (value == -1) {{
const char *error = "{{\"error\":\"{rejected}\",\"payload\":{{}}}}";
memcpy(output->data, error, strlen(error));
output->size = strlen(error);
return DEVER_FFI_ERROR;
}}
if (value == -2) return DEVER_FFI_FAULT;
int size = snprintf(output->data, (size_t)output->capacity, "{{\"output\":%lld}}", fixture_call(value));
if (size < 0 || (uint64_t)size >= output->capacity) return DEVER_FFI_FAULT;
output->size = (uint64_t)size;
return DEVER_FFI_OK;
}}
static void closed(void *context) {{ (void)context; fputs("native-closed\n", stderr); }}
int main(void) {{
const dever_ffi_handler handler = {{DEVER_FFI_ABI_V1, NULL, NULL, invoke, closed}};
return dever_ffi_serve(&handler);
}}
"#)).unwrap();
let workspace = Path::new(env!("CARGO_MANIFEST_DIR")).join("../..");
let output = Command::new(&compiler)
.args(["-std=c11", "-Wall", "-Wextra", "-Werror", "-I"])
.arg(workspace.join("sdk/c"))
.arg(workspace.join("sdk/c/dever_ffi.c"))
.arg(&adapter_source)
.arg("-L")
.arg(&libraries)
.args(["-lfixture", "-Wl,-rpath,$ORIGIN/lib", "-o"])
.arg(&worker)
.output()
.unwrap();
assert!(
output.status.success(),
"{}",
String::from_utf8_lossy(&output.stderr)
);
let mut supplied = Vec::new();
for subdirectory in ["lib"] {
for entry in fs::read_dir(assets.join(subdirectory)).unwrap() {
let entry = entry.unwrap();
assert!(entry.file_type().unwrap().is_file());
let name = entry.file_name().into_string().unwrap();
let path = format!("process-runtime/{subdirectory}/{name}");
let input = entry.path();
let bytes = fs::read(input).unwrap();
supplied.push(EmbeddedResource {
sha256: sha(&bytes),
executable: dever_process::asset_executable(
path.strip_prefix("process-runtime/").unwrap(),
),
path,
bytes,
});
}
}
let mut entry = resource(contract.entry.clone(), fs::read(&worker).unwrap());
entry.executable = true;
supplied.push(entry);
let mut packaged = workers::prepare(directory.path(), &program, &supplied).unwrap();
let manifest = packaged
.iter()
.find(|file| file.path.ends_with("/contract.json"))
.unwrap();
fs::write(
worker.parent().unwrap().join("contract.json"),
&manifest.bytes,
)
.unwrap();
assert_eq!(
serde_json::from_slice::<serde_json::Value>(&manifest.bytes).unwrap(),
contract.sdk_manifest()
);
packaged.extend(
supplied
.into_iter()
.filter(|file| file.path.starts_with("process-runtime/")),
);
let native = dever_core::native::compile_project_with_resources(
&program,
&sources,
std::ffi::OsStr::new("rustc"),
dever_runtime::config::RuntimeProfile::default(),
&packaged,
)
.unwrap();
let executable = directory.path().join("application");
native.save(&executable).unwrap();
fs::create_dir(directory.path().join("config")).unwrap();
fs::write(
directory.path().join("config/setting.json"),
r#"{"adapter":{"media.native":{"use":"default"}}}"#,
)
.unwrap();
Self {
directory,
worker,
executable,
rejected,
hello: json!({"kind":"hello", "version":"dever-component-1", "port":contract.port,
"adapter":contract.adapter, "schema":contract.schema, "capabilities":contract.capabilities,
"operations":contract.operations, "setting":{}}),
}
}
fn capture(
&self,
command: &mut Command,
input: &[u8],
limit: Duration,
) -> (bool, Vec<u8>, String) {
let stdout = self.directory.path().join("stdout");
let stderr = self.directory.path().join("stderr");
let child = command
.env_clear()
.process_group(0)
.stdin(Stdio::piped())
.stdout(fs::File::create(&stdout).unwrap())
.stderr(fs::File::create(&stderr).unwrap())
.spawn()
.unwrap();
let mut owned = OwnedProcessGroup(child);
owned.0.stdin.take().unwrap().write_all(input).unwrap();
let mut descendants = Descendants::default();
let deadline = Instant::now() + limit;
let status = loop {
descendants.observe(owned.0.id());
if let Some(status) = owned.0.try_wait().unwrap() {
break status;
}
assert!(
Instant::now() < deadline,
"fixture exceeded its bounded process deadline"
);
thread::sleep(Duration::from_millis(5));
};
let cleanup_deadline = Instant::now() + Duration::from_secs(3);
loop {
let alive = descendants
.0
.iter()
.any(|(pid, started)| process_start(*pid) == Some(*started));
if !alive {
break;
}
assert!(
Instant::now() < cleanup_deadline,
"native Worker descendants were not reaped: {descendants:?}"
);
thread::sleep(Duration::from_millis(5));
}
(
status.success(),
fs::read(stdout).unwrap(),
fs::read_to_string(stderr).unwrap(),
)
}
}
#[derive(Default, Debug)]
struct Descendants(BTreeMap<u32, u64>);
impl Descendants {
fn observe(&mut self, parent: u32) {
let mut pending = vec![parent];
pending.extend(
self.0.iter().filter_map(|(pid, started)| {
(process_start(*pid) == Some(*started)).then_some(*pid)
}),
);
while let Some(parent) = pending.pop() {
let children = fs::read_to_string(format!("/proc/{parent}/task/{parent}/children"))
.unwrap_or_default();
for child in children
.split_whitespace()
.map(|pid| pid.parse::<u32>().unwrap())
{
if let Some(started) = process_start(child)
&& self.0.insert(child, started).is_none()
{
pending.push(child);
}
}
}
}
}
impl Drop for Descendants {
fn drop(&mut self) {
for (pid, started) in &self.0 {
if process_start(*pid) == Some(*started)
&& let Some(pid) = rustix::process::Pid::from_raw(*pid as i32)
{
let _ = rustix::process::kill_process(pid, rustix::process::Signal::KILL);
}
}
}
}
fn process_start(pid: u32) -> Option<u64> {
let stat = fs::read_to_string(format!("/proc/{pid}/stat")).ok()?;
stat.rsplit_once(") ")?
.1
.split_whitespace()
.nth(19)?
.parse()
.ok()
}
#[test]
#[ignore = "requires explicit DEVER_FFI_CC, DEVER_PROCESS_RUNTIME inputs"]
fn real_c_abi_packages_compiles_and_checks_the_generated_protocol() {
let fixture = Fixture::prepare();
for value in [7, -1, -2, -3] {
let input = [
frame(fixture.hello.clone()),
frame(json!({"kind":"call", "id":1, "operation":"call", "payload":{"value":value}})),
frame(json!({"kind":"shutdown"})),
]
.concat();
let (success, stdout, stderr) = fixture.capture(
Command::new(&fixture.worker).current_dir(fixture.worker.parent().unwrap()),
&input,
Duration::from_secs(5),
);
let mut frames = stdout.as_slice();
assert_eq!(response(&mut frames).unwrap()["kind"], "ready", "{stderr}");
if value >= -1 {
assert!(success, "{stderr}");
let reply = response(&mut frames).unwrap();
if value == 7 {
assert_eq!(reply["payload"]["output"], 21);
} else {
assert_eq!(reply["error"], fixture.rejected);
}
assert_eq!(response(&mut frames).unwrap()["kind"], "shutdown");
assert!(stderr.contains("native-closed"));
} else {
assert!(!success);
assert!(frames.is_empty());
assert_eq!(stderr.contains("native-closed"), value == -2);
}
assert!(frames.is_empty());
}
}
#[test]
#[ignore = "requires explicit DEVER_FFI_CC and DEVER_PROCESS_RUNTIME inputs"]
fn real_c_abi_direct_worker_errors_deadlines_and_cancellation() {
let fixture = Fixture::prepare();
fs::remove_dir_all(fixture.worker.parent().unwrap()).unwrap();
for (entry, input, expected) in [
("call", "{\"value\":7}", Some(21)),
("call", "{\"value\":-1}", None),
("call", "{\"value\":-2}", None),
("call", "{\"value\":-3}", None),
("call", "{\"value\":-4}", None),
("cancel", "{}", None),
("deadline", "{}", None),
] {
let (success, stdout, stderr) = fixture.capture(
Command::new(&fixture.executable)
.current_dir(fixture.directory.path())
.args([&format!("media.native.{entry}"), input]),
&[],
Duration::from_secs(45),
);
if let Some(expected) = expected {
assert!(success, "{entry} {input}: {stderr}");
let reply: serde_json::Value = serde_json::from_slice(&stdout).unwrap();
assert_eq!(reply["code"], 0, "{reply}; {stderr}");
assert_eq!(reply["data"], expected, "{reply}; {stderr}");
} else {
assert!(!success, "{entry} {input}: {stderr}");
if input.contains("-1") {
assert!(stderr.contains("Rejected"), "{stderr}");
}
if entry != "call" || input.contains("-4") {
assert!(stderr.contains("timed out"), "{stderr}");
}
}
}
}
+2 -2
View File
@@ -67,7 +67,7 @@ exports.send = async payload => {
assert_eq!(sha(&bytes), artifact.sha256);
resources.push(resource(artifact.path.clone(), bytes));
}
for asset in sandbox::assets(&workspace) {
for asset in process_runtime::assets(&workspace) {
let mut file = resource(asset.path, asset.bytes);
file.executable = asset.executable;
resources.push(file);
@@ -112,7 +112,7 @@ exports.send = async payload => {
generated.extend(
resources
.into_iter()
.filter(|file| file.path.starts_with("sandbox/")),
.filter(|file| file.path.starts_with("process-runtime/")),
);
fs::remove_file(entry).unwrap();
run_packaged_protocol(&temp, &contract, "npm", &generated);
+43 -22
View File
@@ -38,42 +38,31 @@ mod native_objects;
mod targets;
#[test]
fn sandbox_bootstrap_requires_static_elf_and_complete_guard_libraries() {
let loader = format!("lib/{}", dever_sandbox::loader_name());
fn process_runtime_requires_complete_loader_libraries_and_rejects_helpers() {
let loader = dever_process::loader_name();
let mut files = vec![
("bin/bwrap".into(), native_elf::static_executable()),
(
"bin/guard".into(),
native_elf::elf(None, &["libc.so.6"], None),
),
(
loader,
native_elf::library(dever_sandbox::loader_name(), &[]),
format!("lib/{loader}"),
native_elf::library(loader, &["libc.so.6"]),
),
(
"lib/libc.so.6".into(),
native_elf::library("libc.so.6", &[]),
),
];
dever_sandbox::validate_assets(&files).unwrap();
files[0].1 = native_elf::elf(None, &[], None);
dever_process::validate_assets(&files).unwrap();
files.push(("bin/bwrap".into(), native_elf::static_executable()));
assert!(
dever_sandbox::validate_assets(&files)
dever_process::validate_assets(&files)
.unwrap_err()
.contains("static target ELF")
.contains("invalid")
);
files[0].1 = native_elf::library("bwrap", &["libc.so.6"]);
assert!(
dever_sandbox::validate_assets(&files)
.unwrap_err()
.contains("static target ELF")
);
files[0].1 = native_elf::static_executable();
files.pop();
files.pop();
assert!(
dever_sandbox::validate_assets(&files)
dever_process::validate_assets(&files)
.unwrap_err()
.contains("sandbox dependency 'lib/libc.so.6' is missing")
.contains("lib/libc.so.6")
);
}
@@ -354,6 +343,38 @@ fn compiler_libraries_must_form_a_closed_target_and_search_path() {
assert!(author.reject().contains("inherited RPATH"));
}
#[test]
fn compiler_libraries_follow_the_linked_soname_not_a_distribution_alias() {
let mut author = Author::new();
let name = "libLLVM-18.so.18.1";
fs::write(
author.root.join("inputs/core"),
native_elf::compiler(&[name, "libc.so.6"]),
)
.unwrap();
fs::write(
author.root.join("inputs/llvm"),
native_elf::library(name, &["libc.so.6"]),
)
.unwrap();
author.settings["core"]["sha256"] =
json!(sha256_file(&author.root.join("inputs/core")).unwrap());
author.settings["core_libraries"][0]["path"] = json!(name);
author.settings["core_libraries"][0]["sha256"] =
json!(sha256_file(&author.root.join("inputs/llvm")).unwrap());
author.save();
let output = author.output("linked-soname");
let manifest = packaging::create(&author.root, &output).unwrap();
assert!(
manifest
.artifacts
.iter()
.any(|artifact| artifact.path == format!("lib/{name}"))
);
assert!(!output.join("lib/libLLVM.so.18.1").exists());
assert_signed(&author, &output);
}
#[test]
fn installation_rechecks_the_signed_native_compiler_closure_before_health_execution() {
let author = Author::new();
+112 -23
View File
@@ -17,8 +17,8 @@ mod daemon;
mod inputs;
#[path = "../../dever-tests/tests/support/process.rs"]
mod process;
#[path = "../../dever-tests/tests/support/sandbox.rs"]
mod sandbox;
#[path = "../../dever-tests/tests/support/process_runtime.rs"]
mod process_runtime;
use super::temp::TemporaryDirectory;
#[path = "ecosystem_acceptance.rs"]
@@ -147,11 +147,15 @@ fn author_inputs(author: &Path, workspace: &Path) -> Ed25519KeyPair {
),
);
}
let compiler = input(author, Path::new(env!("CARGO_BIN_EXE_dever")), "dever");
let compiler = input(
author,
&workspace.join("target/native-release-inputs/bin/dever"),
"dever",
);
let core_libraries = inputs::core_libraries()
.into_iter()
.map(|(name, source)| {
let mut value = input(author, &source, name);
let mut value = input(author, &source, &name);
value["path"] = json!(name);
value
})
@@ -171,11 +175,11 @@ fn author_inputs(author: &Path, workspace: &Path) -> Ed25519KeyPair {
.map(|name| links[*name].clone())
.collect::<Vec<_>>()
};
let sandbox = sandbox::assets(workspace).into_iter().map(|asset| {
let process_runtime = process_runtime::assets(workspace).into_iter().map(|asset| {
let relative = format!("inputs/{}", asset.path);
write(author, &relative, &asset.bytes);
fs::set_permissions(author.join(&relative), fs::Permissions::from_mode(if asset.executable { 0o755 } else { 0o644 })).unwrap();
json!({"source":relative, "path":asset.path.strip_prefix("sandbox/").unwrap(), "sha256":sha256_file(&author.join(&relative)).unwrap()})
json!({"source":relative, "path":asset.path.strip_prefix("process-runtime/").unwrap(), "sha256":sha256_file(&author.join(&relative)).unwrap()})
}).collect::<Vec<_>>();
write(
author,
@@ -193,7 +197,7 @@ fn author_inputs(author: &Path, workspace: &Path) -> Ed25519KeyPair {
"skill":[skill],
"start":ordered(&["crt1.o","crti.o","crtbeginT.o"]),
"libraries":ordered(&["libc.a","libm.a","libmvec.a","libgcc.a","libgcc_eh.a"]),
"end":ordered(&["crtend.o","crtn.o"]), "sandbox":sandbox
"end":ordered(&["crtend.o","crtn.o"]), "process_runtime":process_runtime
}))
.unwrap(),
);
@@ -211,10 +215,11 @@ fn isolated_os_command(root: &Path) -> Command {
}
fn isolated_os_namespace(root: &Path) -> Command {
let assets = Path::new(env!("CARGO_MANIFEST_DIR"))
.join("../../target/sandbox-inputs/assets")
.canonicalize()
.expect("prepare the explicit Linux isolation assets");
let isolator = PathBuf::from(
std::env::var_os("DEVER_TEST_ISOLATOR")
.expect("set an absolute test-only namespace isolator path"),
);
assert!(isolator.is_absolute());
let init = Path::new(env!("CARGO_MANIFEST_DIR"))
.join("../../target/debug/examples/native-acceptance-init")
.canonicalize()
@@ -222,10 +227,7 @@ fn isolated_os_namespace(root: &Path) -> Command {
let bytes = fs::read(&init).unwrap();
let executable = goblin::elf::Elf::parse(&bytes).unwrap();
assert!(executable.interpreter.is_none() && executable.libraries.is_empty());
// A plain chroot cannot create a nested user namespace. Use a real mount
// namespace root so the program can retain its normal Worker sandbox.
dever_sandbox::validate_bwrap(&fs::read(assets.join("bin/bwrap")).unwrap()).unwrap();
let mut command = Command::new(assets.join("bin/bwrap"));
let mut command = Command::new(isolator);
command
.env_clear()
.args([
@@ -301,6 +303,33 @@ fn application(root: &Path, profile: &str) {
"exercise() (okay: Bool) { dever.task.sleep(1)\n okay = true }\n"
};
write(root, "module/sample/value/app.dever", app);
if matches!(profile, "sqlite" | "both") {
write(
root,
"module/sample/value/api.dever",
"cmd exercise = app.exercise\nlong cmd stages = app.stages\n",
);
write(
root,
"module/sample/value/app.dever",
format!(
"{app}\nstages() (okay: Bool) {{ id = store()\n dever.task.sleep(1)\n okay = finish(id) }}\ntransaction store() (id: model.id) {{ saved = model.create({{ name = \"stage\" }})\n id = saved.id }}\ntransaction finish(id: model.id) (okay: Bool) {{ count = model.count()\n removed = model.delete(id)\n okay = count == 1 and removed == 1 }}\n"
),
);
}
}
#[test]
fn release_profile_projects_have_valid_source_contracts() {
for profile in PROFILES {
let project = TemporaryDirectory::new();
application(project.path(), profile);
command(
Command::new(env!("CARGO_BIN_EXE_dever"))
.arg("check")
.arg(project.path()),
);
}
}
fn assert_response(output: Output) {
@@ -489,7 +518,7 @@ fn signed_core_builds_without_host_compiler_libraries_or_tools() {
#[test]
#[ignore = "requires four optimized archives and component-fixture; starts only an owned daemon, SQLite and exec Worker fixtures"]
fn optimized_profiles_make_reproducible_signed_release_and_run_through_daemon() {
let workspace = Path::new(env!("CARGO_MANIFEST_DIR")).join("../..");
let workspace = inputs::workspace();
let profiles = archive_report(&workspace);
let author = TemporaryDirectory::new();
let key = author_inputs(author.path(), &workspace);
@@ -505,7 +534,7 @@ fn optimized_profiles_make_reproducible_signed_release_and_run_through_daemon()
let version = Version::parse(env!("CARGO_PKG_VERSION")).unwrap();
let release = layout.downloads().join(version.as_str());
command(
Command::new(workspace.join("target/debug/examples/native-release"))
Command::new(workspace.join("target/native-release-inputs/bin/native-release"))
.arg(author.path())
.arg("--output")
.arg(&release),
@@ -523,10 +552,13 @@ fn optimized_profiles_make_reproducible_signed_release_and_run_through_daemon()
fs::remove_dir_all(&release).unwrap();
drop(author);
let compiler = layout.versions().join(version.as_str()).join("dever-core");
let mut daemon = daemon::TestDaemon::start(&layout);
let mut daemon = daemon::TestDaemon::start_with_executable(
&layout,
&workspace.join("target/native-release-inputs/bin/deverd"),
);
let projects = TemporaryDirectory::new();
let mut binaries = serde_json::Map::new();
let mut programs: Vec<(PathBuf, PathBuf)> = Vec::new();
let mut programs = Vec::new();
for profile in PROFILES {
let project = projects.path().join(profile);
application(&project, profile);
@@ -538,6 +570,15 @@ fn optimized_profiles_make_reproducible_signed_release_and_run_through_daemon()
"{}",
]),
));
if matches!(profile, "sqlite" | "both") {
assert_response(command(
Command::new(&compiler).arg("run").arg(&project).args([
"--",
"sample.value.stages",
"{}",
]),
));
}
let executable = project.join("program");
command(
Command::new(&compiler)
@@ -547,7 +588,7 @@ fn optimized_profiles_make_reproducible_signed_release_and_run_through_daemon()
.arg(&executable),
);
binaries.insert(profile.into(), json!({"bytes":fs::metadata(&executable).unwrap().len(),"sha256":sha256_file(&executable).unwrap()}));
programs.push((project, executable));
programs.push((profile, project, executable));
}
assert_eq!(cache_status(&layout).unwrap().entries, 4);
let exec_project = TemporaryDirectory::new();
@@ -595,19 +636,25 @@ fn optimized_profiles_make_reproducible_signed_release_and_run_through_daemon()
"managed_run": true, "standalone_run": true,
"source_project_removed": true, "response": 7
});
for (project, executable) in programs {
for (profile, project, executable) in programs {
fs::remove_dir_all(project.join("module")).unwrap();
assert_response(command(
Command::new(executable).args(["sample.value.exercise", "{}"]),
Command::new(&executable).args(["sample.value.exercise", "{}"]),
));
if matches!(profile, "sqlite" | "both") {
assert_response(command(
Command::new(executable).args(["sample.value.stages", "{}"]),
));
}
}
let report = json!({
"format":"dever-native-release-acceptance-v1", "platform":"linux-x86_64",
"version":version.as_str(), "profiles":profiles,"programs":binaries,"exec":exec,
"manifest_sha256":format!("{:x}", sha2::Sha256::digest(&manifest)),
"checks":{"pack_determinism":true,"signed_install":true,"managed_profiles":4,"standalone_profiles":4,"sqlite_crud":true,"signed_exec_worker":true},
"checks":{"pack_determinism":true,"signed_install":true,"managed_profiles":4,"standalone_profiles":4,"sqlite_crud":true,"long_staged_transactions":true,"signed_exec_worker":true},
"postgres_server":"not_run", "cross_host_source_reproducibility":"not_run"
});
retain_candidate(&layout, &version, "native", &report);
use sha2::Digest;
write(
&workspace,
@@ -615,3 +662,45 @@ fn optimized_profiles_make_reproducible_signed_release_and_run_through_daemon()
serde_json::to_vec_pretty(&report).unwrap(),
);
}
fn retain_candidate(layout: &Layout, version: &Version, name: &str, report: &Value) {
let Some(destination) = std::env::var_os("DEVER_NATIVE_ACCEPTANCE_OUTPUT") else {
return;
};
let destination = PathBuf::from(destination);
assert!(destination.is_absolute());
fs::create_dir_all(&destination).unwrap();
let destination = destination.join(name);
fs::create_dir(&destination).unwrap();
let payload = destination.join("release");
let resources = dever_cli::toolchain::SignedResources::load(layout, version).unwrap();
let manifest = resources.manifest();
for artifact in manifest.artifacts.iter().chain(
manifest
.extensions
.iter()
.flat_map(|extension| &extension.artifacts),
) {
let output = payload.join(&artifact.path);
fs::create_dir_all(output.parent().unwrap()).unwrap();
fs::copy(resources.path(&artifact.path).unwrap(), &output).unwrap();
assert_eq!(sha256_file(&output).unwrap(), artifact.sha256);
}
for file in ["manifest.json", "manifest.sig"] {
fs::copy(
layout.versions().join(version.as_str()).join(file),
payload.join(file),
)
.unwrap();
}
fs::copy(
layout.state().join("trusted-release-key"),
destination.join("test-release.pub"),
)
.unwrap();
fs::write(
destination.join("verification.json"),
serde_json::to_vec_pretty(report).unwrap(),
)
.unwrap();
}
@@ -64,7 +64,6 @@ fn bootstrap_payload_has_a_separate_signed_complete_loader_closure() {
"bootstrap/deverd",
"bootstrap/lib/libgcc_s.so.1",
"bootstrap/deverd.service",
"bootstrap/dever-sandbox",
] {
let artifact = manifest
.artifacts
@@ -78,9 +77,12 @@ fn bootstrap_payload_has_a_separate_signed_complete_loader_closure() {
assert!(unit.contains("RuntimeDirectory=dever"));
assert!(!unit.contains("Environment"));
assert!(!unit.contains("PATH"));
assert_eq!(
fs::read_to_string(release.join("bootstrap/dever-sandbox")).unwrap(),
dever_sandbox::APPARMOR_PROFILE
assert!(!release.join("bootstrap/dever-process").exists());
assert!(
!manifest
.artifacts
.iter()
.any(|artifact| artifact.path.contains("bwrap") || artifact.path.contains("apparmor"))
);
}
@@ -174,6 +176,73 @@ fn signed_installed_versions_keep_their_original_bootstrap_service_template() {
);
}
#[test]
fn signed_historical_bootstrap_profiles_are_read_only_and_version_bound() {
use dever_cli::toolchain::{Artifact, Layout, MachineManager, Version};
for release_version in ["0.1.4", "0.1.5"] {
let mut author = Author::new();
author.settings["version"] = json!(release_version);
add_bootstrap(&mut author);
author.save();
let machine = TemporaryDirectory::new();
let layout = Layout::new(machine.path().join("machine"));
layout.initialize().unwrap();
let version = Version::parse(release_version).unwrap();
let installed = layout.versions().join(release_version);
let mut manifest = packaging::create(&author.root, &installed).unwrap();
let profile = installed.join("bootstrap/dever-sandbox");
fs::write(&profile, b"historical profile fixture\n").unwrap();
manifest.artifacts.push(Artifact {
path: "bootstrap/dever-sandbox".into(),
bytes: fs::metadata(&profile).unwrap().len(),
sha256: sha256_file(&profile).unwrap(),
});
let encoded = serde_json::to_vec(&manifest).unwrap();
let key = Ed25519KeyPair::from_pkcs8(&fs::read(author.root.join("signing.pk8")).unwrap())
.unwrap();
fs::write(installed.join("manifest.json"), &encoded).unwrap();
fs::write(
installed.join("manifest.sig"),
key.sign(&encoded)
.as_ref()
.iter()
.map(|byte| format!("{byte:02x}"))
.collect::<String>(),
)
.unwrap();
fs::write(
layout.state().join("trusted-release-key"),
author
.public_key
.iter()
.map(|byte| format!("{byte:02x}"))
.collect::<String>(),
)
.unwrap();
let manager = MachineManager::new(layout);
if release_version == "0.1.4" {
assert_eq!(
manager.resolve_core(&version).unwrap(),
installed.join("dever-core")
);
fs::write(profile, b"unsigned change").unwrap();
assert!(
manager
.resolve_core(&version)
.unwrap_err()
.contains("integrity verification")
);
} else {
assert!(
manager
.resolve_core(&version)
.unwrap_err()
.contains("unsupported artifact")
);
}
}
}
#[test]
#[ignore = "root-only installer trust rejection; never touches the actual machine installation"]
fn bootstrap_installer_rejects_untrusted_inputs_before_publication() {
@@ -313,7 +382,6 @@ fn bootstrap_recovery_is_idempotent_and_rejects_unrecorded_targets() {
for directory in [
"usr/local/bin",
"etc/systemd/system/multi-user.target.wants",
"etc/apparmor.d",
] {
fs::create_dir_all(root.join(directory)).unwrap();
}
@@ -331,20 +399,12 @@ fn bootstrap_recovery_is_idempotent_and_rejects_unrecorded_targets() {
let link_hash =
|target: &str| format!("{:x}", Sha256::digest(format!("link\0{target}").as_bytes()));
let journal = json!({"format":"dever-bootstrap-transaction-v1","phase":"prepared","service_was_active":null,"service_start_requested":false,"changes":[
{"target":"sandbox","before":null,"after":node_hash("directory",b"")},
{"target":"apparmor","before":null,"after":node_hash("file",b"new profile")},
{"target":"bin","before":node_hash("directory",b""),"after":node_hash("directory",b"")},
{"target":"service","before":null,"after":node_hash("file",b"new service")},
{"target":"entry","before":null,"after":link_hash("/opt/dever/bin/dever")},
{"target":"enable","before":null,"after":link_hash("../deverd.service")},
{"target":"active","before":null,"after":node_hash("file",b"new active")}
]});
fs::create_dir(root.join("opt/dever/.dever-bootstrap-sandbox.next")).unwrap();
fs::write(
root.join("etc/apparmor.d/.dever-bootstrap-dever-sandbox.next"),
b"new profile",
)
.unwrap();
let path = layout.state().join("bootstrap-install.json");
fs::write(&path, serde_json::to_vec(&journal).unwrap()).unwrap();
let mut invalid = journal.clone();
@@ -374,8 +434,6 @@ fn bootstrap_recovery_is_idempotent_and_rejects_unrecorded_targets() {
// undo the new active version, including after a failed cleanup retry.
fs::write(&service, b"new service").unwrap();
symlink("/opt/dever/bin/dever", &entry).unwrap();
fs::create_dir(root.join("opt/dever/sandbox")).unwrap();
fs::write(root.join("etc/apparmor.d/dever-sandbox"), b"new profile").unwrap();
symlink(
"../deverd.service",
root.join("etc/systemd/system/multi-user.target.wants/deverd.service"),
@@ -212,24 +212,13 @@ fn signed_bootstrap_installs_and_serves_two_projects_without_host_libraries() {
.arg(installer.path()),
);
let layout = Layout::new(image.path().join("opt/dever"));
let sandbox_artifact = manifest
.artifacts
.iter()
.find(|artifact| artifact.path == "sandbox/linux-x86_64/bin/bwrap")
.unwrap();
let sandbox_store = layout.root().join("sandbox");
let sandbox_helper = sandbox_store.join(&sandbox_artifact.sha256).join("bwrap");
assert_eq!(
sha256_file(&sandbox_helper).unwrap(),
sandbox_artifact.sha256
);
assert_eq!(
fs::metadata(&sandbox_helper).unwrap().mode() & 0o7777,
0o755
);
assert_eq!(
fs::read_to_string(image.path().join("etc/apparmor.d/dever-sandbox")).unwrap(),
dever_sandbox::APPARMOR_PROFILE
assert!(!image.path().join("etc/apparmor.d/dever-process").exists());
assert!(
!manifest
.artifacts
.iter()
.any(|artifact| artifact.path.ends_with("/bin/bwrap")
|| artifact.path.ends_with("/bin/guard"))
);
let launcher = layout.bin().join("dever");
let executable = layout.bin().join("deverd");
@@ -305,38 +294,6 @@ fn signed_bootstrap_installs_and_serves_two_projects_without_host_libraries() {
active
);
fs::write(&unit_path, old_unit).unwrap();
// Retain a different, correctly signed static helper identity through an
// upgrade. ELF trailing padding keeps this executable's actual behavior.
let mut previous_helper = fs::read(&sandbox_helper).unwrap();
previous_helper.push(0);
let previous_hash = dever_sandbox::validate_bwrap(&previous_helper).unwrap();
let previous_root = sandbox_store.join(&previous_hash);
fs::create_dir(&previous_root).unwrap();
fs::write(previous_root.join("bwrap"), &previous_helper).unwrap();
fs::set_permissions(
previous_root.join("bwrap"),
fs::Permissions::from_mode(0o755),
)
.unwrap();
let mut previous_manifest = manifest.clone();
let previous_artifact = previous_manifest
.artifacts
.iter_mut()
.find(|artifact| artifact.path == sandbox_artifact.path)
.unwrap();
previous_artifact.sha256 = previous_hash.clone();
previous_artifact.bytes = previous_helper.len() as u64;
let previous_encoded = serde_json::to_vec(&previous_manifest).unwrap();
fs::write(
previous_root.join("manifest.sig"),
key.sign(&previous_encoded)
.as_ref()
.iter()
.map(|byte| format!("{byte:02x}"))
.collect::<String>(),
)
.unwrap();
fs::write(previous_root.join("manifest.json"), previous_encoded).unwrap();
// Retry is supported, but a later damaged package cannot damage the
// published launcher or compiler selection.
command(
@@ -345,15 +302,6 @@ fn signed_bootstrap_installs_and_serves_two_projects_without_host_libraries() {
.arg(installer.path()),
);
assert_eq!(fs::read_to_string(&unit_path).unwrap(), current_unit);
assert_eq!(
sha256_file(&sandbox_helper).unwrap(),
sandbox_artifact.sha256
);
assert_eq!(
sha256_file(&previous_root.join("bwrap")).unwrap(),
previous_hash
);
assert_eq!(fs::read_dir(&sandbox_store).unwrap().count(), 2);
let before_bad_install = fs::read(layout.state().join("active-version")).unwrap();
fs::write(
release.join("bootstrap/lib/libgcc_s.so.1"),
@@ -98,23 +98,11 @@ fn packaged_project(root: &Path, source: &Path, layout: &Layout) {
}
#[test]
#[ignore = "requires explicit current runtime archive, author C/LLVM inputs and owned daemon/sandbox"]
#[ignore = "requires explicit current runtime archive, author C/LLVM inputs and owned daemon/process_runtime"]
fn signed_cli_runs_and_packages_ordinary_command_with_private_library() {
let workspace = Path::new(env!("CARGO_MANIFEST_DIR")).join("../..");
let workspace = inputs::workspace();
let author = TemporaryDirectory::new();
let key = author_inputs(author.path(), &workspace);
// This test validates the command boundary, not release-profile trimming.
// Use the explicitly rebuilt archive containing the current wire ABI.
let current = input(
author.path(),
&workspace.join("target/native-runtime-abi/debug/libdever_backend_bridge.a"),
"command-base.a",
);
let settings_path = author.path().join("config/setting.json");
let mut settings: Value = serde_json::from_slice(&fs::read(&settings_path).unwrap()).unwrap();
settings["profiles"]["base"] = current;
fs::write(settings_path, serde_json::to_vec(&settings).unwrap()).unwrap();
let machine = machine_temporary_directory(&workspace);
let layout = Layout::new(machine.path());
layout.initialize().unwrap();
@@ -123,7 +111,10 @@ fn signed_cli_runs_and_packages_ordinary_command_with_private_library() {
packaging::create(author.path(), &release).unwrap();
install_release(&layout, &version, &key);
let compiler = layout.versions().join(version.as_str()).join("dever-core");
let mut daemon = daemon::TestDaemon::start(&layout);
let mut daemon = daemon::TestDaemon::start_with_executable(
&layout,
&workspace.join("target/native-release-inputs/bin/deverd"),
);
let source = TemporaryDirectory::new();
project(source.path(), &workspace);
command(Command::new(&compiler).arg("check").arg(source.path()));
@@ -1,5 +1,5 @@
//! Explicit x86 compiler -> ARM application acceptance; QEMU user mode does
//! not establish the namespace/seccomp contract of external Workers.
//! not establish external Worker lifecycle behavior on a real target kernel.
use super::*;
use dever_cli::toolchain::BuildTarget;
use dever_cli::toolchain::compilation::{CompileKind, CompileRequest};
@@ -281,7 +281,7 @@ fn signed_host_compiler_builds_arm_profiles_and_runs_standalone_under_qemu() {
"format":"dever-cross-application-acceptance-v1", "host":"linux-x86_64", "target":"linux-aarch64",
"programs":evidence, "http":http, "qemu_sha256":sha256_file(&emulator).unwrap(),
"checks":{"signed_install":true,"profiles":4,"private_arm_build":true,"standalone_after_source_and_machine_removal":true,"sqlite_crud":true,"cache_entries":6,"cache_target_isolation":true,"tampered_target_cache_hit_rejected":true},
"postgres_server":"not_run", "worker_sandbox":"requires_arm_kernel_system_emulation"
"postgres_server":"not_run", "worker_execution":"requires_arm_kernel_system_emulation"
});
write(
&workspace,
@@ -15,8 +15,8 @@ fn digest(bytes: &[u8]) -> String {
format!("{:x}", Sha256::digest(bytes))
}
fn add_target_sandbox(author: &Path, workspace: &Path) {
let root = workspace.join("target/arm64-cross/sandbox");
fn add_target_process_runtime(author: &Path, workspace: &Path) {
let root = workspace.join("target/arm64-cross/process_runtime");
let mut files = Vec::new();
let mut inputs = Vec::new();
for directory in ["bin", "lib"] {
@@ -25,16 +25,16 @@ fn add_target_sandbox(author: &Path, workspace: &Path) {
assert!(entry.file_type().unwrap().is_file());
let relative = format!("{directory}/{}", entry.file_name().to_str().unwrap());
let bytes = fs::read(entry.path()).unwrap();
let source = format!("inputs/arm-sandbox/{relative}");
let source = format!("inputs/arm-process-runtime/{relative}");
write(author, &source, &bytes);
inputs.push(json!({"source":source,"path":relative,"sha256":digest(&bytes)}));
files.push((relative, bytes));
}
}
dever_sandbox::validate_assets_for_target(&files, TARGET.platform()).unwrap();
dever_process::validate_assets_for_target(&files, TARGET.platform()).unwrap();
let path = author.join("config/setting.json");
let mut settings: Value = serde_json::from_slice(&fs::read(&path).unwrap()).unwrap();
settings["sandbox_targets"] = json!({"linux-aarch64":inputs});
settings["process_runtime_targets"] = json!({"linux-aarch64":inputs});
fs::write(path, serde_json::to_vec(&settings).unwrap()).unwrap();
}
@@ -267,7 +267,7 @@ fn prepare_locked_application(
}
#[test]
#[ignore = "requires explicit ARM runtimes/dependencies/sandbox and native archives; builds with an owned host daemon and exports apps for ARM system QEMU"]
#[ignore = "requires explicit ARM runtimes/dependencies/process_runtime and native archives; builds with an owned host daemon and exports apps for ARM system QEMU"]
fn signed_arm_ecosystems_build_offline_standalone_apps_for_system_guest() {
build_guest_apps(&ECOSYSTEMS, "ecosystem-build-acceptance.json");
}
@@ -294,7 +294,7 @@ fn build_guest_apps(ecosystems: &[&str], report: &str) {
let author = TemporaryDirectory::new();
let key = author_inputs(author.path(), &workspace);
add_arm_inputs(author.path(), &workspace);
add_target_sandbox(author.path(), &workspace);
add_target_process_runtime(author.path(), &workspace);
let machine = machine_temporary_directory(&workspace);
let layout = Layout::new(machine.path());
layout.initialize().unwrap();
@@ -349,6 +349,6 @@ fn build_guest_apps(ecosystems: &[&str], report: &str) {
"checks":{"signed_runtime_install":true,"offline_lib_doctor":ecosystems.len(),"actual_cli_arm_build":ecosystems.len(),"source_and_build_machine_removed":true},
"guest_probe":{"arguments":["sample.worker.probe","{}"],"expected_data":true},
"guest_failure":{"arguments":["sample.worker.reject","{}"],"expected":"runtime fault; next fresh probe still succeeds; no remaining Worker"},
"guest_execution":"not_run", "sandbox_execution":"requires full ARM kernel guest"
"guest_execution":"not_run", "worker_execution":"requires ARM system guest verification"
})).unwrap());
}
@@ -204,6 +204,115 @@ fn ecosystem_inputs(author: &Path, workspace: &Path) {
fs::write(config, serde_json::to_vec(&settings).unwrap()).unwrap();
}
#[test]
#[ignore = "requires explicit native and zero-Lib ecosystem packs; uses an owned root installation and daemon"]
fn signed_zero_lib_ecosystems_run_and_build_in_the_current_container() {
let workspace = inputs::workspace();
let author = TemporaryDirectory::new();
let key = author_inputs(author.path(), &workspace);
ecosystem_inputs(author.path(), &workspace);
let machine = machine_temporary_directory(&workspace);
let layout = Layout::new(machine.path());
layout.initialize().unwrap();
let version = Version::parse(env!("CARGO_PKG_VERSION")).unwrap();
let release = layout.downloads().join(version.as_str());
packaging::create(author.path(), &release).unwrap();
install_release(&layout, &version, &key);
fs::remove_dir_all(&release).unwrap();
drop(author);
let core = layout.versions().join(version.as_str()).join("dever-core");
let mut daemon = daemon::TestDaemon::start_with_executable(
&layout,
&workspace.join("target/native-release-inputs/bin/deverd"),
);
let deployments = TemporaryDirectory::new();
let mut programs = Vec::new();
for ecosystem in ECOSYSTEMS {
let project = TemporaryDirectory::new();
let (extension, source) = match ecosystem {
"pip" => (
"py",
"import gzip\nasync def probe(payload, setting):\n return {\"okay\": gzip.decompress(gzip.compress(b'dever')) == b'dever'}\n",
),
"npm" => (
"mjs",
"import {createHash} from 'node:crypto';\nexport async function probe() { return {okay:createHash('sha256').update('dever').digest().length === 32}; }\n",
),
"go" => ("go", include_str!("../../ecosystem-release/probe.go")),
_ => unreachable!(),
};
write(project.path(), "config/setting.json", "{}");
write(
project.path(),
"module/sample/worker/port.dever",
"probe() (okay: Bool) fails app.ProbeResult\n",
);
write(
project.path(),
"module/sample/worker/app.dever",
"type ProbeResult { error Unavailable(message: Text) }\nprobe() (okay: Bool) { okay = port.probe() }\n",
);
write(
project.path(),
"module/sample/worker/api.dever",
"cmd probe = app.probe\n",
);
write(
project.path(),
"module/sample/worker/adapter.dever",
format!("external {ecosystem} \"worker/probe.{extension}\" {{}}\n"),
);
write(
project.path(),
&format!("module/sample/worker/worker/probe.{extension}"),
source,
);
command(Command::new(&core).arg("check").arg(project.path()));
command(Command::new(&core).args(["lib", "add"]).arg(project.path()));
assert_response(command(
Command::new(&core).arg("run").arg(project.path()).args([
"--",
"sample.worker.probe",
"{}",
]),
));
let destination = deployments.path().join(ecosystem);
write(&destination, "config/setting.json", "{}");
let program = destination.join("program");
command(
Command::new(&core)
.arg("build")
.arg(project.path())
.arg("--output")
.arg(&program),
);
programs.push((ecosystem, destination, program));
}
daemon.stop();
for (ecosystem, destination, program) in programs {
assert_response(command(
Command::new(program)
.current_dir(destination)
.args(["sample.worker.probe", "{}"]),
));
eprintln!(
"{ecosystem}: signed runtime, managed execution and source-free standalone execution passed"
);
}
retain_candidate(
&layout,
&version,
"ecosystems",
&json!({
"format":"dever-zero-lib-release-acceptance-v1",
"platform":"linux-x86_64", "version":version.as_str(),
"checks":{"signed_install":true,"managed_ecosystems":3,"standalone_ecosystems":3,"source_projects_removed":true},
"third_party_packages":"not_run", "host_language_absence":"not_verified",
"gpu":"not_run", "public_release":"not_published"
}),
);
}
fn application(root: &Path, ecosystem: &str) {
let (extension, source) = match ecosystem {
"pip" => ("py", include_str!("../../ecosystem-release/probe.py")),
@@ -389,10 +498,10 @@ fn signed_ecosystems_build_and_run_without_system_language_installations() {
nonroot_standalone(&isolated.path().join(ecosystem), &programs[ecosystem]),
);
}
write(&workspace, "target/closure-sandbox-ecosystem-release-acceptance.json", serde_json::to_vec_pretty(&json!({
write(&workspace, "target/closure-process_runtime-ecosystem-release-acceptance.json", serde_json::to_vec_pretty(&json!({
"format":"dever-ecosystem-release-acceptance-v1", "platform":"linux-x86_64",
"runtimes":runtimes,"programs":programs,"standalone_nonroot":standalone_nonroot,
"checks":{"signed_install":true,"locked_runtimes":3,"managed_check_run_build":3,"standalone_isolated_root":3,"standalone_nonroot":3,"shared_compile_entries":3,"python_stdlib_paths_in_bundle":true,"os_sandbox":true,"offline_build_receipt_replay":true},
"checks":{"signed_install":true,"locked_runtimes":3,"managed_check_run_build":3,"standalone_isolated_root":3,"standalone_nonroot":3,"shared_compile_entries":3,"python_stdlib_paths_in_bundle":true,"direct_workers":true,"offline_build_receipt_replay":true},
"os_baseline":"explicit Linux GNU OS libraries and named device nodes inside an owned root",
"runtime_pack_provenance":"test author signed the exact pip/npm packs bound to the completed source builds; maker determinism is verified separately",
"third_party_packages":{"pip":"simplejson@3.20.1 _speedups C extension","npm":"bufferutil@4.0.9 build/Release addon"}, "other_platforms":"not_run", "public_release":"not_published"
@@ -298,10 +298,8 @@ fn go_cross_pack_binds_host_tools_and_arm_standard_library_separately() {
#[test]
#[cfg(target_arch = "x86_64")]
fn sandbox_cross_assets_preserve_loader_mode_and_validate_deployment_target() {
fn process_runtime_cross_assets_preserve_loader_mode_and_validate_deployment_target() {
let mut files = vec![
("bin/bwrap".into(), native_elf::static_executable()),
("bin/guard".into(), native_elf::static_executable()),
(
"lib/ld-linux-aarch64.so.1".into(),
native_elf::library("ld-linux-aarch64.so.1", &[]),
@@ -314,12 +312,12 @@ fn sandbox_cross_assets_preserve_loader_mode_and_validate_deployment_target() {
for (_, bytes) in &mut files {
bytes[18..20].copy_from_slice(&183u16.to_le_bytes());
}
dever_sandbox::validate_assets_for_target(&files, "linux-aarch64").unwrap();
dever_process::validate_assets_for_target(&files, "linux-aarch64").unwrap();
let mut author = super::targets::cross_author();
let inputs = files
.iter()
.map(|(path, bytes)| {
let source = format!("inputs/arm-sandbox/{path}");
let source = format!("inputs/arm-process-runtime/{path}");
let file = author.root.join(&source);
fs::create_dir_all(file.parent().unwrap()).unwrap();
fs::write(&file, bytes).unwrap();
@@ -327,18 +325,16 @@ fn sandbox_cross_assets_preserve_loader_mode_and_validate_deployment_target() {
json!({"source":source,"path":path,"sha256":sha256_file(&file).unwrap()})
})
.collect::<Vec<_>>();
author.settings["sandbox_targets"] = json!({"linux-aarch64":inputs});
author.settings["process_runtime_targets"] = json!({"linux-aarch64":inputs});
author.save();
let output = author.output("cross-sandbox");
let output = author.output("cross-process_runtime");
packaging::create(&author.root, &output).unwrap();
for (path, mode) in [
("bin/bwrap", 0o755),
("bin/guard", 0o755),
("lib/ld-linux-aarch64.so.1", 0o755),
("lib/libc.so.6", 0o644),
] {
assert_eq!(
fs::metadata(output.join("sandbox/linux-aarch64").join(path))
fs::metadata(output.join("process-runtime/linux-aarch64").join(path))
.unwrap()
.permissions()
.mode()
@@ -349,7 +345,7 @@ fn sandbox_cross_assets_preserve_loader_mode_and_validate_deployment_target() {
}
files[1].1[18..20].copy_from_slice(&62u16.to_le_bytes());
assert!(
dever_sandbox::validate_assets_for_target(&files, "linux-aarch64")
dever_process::validate_assets_for_target(&files, "linux-aarch64")
.unwrap_err()
.contains("incompatible target")
);
@@ -2,6 +2,45 @@ use super::*;
use dever_cli::toolchain::runtime_pack::RuntimePack;
use dever_runtime::config::RuntimeProfile;
#[test]
#[cfg(target_arch = "x86_64")]
fn target_resource_namespaces_are_bound_to_the_signed_release_version() {
use dever_cli::toolchain::{
Artifact, Extension, ExtensionKind, ReleaseManifest, Version, validate_catalog,
};
let mut manifest = ReleaseManifest::new(
Version::parse("0.1.4").unwrap(),
vec![Artifact {
path: "dever-core".into(),
bytes: 1,
sha256: "0".repeat(64),
}],
);
manifest.extensions.push(Extension {
kind: ExtensionKind::Target,
target: BuildTarget::LinuxAarch64,
artifacts: vec![Artifact {
path: "sandbox/linux-aarch64/lib/libc.so.6".into(),
bytes: 1,
sha256: "0".repeat(64),
}],
});
validate_catalog(&manifest).unwrap();
manifest.version = Version::parse("0.1.5").unwrap();
assert!(
validate_catalog(&manifest)
.unwrap_err()
.contains("resource namespace")
);
manifest.extensions[0].artifacts[0].path = "process-runtime/linux-aarch64/lib/libc.so.6".into();
validate_catalog(&manifest).unwrap();
manifest.version = Version::parse("0.1.4").unwrap();
assert!(
validate_catalog(&manifest)
.unwrap_err()
.contains("resource namespace")
);
}
#[cfg(target_arch = "x86_64")]
pub(super) fn cross_author() -> Author {
let mut author = Author::new();
+8 -2
View File
@@ -31,7 +31,11 @@ impl Project {
MachineManager::new(layout.clone())
.install_requested("1.0.0")
.unwrap();
fs::copy(env!("CARGO_BIN_EXE_dever-launcher"), layout.bin().join("dever")).unwrap();
fs::copy(
fixture_binary(env!("CARGO_BIN_EXE_dever-launcher")),
layout.bin().join("dever"),
)
.unwrap();
let root = temporary.path().join("project");
fs::create_dir_all(root.join("module")).unwrap();
fs::create_dir(root.join("config")).unwrap();
@@ -122,7 +126,9 @@ fn project_skill_preserves_user_bytes_permissions_and_updates_only_its_block() {
let suffix = "\r\n# 自定义后文\r\n原样保留";
fs::write(
&agents,
format!("{prefix}<!-- dever-language:start -->\n旧规则\n<!-- dever-language:end -->{suffix}"),
format!(
"{prefix}<!-- dever-language:start -->\n旧规则\n<!-- dever-language:end -->{suffix}"
),
)
.unwrap();
fixture.succeeds();
+106 -7
View File
@@ -18,6 +18,18 @@ use sha2::{Digest, Sha256};
static NEXT_TEMPORARY: AtomicU64 = AtomicU64::new(0);
fn fixture_binary(cargo_path: &str) -> PathBuf {
match std::env::var_os("DEVER_NATIVE_ACCEPTANCE_WORKSPACE") {
Some(root) => {
let root = PathBuf::from(root);
assert!(root.is_absolute());
root.join("target/native-release-inputs/bin")
.join(Path::new(cargo_path).file_name().unwrap())
}
None => cargo_path.into(),
}
}
#[cfg(unix)]
#[path = "shared_toolchain_extensions.rs"]
mod extensions;
@@ -61,7 +73,7 @@ struct TestDaemon(Child);
#[cfg(unix)]
impl TestDaemon {
fn start(layout: &Layout) -> Self {
let child = Command::new(env!("CARGO_BIN_EXE_deverd"))
let child = Command::new(fixture_binary(env!("CARGO_BIN_EXE_deverd")))
.env_clear()
.args(["--root", layout.root().to_str().unwrap()])
.spawn()
@@ -228,6 +240,77 @@ fn all_supported_platforms_have_one_machine_entry_and_service_contract() {
assert!(platform_contract("freebsd").is_none());
}
#[cfg(unix)]
#[test]
fn signed_installation_uses_declared_modes_under_restrictive_umask() {
use std::os::unix::fs::PermissionsExt;
if std::env::var_os("DEVER_TEST_RESTRICTIVE_UMASK").is_none() {
let output = Command::new("/bin/sh")
.args(["-c", "umask 077; exec \"$@\"", "dever-umask-probe"])
.arg(std::env::current_exe().unwrap())
.args([
"--exact",
"signed_installation_uses_declared_modes_under_restrictive_umask",
"--nocapture",
])
.env("DEVER_TEST_RESTRICTIVE_UMASK", "1")
.output()
.unwrap();
assert!(
output.status.success(),
"{}\n{}",
String::from_utf8_lossy(&output.stdout),
String::from_utf8_lossy(&output.stderr)
);
return;
}
let temporary = TemporaryDirectory::new();
let layout = Layout::new(temporary.path().join("machine"));
layout.initialize().unwrap();
let key = signing_key();
fs::write(
layout.state().join("trusted-release-key"),
hex(key.public_key().as_ref()),
)
.unwrap();
fs::set_permissions(
layout.state().join("trusted-release-key"),
fs::Permissions::from_mode(0o644),
)
.unwrap();
write_release(
&layout,
&key,
"1.0.0",
&temporary.path().join("dispatch"),
true,
true,
);
MachineManager::new(layout.clone())
.install_requested("1.0.0")
.unwrap();
for directory in [
layout.bin(),
layout.versions(),
layout.versions().join("1.0.0"),
layout
.versions()
.join("1.0.0/skills/dever-language/assets/template"),
] {
assert_eq!(
fs::metadata(directory).unwrap().permissions().mode() & 0o777,
0o755
);
}
fs::set_permissions(layout.state(), fs::Permissions::from_mode(0o700)).unwrap();
layout.initialize().unwrap();
assert_eq!(
fs::metadata(layout.state()).unwrap().permissions().mode() & 0o777,
0o700
);
}
#[cfg(unix)]
#[test]
fn signed_versions_install_select_dispatch_update_and_rollback_atomically() {
@@ -310,7 +393,11 @@ fn signed_versions_install_select_dispatch_update_and_rollback_atomically() {
fs::read_to_string(skill_path(&layout).join("SKILL.md")).unwrap(),
"skill 1.0.0\n"
);
fs::copy(env!("CARGO_BIN_EXE_dever-launcher"), layout.bin().join("dever")).unwrap();
fs::copy(
fixture_binary(env!("CARGO_BIN_EXE_dever-launcher")),
layout.bin().join("dever"),
)
.unwrap();
let active_project = temporary.path().join("active-project");
fs::create_dir_all(active_project.join("module")).unwrap();
fs::create_dir(active_project.join("config")).unwrap();
@@ -510,7 +597,11 @@ fn installed_launcher_is_one_shared_entry_for_multiple_projects() {
.unwrap();
let launcher = layout.bin().join("dever");
fs::copy(env!("CARGO_BIN_EXE_dever-launcher"), &launcher).unwrap();
fs::copy(
fixture_binary(env!("CARGO_BIN_EXE_dever-launcher")),
&launcher,
)
.unwrap();
let version = Command::new(&launcher)
.env_clear()
.arg("version")
@@ -721,7 +812,7 @@ fn deverd_rejects_a_second_instance_and_recovers_its_socket_after_exit() {
layout.initialize().unwrap();
let mut daemon = TestDaemon::start(&layout);
let socket_inode = fs::symlink_metadata(layout.service_socket()).unwrap().ino();
let second = Command::new(env!("CARGO_BIN_EXE_deverd"))
let second = Command::new(fixture_binary(env!("CARGO_BIN_EXE_deverd")))
.args(["--root", layout.root().to_str().unwrap()])
.output()
.unwrap();
@@ -737,7 +828,7 @@ fn deverd_rejects_a_second_instance_and_recovers_its_socket_after_exit() {
assert!(layout.service_socket().exists());
let lock = layout.state().join("deverd.lock");
fs::set_permissions(&lock, fs::Permissions::from_mode(0o644)).unwrap();
let insecure = Command::new(env!("CARGO_BIN_EXE_deverd"))
let insecure = Command::new(fixture_binary(env!("CARGO_BIN_EXE_deverd")))
.args(["--root", layout.root().to_str().unwrap()])
.output()
.unwrap();
@@ -1097,7 +1188,11 @@ fn active_skill_is_readable_by_other_machine_users_without_install_lock_access()
.install_requested("1.0.0")
.unwrap();
let launcher = layout.bin().join("dever");
fs::copy(env!("CARGO_BIN_EXE_dever-launcher"), &launcher).unwrap();
fs::copy(
fixture_binary(env!("CARGO_BIN_EXE_dever-launcher")),
&launcher,
)
.unwrap();
fs::set_permissions(&launcher, fs::Permissions::from_mode(0o755)).unwrap();
let output = Command::new(&launcher)
.args(["skill", "path"])
@@ -1193,7 +1288,11 @@ fn deverd_uses_kernel_credentials_for_other_machine_users() {
layout.initialize().unwrap();
let _daemon = TestDaemon::start(&layout);
let launcher = layout.bin().join("dever");
fs::copy(env!("CARGO_BIN_EXE_dever-launcher"), &launcher).unwrap();
fs::copy(
fixture_binary(env!("CARGO_BIN_EXE_dever-launcher")),
&launcher,
)
.unwrap();
fs::set_permissions(&launcher, fs::Permissions::from_mode(0o755)).unwrap();
let status = Command::new(&launcher)
.args(["cache", "status"])
+5 -1
View File
@@ -8,7 +8,11 @@ pub struct TestDaemon(Child);
impl TestDaemon {
pub fn start(layout: &Layout) -> Self {
let child = Command::new(env!("CARGO_BIN_EXE_deverd"))
Self::start_with_executable(layout, std::path::Path::new(env!("CARGO_BIN_EXE_deverd")))
}
pub fn start_with_executable(layout: &Layout, executable: &std::path::Path) -> Self {
let child = Command::new(executable)
.env_clear()
.current_dir(layout.root())
.args(["--root", "."])
+38 -57
View File
@@ -1,67 +1,48 @@
//! Explicit Linux author inputs shared by opt-in native fixtures.
use std::path::{Path, PathBuf};
pub const LLVM_LIBRARY: &str = "/usr/lib/llvm-18/lib/libLLVM.so.1";
pub fn workspace() -> PathBuf {
let root = std::env::var_os("DEVER_NATIVE_ACCEPTANCE_WORKSPACE")
.map(PathBuf::from)
.unwrap_or_else(|| Path::new(env!("CARGO_MANIFEST_DIR")).join("../.."));
assert!(root.is_absolute());
root
}
pub fn core_libraries() -> Vec<(&'static str, PathBuf)> {
let mut inputs = vec![("libLLVM.so.18.1", PathBuf::from(LLVM_LIBRARY))];
for name in [
"libstdc++.so.6",
"libgcc_s.so.1",
"libz.so.1",
"libffi.so.8",
"libedit.so.2",
"libzstd.so.1",
"libtinfo.so.6",
"libxml2.so.2",
"libbsd.so.0",
"libmd.so.0",
"libicuuc.so.74",
"libicudata.so.74",
"liblzma.so.5",
] {
inputs.push((name, Path::new("/usr/lib/x86_64-linux-gnu").join(name)));
}
inputs
.into_iter()
.map(|(name, path)| {
(
name,
path.canonicalize()
.expect("prepare the explicit GNU compiler-library inputs"),
)
fn prepared_root() -> PathBuf {
workspace().join("target/native-release-inputs")
}
pub fn core_libraries() -> Vec<(String, PathBuf)> {
let mut inputs = std::fs::read_dir(prepared_root().join("core-libraries"))
.expect("prepare the explicit GNU compiler-library closure")
.map(|entry| {
let entry = entry.unwrap();
assert!(entry.file_type().unwrap().is_file());
(entry.file_name().into_string().unwrap(), entry.path())
})
.collect()
.collect::<Vec<_>>();
assert!(!inputs.is_empty(), "compiler-library closure is empty");
inputs.sort_by(|left, right| left.0.cmp(&right.0));
inputs
}
pub fn native_inputs() -> Vec<(&'static str, PathBuf)> {
let workspace = Path::new(env!("CARGO_MANIFEST_DIR")).join("../..");
vec![
(
"runtime.a",
workspace.join("target/native-runtime-abi/debug/libdever_backend_bridge.a"),
),
("crt1.o", "/usr/lib/x86_64-linux-gnu/crt1.o".into()),
("crti.o", "/usr/lib/x86_64-linux-gnu/crti.o".into()),
(
"crtbeginT.o",
"/usr/lib/gcc/x86_64-linux-gnu/13/crtbeginT.o".into(),
),
(
"crtend.o",
"/usr/lib/gcc/x86_64-linux-gnu/13/crtend.o".into(),
),
("crtn.o", "/usr/lib/x86_64-linux-gnu/crtn.o".into()),
("libc.a", "/usr/lib/x86_64-linux-gnu/libc.a".into()),
("libm.a", "/usr/lib/x86_64-linux-gnu/libm-2.39.a".into()),
("libmvec.a", "/usr/lib/x86_64-linux-gnu/libmvec.a".into()),
(
"libgcc.a",
"/usr/lib/gcc/x86_64-linux-gnu/13/libgcc.a".into(),
),
(
"libgcc_eh.a",
"/usr/lib/gcc/x86_64-linux-gnu/13/libgcc_eh.a".into(),
),
let native = prepared_root().join("native");
[
"runtime.a",
"crt1.o",
"crti.o",
"crtbeginT.o",
"crtend.o",
"crtn.o",
"libc.a",
"libm.a",
"libmvec.a",
"libgcc.a",
"libgcc_eh.a",
]
.into_iter()
.map(|name| (name, native.join(name)))
.collect()
}
+85
View File
@@ -0,0 +1,85 @@
#[path = "../dever-tests/tests/support/process.rs"]
mod process;
#[path = "../dever-tests/tests/support/temp.rs"]
mod temp;
use std::fs;
use std::process::Command;
#[test]
fn runtime_assets_have_no_sandbox_executables() {
assert!(dever_process::asset_executable("lib/ld-linux-x86-64.so.2"));
assert!(dever_process::asset_executable("lib/ld-linux-aarch64.so.1"));
assert!(!dever_process::asset_executable("bin/bwrap"));
assert!(!dever_process::asset_executable("bin/guard"));
assert!(!dever_process::asset_executable("lib/libc.so.6"));
}
#[cfg(target_os = "linux")]
#[test]
fn direct_child_keeps_the_container_namespace_and_explicit_paths() {
let directory = temp::TemporaryDirectory::new();
let input = directory.path().join("input");
fs::write(&input, "trusted input").unwrap();
let mut command = Command::new(std::env::current_exe().unwrap());
command.env("DEVER_TEST_AMBIENT", "must not reach the child");
dever_process::configure(&mut command);
command
.args(["--ignored", "--exact", "direct_child_probe"])
.env("DEVER_TEST_INPUT", &input)
.env(
"DEVER_TEST_MOUNT",
fs::read_link("/proc/self/ns/mnt").unwrap(),
);
assert!(
process::status(&mut command, std::time::Duration::from_secs(5))
.unwrap()
.success()
);
assert_eq!(fs::read(input).unwrap(), b"trusted output");
}
#[cfg(target_os = "linux")]
#[test]
fn process_group_observes_exit_before_reaping_the_leader() {
let mut command = Command::new(std::env::current_exe().unwrap());
dever_process::configure(&mut command);
let mut child = command
.args(["--ignored", "--exact", "direct_child_probe"])
.env("DEVER_TEST_WAIT", "1")
.spawn()
.unwrap();
let group = dever_process::ProcessGroup::pin(child.id()).unwrap();
assert!(!group.exited().unwrap());
group.terminate().unwrap();
let deadline = std::time::Instant::now() + std::time::Duration::from_secs(2);
while !group.exited().unwrap() && std::time::Instant::now() < deadline {
std::thread::sleep(std::time::Duration::from_millis(5));
}
let exited = group.exited().unwrap();
drop(group);
assert!(
!process::wait(&mut child, std::time::Duration::from_secs(2))
.unwrap()
.success()
);
assert!(exited);
}
#[cfg(target_os = "linux")]
#[test]
#[ignore = "internal direct-process probe"]
fn direct_child_probe() {
if std::env::var_os("DEVER_TEST_WAIT").is_some() {
std::thread::sleep(std::time::Duration::from_secs(2));
return;
}
assert!(std::env::var_os("DEVER_TEST_AMBIENT").is_none());
assert_eq!(
fs::read_link("/proc/self/ns/mnt").unwrap().as_os_str(),
std::env::var_os("DEVER_TEST_MOUNT").unwrap()
);
let path = std::env::var_os("DEVER_TEST_INPUT").unwrap();
assert_eq!(fs::read(&path).unwrap(), b"trusted input");
fs::write(path, "trusted output").unwrap();
}
@@ -1,109 +0,0 @@
/* Explicit AppArmor acceptance probe. Run only after host-policy approval.
* One process attempts capabilities in its own user/network namespaces; it
* never execs another program and never changes the host network or policy.
* Exit 0: kernel denied a required capability; 1: unsafe access; 2: invalid or
* unavailable setup. aa-exec refusing the transition before main is separate
* evidence and must be recorded by the caller, not mistaken for this result.
*/
#define _GNU_SOURCE
#include <errno.h>
#include <fcntl.h>
#include <net/if.h>
#include <sched.h>
#include <stdio.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/socket.h>
#include <unistd.h>
static int failure(const char *stage)
{
int cause = errno;
int denied = cause == EPERM || cause == EACCES;
printf("%s|%s|errno=%d\n", denied ? "DENIED" : "UNAVAILABLE", stage, cause);
return denied ? 0 : 2;
}
static int unavailable(const char *stage)
{
printf("UNAVAILABLE|%s|errno=%d\n", stage, errno);
return 2;
}
static int write_mapping(const char *path, const char *text)
{
int fd = open(path, O_WRONLY | O_CLOEXEC | O_NOFOLLOW);
if (fd < 0)
return -1;
size_t size = strlen(text);
ssize_t written = write(fd, text, size);
int cause = errno;
close(fd);
if (written != (ssize_t)size) {
errno = written < 0 ? cause : EIO;
return -1;
}
return 0;
}
int main(void)
{
uid_t uid = getuid();
gid_t gid = getgid();
if (uid == 0 || geteuid() == 0) {
fputs("UNAVAILABLE|requires_real_nonzero_uid\n", stderr);
return 2;
}
char label[4096];
int fd = open("/proc/self/attr/current", O_RDONLY | O_CLOEXEC);
if (fd < 0)
return unavailable("read_initial_label");
ssize_t length = read(fd, label, sizeof(label) - 1);
int cause = errno;
close(fd);
if (length < 0) {
errno = cause;
return unavailable("read_initial_label");
}
label[length] = '\0';
printf("INITIAL_LABEL|%s\n", label);
if (unshare(CLONE_NEWUSER) != 0)
return failure("unshare_user");
char mapping[80];
snprintf(mapping, sizeof(mapping), "0 %u 1\n", (unsigned int)uid);
if (write_mapping("/proc/self/uid_map", mapping) != 0)
return failure("uid_map");
if (write_mapping("/proc/self/setgroups", "deny\n") != 0)
return failure("setgroups");
snprintf(mapping, sizeof(mapping), "0 %u 1\n", (unsigned int)gid);
if (write_mapping("/proc/self/gid_map", mapping) != 0)
return failure("gid_map");
if (unshare(CLONE_NEWNET) != 0)
return failure("unshare_net");
puts("UNSAFE_ALLOWED|network_namespace|CAP_SYS_ADMIN_check_passed");
fd = socket(AF_INET, SOCK_DGRAM | SOCK_CLOEXEC, 0);
if (fd < 0) {
unavailable("network_socket");
return 1;
}
struct ifreq interface = {0};
memcpy(interface.ifr_name, "lo", 3);
if (ioctl(fd, SIOCGIFFLAGS, &interface) != 0) {
cause = errno;
close(fd);
errno = cause;
unavailable("read_loopback_flags");
return 1;
}
interface.ifr_flags |= IFF_UP;
int result = ioctl(fd, SIOCSIFFLAGS, &interface);
cause = errno;
close(fd);
if (result != 0) {
errno = cause;
failure("raise_loopback");
return 1;
}
puts("UNSAFE_ALLOWED|userns_and_net_admin_in_single_process");
return 1;
}
-538
View File
@@ -1,538 +0,0 @@
//! Real process tests are supplied with explicit author-owned sandbox assets.
#[path = "../dever-tests/tests/support/process.rs"]
pub mod process;
#[path = "../dever-tests/tests/support/temp.rs"]
mod temp;
use dever_sandbox::{Capabilities, Grant, Launch};
use std::fs;
use std::path::{Path, PathBuf};
use std::process::{Command, Stdio};
use std::time::Duration;
#[test]
fn bootstrap_modes_include_the_loader_and_only_declared_executables() {
assert!(dever_sandbox::asset_executable("bin/bwrap"));
assert!(dever_sandbox::asset_executable("bin/guard"));
assert!(dever_sandbox::asset_executable("lib/ld-linux-x86-64.so.2"));
assert!(dever_sandbox::asset_executable("lib/ld-linux-aarch64.so.1"));
assert!(!dever_sandbox::asset_executable("lib/libc.so.6"));
assert!(!dever_sandbox::asset_executable(
"other/ld-linux-x86-64.so.2"
));
assert!(!dever_sandbox::asset_executable(
"other/ld-linux-aarch64.so.1"
));
assert!(!dever_sandbox::asset_executable("lib/ld-linux-armhf.so.3"));
assert!(!dever_sandbox::asset_executable(
"lib/ld-linux-aarch64.so.1.backup"
));
assert!(!dever_sandbox::asset_executable("bin/unverified"));
}
#[test]
fn sandbox_paths_reject_relative_inputs_before_starting_a_process() {
use std::path::Path;
let launch = dever_sandbox::Launch {
assets: Path::new("relative"),
worker: Path::new("/missing"),
executable: Path::new("/missing/probe"),
arguments: &[],
working_directory: Path::new("/missing"),
capabilities: dever_sandbox::Capabilities::default(),
grants: &[],
};
assert!(
launch
.command()
.unwrap_err()
.contains("absolute and normalized")
);
}
#[cfg(target_os = "linux")]
struct Fixture {
directory: temp::TemporaryDirectory,
assets: PathBuf,
worker: PathBuf,
}
#[cfg(target_os = "linux")]
struct SandboxProcess(std::process::Child);
#[cfg(target_os = "linux")]
impl Drop for SandboxProcess {
fn drop(&mut self) {
// A failing assertion must not leave the owned namespace running.
let _ = process::terminate(&mut self.0);
}
}
#[cfg(target_os = "linux")]
impl Fixture {
fn new() -> Self {
use std::os::unix::fs::PermissionsExt;
let directory = temp::TemporaryDirectory::new();
let assets = directory.path().join("assets");
let worker = directory.path().join("worker");
let prepared =
Path::new(env!("CARGO_MANIFEST_DIR")).join("../../target/sandbox-inputs/assets");
fs::create_dir_all(assets.join("bin")).unwrap();
fs::create_dir_all(assets.join("lib")).unwrap();
fs::create_dir(&worker).unwrap();
fs::copy(prepared.join("bin/bwrap"), assets.join("bin/bwrap"))
.expect("prepare explicit target/sandbox-inputs first");
fs::copy(
env!("CARGO_BIN_EXE_dever-sandbox-guard"),
assets.join("bin/guard"),
)
.unwrap();
for entry in fs::read_dir(prepared.join("lib")).unwrap() {
let entry = entry.unwrap();
assert!(entry.file_type().unwrap().is_file());
fs::copy(entry.path(), assets.join("lib").join(entry.file_name())).unwrap();
}
fs::copy(std::env::current_exe().unwrap(), worker.join("probe")).unwrap();
// The unprivileged case can traverse only these explicitly owned trees.
fs::set_permissions(directory.path(), fs::Permissions::from_mode(0o755)).unwrap();
Self {
directory,
assets,
worker,
}
}
fn command(&self, probe: &str, capabilities: Capabilities, grants: &[Grant]) -> Command {
Launch {
assets: &self.assets,
worker: &self.worker,
executable: &self.worker.join("probe"),
arguments: &[
"--ignored".into(),
"--exact".into(),
probe.into(),
"--nocapture".into(),
],
working_directory: &self.worker,
capabilities,
grants,
}
.command()
.unwrap()
}
fn assert_success(&self, command: &mut Command) {
let stdout = self.directory.path().join("stdout");
let stderr = self.directory.path().join("stderr");
command
.stdin(Stdio::null())
.stdout(fs::File::create(&stdout).unwrap())
.stderr(fs::File::create(&stderr).unwrap());
let result = process::status(command, Duration::from_secs(10));
assert!(
result.as_ref().is_ok_and(|status| status.success()),
"sandbox failed: {result:?}\nstdout: {}\nstderr: {}",
fs::read_to_string(stdout).unwrap(),
fs::read_to_string(stderr).unwrap()
);
}
}
#[cfg(target_os = "linux")]
#[test]
#[ignore = "requires explicitly prepared target/sandbox-inputs assets; owns only test processes"]
fn enforces_files_network_process_and_threads() {
let fixture = Fixture::new();
fixture.assert_success(&mut fixture.command("restricted_probe", Capabilities::default(), &[]));
let output = fixture.directory.path().join("output");
fs::create_dir(&output).unwrap();
fixture.assert_success(&mut fixture.command(
"granted_probe",
Capabilities {
network: true,
process: true,
},
&[Grant {
source: output.clone(),
destination: "/data/output".into(),
writable: true,
}],
));
assert_eq!(fs::read(output.join("result")).unwrap(), b"granted");
}
#[cfg(target_os = "linux")]
#[test]
#[ignore = "requires explicitly prepared sandbox assets; mutates only owned fixture paths"]
fn pins_grants_against_directory_replacement() {
use std::os::unix::fs::symlink;
let fixture = Fixture::new();
let granted = fixture.directory.path().join("granted");
let outside = fixture.directory.path().join("outside");
fs::create_dir(&granted).unwrap();
fs::create_dir(&outside).unwrap();
fs::write(granted.join("value"), "authorized").unwrap();
fs::write(outside.join("value"), "outside").unwrap();
let mut command = fixture.command(
"pinned_grant_probe",
Capabilities::default(),
&[Grant {
source: granted.clone(),
destination: "/data/input".into(),
writable: false,
}],
);
fs::rename(&granted, fixture.directory.path().join("moved")).unwrap();
symlink(&outside, &granted).unwrap();
fixture.assert_success(&mut command);
}
#[cfg(target_os = "linux")]
#[test]
#[ignore = "internal sandbox descriptor-pinning probe"]
fn pinned_grant_probe() {
assert_eq!(
fs::read_to_string("/data/input/value").unwrap(),
"authorized"
);
}
#[cfg(target_os = "linux")]
#[test]
#[ignore = "requires explicitly prepared sandbox assets"]
fn rejects_writable_aliases_of_executable_inputs() {
let fixture = Fixture::new();
for source in [
&fixture.assets,
&fixture.worker,
&fixture.directory.path().to_owned(),
] {
let error = Launch {
assets: &fixture.assets,
worker: &fixture.worker,
executable: &fixture.worker.join("probe"),
arguments: &[],
working_directory: &fixture.worker,
capabilities: Capabilities::default(),
grants: &[Grant {
source: source.to_owned(),
destination: "/data/alias".into(),
writable: true,
}],
}
.command()
.unwrap_err();
assert!(error.contains("overlaps executable inputs"), "{error}");
}
}
#[cfg(target_os = "linux")]
#[test]
#[ignore = "requires explicitly prepared signed-release Python/Node inputs and sandbox assets"]
fn prepared_python_and_node_run_without_host_files_or_process_creation() {
use std::os::unix::fs::PermissionsExt;
let fixture = Fixture::new();
let root = Path::new(env!("CARGO_MANIFEST_DIR"))
.join("../../target/ecosystem-release-inputs/prepared")
.canonicalize()
.unwrap();
let python = "import ctypes, hashlib, json, math, os, pathlib, sqlite3, ssl, sys, threading, zlib; assert all(not p or pathlib.Path(p).is_relative_to('/worker') for p in sys.path); assert not os.path.exists('/usr/bin/python3'); assert json.loads(zlib.decompress(zlib.compress(b'{\"value\":42}')))['value'] == 42; t = threading.Thread(target=lambda: None); t.start(); t.join(); print('python okay')";
let javascript = "const assert = require('node:assert/strict'); const fs = require('node:fs'); const crypto = require('node:crypto'); const zlib = require('node:zlib'); assert.equal(fs.existsSync('/usr/bin/node'), false); assert.equal(zlib.gunzipSync(zlib.gzipSync('42')).toString(), '42'); assert.equal(crypto.createHash('sha256').update('probe').digest().length, 32); console.log('node okay');";
for (ecosystem, executable, args) in [
(
"pip",
"bin/python3.12",
vec!["-I", "-S", "-B", "-c", python],
),
("npm", "bin/node", vec!["-e", javascript]),
] {
let source = root.join(ecosystem);
let worker = fixture.directory.path().join(ecosystem);
let mut pending = vec![PathBuf::new()];
while let Some(relative) = pending.pop() {
fs::create_dir_all(worker.join(&relative)).unwrap();
for entry in fs::read_dir(source.join(&relative)).unwrap() {
let entry = entry.unwrap();
let path = relative.join(entry.file_name());
let kind = entry.file_type().unwrap();
if kind.is_dir() {
pending.push(path);
} else {
assert!(kind.is_file(), "prepared inputs must not contain links");
if path == Path::new(executable) {
fs::copy(entry.path(), worker.join(&path)).unwrap();
fs::set_permissions(worker.join(path), fs::Permissions::from_mode(0o700))
.unwrap();
} else {
fs::hard_link(entry.path(), worker.join(path)).unwrap();
}
}
}
}
let arguments = args
.into_iter()
.map(std::ffi::OsString::from)
.collect::<Vec<_>>();
let mut command = Launch {
assets: &fixture.assets,
worker: &worker,
executable: &worker.join(executable),
arguments: &arguments,
working_directory: &worker,
capabilities: Capabilities::default(),
grants: &[],
}
.command()
.unwrap();
fixture.assert_success(&mut command);
}
}
#[cfg(target_os = "linux")]
#[test]
#[ignore = "requires explicitly prepared assets and host permission for unprivileged user namespaces"]
fn enforces_unprivileged_namespace() {
if rustix::process::geteuid().as_raw() != 0 {
unprivileged_launch();
return;
}
// Select the product helper after the real UID transition. Changing only
// Command.uid after root constructed Launch tests a different call flow.
let mut command = Command::new("/usr/bin/setpriv");
command
.env_clear()
.args([
"--reuid=65534",
"--regid=65534",
"--clear-groups",
"--inh-caps=-all",
"--ambient-caps=-all",
"--",
])
.arg(std::env::current_exe().unwrap())
.args([
"--ignored",
"--exact",
"unprivileged_launcher_probe",
"--nocapture",
])
.stdin(Stdio::null());
assert!(
process::status(&mut command, Duration::from_secs(20))
.unwrap()
.success()
);
}
#[cfg(target_os = "linux")]
fn unprivileged_launch() {
assert_ne!(rustix::process::geteuid().as_raw(), 0);
let fixture = Fixture::new();
let mut command = fixture.command("restricted_probe", Capabilities::default(), &[]);
fixture.assert_success(&mut command);
}
#[cfg(target_os = "linux")]
#[test]
#[ignore = "internal non-root launcher probe, invoked by enforces_unprivileged_namespace"]
fn unprivileged_launcher_probe() {
assert!(rustix::process::getgroups().unwrap().is_empty());
unprivileged_launch();
}
#[cfg(target_os = "linux")]
#[test]
#[ignore = "internal sandbox probe, invoked only inside an owned namespace"]
fn restricted_probe() {
assert_eq!(std::env::current_dir().unwrap(), Path::new("/worker"));
assert!(std::env::vars_os().next().is_none());
assert!(fs::read("/etc/passwd").is_err());
assert!(fs::read("/proc/1/root/etc/passwd").is_err());
assert!(fs::write("/worker/forbidden", b"no").is_err());
assert!(std::net::TcpListener::bind("127.0.0.1:0").is_err());
assert!(std::os::unix::net::UnixListener::bind("/tmp/denied.sock").is_err());
assert!(
Command::new("/worker/probe")
.arg("--list")
.status()
.is_err()
);
assert_eq!(std::thread::spawn(|| 42).join().unwrap(), 42);
fs::write("/tmp/private", b"temporary").unwrap();
}
#[cfg(target_os = "linux")]
#[test]
#[ignore = "requires explicitly prepared assets; kills only this test's namespace"]
fn terminates_detached_descendants_on_kill() {
let fixture = Fixture::new();
let output = fixture.directory.path().join("output");
fs::create_dir(&output).unwrap();
let mut command = fixture.command(
"tree_probe",
Capabilities {
network: false,
process: true,
},
&[Grant {
source: output.clone(),
destination: "/data/output".into(),
writable: true,
}],
);
command
.stdin(Stdio::null())
.stdout(Stdio::null())
.stderr(Stdio::null());
let mut child = SandboxProcess(command.spawn().unwrap());
let deadline = std::time::Instant::now() + Duration::from_secs(5);
while !output.join("child-ready").is_file() {
if std::time::Instant::now() >= deadline || child.0.try_wait().unwrap().is_some() {
panic!("sandbox descendant did not start");
}
std::thread::sleep(Duration::from_millis(5));
}
let mut descendants = Vec::new();
let mut pending = vec![child.0.id()];
while let Some(pid) = pending.pop() {
let children =
fs::read_to_string(format!("/proc/{pid}/task/{pid}/children")).unwrap_or_default();
for child in children
.split_whitespace()
.map(|value| value.parse::<u32>().unwrap())
{
descendants.push(child);
pending.push(child);
}
}
assert!(
descendants.len() >= 2,
"the probe must own a real descendant tree"
);
process::terminate(&mut child.0).unwrap();
let deadline = std::time::Instant::now() + Duration::from_secs(3);
loop {
let running = descendants
.iter()
.filter(|pid| {
fs::read_to_string(format!("/proc/{pid}/stat")).is_ok_and(|stat| {
!stat.rsplit_once(") ").is_some_and(|(_, fields)| {
fields.starts_with("Z ") || fields.starts_with("X ")
})
})
})
.count();
if running == 0 {
break;
}
assert!(
std::time::Instant::now() < deadline,
"{running} owned sandbox descendants survived termination"
);
std::thread::sleep(Duration::from_millis(5));
}
}
#[cfg(target_os = "linux")]
#[test]
#[ignore = "internal sandbox process-tree probe"]
fn tree_probe() {
use std::os::unix::process::CommandExt;
let mut child = Command::new("/worker/probe")
.args(["--ignored", "--exact", "descendant_probe"])
.process_group(0)
.spawn()
.unwrap();
assert!(child.wait().unwrap().success());
}
#[cfg(target_os = "linux")]
#[test]
#[ignore = "internal sandbox descendant probe"]
fn descendant_probe() {
fs::write("/data/output/child-ready", b"ready").unwrap();
std::thread::sleep(Duration::from_secs(30));
}
#[cfg(target_os = "linux")]
#[test]
#[ignore = "internal sandbox probe, invoked only inside an owned namespace"]
fn granted_probe() {
use std::net::ToSocketAddrs;
assert!(
("localhost", 80)
.to_socket_addrs()
.unwrap()
.next()
.is_some()
);
let listener = std::net::TcpListener::bind("127.0.0.1:0").unwrap();
assert_ne!(listener.local_addr().unwrap().port(), 0);
assert!(std::os::unix::net::UnixListener::bind("/tmp/denied.sock").is_err());
assert!(
Command::new("/worker/probe")
.arg("--list")
.stdout(Stdio::null())
.status()
.unwrap()
.success()
);
fs::write("/data/output/result", b"granted").unwrap();
assert!(fs::read("/etc/passwd").is_err());
}
#[cfg(target_os = "linux")]
#[test]
#[ignore = "requires explicitly prepared target/sandbox-inputs assets"]
fn rejects_missing_dependency_before_host_loader_can_supply_it() {
let fixture = Fixture::new();
fs::remove_file(fixture.assets.join("lib/libc.so.6")).unwrap();
let error = Launch {
assets: &fixture.assets,
worker: &fixture.worker,
executable: &fixture.worker.join("probe"),
arguments: &[],
working_directory: &fixture.worker,
capabilities: Capabilities::default(),
grants: &[],
}
.command()
.unwrap_err();
assert!(
error.contains("sandbox dependency 'lib/libc.so.6' is missing"),
"{error}"
);
}
#[cfg(target_os = "linux")]
#[test]
#[ignore = "requires explicit assets and the prepared C syscall probe"]
fn kernel_rejects_namespace_ptrace_and_truncated_ioctl_requests() {
let fixture = Fixture::new();
let probe = Path::new(env!("CARGO_MANIFEST_DIR")).join("../../target/sandbox-inputs/syscalls");
let executable = fixture.worker.join("syscalls");
fs::copy(probe, &executable).expect("prepare the explicit C syscall probe first");
for process in [false, true] {
let arguments = if process {
vec![]
} else {
vec!["no-process".into()]
};
let mut command = Launch {
assets: &fixture.assets,
worker: &fixture.worker,
executable: &executable,
arguments: &arguments,
working_directory: &fixture.worker,
capabilities: Capabilities {
network: true,
process,
},
grants: &[],
}
.command()
.unwrap();
fixture.assert_success(&mut command);
}
}
-60
View File
@@ -1,60 +0,0 @@
/* Kernel-level probes for argument truncation and fixed escape denials. */
#define _GNU_SOURCE
#include <assert.h>
#include <errno.h>
#include <fcntl.h>
#include <linux/sched.h>
#include <stdint.h>
#include <sys/ioctl.h>
#include <sys/ptrace.h>
#include <sys/socket.h>
#include <sys/syscall.h>
#include <sys/xattr.h>
#include <unistd.h>
int main(int argc, char **argv) {
(void)argv;
/* Spawn may reset only the effective ID to this sandbox's real identity. */
const long identity_calls[] = {SYS_setresuid, SYS_setresgid};
const uint32_t identities[] = {getuid(), getgid()};
for (unsigned i = 0; i < sizeof(identity_calls) / sizeof(identity_calls[0]); i++) {
const long call = identity_calls[i];
const uint32_t identity = identities[i];
errno = 0;
const long reset = syscall(call, UINT32_MAX, identity, UINT32_MAX);
assert(argc == 1 ? reset == 0 : (reset == -1 && errno == EPERM));
if (argc == 1) {
assert(syscall(call, UINT64_MAX, UINT64_C(0x100000000) | identity, UINT64_MAX) == 0);
}
const uint64_t denied[][3] = {
{UINT32_MAX, identity ^ 1U, UINT32_MAX},
{identity, identity, UINT32_MAX},
{UINT32_MAX, identity, identity},
};
for (unsigned j = 0; j < sizeof(denied) / sizeof(denied[0]); j++) {
errno = 0;
assert(syscall(call, denied[j][0], denied[j][1], denied[j][2]) == -1 && errno == EPERM);
}
}
/* Python copy2 must be able to inspect metadata of its mounted source. */
assert(listxattr("/worker/syscalls", NULL, 0) >= 0);
int fd = open("/dev/null", O_RDONLY);
assert(fd >= 0);
const uint64_t forbidden[] = {TIOCSTI, TIOCLINUX, UINT64_C(0x100000000) | TIOCSTI, UINT64_C(0x100000000) | TIOCLINUX};
for (unsigned i = 0; i < sizeof(forbidden) / sizeof(forbidden[0]); i++) {
errno = 0;
assert(syscall(SYS_ioctl, fd, forbidden[i], 0) == -1 && errno == EPERM);
}
errno = 0;
assert(ioctl(fd, TCGETS, 0) == -1 && errno == ENOTTY);
assert(close(fd) == 0);
errno = 0;
assert(syscall(SYS_unshare, CLONE_NEWUSER) == -1 && errno == EPERM);
errno = 0;
assert(ptrace(PTRACE_TRACEME, 0, 0, 0) == -1 && errno == EPERM);
errno = 0;
assert(socket(AF_UNIX, SOCK_STREAM, 0) == -1 && errno == EPERM);
errno = 0;
assert(syscall(SYS_clone3, 0, 0) == -1 && errno == ENOSYS);
return 0;
}
-140
View File
@@ -1,140 +0,0 @@
//! Trust-chain regression checks touch only an explicitly owned target subtree.
use super::*;
use std::fs;
use std::os::unix::fs::{PermissionsExt, chown, symlink};
#[path = "../dever-tests/tests/support/temp.rs"]
mod temp;
#[test]
fn restricted_hosts_require_strict_profile_transitions_before_helper_selection() {
use std::io::{Error, ErrorKind};
for userns in [None, Some("0\n")] {
assert!(
!host_policy(|path| {
assert!(
path.ends_with("apparmor_restrict_unprivileged_userns"),
"unrestricted hosts must not read the strict flag"
);
userns
.map(str::to_owned)
.ok_or_else(|| Error::from(ErrorKind::NotFound))
})
.unwrap()
);
}
for strict in [None, Some("0\n"), Some("1\n")] {
let result = host_policy(|path| {
if path.ends_with("apparmor_restrict_unprivileged_userns") {
return Ok("1\n".into());
}
assert!(path.ends_with("apparmor_restrict_unprivileged_unconfined"));
strict
.map(str::to_owned)
.ok_or_else(|| Error::from(ErrorKind::NotFound))
});
if strict == Some("1\n") {
assert!(result.unwrap());
} else {
assert!(
result
.unwrap_err()
.contains("kernel.apparmor_restrict_unprivileged_unconfined=1")
);
}
}
for failure in [ErrorKind::PermissionDenied, ErrorKind::InvalidData] {
assert!(
host_policy(|_| Err(Error::from(failure)))
.unwrap_err()
.contains("cannot inspect")
);
}
assert!(
host_policy(|_| Ok("2\n".into()))
.unwrap_err()
.contains("invalid AppArmor kernel flag")
);
assert!(
host_policy(
|path| if path.ends_with("apparmor_restrict_unprivileged_userns") {
Ok("1".into())
} else {
Err(Error::from(ErrorKind::PermissionDenied))
}
)
.unwrap_err()
.contains("cannot inspect")
);
}
#[test]
#[ignore = "requires root and the prepared static author bwrap; touches only owned target files"]
fn rejects_untrusted_helper_paths_bytes_modes_owners_and_capabilities() {
assert_eq!(rustix::process::geteuid().as_raw(), 0);
let target = Path::new(env!("CARGO_MANIFEST_DIR"))
.join("../../target")
.canonicalize()
.unwrap();
let root = temp::TemporaryDirectory::new_in(&target);
let bytes = fs::read(target.join("sandbox-inputs/assets/bin/bwrap")).unwrap();
let hash = crate::validate_bwrap(&bytes).unwrap();
let directory = root.path().join(&hash);
fs::create_dir(&directory).unwrap();
let helper = directory.join("bwrap");
assert!(
verify_helper(&helper, &hash)
.unwrap_err()
.contains("missing")
);
fs::write(&helper, &bytes).unwrap();
fs::set_permissions(&helper, fs::Permissions::from_mode(0o755)).unwrap();
verify_helper(&helper, &hash).unwrap();
assert!(
verify_helper(&helper, &"0".repeat(64))
.unwrap_err()
.contains("differs")
);
for mode in [0o775, 0o4755, 0o2755, 0o644] {
fs::set_permissions(&helper, fs::Permissions::from_mode(mode)).unwrap();
assert!(verify_helper(&helper, &hash).is_err());
}
fs::set_permissions(&helper, fs::Permissions::from_mode(0o755)).unwrap();
chown(&helper, Some(65534), Some(65534)).unwrap();
assert!(verify_helper(&helper, &hash).is_err());
chown(&helper, Some(0), Some(0)).unwrap();
fs::set_permissions(&directory, fs::Permissions::from_mode(0o777)).unwrap();
assert!(verify_helper(&helper, &hash).is_err());
fs::set_permissions(&directory, fs::Permissions::from_mode(0o755)).unwrap();
let renamed = root.path().join("retained");
fs::rename(&directory, &renamed).unwrap();
symlink(&renamed, &directory).unwrap();
assert!(verify_helper(&helper, &hash).is_err());
fs::remove_file(&directory).unwrap();
fs::rename(&renamed, &directory).unwrap();
// VFS_CAP_REVISION_2 with only CAP_NET_ADMIN permitted; never execute it.
let mut capability = [0u8; 20];
capability[..4].copy_from_slice(&0x0200_0001u32.to_le_bytes());
capability[4..8].copy_from_slice(&(1u32 << 12).to_le_bytes());
rustix::fs::setxattr(
&helper,
"security.capability",
&capability,
rustix::fs::XattrFlags::empty(),
)
.unwrap();
assert!(
verify_helper(&helper, &hash)
.unwrap_err()
.contains("file capabilities")
);
rustix::fs::removexattr(&helper, "security.capability").unwrap();
verify_helper(&helper, &hash).unwrap();
// A sticky /tmp ancestor is acceptable for installer image fixtures, but
// cannot be part of the runtime's administrator-controlled attachment path.
let shared_ancestor = temp::TemporaryDirectory::new();
let shared_helper = shared_ancestor.path().join("bwrap");
fs::write(&shared_helper, &bytes).unwrap();
fs::set_permissions(&shared_helper, fs::Permissions::from_mode(0o755)).unwrap();
assert!(verify_helper(&shared_helper, &hash).is_err());
}
+15 -4
View File
@@ -25,7 +25,8 @@ fn main() {
fn run() -> Result<i32, Box<dyn std::error::Error>> {
let mut arguments = std::env::args().skip(1);
match arguments.next().as_deref() {
let mode = arguments.next();
match mode.as_deref() {
Some("echo") => {
for argument in arguments {
std::io::stdout().write_all(argument.as_bytes())?;
@@ -63,7 +64,7 @@ fn run() -> Result<i32, Box<dyn std::error::Error>> {
stdout.join().map_err(|_| "stdout thread panicked")??;
stderr.join().map_err(|_| "stderr thread panicked")??;
}
Some("wait") | Some("tree") => {
Some("wait") | Some("tree") | Some("tree-exit") => {
let directory = arguments.next().ok_or("missing marker directory")?;
let directory = Path::new(&directory);
let mut starts = std::fs::OpenOptions::new()
@@ -71,13 +72,23 @@ fn run() -> Result<i32, Box<dyn std::error::Error>> {
.append(true)
.open(directory.join("starts"))?;
starts.write_all(b"started\n")?;
// Every lifecycle case gets a real descendant. PID namespace exit
// must stop its heartbeat before the runtime returns cancellation.
let mut child = std::process::Command::new(std::env::current_exe()?)
.arg("heartbeat")
.arg(directory)
.spawn()?;
std::fs::write(directory.join("ready"), b"ready")?;
if mode.as_deref() == Some("tree-exit") {
let deadline = std::time::Instant::now() + Duration::from_secs(2);
while !directory.join("heartbeat").exists() {
if std::time::Instant::now() >= deadline {
let _ = child.kill();
let _ = child.wait();
return Err("descendant did not start".into());
}
std::thread::sleep(Duration::from_millis(5));
}
return Ok(0);
}
let status = child.wait()?;
return Ok(status.code().unwrap_or(92));
}
@@ -1 +1 @@
cmd fixture_member = app.fixture_member
long cmd fixture_member = app.fixture_member
@@ -4,8 +4,17 @@ fixture_member(
display_name: Text,
password: Secret
) (result: BootstrapView) {
input = domain.prepare(email, display_name)
hash = blocking(dever.crypto.password_hash(password))
result = fixture_store_member(selected_tenant_id, email, display_name, hash)
}
transaction fixture_store_member(
selected_tenant_id: platform.tenant.model.id,
email: Text,
display_name: Text,
hash: Text
) (result: BootstrapView) {
input = domain.prepare(email, display_name)
principal = model.principal.create(
{
display_name = input.name
+151 -2
View File
@@ -40,6 +40,155 @@ fn reject(files: &[(&str, &str)], expected: &str) {
);
}
#[test]
fn ordinary_and_long_entries_have_explicit_transaction_policies() {
for entry in ["public post create = app.create", "cmd create = app.create"] {
let files = [
(
"content/item/model.dever",
"global type Item { name: Text }",
),
(
"content/item/app.dever",
"create(name: Text) (value: Text) { saved = model.create({ name = name })\nvalue = saved.name }",
),
("content/item/api.dever", entry),
];
check(&files);
let long_entry = if entry.starts_with("public ") {
entry.replace("public ", "public long ")
} else {
format!("long {entry}")
};
let mut long = files;
long[2].1 = &long_entry;
reject(
&long,
"long entry database writes must be enclosed in an explicit transaction",
);
let mut transactional = long;
let app = format!("transaction {}", files[1].1);
transactional[1].1 = &app;
check(&transactional);
}
}
#[test]
fn entry_orchestration_can_wait_between_explicit_transactions() {
let files = [
(
"content/item/model.dever",
"global type Item { name: Text }",
),
(
"content/item/app.dever",
"create() (count: Int) { first = store(\"first\")\ndever.task.sleep(1)\nsecond = store(\"second\")\ncount = first + second }\ntransaction store(name: Text) (count: Int) { saved = model.create({ name = name })\ncount = model.count() }",
),
(
"content/item/api.dever",
"public long post create = app.create\nlong cmd create = app.create",
),
];
let source_map = sources(&files);
let program = dever_core::check(&source_map).unwrap_or_else(|errors| panic!("{errors:?}"));
let native = dever_core::native::emit_project(
&program,
&source_map,
dever_runtime::config::RuntimeProfile {
sqlite: true,
postgres: false,
},
)
.unwrap();
assert!(!native.contains("let transaction = transaction_database.begin()"));
assert!(native.contains("let transaction_owner = if _database.is_none()"));
let (_root, settings) = auth_settings();
let mut configured = identity_sources().to_vec();
configured.extend([
files[0],
files[1],
("content/item/api/admin/create.dever", files[2].1),
]);
let configured_sources = sources(&configured);
let configured_program = dever_core::check_with_settings(&configured_sources, &settings)
.unwrap_or_else(|errors| panic!("{errors:?}"));
dever_core::llvm::emit_application(&configured_program, &configured_sources).unwrap();
let ordinary_entry = files[2].1.replace("long ", "");
let mut ordinary = files;
ordinary[2].1 = &ordinary_entry;
reject(&ordinary, "automatic entry transactions cannot contain");
let mut invalid = files;
let app = files[1].1.replace("create()", "transaction create()");
invalid[1].1 = &app;
reject(
&invalid,
"transaction functions cannot start concurrent work",
);
}
#[test]
fn job_entries_require_long_before_using_staged_transactions() {
let model = "global type Item { name: Text }";
let ordinary = "apply() () { saved = model.create({ name = \"job\" }) }";
let transactional = "transaction apply() () { saved = model.create({ name = \"job\" }) }";
for long in [false, true] {
let prefix = if long { "long " } else { "" };
let job = format!(
"database default\n{prefix}job apply() () retry(3) timeout(30000) {{ app.apply() }}"
);
let mut files = [
("content/item/model.dever", model),
("content/item/app.dever", ordinary),
("content/item/job.dever", job.as_str()),
];
if long {
reject(&files, "long entry database writes must be enclosed");
files[1].1 = transactional;
}
check(&files);
}
let files = [
("content/item/model.dever", model),
(
"content/item/app.dever",
"apply() () { save()\ndever.task.sleep(1) }\ntransaction save() () { saved = model.create({ name = \"job\" }) }",
),
(
"content/item/job.dever",
"database default\njob apply() () retry(3) timeout(30000) { app.apply() }",
),
];
reject(&files, "declare the entry 'long'");
let mut long = files;
long[2].1 = "database default\nlong job apply() () retry(3) timeout(30000) { app.apply() }";
check(&long);
}
#[test]
fn long_is_an_entry_modifier_not_an_app_function_modifier() {
for (path, text) in [
("content/item/api.dever", "long rest"),
(
"content/item/api.dever",
"long public post create = app.create",
),
("content/item/app.dever", "long create() () {}"),
("content/item/app.dever", "long transaction create() () {}"),
(
"content/item/api.dever",
"public long cmd create = app.create",
),
] {
let mut source = SourceMap::default();
let id = source.add(path, text);
assert!(dever_core::parse(source.get(id)).is_err());
}
check(&[(
"content/item/app.dever",
"long() (value: Int) { value = 1 }",
)]);
}
fn auth_settings() -> (temp::TemporaryDirectory, Settings) {
let root = temp::TemporaryDirectory::new();
fs::create_dir(root.path().join("config")).unwrap();
@@ -698,7 +847,7 @@ fn sql_projection_files(sqlite: &str, postgres: &str, method: &str) -> [(&'stati
(
"news/article/app.dever",
"type View { name: Text }\n\
read(name: Text, size: Int) (rows: List<View>) {\n\
transaction read(name: Text, size: Int) (rows: List<View>) {\n\
rows = model.lookup(name, size)\n}\n"
.into(),
),
@@ -767,7 +916,7 @@ fn unproven_sql_keeps_its_write_effect_in_each_dialect() {
#[test]
fn get_rejects_writes_reached_through_cross_domain_app_calls() {
let writer = "create(name: Text) (value: Text) { account = model.create({ name = text.trim(name) })\nvalue = account.name }";
let writer = "transaction create(name: Text) (value: Text) { account = model.create({ name = text.trim(name) })\nvalue = account.name }";
reject(
&[
(
+11 -45
View File
@@ -37,54 +37,20 @@ fn project(setting: &str) -> support::temp::TemporaryDirectory {
root
}
#[cfg(feature = "external")]
#[test]
fn worker_file_grants_are_explicit_and_cannot_write_executable_cache() {
let setting = |path: &str, write: bool| {
serde_json::json!({
"adapter": {"sample.worker": {"files": {"content": {"path": path, "write": write}}}}
})
.to_string()
};
let capabilities = vec!["file".into()];
let root = project(&setting("data/upload", true));
let settings = Settings::load_project(root.path()).unwrap();
let grants = settings
.external_grants("sample.worker", &capabilities)
.unwrap();
assert_eq!(grants.len(), 1);
assert_eq!(grants[0].source, root.path().join("data/upload"));
assert_eq!(grants[0].destination, std::path::Path::new("/data/content"));
assert!(grants[0].writable);
assert!(
settings
.external_grants("sample.worker", &[])
.unwrap_err()
.contains("file capability")
);
for path in [
"data",
"data/cache",
"data/cache/lib",
"data/cache/lib/worker",
] {
let root = project(&setting(path, true));
let settings = Settings::load_project(root.path()).unwrap();
assert!(
settings
.external_grants("sample.worker", &capabilities)
.unwrap_err()
.contains("executable resource cache")
fn obsolete_worker_mount_and_gpu_grants_are_rejected() {
for field in ["files", "gpu"] {
let root = project(
&serde_json::json!({
"adapter": {"sample.worker": {field: {}}}
})
.to_string(),
);
}
for path in ["../private", "/etc", ""] {
let root = project(&setting(path, false));
let settings = Settings::load_project(root.path()).unwrap();
assert!(
settings
.external_grants("sample.worker", &capabilities)
.unwrap_err()
.contains("relative to the application")
Settings::load_project(root.path())
.unwrap()
.select_adapter("sample.worker", &["default"])
.is_err()
);
}
}
+21 -6
View File
@@ -1060,8 +1060,13 @@ deliver(input: Work) () {
);
assert!(
std::process::Command::new("kill")
.args(["-TERM", &worker.0.id().to_string()])
std::process::Command::new("/bin/sh")
.args([
"-c",
"kill -TERM \"$1\"",
"dever-test-signal",
&worker.0.id().to_string()
])
.status()
.unwrap()
.success()
@@ -1202,8 +1207,13 @@ fn one_native_application_gates_api_worker_and_all_modes() {
panic!("{mode} did not reach its service state: {stderr}");
}
assert!(
std::process::Command::new("kill")
.args(["-TERM", &child.0.id().to_string()])
std::process::Command::new("/bin/sh")
.args([
"-c",
"kill -TERM \"$1\"",
"dever-test-signal",
&child.0.id().to_string()
])
.status()
.unwrap()
.success()
@@ -1269,8 +1279,13 @@ fn worker_signal_stops_claiming_and_drains_or_cancels_before_database_close() {
);
}
assert!(
std::process::Command::new("kill")
.args(["-TERM", &child.id().to_string()])
std::process::Command::new("/bin/sh")
.args([
"-c",
"kill -TERM \"$1\"",
"dever-test-signal",
&child.id().to_string()
])
.status()
.unwrap()
.success()
+34 -36
View File
@@ -9,8 +9,8 @@ use dever_runtime::wire::Encoder;
use serde_json::json;
use sha2::{Digest, Sha256};
#[path = "support/sandbox.rs"]
mod sandbox;
#[path = "support/process_runtime.rs"]
mod process_runtime;
#[path = "support/temp.rs"]
mod temp;
@@ -35,7 +35,6 @@ impl Fixture {
fs::create_dir(directory.path().join("markers")).unwrap();
fs::create_dir(directory.path().join("config")).unwrap();
let settings = json!({"adapter": {PORT: {
"files": {"markers": {"path":"markers", "write":true}},
"command": {"timeout_ms":timeout_ms, "output_limit":output_limit}
}}});
fs::write(
@@ -44,13 +43,13 @@ impl Fixture {
)
.unwrap();
let workspace = Path::new(env!("CARGO_MANIFEST_DIR")).join("../..");
let mut assets = sandbox::assets(&workspace);
assets.push(sandbox::Asset {
let mut assets = process_runtime::assets(&workspace);
assets.push(process_runtime::Asset {
path: "commands/fixture/tool".into(),
bytes: program,
executable: true,
});
assets.push(sandbox::Asset {
assets.push(process_runtime::Asset {
path: "tool.dever-command.json".into(),
bytes: json!({"format":"dever-command-launch-v1", "ecosystem":"command", "entry":"tool",
"executable":"commands/fixture/tool", "arguments":[], "working_directory":"commands/fixture"})
@@ -104,6 +103,12 @@ struct Output {
stderr: Vec<u8>,
}
async fn tree(directory: std::path::PathBuf) -> Result<(), String> {
invoke(&["tree", directory.to_str().unwrap()], None)
.await
.map(|_| ())
}
async fn invoke(args: &[&str], stdin: Option<&[u8]>) -> Result<Output, String> {
let mut writer = Encoder::default();
writer.begin_object()?;
@@ -255,18 +260,12 @@ async fn heartbeat(fixture: &Fixture) -> Result<(), String> {
}
#[test]
fn guard_exec_failure_is_a_fault_not_a_program_exit_code() {
fn invalid_executable_is_rejected_before_a_program_exit_code_exists() {
let fixture = Fixture::with_program(5000, 1024 * 1024, b"not an ELF executable\n".to_vec());
dever_runtime::task::run_entry(async {
fixture.start().await?;
let error = invoke(&[], None)
.await
.err()
.ok_or("exec failure was returned as an exit code")?;
assert!(
error.contains("startup failed") && error.contains("cannot execute"),
"{error}"
);
let error = fixture.start().await.unwrap_err();
assert!(error.contains("invalid executable ELF"), "{error}");
assert!(!fixture.marker("starts").exists());
component::shutdown().await
})
.unwrap();
@@ -294,15 +293,26 @@ async fn assert_reaped(fixture: &Fixture) {
);
}
#[test]
fn successful_root_exit_stops_descendants_holding_output_pipes() {
let fixture = Fixture::new(5000, 1024 * 1024);
dever_runtime::task::run_entry(async {
fixture.start().await?;
let directory = fixture.marker("");
let output = invoke(&["tree-exit", directory.to_str().unwrap()], None).await?;
assert_eq!(output.code, 0);
assert_reaped(&fixture).await;
component::shutdown().await
})
.unwrap();
}
#[test]
fn timeout_reaps_descendants_and_keeps_the_adapter_available() {
let fixture = Fixture::new(800, 1024 * 1024);
dever_runtime::task::run_entry(async {
fixture.start().await?;
let call = dever_runtime::task::run(async {
invoke(&["tree", "/data/markers"], None).await.map(|_| ())
})
.await?;
let call = dever_runtime::task::run(tree(fixture.marker(""))).await?;
heartbeat(&fixture).await?;
let error = dever_runtime::task::wait(call)
.await
@@ -321,10 +331,7 @@ fn cancellation_and_root_close_reap_descendants_without_replay() {
let fixture = Fixture::new(5000, 1024 * 1024);
dever_runtime::task::run_entry(async {
fixture.start().await?;
let call = dever_runtime::task::run(async {
invoke(&["tree", "/data/markers"], None).await.map(|_| ())
})
.await?;
let call = dever_runtime::task::run(tree(fixture.marker(""))).await?;
heartbeat(&fixture).await?;
if root_close {
// Return while the call is active: the invocation closes its
@@ -361,10 +368,7 @@ fn queued_cancellation_does_not_wait_for_an_unrelated_running_command() {
let fixture = Fixture::new(5000, 1024 * 1024);
dever_runtime::task::run_entry(async {
fixture.start().await?;
let active = dever_runtime::task::run(async {
invoke(&["tree", "/data/markers"], None).await.map(|_| ())
})
.await?;
let active = dever_runtime::task::run(tree(fixture.marker(""))).await?;
heartbeat(&fixture).await?;
let (entered, admitted) = tokio::sync::oneshot::channel();
let queued = dever_runtime::task::run(async move {
@@ -395,10 +399,7 @@ fn task_deadline_waits_for_command_process_cleanup() {
let fixture = Fixture::new(5000, 1024 * 1024);
dever_runtime::task::run_entry(async {
fixture.start().await?;
let call = dever_runtime::task::run(async {
invoke(&["tree", "/data/markers"], None).await.map(|_| ())
})
.await?;
let call = dever_runtime::task::run(tree(fixture.marker(""))).await?;
heartbeat(&fixture).await?;
assert_eq!(dever_runtime::task::wait_timeout(call, 10).await?, None);
assert_reaped(&fixture).await;
@@ -412,10 +413,7 @@ fn explicit_shutdown_drains_active_and_queued_commands() {
let fixture = Fixture::new(5000, 1024 * 1024);
dever_runtime::task::run_entry(async {
fixture.start().await?;
let active = dever_runtime::task::run(async {
invoke(&["tree", "/data/markers"], None).await.map(|_| ())
})
.await?;
let active = dever_runtime::task::run(tree(fixture.marker(""))).await?;
heartbeat(&fixture).await?;
let queued =
dever_runtime::task::run(async { invoke(&["echo", "queued"], None).await.map(|_| ()) })
+13 -11
View File
@@ -5,8 +5,8 @@ use dever_runtime::component::{self, Definition, Reply};
use dever_runtime::wire::{Encoded, Encoder};
use sha2::{Digest, Sha256};
#[path = "support/sandbox.rs"]
mod sandbox;
#[path = "support/process_runtime.rs"]
mod process_runtime;
#[path = "support/temp.rs"]
mod temp;
@@ -28,8 +28,6 @@ fn component_protocol_and_lifecycle_are_bounded() {
let fixture = install_fixture();
let result = dever_runtime::task::run_entry(async {
fixture.prepare()?;
// One invocation keeps the same deployment grants even if settings are
// replaced between Worker startup, cancellation and restart.
fs::write(
fixture.directory.path().join("config/setting.json"),
"invalid",
@@ -252,6 +250,12 @@ fn definition(
mode: &'static str,
timeout_ms: u64,
) -> Definition {
let (_, setting) = component::adapter_settings()
.unwrap()
.select_adapter("notification.delivery", &["fixture"])
.unwrap();
let mut setting: serde_json::Value = serde_json::from_str(&setting.unwrap()).unwrap();
setting["mode"] = serde_json::json!(mode);
Definition {
ecosystem: "exec".into(),
key: key.into(),
@@ -261,9 +265,7 @@ fn definition(
schema: "a19ca3e615ce5e71".into(),
capabilities: vec!["network".into(), "file".into()],
operations: OPERATIONS.iter().map(|name| (*name).into()).collect(),
setting: Some(format!(
r#"{{"mode":"{mode}","marker":"/data/lifecycle/worker"}}"#
)),
setting: Some(setting.to_string()),
timeout_ms,
}
}
@@ -295,15 +297,15 @@ fn install_fixture() -> InstalledFixture {
let directory = temp::TemporaryDirectory::new();
fs::create_dir(directory.path().join("lifecycle")).unwrap();
fs::create_dir(directory.path().join("config")).unwrap();
fs::write(directory.path().join("config/setting.json"), r#"{"adapter":{"notification.delivery":{"files":{"lifecycle":{"path":"lifecycle","write":true}}}}}"#).unwrap();
fs::write(directory.path().join("config/setting.json"), serde_json::json!({"adapter":{"notification.delivery":{"setting":{"marker":directory.path().join("lifecycle/worker")}}}}).to_string()).unwrap();
let workspace = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../..");
let mut assets = sandbox::assets(&workspace);
assets.push(sandbox::Asset {
let mut assets = process_runtime::assets(&workspace);
assets.push(process_runtime::Asset {
path: "workers/fixture/worker".into(),
bytes: fs::read(env!("CARGO_BIN_EXE_component-fixture")).unwrap(),
executable: true,
});
assets.push(sandbox::Asset {
assets.push(process_runtime::Asset {
path: "worker.dever-worker.json".into(),
bytes: br#"{"format":"dever-worker-launch-v1","ecosystem":"exec","entry":"worker","executable":"workers/fixture/worker","arguments":[],"working_directory":"workers/fixture"}"#.to_vec(), executable: false,
});
+7 -5
View File
@@ -1,5 +1,5 @@
#[path = "support/sandbox.rs"]
mod sandbox;
#[path = "support/process_runtime.rs"]
mod process_runtime;
#[path = "support/temp.rs"]
mod temp;
@@ -385,7 +385,7 @@ fn existing_cache_permissions_are_checked_before_reuse_and_launch() {
}
#[test]
#[ignore = "requires explicitly prepared sandbox assets and the private Rust bootstrap compiler"]
#[ignore = "requires explicitly prepared process_runtime assets and the private Rust bootstrap compiler"]
fn packaged_exec_runs_without_source_worker_or_inherited_environment() {
let root = TemporaryDirectory::new();
let mut sources = dever_core::source::SourceMap::default();
@@ -424,7 +424,9 @@ fn packaged_exec_runs_without_source_worker_or_inherited_environment() {
bytes: manifest,
executable: false,
});
for asset in sandbox::assets(&std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../..")) {
for asset in
process_runtime::assets(&std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("../.."))
{
resources.push(dever_core::native::EmbeddedResource {
path: asset.path,
sha256: digest(&asset.bytes),
@@ -446,7 +448,7 @@ fn packaged_exec_runs_without_source_worker_or_inherited_environment() {
fs::create_dir(root.path().join("lifecycle")).unwrap();
fs::write(
root.path().join("config/setting.json"),
r#"{"adapter":{"notification.mail":{"use":"default","setting":{"mode":"normal","marker":"/data/lifecycle/worker"},"files":{"lifecycle":{"path":"lifecycle","write":true}}}}}"#,
serde_json::json!({"adapter":{"notification.mail":{"use":"default","setting":{"mode":"normal","marker":root.path().join("lifecycle/worker")}}}}).to_string(),
)
.unwrap();
assert!(!root.path().join(&entry).exists());
+12
View File
@@ -56,6 +56,18 @@ fn assert_roundtrip_at(path: &FilePath, source: &str) -> String {
output
}
#[test]
fn long_entry_modifiers_roundtrip_without_changing_visibility_or_job_policy() {
let api = "public long post generate=app.generate\nlong cmd generate=app.generate\npost create=app.create\n";
let formatted = assert_roundtrip_at(FilePath::new("content/item/api.dever"), api);
assert!(formatted.contains("public long post generate = app.generate"));
assert!(formatted.contains("long cmd generate = app.generate"));
assert!(formatted.contains("post create = app.create"));
let job = "long job generate()()retry(3)timeout(30000){app.generate()}";
let formatted = assert_roundtrip_at(FilePath::new("content/item/job.dever"), job);
assert!(formatted.contains("long job generate() () retry(3) timeout(30000)"));
}
#[test]
fn canonical_layout_preserves_field_evaluation_order_and_groups() {
let source = "type Pair{a:Int\nb:Int}\npublic main( )(result:Pair){result=Pair{b=second()\na=first()}}\n";
+84 -3
View File
@@ -6,6 +6,9 @@ use dever_runtime::http::{Header, Request};
use std::fs;
use std::process::Command;
#[path = "support/process.rs"]
#[allow(dead_code)]
mod process;
#[path = "support/temp.rs"]
mod temp;
@@ -17,6 +20,81 @@ fn source(files: &[(&str, &str)]) -> SourceMap {
sources
}
#[cfg(feature = "sqlite")]
#[test]
fn command_releases_short_transactions_before_waiting_and_preserves_committed_stages() {
let sources = source(&[
(
"content/item/model.dever",
"global type Item { name: Text unique }",
),
(
"content/item/app.dever",
"transaction store(name: Text) (value: Text) { saved = model.create({ name = name })\nvalue = saved.name }\ncount() (total: Int) { total = model.count() }\nperform() (total: Int) { saved = store(\"first\")\ntask = run(count())\ntotal = wait(task) }\nfail_later() (value: Text) { committed = store(\"second\")\nvalue = store(committed) }",
),
(
"content/item/api.dever",
"long cmd perform = app.perform\nlong cmd fail_later = app.fail_later\ncmd fail_atomic = app.fail_later\ncmd count = app.count",
),
]);
let program = dever_core::check(&sources).unwrap_or_else(|errors| panic!("{errors:?}"));
let rustc = std::env::var_os("RUSTC").unwrap_or_else(|| "rustc".into());
let native = dever_core::native::compile_project(
&program,
&sources,
&rustc,
RuntimeProfile {
sqlite: true,
postgres: false,
},
)
.unwrap();
let root = native.executable().parent().unwrap();
fs::create_dir_all(root.join("config")).unwrap();
fs::write(
root.join("config/setting.json"),
r#"{"database":{"default":{"type":"sqlite","path":"data/db/app.db","max_connections":1}}}"#,
)
.unwrap();
let invoke = |operation: &str| {
let mut child = Command::new(native.executable())
.current_dir(root)
.args([operation, "{}"])
.stdout(std::process::Stdio::piped())
.stderr(std::process::Stdio::piped())
.spawn()
.unwrap();
process::wait(&mut child, std::time::Duration::from_secs(10)).unwrap();
child.wait_with_output().unwrap()
};
let first = invoke("content.item.perform");
assert!(
first.status.success(),
"{}",
String::from_utf8_lossy(&first.stderr)
);
let first: serde_json::Value = serde_json::from_slice(&first.stdout).unwrap();
assert_eq!(first["data"], 1);
let atomic = invoke("content.item.fail_atomic");
assert!(!atomic.status.success());
assert!(String::from_utf8_lossy(&atomic.stderr).contains("UNIQUE"));
let retained = invoke("content.item.count");
assert!(retained.status.success());
let retained: serde_json::Value = serde_json::from_slice(&retained.stdout).unwrap();
assert_eq!(retained["data"], 1);
let failed = invoke("content.item.fail_later");
assert!(!failed.status.success());
assert!(String::from_utf8_lossy(&failed.stderr).contains("UNIQUE"));
let count = invoke("content.item.count");
assert!(
count.status.success(),
"{}",
String::from_utf8_lossy(&count.stderr)
);
let count: serde_json::Value = serde_json::from_slice(&count.stdout).unwrap();
assert_eq!(count["data"], 2);
}
#[test]
fn project_entry_collects_api_and_command_without_main() {
let sources = source(&[
@@ -175,13 +253,13 @@ fn generated_write_entry_rejects_detached_concurrency() {
.map(|error| error.render(&sources))
.collect::<String>();
assert!(
rendered.contains("transaction cannot start concurrent work"),
rendered.contains("automatic entry transactions cannot contain"),
"{rendered}"
);
}
#[test]
fn generated_write_entry_allows_structured_blocking_work() {
fn generated_long_entry_keeps_password_work_outside_its_write_transaction() {
let sources = source(&[
(
"user/account/model.dever",
@@ -201,7 +279,10 @@ transaction persist(hash: Text) (id: model.id) {
id = stored.id
}"#,
),
("user/account/api.dever", "post register = app.register"),
(
"user/account/api.dever",
"long post register = app.register",
),
]);
let program = dever_core::check(&sources).unwrap_or_else(|errors| {
panic!(
+70
View File
@@ -11,6 +11,76 @@ const ADAPTER: &str = "port.read() (value: Int) { value = 7 }";
const COMMAND_PORT: &str = "execute(args: List<Text>, stdin: Bytes?) (output: dever.process.Output) fails dever.process.Error";
const COMMAND_APP: &str = "read() (value: Int) { output = port.execute([\"argument with spaces\"], null)\n value = output.code }";
#[test]
fn external_worker_orchestration_requires_an_explicit_long_entry() {
for adapter in [
"external exec \"worker/mail\" {}",
"external exec \"worker/mail\" { allow gpu }",
] {
for long in [false, true] {
let entry = if long {
"long cmd read = app.read"
} else {
"cmd read = app.read"
};
let source = sources(
PORT,
adapter,
&[
("notification/mail/model.dever", "type Mail { value: Int }"),
(
"notification/mail/app.dever",
"type ReadResult { Read(value: Int)\n error Unavailable(message: Text) }\nread() (value: Int) { value = port.read()\nstore(value) }\ntransaction store(value: Int) () { stored = model.create({ value = value }) }",
),
("notification/mail/api.dever", entry),
],
);
if long {
let program =
dever_core::check(&source).unwrap_or_else(|errors| panic!("{errors:?}"));
let native = dever_core::native::emit_project(
&program,
&source,
dever_runtime::config::RuntimeProfile {
sqlite: true,
postgres: false,
},
)
.unwrap();
assert!(!native.contains("let transaction = transaction_database.begin()"));
dever_core::llvm::emit_application(&program, &source).unwrap();
} else {
rejected(&source, "declare the entry 'long'");
}
}
}
}
#[test]
fn transactions_reject_external_workers_even_without_declared_io_capabilities() {
for adapter in [
"external exec \"worker/mail\" {}",
"external exec \"worker/mail\" { allow gpu }",
] {
let source = sources(
PORT,
adapter,
&[
("notification/mail/model.dever", "type Mail { value: Int }"),
(
"notification/mail/app.dever",
"type ReadResult { Read(value: Int)\n error Unavailable(message: Text) }\ntransaction save() (value: Int) { value = port.read()\nstored = model.create({ value = value }) }",
),
("notification/mail/api.dever", "cmd save = app.save"),
],
);
rejected(
&source,
"transaction functions cannot start concurrent work or external I/O",
);
}
}
fn command_sources(port: &str, adapter: &str, app: &str) -> SourceMap {
sources(port, adapter, &[("notification/mail/app.dever", app)])
}
+13 -11
View File
@@ -22,7 +22,7 @@ fn diagnostics(sources: &SourceMap) -> String {
// Code-generation assertions may inspect unbound HIR; an executable API must
// use the same configured site/auth contract as the CLI.
fn compile_authenticated_rest(mut sources: SourceMap) {
fn compile_authenticated_rest(mut sources: SourceMap, user_id_type: &str) {
sources.add("access/user/model.dever", "global type User { name: Text }");
sources.add(
"access/tenant/model.dever",
@@ -30,16 +30,18 @@ fn compile_authenticated_rest(mut sources: SourceMap) {
);
sources.add(
"access/session/app.dever",
r#"type Identity {
format!(
r#"type Identity {{
id: Text
user_id: access.user.model.id?
user_id: {user_id_type}?
tenant_id: access.tenant.model.id?
}
verify(claims: dever.auth.Claims) (identity: Identity) {
identity = Identity { id = claims.subject
}}
verify(claims: dever.auth.Claims) (identity: Identity) {{
identity = Identity {{ id = claims.subject
user_id = null
tenant_id = null }
}"#,
tenant_id = null }}
}}"#
),
);
let directory = TemporaryDirectory::new();
std::fs::create_dir(directory.path().join("config")).unwrap();
@@ -109,7 +111,7 @@ fn rest_derives_the_public_model_without_an_app_wrapper() {
assert!(generated.contains("transaction.commit().await"));
assert!(generated.contains("rest_rollback(transaction).await"));
assert!(generated.contains("dever_runtime::api::text_bounds(\"name\""));
compile_authenticated_rest(sources);
compile_authenticated_rest(sources, "access.user.model.id");
}
#[test]
@@ -184,7 +186,7 @@ fn rest_owner_is_server_bound_and_scopes_every_database_statement() {
] {
assert!(generated.contains(sql), "missing {sql}: {generated}");
}
compile_authenticated_rest(sources);
compile_authenticated_rest(sources, "user.account.model.id");
}
#[test]
@@ -378,7 +380,7 @@ fn rest_field_bindings_are_checked_and_emit_static_search_sql() {
generated.contains("SELECT COUNT(*) FROM \\\"profile\\\" WHERE \\\"user_id\\\" = ?1"),
"{generated}"
);
compile_authenticated_rest(sources);
compile_authenticated_rest(sources, "user.account.model.id");
}
#[test]
+2 -2
View File
@@ -67,7 +67,7 @@ transaction store(credential_id: model.id) (id: model.id) {
),
(
"user/account/api/front/account.dever",
"public post login = app.login",
"public long post login = app.login",
),
]);
let program = checked_platform(&sources);
@@ -106,7 +106,7 @@ login(credential_id: model.id, password: Secret) (id: model.id) {
),
(
"user/account/api/front/account.dever",
"public post login = app.login",
"public long post login = app.login",
),
],
"explicit transaction function",
@@ -0,0 +1,29 @@
use std::fs;
use std::path::Path;
pub struct Asset {
pub path: String,
pub bytes: Vec<u8>,
pub executable: bool,
}
pub fn assets(_workspace: &Path) -> Vec<Asset> {
let prepared = std::path::PathBuf::from(
std::env::var_os("DEVER_PROCESS_RUNTIME")
.expect("set DEVER_PROCESS_RUNTIME to explicit native runtime assets"),
);
assert!(prepared.is_absolute());
let mut assets = Vec::new();
for entry in fs::read_dir(prepared.join("lib")).unwrap() {
let entry = entry.unwrap();
assert!(entry.file_type().unwrap().is_file());
let name = entry.file_name().into_string().unwrap();
assets.push(Asset {
path: format!("process-runtime/lib/{name}"),
bytes: fs::read(entry.path()).unwrap(),
executable: name.starts_with("ld-linux"),
});
}
assets.sort_by(|left, right| left.path.cmp(&right.path));
assets
}
-40
View File
@@ -1,40 +0,0 @@
//! Explicit private fixture assets; never consult PATH or download a tool.
use std::fs;
use std::path::Path;
pub struct Asset {
pub path: String,
pub bytes: Vec<u8>,
pub executable: bool,
}
pub fn assets(workspace: &Path) -> Vec<Asset> {
let prepared = workspace.join("target/sandbox-inputs/assets");
let mut assets = Vec::new();
for directory in ["bin", "lib"] {
for entry in fs::read_dir(prepared.join(directory))
.expect("prepare target/sandbox-inputs assets explicitly")
{
let entry = entry.unwrap();
assert!(entry.file_type().unwrap().is_file());
let name = entry.file_name().into_string().unwrap();
if directory == "bin" && name == "guard" {
continue;
}
assets.push(Asset {
path: format!("sandbox/{directory}/{name}"),
bytes: fs::read(entry.path()).unwrap(),
executable: directory == "bin" || name.starts_with("ld-linux"),
});
}
}
assets.push(Asset {
path: "sandbox/bin/guard".into(),
bytes: fs::read(workspace.join("target/debug/dever-sandbox-guard"))
.expect("build the current dever-sandbox-guard fixture explicitly"),
executable: true,
});
assets.sort_by(|left, right| left.path.cmp(&right.path));
assets
}
+1 -1
View File
@@ -267,7 +267,7 @@ fn upload_route_native_compiles_with_owned_resource() {
let native = dever_core::native::compile_project(
&program,
&sources,
std::ffi::OsStr::new("/root/.cargo/bin/rustc"),
std::ffi::OsStr::new("rustc"),
dever_runtime::config::RuntimeProfile::default(),
)
.unwrap();
+1 -1
View File
@@ -8,7 +8,7 @@ check_cleanup() {
for process in /proc/[0-9]*/exe; do
executable=$(readlink "$process" || true)
case "$executable" in
/worker/*|*/workers/*|*/sandbox/bin/*|*/sandbox/lib/*)
*/workers/*|*/process-runtime/lib/*)
echo "ERROR: Worker process leaked: $executable"
exit 1
;;
+40
View File
@@ -0,0 +1,40 @@
#include "dever_ffi.h"
#include <stdio.h>
#include <string.h>
extern long long fixture_triple(long long value);
static int invoke(void *context, dever_ffi_slice operation, dever_ffi_slice payload, dever_ffi_output *output) {
(void)context;
if (operation.size == 6 && memcmp(operation.data, "reject", 6) == 0) {
const char *error = "{\"error\":\"example.Rejected\",\"payload\":null}";
memcpy(output->data, error, strlen(error));
output->size = strlen(error);
return DEVER_FFI_ERROR;
}
if (operation.size == 10 && memcmp(operation.data, "bad_length", 10) == 0) {
output->size = output->capacity + 1;
return DEVER_FFI_OK;
}
if (operation.size == 5 && memcmp(operation.data, "fault", 5) == 0) return DEVER_FFI_FAULT;
if (payload.size >= 64) return DEVER_FFI_FAULT;
char input[64];
memcpy(input, payload.data, (size_t)payload.size);
input[payload.size] = 0;
long long value = 0;
if (sscanf(input, "{\"value\":%lld}", &value) != 1 || value > 1000000 || value < -1000000) return DEVER_FFI_FAULT;
int size = snprintf(output->data, (size_t)output->capacity, "{\"value\":%lld}", fixture_triple(value));
if (size < 0 || (uint64_t)size >= output->capacity) return DEVER_FFI_FAULT;
output->size = (uint64_t)size;
return DEVER_FFI_OK;
}
static void closed(void *context) {
(void)context;
fputs("closed\n", stderr);
}
int main(void) {
const dever_ffi_handler handler = {DEVER_FFI_ABI_V1, NULL, NULL, invoke, closed};
return dever_ffi_serve(&handler);
}
+121
View File
@@ -0,0 +1,121 @@
import argparse
import json
from pathlib import Path
import struct
import subprocess
import tempfile
import unittest
COMPILER = None
ROOT = Path(__file__).resolve().parents[2]
MANIFEST = {
"port": "example.Native", "adapter": "example.C", "schema": "ffi-test-v1",
"capabilities": [], "operations": ["triple", "reject", "fault", "bad_length"],
}
HELLO = MANIFEST | {"kind": "hello", "version": "dever-component-1", "setting": {}}
def frame(message):
data = json.dumps(message, separators=(",", ":")).encode()
return struct.pack(">I", len(data)) + data
def responses(data):
values = []
while data:
if len(data) < 4:
raise AssertionError("truncated frame")
size = struct.unpack(">I", data[:4])[0]
if not 0 < size <= 16 * 1024 * 1024 or size > len(data) - 4:
raise AssertionError("invalid frame size")
values.append(json.loads(data[4:4 + size]))
data = data[4 + size:]
return values
class NativeFfi(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.temporary = tempfile.TemporaryDirectory(prefix="dever-c-ffi-")
cls.addClassCleanup(cls.temporary.cleanup)
cls.directory = Path(cls.temporary.name)
(cls.directory / "lib").mkdir()
library = cls.directory / "library.c"
library.write_text("long long fixture_triple(long long value) { return value * 3; }\n")
subprocess.run([COMPILER, "-shared", "-fPIC", "-Wl,-soname,libfixture.so", str(library), "-o", str(cls.directory / "lib/libfixture.so")], check=True, timeout=30)
cls.binary = cls.directory / "worker"
subprocess.run([
COMPILER, "-std=c11", "-Wall", "-Wextra", "-Werror", "-I", str(ROOT / "sdk/c"),
str(ROOT / "sdk/c/dever_ffi.c"), str(ROOT / "test/external-sdk/c_ffi_adapter.c"),
"-L", str(cls.directory / "lib"), "-lfixture", "-Wl,-rpath,$ORIGIN/lib", "-o", str(cls.binary),
], check=True, timeout=30)
(cls.directory / "contract.json").write_text(json.dumps(MANIFEST, indent=2))
def execute(self, messages, suffix=b""):
result = subprocess.run([str(self.binary)], input=b"".join(frame(message) for message in messages) + suffix, cwd=self.directory, stdout=subprocess.PIPE, stderr=subprocess.PIPE, timeout=5, env={})
return result, responses(result.stdout)
def test_c_abi_calls_a_shared_library_and_preserves_protocol_lifecycle(self):
result, values = self.execute([
HELLO, {"kind": "health", "id": 0},
{"kind": "call", "id": 1, "operation": "triple", "payload": {"value": 7}},
{"kind": "cancel", "id": 1},
{"kind": "call", "id": 2, "operation": "reject", "payload": {}},
{"kind": "call", "id": 3, "operation": "triple", "payload": {"value": -9}},
{"kind": "shutdown"},
])
self.assertEqual(result.returncode, 0, result.stderr)
self.assertEqual(values, [
MANIFEST | {"kind": "ready", "version": "dever-component-1"},
{"kind": "health", "id": 0},
{"kind": "result", "id": 1, "payload": {"value": 21}},
{"kind": "error", "id": 2, "error": "example.Rejected", "payload": None},
{"kind": "result", "id": 3, "payload": {"value": -27}},
{"kind": "shutdown"},
])
self.assertEqual(result.stderr, b"closed\n")
def test_identity_cannot_be_supplied_by_the_caller(self):
for changed in [{"schema": "wrong"}, {"capabilities": ["gpu"]}, {"port": "wrong"}, {"operations": ["other"]}]:
with self.subTest(changed=changed):
result, values = self.execute([HELLO | changed])
self.assertNotEqual(result.returncode, 0)
self.assertEqual(values, [])
self.assertNotIn(b"closed", result.stderr)
def test_invalid_calls_and_native_results_fail_closed(self):
for call in [
{"id": 2, "operation": "triple"},
{"id": 1, "operation": "missing"},
{"id": 1, "operation": "bad_length"},
{"id": 1, "operation": "fault"},
{"id": -1, "operation": "triple"},
{"id": 9223372036854775808, "operation": "triple"},
]:
with self.subTest(call=call):
result, values = self.execute([HELLO, {"kind": "call", "payload": {"value": 7}} | call])
self.assertNotEqual(result.returncode, 0)
self.assertEqual(len(values), 1)
self.assertIn(b"closed\n", result.stderr)
def test_truncated_oversized_and_malformed_frames_fail_closed(self):
for suffix in [b"\x00", struct.pack(">I", 0), struct.pack(">I", 16777217), struct.pack(">I", 5) + b"{}", frame({"kind": "shutdown", "extra": 1})]:
with self.subTest(suffix=suffix):
result, values = self.execute([HELLO], suffix)
self.assertNotEqual(result.returncode, 0)
self.assertEqual(len(values), 1)
for malformed in [b"{", b'{"kind":"call","id":1e}', b'{"kind":"\\q"}', b"[" * 65 + b"]" * 65, b'{"kind":"shutdown"}\x00']:
result, values = self.execute([HELLO], struct.pack(">I", len(malformed)) + malformed)
self.assertNotEqual(result.returncode, 0)
self.assertEqual(len(values), 1)
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--cc", required=True)
options = parser.parse_args()
if not Path(options.cc).is_absolute() or not Path(options.cc).is_file():
parser.error("--cc must be an explicit absolute compiler file")
COMPILER = options.cc
unittest.main(argv=[__file__], verbosity=2)
+43
View File
@@ -340,6 +340,18 @@ inspect() (okay: Bool) { okay = model.count() == 0 }
body: "{}",
};
const ENCODING_COMMIT: Fixture<'static> = Fixture {
model: ITEM,
app: r#"transaction exercise(denominator: Float) (value: Float) {
removed = model.delete({ where = name == "encoding commit" })
saved = model.create({ name = "encoding commit" })
value = 1.0 / denominator
}
inspect() (okay: Bool) { okay = model.count() == 1 }
"#,
body: r#"{"denominator":0}"#,
};
const ENCODING_ROLLBACK: Fixture<'static> = Fixture {
model: ITEM,
app: r#"exercise(denominator: Float) (value: Float) {
@@ -847,6 +859,37 @@ validate(first: model.user.User, second: model.user.User) (okay: Bool) { okay =
directory
}
fn encoding_commit_sequence(setting: &str, timeout: Duration) {
let directory = configured_directory(setting);
let sources = fixture_sources_with_api(
&ENCODING_COMMIT,
"long cmd exercise = app.exercise\ncmd inspect = app.inspect",
);
for (command, body, fault) in [
("store.item.exercise", ENCODING_COMMIT.body, Some(3)),
("store.item.inspect", "{}", None),
] {
let ir = entry_ir(&sources, fault);
assert_output(
llvm_module::execute_in(
&directory,
&ir,
"encoding failure preserves committed row",
64,
timeout,
&[command, body],
),
fault.is_some(),
);
}
}
#[test]
#[ignore = "requires explicit runtime archive; committed long-entry SQLite stages"]
fn sqlite_long_command_preserves_commit_on_encoding_failure() {
encoding_commit_sequence(SETTING, Duration::from_secs(20));
}
fn constraint_sequence(setting: &str, timeout: Duration) {
let directory = configured_directory(setting);
let sources = fixture_sources_with_api(
+12 -9
View File
@@ -6,8 +6,8 @@ mod llvm_module;
#[path = "dever-tests/tests/support/process.rs"]
mod process;
#[cfg(all(target_os = "linux", target_arch = "x86_64"))]
#[path = "dever-tests/tests/support/sandbox.rs"]
mod sandbox;
#[path = "dever-tests/tests/support/process_runtime.rs"]
mod process_runtime;
#[cfg(all(target_os = "linux", target_arch = "x86_64"))]
#[path = "dever-tests/tests/support/temp.rs"]
mod temp;
@@ -61,10 +61,10 @@ capture() (text: Text) { text = message(result(read())) }
checked(ecosystem, READ_APP, READ_PORT, READ_API)
}
fn settings(program: &Program, mode: &str) -> String {
fn settings(program: &Program, mode: &str, root: &Path) -> String {
let contracts = program.external_worker_contracts();
let identity = contracts[0].errors.keys().next().unwrap();
json!({"adapter":{"notification.delivery":{"setting":{"mode":mode,"identity":identity,"marker":"/data/lifecycle/worker"},"files":{"lifecycle":{"path":"lifecycle","write":true}}}}})
json!({"adapter":{"notification.delivery":{"setting":{"mode":mode,"identity":identity,"marker":root.join("lifecycle/worker")}}}})
.to_string()
}
@@ -100,7 +100,7 @@ capture() (text: Text) { text = message(result(read())) }
"arguments":[], "working_directory":"workers/fixture"
});
let mut resources =
super::sandbox::assets(&Path::new(env!("CARGO_MANIFEST_DIR")).join("../.."))
super::process_runtime::assets(&Path::new(env!("CARGO_MANIFEST_DIR")).join("../.."))
.into_iter()
.map(|asset| resource(asset.path, asset.bytes, asset.executable))
.collect::<Vec<_>>();
@@ -130,7 +130,7 @@ capture() (text: Text) { text = message(result(read())) }
) -> std::process::Output {
let (_, program) = source;
let directory = temp::TemporaryDirectory::new();
stage(&directory, &settings(program, mode));
stage(&directory, &settings(program, mode, directory.path()));
let resources = resources(program);
let output = execute(
source,
@@ -232,12 +232,12 @@ read() (summary: Summary) {
}
#[test]
#[ignore = "requires fresh runtime-external archive, command-fixture and sandbox assets"]
#[ignore = "requires fresh runtime-external archive, command-fixture and process_runtime assets"]
fn command_binary_io_and_exit_status_use_real_typed_llvm_calls() {
let source = command_source();
let contract = &source.1.external_command_contracts()[0];
let workspace = Path::new(env!("CARGO_MANIFEST_DIR")).join("../..");
let mut resources = super::sandbox::assets(&workspace)
let mut resources = super::process_runtime::assets(&workspace)
.into_iter()
.map(|asset| resource(asset.path, asset.bytes, asset.executable))
.collect::<Vec<_>>();
@@ -590,7 +590,10 @@ read() (value: Int) {
let relocated = directory.path().join("program");
fs::copy(executable, &relocated).unwrap();
drop(build);
stage(&directory, &settings(&program, "embedded"));
stage(
&directory,
&settings(&program, "embedded", directory.path()),
);
result(
&llvm_module::execute_program(
&relocated,
+15 -10
View File
@@ -95,17 +95,22 @@ transaction abort() () {
}
apply(input: Publish) () {
original = model.get(input.article_id)
title = port.title(original.title)
store(input.article_id, title)
}
transaction store(article_id: model.id, title: Text) () {
original = model.get(article_id)
marker = model.create({ title = "started" })
article = model.create({ title = port.title(original.title) })
article = model.create({ title = title })
}
count(title: Text) (total: Int) { total = model.count({ where = title == title }) }
arm() (id: Id) { id = dever.job.enqueue(job.cleanup, "manual") }
clean() () { article = model.create({ title = "cron" }) }
transaction arm() (id: Id) { id = dever.job.enqueue(job.cleanup, "manual") }
transaction clean() () { article = model.create({ title = "cron" }) }
"#,
),
(
"content/article/job.dever",
"database default\njob publish(input: app.Publish) () retry(2) timeout(1000) { app.apply(input) }\njob cleanup() () retry(2) timeout(1000) { app.clean() }\nschedule cleanup = \"* * * * *\"",
"database default\nlong job publish(input: app.Publish) () retry(2) timeout(1000) { app.apply(input) }\njob cleanup() () retry(2) timeout(1000) { app.clean() }\nschedule cleanup = \"* * * * *\"",
),
(
"content/article/port.dever",
@@ -566,8 +571,8 @@ retry() () {
}
#[test]
#[ignore = "requires explicit runtime archive; bounded task cancellation inside owned SQLite transactions"]
fn timed_out_job_drains_children_and_rolls_back_before_retry() {
#[ignore = "requires explicit runtime archive; bounded cancellation between committed SQLite stages"]
fn timed_out_job_drains_children_and_preserves_committed_stages() {
use dever_runtime::orm::Value;
let module = [
("main.dever", "main() () {}"),
@@ -577,7 +582,7 @@ retry() () {
),
(
"content/article/app.dever",
"submit() (id: Id) { id = dever.job.enqueue(job.publish, \"timeout\") }\napply() () { saved = model.create({ title = \"rollback\" })\n port.pause() }\ncount() (total: Int) { total = model.count() }",
"transaction submit() (id: Id) { id = dever.job.enqueue(job.publish, \"timeout\") }\ntransaction save() () { saved = model.create({ title = \"committed\" }) }\napply() () { save()\n port.pause() }\ncount() (total: Int) { total = model.count() }",
),
(
"content/article/port.dever",
@@ -586,7 +591,7 @@ retry() () {
("content/article/adapter.dever", "port.pause() () {}"),
(
"content/article/job.dever",
"database default\njob publish() () retry(2) timeout(10) { app.apply() }",
"database default\nlong job publish() () retry(2) timeout(10) { app.apply() }",
),
];
let tests = [(
@@ -595,10 +600,10 @@ retry() () {
timeout() () {
id = app.submit()
assert_eq(dever.test.drain_jobs(5), 1)
assert_eq(app.count(), 0)
assert_eq(app.count(), 1)
dever.test.advance_clock(1000)
assert_eq(dever.test.drain_jobs(5), 1)
assert_eq(app.count(), 0)
assert_eq(app.count(), 2)
dever.test.advance_clock(10000)
assert_eq(dever.test.drain_jobs(5), 0)
}"#,
+13 -10
View File
@@ -1,5 +1,5 @@
//! Developer-only C ABI acceptance; normal execution never invokes these tools.
use std::path::Path;
use std::path::{Path, PathBuf};
use std::process::{Command, Stdio};
use std::time::Duration;
@@ -12,7 +12,7 @@ mod temp;
// from the default tests because Cargo must not recursively build from a test.
#[cfg(all(target_os = "linux", target_arch = "x86_64"))]
#[test]
#[ignore = "build runtime-abi-only staticlib in target/native-runtime-abi before running"]
#[ignore = "requires an explicit runtime-only DEVER_RUNTIME_ABI_ARCHIVE"]
fn canonical_c_header_links_and_executes_the_real_runtime_archive() {
run_driver("driver.c", None, false);
}
@@ -20,10 +20,13 @@ fn canonical_c_header_links_and_executes_the_real_runtime_archive() {
#[cfg(all(target_os = "linux", target_arch = "x86_64"))]
fn run_driver(source: &str, llvm_object: Option<&Path>, allocation_ledger: bool) {
let root = Path::new(env!("CARGO_MANIFEST_DIR")).join("../..");
let archive = root.join("target/native-runtime-abi/debug/libdever_backend_bridge.a");
let archive = PathBuf::from(
std::env::var_os("DEVER_RUNTIME_ABI_ARCHIVE")
.expect("set DEVER_RUNTIME_ABI_ARCHIVE to the runtime-only archive being validated"),
);
assert!(
archive.is_file(),
"missing explicit runtime ABI fixture archive"
archive.is_absolute() && archive.is_file(),
"runtime ABI fixture must be an existing absolute archive path"
);
let directory = temp::TemporaryDirectory::new();
let program = directory.path().join("abi-acceptance");
@@ -75,7 +78,7 @@ fn run_driver(source: &str, llvm_object: Option<&Path>, allocation_ledger: bool)
#[cfg(all(feature = "embedded", target_os = "linux", target_arch = "x86_64"))]
#[test]
#[ignore = "build runtime-abi-only staticlib in target/native-runtime-abi before running"]
#[ignore = "requires an explicit runtime-only DEVER_RUNTIME_ABI_ARCHIVE"]
fn llvm_object_calls_the_versioned_runtime_abi_without_rust_layouts() {
use dever_backend_bridge::{Target, emit_object};
let directory = temp::TemporaryDirectory::new();
@@ -90,21 +93,21 @@ fn llvm_object_calls_the_versioned_runtime_abi_without_rust_layouts() {
#[cfg(all(target_os = "linux", target_arch = "x86_64"))]
#[test]
#[ignore = "build runtime-abi-only staticlib in target/native-runtime-abi before running"]
#[ignore = "requires an explicit runtime-only DEVER_RUNTIME_ABI_ARCHIVE"]
fn protocol_poll_rejects_invalid_output_before_advancing_the_operation() {
run_driver("protocol-driver.c", None, false);
}
#[cfg(all(target_os = "linux", target_arch = "x86_64"))]
#[test]
#[ignore = "build runtime-abi-only staticlib in target/native-runtime-abi before running"]
#[ignore = "requires an explicit runtime-only DEVER_RUNTIME_ABI_ARCHIVE"]
fn logger_flush_is_a_stable_native_allocation_barrier() {
run_driver("log-driver.c", None, true);
}
#[cfg(all(target_os = "linux", target_arch = "x86_64"))]
#[test]
#[ignore = "build runtime-abi-only staticlib in target/native-runtime-abi before running"]
#[ignore = "requires an explicit runtime-only DEVER_RUNTIME_ABI_ARCHIVE"]
fn wire_codecs_validate_slots_before_callback_and_release_partial_values() {
run_driver("wire-driver.c", None, true);
}
@@ -115,7 +118,7 @@ fn wire_codecs_validate_slots_before_callback_and_release_partial_values() {
target_arch = "x86_64"
))]
#[test]
#[ignore = "build a runtime-sqlite or runtime-postgres staticlib in target/native-runtime-abi before running"]
#[ignore = "requires a runtime-sqlite or runtime-postgres DEVER_RUNTIME_ABI_ARCHIVE"]
fn database_abi_preserves_typed_owners_and_rejects_invalid_slots_without_io() {
run_driver("database-driver.c", None, true);
}
+23 -13
View File
@@ -6,6 +6,7 @@ import hashlib
import importlib.util
import io
import json
import os
from pathlib import Path
import subprocess
import tarfile
@@ -17,6 +18,7 @@ from install_support import load_installer
ROOT = Path(__file__).resolve().parents[2]
ZSTD = Path(os.environ.get("DEVER_TEST_ZSTD", "/usr/bin/zstd"))
def load_module(name, path):
@@ -86,7 +88,7 @@ class InstallerTests(unittest.TestCase):
self.platform, "1.2.3")
def create_assets(self, output):
return assets.create(self.release, output, Path("/usr/bin/zstd"))
return assets.create(self.release, output, ZSTD)
def release_record(self, directory, version="1.2.3"):
return {
@@ -98,7 +100,7 @@ class InstallerTests(unittest.TestCase):
}
def archive_payloads(self, path):
contents = subprocess.run(["/usr/bin/zstd", "-q", "-d", "-c", str(path)],
contents = subprocess.run([str(ZSTD), "-q", "-d", "-c", str(path)],
env={}, capture_output=True, check=True, timeout=15).stdout
with tarfile.open(fileobj=io.BytesIO(contents)) as archive:
return {member.name: archive.extractfile(member).read() for member in archive}
@@ -135,24 +137,32 @@ class InstallerTests(unittest.TestCase):
self.create_assets(first)
def test_extension_assets_are_separate_and_cannot_overlap_base(self):
base_only = self.root / "base-only"
self.create_assets(base_only)
manifest_path = self.release / "manifest.json"
manifest = json.loads(manifest_path.read_text())
name = f"runtime/pip/{self.platform}/runtime.pack"
path = self.release / name
path.parent.mkdir(parents=True)
contents = b"optional Python bytes"
path.write_bytes(contents)
entry = {"path": name, "bytes": len(contents), "sha256": hashlib.sha256(contents).hexdigest()}
manifest["extensions"] = [{"kind": {"type": "runtime", "ecosystem": "pip"},
"target": self.platform, "artifacts": [entry]}]
expected = {}
for ecosystem in ("pip", "npm", "go"):
name = f"runtime/{ecosystem}/{self.platform}/runtime.pack"
path = self.release / name
path.parent.mkdir(parents=True)
contents = f"optional {ecosystem} bytes".encode()
path.write_bytes(contents)
entry = {"path": name, "bytes": len(contents), "sha256": hashlib.sha256(contents).hexdigest()}
manifest["extensions"].append({"kind": {"type": "runtime", "ecosystem": ecosystem},
"target": self.platform, "artifacts": [entry]})
expected[ecosystem] = {name: contents}
manifest_path.write_text(json.dumps(manifest))
self.sign_manifest()
self.authenticate()
output = self.root / "split"
self.create_assets(output)
self.assertEqual(self.archive_payloads(output / f"dever-{self.platform}.tar.zst"), self.payloads)
extension = output / f"dever-{self.platform}.ext-runtime-pip-{self.platform}.tar.zst"
self.assertEqual(self.archive_payloads(extension), {name: contents})
base_name = f"dever-{self.platform}.tar.zst"
self.assertEqual(self.archive_payloads(output / base_name), self.payloads)
self.assertEqual((output / base_name).read_bytes(), (base_only / base_name).read_bytes())
for ecosystem, payloads in expected.items():
extension = output / f"dever-{self.platform}.ext-runtime-{ecosystem}-{self.platform}.tar.zst"
self.assertEqual(self.archive_payloads(extension), payloads)
manifest["extensions"][0]["artifacts"].append(self.catalog["dever-core"])
manifest_path.write_text(json.dumps(manifest))
with self.assertRaisesRegex(ValueError, "resource namespace"):
+7 -4
View File
@@ -78,8 +78,11 @@ def worker_project(image, ecosystem):
def isolated(image, init):
isolator = Path(os.environ["DEVER_TEST_ISOLATOR"])
if not isolator.is_absolute() or not isolator.is_file():
raise ValueError("DEVER_TEST_ISOLATOR requires an absolute test-only executable")
return [
WORKSPACE / "target/sandbox-inputs/assets/bin/bwrap",
isolator,
"--unshare-all", "--die-with-parent", "--new-session", "--bind", image,
"/", "--proc", "/proc", "--dev", "/dev", "--chdir", "/", "--ro-bind",
init, "/fixture-proc-init",
@@ -195,7 +198,7 @@ def ordinary_uid_target_reuse(image):
child.stderr.close()
def install_assets(assets, image, manifest):
def install_assets(assets, image, manifest, trusted_key):
# Only transport is local: signature verification, full tar extraction and
# the final bootstrap executable are the real first-install implementation.
installer = load_installer()
@@ -215,7 +218,7 @@ def install_assets(assets, image, manifest):
installer.download = download
installer.install(argparse.Namespace(
version="latest", release=None, system_root=image, no_start=True,
trusted_key=WORKSPACE / "skills/assets/release.pub",
trusted_key=trusted_key,
))
@@ -238,7 +241,7 @@ def accept(release, assets, image, report, init, extension_helper=None):
try:
minimal_os(image)
if assets:
install_assets(assets, image, manifest)
install_assets(assets, image, manifest, WORKSPACE / "skills/assets/release.pub")
else:
execute([
"/bin/bash", WORKSPACE / "install",
+210 -79
View File
@@ -1,113 +1,244 @@
#!/usr/bin/env python3
"""Actual-asset update transaction in an owned image; never publishes the fixture version."""
"""Actual-version install, upgrade and rollback in an owned container directory."""
import argparse
import hashlib
import json
import os
from pathlib import Path
import shutil
import subprocess
import tempfile
import time
from verify_prepared_release import InstalledDaemon, execute, install_assets, minimal_os, response
from install_support import load_installer
from verify_prepared_release import execute, install_assets, response, worker_project
def prepare_update(image, manifest, key, helper):
# Reuse real payloads under a newly signed, acceptance-only catalog identity.
# This tests the update owner without claiming a second public release exists.
next_manifest = dict(manifest)
major, minor, patch = map(int, manifest["version"].split("."))
version = f"{major}.{minor}.{patch + 1}"
next_manifest["version"] = version
class LocalDaemon:
def __init__(self, machine):
self.machine = machine
self.launcher = machine / "bin/dever"
self.child = None
self.errors = None
def __enter__(self):
self.errors = tempfile.TemporaryFile()
self.child = subprocess.Popen(
[str(self.machine / "bin/deverd"), "--root", str(self.machine)],
env={}, stdout=subprocess.DEVNULL, stderr=self.errors,
)
try:
deadline = time.monotonic() + 10
while time.monotonic() < deadline:
if self.child.poll() is not None:
self.errors.seek(0)
raise RuntimeError(self.errors.read(65536).decode(errors="replace"))
if (self.machine / "state/deverd.sock").is_socket():
self.command("cache", "status")
return self
time.sleep(0.05)
raise RuntimeError("owned compiler daemon did not become ready")
except BaseException:
self.__exit__(None, None, None)
raise
def command(self, *arguments):
return execute([self.launcher, *arguments], timeout=240)
def __exit__(self, *_):
if self.child is not None and self.child.poll() is None:
self.child.terminate()
try:
self.child.wait(timeout=10)
except subprocess.TimeoutExpired:
self.child.kill()
self.child.wait(timeout=10)
if self.errors is not None:
self.errors.close()
def write(root, relative, contents):
path = root / relative
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(contents)
def profile_project(root, profile):
sqlite = {"type": "sqlite", "path": "data/owned.sqlite", "max_connections": 1}
postgres = {"type": "postgres", "url": "postgres://fixture@127.0.0.1:1/unused", "tls": "disabled"}
settings = {
"base": {}, "sqlite": {"database": {"default": sqlite}},
"postgres": {"database": {"default": postgres}},
"both": {"database": {"default": sqlite, "unused": postgres}},
}[profile]
write(root, "config/setting.json", json.dumps(settings))
if profile in ("sqlite", "both"):
write(root, "module/sample/value/model.dever", "type Value { name: Text(1, 32) }\n")
write(root, "module/sample/value/api.dever", "cmd exercise = app.exercise\nlong cmd stages = app.stages\n")
source = '''exercise() (okay: Bool) {
saved = model.create({ name = "pack" })
count = model.count()
removed = model.delete(saved.id)
okay = count == 1 and removed == 1
}
stages() (okay: Bool) {
id = store()
dever.task.sleep(1)
okay = finish(id)
}
transaction store() (id: model.id) {
saved = model.create({ name = "stage" })
id = saved.id
}
transaction finish(id: model.id) (okay: Bool) {
count = model.count()
removed = model.delete(id)
okay = count == 1 and removed == 1
}
'''
else:
write(root, "module/sample/value/api.dever", "cmd exercise = app.exercise\n")
source = "exercise() (okay: Bool) { dever.task.sleep(1)\n okay = true }\n"
write(root, "module/sample/value/app.dever", source)
def fresh_install(args, manifest, passed):
image = args.work / "fresh"
image.mkdir()
install_assets(args.assets, image, manifest, args.trusted_key)
machine = image / "opt/dever"
package = machine / "cache/downloads" / version
package.mkdir()
installed = machine / "versions" / manifest["version"]
for artifact in manifest["artifacts"]:
destination = package / artifact["path"]
destination.parent.mkdir(parents=True, exist_ok=True)
os.link(installed / artifact["path"], destination)
catalog = package / "manifest.json"
catalog.write_text(json.dumps(next_manifest, separators=(",", ":")))
execute([helper, "--sign-catalog", catalog, key])
(machine / "cache/downloads/latest").write_text(version)
return version, hashlib.sha256(catalog.read_bytes()).hexdigest()
programs = []
with LocalDaemon(machine) as daemon:
if daemon.command("version") != manifest["version"]:
raise RuntimeError("fresh installation selected the wrong version")
skill = machine / "versions" / manifest["version"] / "skills/dever-language"
if daemon.command("skill", "path") != str(skill):
raise RuntimeError("installed skill is not paired with the compiler")
if not (skill / "assets/template/module/hello/greeting/app.dever").is_file():
raise RuntimeError("release has no complete project template")
extensions = machine / "cache/extensions"
if extensions.exists() and any(extensions.iterdir()):
raise RuntimeError("fresh base installation installed optional extensions")
project = image / "hello"
daemon.command("new", project)
response(daemon.command("run", project, "--", "hello.greeting.greet", '{"name":"Dever"}'))
passed("fresh_split_base_install_and_full_skill_template")
for profile in ("base", "sqlite", "postgres", "both"):
project = image / f"profile-{profile}"
profile_project(project, profile)
response(daemon.command("run", project, "--", "sample.value.exercise", "{}"), True)
if profile in ("sqlite", "both"):
response(daemon.command("run", project, "--", "sample.value.stages", "{}"), True)
program = project / "program"
daemon.command("build", project, "--output", program)
shutil.rmtree(project / "module")
programs.append((program, "sample.value.exercise", True))
passed("four_runtime_profiles_sqlite_and_long_transactions")
for ecosystem in ("pip", "npm", "go"):
execute([args.extension_helper, machine, args.assets, manifest["version"], f"runtime-{ecosystem}-{manifest['platform']}"])
project = worker_project(image, ecosystem)
daemon.command("lib", "add", project)
response(daemon.command("run", project, "--", "sample.worker.probe", "{}"), True)
program = project / "program"
daemon.command("build", project, "--output", program)
shutil.rmtree(project / "module")
programs.append((program, "sample.worker.probe", True))
passed(f"signed_optional_{ecosystem}_managed_run_and_build")
for program, command, expected in programs:
response(execute([program, command, "{}"]), expected)
passed("source_free_standalone_programs_without_running_daemon")
shutil.rmtree(image)
def upgrade(args, manifest, previous, passed):
image = args.work / "upgrade"
image.mkdir()
original_umask = os.umask(0o022)
try:
load_installer().install(argparse.Namespace(
version=previous["version"], release=args.previous_release,
system_root=image, no_start=True, trusted_key=args.trusted_key,
))
finally:
os.umask(original_umask)
machine = image / "opt/dever"
project = image / "existing-project"
with LocalDaemon(machine) as daemon:
if daemon.command("version") != previous["version"]:
raise RuntimeError("previous installation did not select its actual binary version")
daemon.command("new", project)
response(daemon.command("run", project, "--", "hello.greeting.greet", '{"name":"Dever"}'))
original = (project / "config/setting.json").read_bytes()
passed("actual_previous_release_installed_and_program_runs")
install_assets(args.assets, image, manifest, args.trusted_key)
with LocalDaemon(machine) as daemon:
if daemon.command("version") != manifest["version"]:
raise RuntimeError("upgrade did not select the actual new compiler version")
response(daemon.command("run", project, "--", "hello.greeting.greet", '{"name":"Dever"}'))
if (project / "config/setting.json").read_bytes() != original:
raise RuntimeError("upgrade modified project configuration")
passed("actual_new_release_upgrade_preserves_existing_project")
daemon.command("use", previous["version"])
if daemon.command("version") != previous["version"]:
raise RuntimeError("rollback did not restore the previous compiler")
response(daemon.command("run", project, "--", "hello.greeting.greet", '{"name":"Dever"}'))
passed("rollback_runs_actual_previous_compiler")
daemon.command("use", manifest["version"])
if daemon.command("version") != manifest["version"]:
raise RuntimeError("switch after rollback did not restore the new compiler")
response(daemon.command("run", project, "--", "hello.greeting.greet", '{"name":"Dever"}'))
passed("new_release_runs_again_after_rollback")
shutil.rmtree(image)
def accept(args):
manifest = json.loads((args.release / "manifest.json").read_bytes())
result = {"version": manifest["version"], "checks": [],
"transport": "local actual assets; acceptance-only next-version catalog",
"online_update": "not_run"}
previous = json.loads((args.previous_release / "manifest.json").read_bytes())
if tuple(map(int, manifest["version"].split("."))) <= tuple(map(int, previous["version"].split("."))):
raise ValueError("upgrade requires distinct increasing actual release versions")
args.work.mkdir(mode=0o700)
result = {
"previous_version": previous["version"], "version": manifest["version"],
"checks": [], "errors": {}, "transport": "local signed split assets",
"trust": "explicit acceptance key; not proof of production signing",
"host_language_absence": "not_verified", "gpu": "not_run",
"postgres_server": "not_run", "online_update": "not_run",
"global_service_actions": "not_run",
}
def passed(name):
result["checks"].append(name)
args.report.write_text(json.dumps(result, indent=2) + "\n")
print(f"PASS {name}", flush=True)
try:
args.image.mkdir()
minimal_os(args.image)
install_assets(args.assets, args.image, manifest)
machine = args.image / "opt/dever"
extension = f"runtime-pip-{manifest['platform']}"
execute([args.extension_helper, machine, args.assets, manifest["version"], extension])
records = args.report.parent / f"{args.report.stem}-process"
with InstalledDaemon(args.image, records, args.init) as daemon:
daemon.command("new", "/project")
response(daemon.command("run", "/project", "--", "hello.greeting.greet", '{"name":"Dever"}'))
passed("actual_release_and_selected_extension_before_update")
version, catalog = prepare_update(args.image, manifest, args.key, args.extension_helper)
result["fixture_version"] = version
previous_active = (machine / "state/active-version").read_bytes()
empty_assets = args.image / "missing-extension-assets"
empty_assets.mkdir()
for name, stage in (
("fresh", lambda: fresh_install(args, manifest, passed)),
("upgrade", lambda: upgrade(args, manifest, previous, passed)),
):
try:
execute([args.extension_helper, machine, empty_assets, version, "--update"])
except RuntimeError as error:
if "cannot open actual extension asset" not in str(error):
raise
else:
raise RuntimeError("update with a missing extension should fail")
if (machine / "state/active-version").read_bytes() != previous_active:
raise RuntimeError("failed update changed active version")
if daemon.command("version") != manifest["version"]:
raise RuntimeError("failed update changed the public launcher selection")
response(daemon.command("run", "/project", "--", "hello.greeting.greet", '{"name":"Dever"}'))
passed("failed_extension_update_keeps_old_active_and_application_working")
execute([args.extension_helper, machine, args.assets, version, "--update"])
if daemon.command("version") != version:
raise RuntimeError("successful update did not activate the new version")
if daemon.command("skill", "path") != f"/opt/dever/versions/{version}/skills/dever-language":
raise RuntimeError("skill did not follow the updated core")
extensions = machine / "cache/extensions" / version / catalog
if {path.name for path in extensions.iterdir()} != {extension}:
raise RuntimeError("update did not preserve exactly the installed extension selection")
passed("successful_update_activates_core_skill_and_only_installed_extensions")
# Only the catalog version changes in this transaction fixture.
# The managed compiler correctly requires its real build version;
# execute it again only after selecting the actual release.
daemon.command("use", manifest["version"])
response(daemon.command("run", "/project", "--", "hello.greeting.greet", '{"name":"Dever"}'))
passed("actual_release_remains_usable_after_update_and_explicit_switch_back")
result["status"] = "passed"
except BaseException as error:
result["status"] = "failed"
result["error"] = str(error)
raise
stage()
except (OSError, ValueError, RuntimeError, subprocess.SubprocessError) as error:
result["errors"][name] = str(error)
print(f"FAIL {name}: {error}", flush=True)
result["status"] = "failed" if result["errors"] else "passed"
return not result["errors"]
finally:
args.report.write_text(json.dumps(result, indent=2) + "\n")
def main():
parser = argparse.ArgumentParser(description=__doc__)
for name in ("release", "assets", "image", "report", "init", "extension-helper", "key"):
for name in ("previous-release", "release", "assets", "trusted-key", "work", "report", "extension-helper"):
parser.add_argument(f"--{name}", required=True, type=Path)
args = parser.parse_args()
if os.geteuid() != 0:
parser.error("requires root for an owned image; never installs host services")
parser.error("requires root in an owned validation container; never touches its installed service")
for name, value in vars(args).items():
setattr(args, name, value.absolute())
args.report.parent.mkdir(parents=True, exist_ok=True)
accept(args)
if not value.is_absolute():
parser.error(f"{name} must be an explicit absolute path")
if args.work.exists() or args.work.is_symlink() or args.report.exists():
parser.error("acceptance work/report must be new owned paths")
raise SystemExit(0 if accept(args) else 1)
if __name__ == "__main__":