From 551d7472f51be76fec1fc817a129075081db1433 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Sat, 26 Sep 2026 06:03:49 +0800 Subject: [PATCH 01/26] 2026.9.27.1: a build.mcpp can declare a runtime library search directory, joining `[runtime] library_dirs` `mcpp::runtime_library_dir(dir)` (`mcpp:runtime-library-dir=`, protocol 12) is the build-program form of `[runtime] library_dirs`: a launch-time search directory for a dependency a build.mcpp discovers (a vcpkg prefix's `bin/`, a Qt SDK's `bin/`) rather than one an author can write into a fixed TOML array. It joins the SAME field the manifest key populates (`RuntimeConfig::libraryDirs`), so every existing consumer of that field sees a directive-declared entry without any of them changing: `mcpp run`'s loader path, `mcpp pack`'s closure search, and the ELF/Mach-O `-Wl,-rpath` rendering. Persisted like `deploy`/`warning`/`pack-format` (non-empty cache tag, `kCacheEpoch` not bumped, same reasoning), and the root package's own build.mcpp needed the same manifest-snapshot mirror `deploy` already has in prepare.cppm, or `resolve_runtime_contract` never sees a directive-sourced entry. Adds unit coverage in test_build_directives.cpp and an e2e script (779) that links an executable against a prebuilt shared library only the directive's RUNPATH makes loadable, checks the build.mcpp cache-hit replay, and checks `mcpp pack --format dir`'s closure staging. Docs: docs/30-build-mcpp.md and docs/zh/30-build-mcpp.md gain the directive, the typed API entry and a worked section; docs/04-mcpp-toml.md and its zh counterpart point `[runtime] library_dirs` at the build-program form. --- docs/04-mcpp-toml.md | 5 + docs/30-build-mcpp.md | 34 +++++ docs/zh/04-mcpp-toml.md | 5 + docs/zh/30-build-mcpp.md | 31 +++++ mcpp.toml | 2 +- modules/buildmcpp/src/directives.cppm | 45 ++++++- modules/buildmcpp/src/program_protocol.cppm | 9 +- modules/versioning/src/version.cppm | 2 +- src/build/hostprogram.cppm | 11 ++ src/build/prepare.cppm | 12 ++ ..._program_declares_a_runtime_library_dir.sh | 125 ++++++++++++++++++ tests/unit/test_build_directives.cpp | 101 +++++++++++++- 12 files changed, 374 insertions(+), 8 deletions(-) create mode 100755 tests/e2e/779_a_build_program_declares_a_runtime_library_dir.sh diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index df94f689..6737e9be 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -1473,6 +1473,11 @@ For one compatibility train, `library_dirs` maps only to runtime search, `capabilities` maps to required run-phase capability requirements. None of these legacy fields creates a provider. +`library_dirs` has a build-program form for a directory only a build.mcpp can +locate (a vcpkg prefix's `bin/`, a Qt SDK's `bin/`): `mcpp::runtime_library_dir(dir)` +(2026.9.27.1+, protocol 12; [30 — Build Programs](30-build-mcpp.md)), which +joins this same field. + `target///resolution.json` schema 2 stores the RuntimeBinding, canonical requirements/providers/artifacts, LinkIntent, platform search mechanism, and post-link verdict. `mcpp why runtime` is a pure interpreter of diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index a0b365e5..1c42f724 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -67,6 +67,7 @@ is ignored, so diagnostics may be logged freely. | `mcpp:windows-subsystem=:` *(2026.9.12.2+)* | set the PE subsystem (`console` or `windows`) of the executable `` of **this** package, the same field as `[targets.] windows_subsystem` (docs/04). Reaches that target's link and no other, never a consumer, and renders nothing on a target that is not PE. A target the package does not declare with `kind = "bin"`, a value outside the set, and a value that contradicts mcpp.toml are each refused before any directive is applied | | `mcpp:windows-entry=:` *(2026.9.12.2+)* | set the entry function (`main`, `wmain`, `WinMain` or `wWinMain`) of the executable ``, the same field as `windows_entry`; the scope and the refusals are those of `windows-subsystem` | | `mcpp:deploy=\t` *(2026.9.12.3+, protocol 11)* | place a file this program produced or selected beside the artifact, at ``, relative to the executable's directory — the build-program form of `[runtime] deploy` (docs/04 §2.11). `` may be absolute (an action's own declared output) or resolved against the package root; TAB-separated, because an absolute Windows `` contains a colon. **Reaches the consumer**, joining the same `LinkIntent` `link-lib`/`link-search`/`link-flag` feed — see below | +| `mcpp:runtime-library-dir=` *(2026.9.27.1+, protocol 12)* | add `` to the launch-time search path — the build-program form of `[runtime] library_dirs` (docs/04 §2.11). Relative resolves against the package root. **Reaches the consumer**, joining the same `LinkIntent` field the manifest key populates: RUNPATH/rpath on ELF and Mach-O, never `-L`, and `mcpp pack`'s closure search — see below | | `mcpp:link-script=` *(2026.8.19+)* | link with this **linker script** (`-T`; relative resolves against the package root, and the emitted path is absolute because the link runs in the build directory). Reaches the **consumer**, unlike `include-dir` — a board's memory layout is the one thing a consumer cannot write for itself | | `mcpp:warning=` *(2026.8.21.2+)* | say something to the user and **keep going**. The one directive that changes no compile line, no link line and no source set. Survives the build cache — see below | | `mcpp:fact==` *(2026.9.5.2+)* | state something the program **established about the machine** (`cuda.driver=12.4`). Compared against floors before anything is compiled; see below | @@ -134,6 +135,7 @@ int main() { | `mcpp::link_flag(s)` *(2026.9.6.5+)* | `mcpp:link-flag=` | | `mcpp::windows_subsystem(target, value)` / `mcpp::windows_entry(target, value)` *(2026.9.12.2+)* | `mcpp:windows-subsystem=` / `mcpp:windows-entry=` | | `mcpp::deploy(from, to)` *(2026.9.12.3+, protocol 11)* | `mcpp:deploy=\t` — see below | +| `mcpp::runtime_library_dir(dir)` *(2026.9.27.1+, protocol 12)* | `mcpp:runtime-library-dir=` — see below | | `mcpp::link_script(p)` *(2026.8.19+)* | `mcpp:link-script=` | | `mcpp::runner(tok)` *(2026.8.19.2+)* | `mcpp:runner=` — see below | | `mcpp::xpkg_dir(ns, name)` / `mcpp::xpkg_dir(name)` *(2026.8.19+)* | the payload directory of a package declared in `[xlings.workspace]` — by this manifest, or by a dependency compiled into this build program *(2026.9.6.6+)*; `""` when it was not declared or is not installed (see below) | @@ -656,6 +658,38 @@ int main() { | `.apk` | `assets/myapp.resources/` | | web | the same relative path in the static directory, served beside `.js`. A project that wants the files inside the `.data` preload instead links with `--preload-file @/`, an ordinary link flag | +### A launch-time search directory: `runtime_library_dir` (2026.9.27.1+, protocol 12) + +`[runtime] library_dirs` (docs/04 §2.11) names a directory to search when the +artifact runs. It is a fixed TOML array, so it cannot name a directory a +build.mcpp only discovers — a vcpkg prefix's `bin/`, a Qt SDK's `bin/`, or any +other prebuilt-dependency layout a build-time probe locates. `mcpp::runtime_library_dir` +is that same declaration, reached from a build program: + +```cpp +import mcpp; +#include + +int main() { + const std::string qtBin = locate_qt_prefix() + "/bin"; // however this + // package finds it + mcpp::link_search(qtBin.c_str()); + mcpp::runtime_library_dir(qtBin.c_str()); +} +``` + +- **Joins the same field the manifest key populates.** A directive-declared + directory reaches every consumer of `[runtime] library_dirs` exactly as one + written in `mcpp.toml` would: `mcpp run`'s loader path, `mcpp pack`'s closure + search, and RUNPATH/rpath on ELF and Mach-O (never `-L` — a launch-time + search directory is not a link-library search path). +- **`dir` may be absolute or package-relative.** A relative value resolves + against the package root, like every other `AbsPath` directive + (`include-dir`, `deploy`'s `from`). +- **Replayed on a cache hit.** A `runtime-library-dir` directive is persisted + in the build cache like `deploy` and `warning`; a cached run restores it + exactly as a fresh run would. + ### Producing a distributable: `pack_format` / `stage_dir` (2026.9.11.1+) An `.msi`, a `.deb`, an AppImage and a signed `.app` are none of the four roles' diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 73057d18..70fc7adb 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -1405,6 +1405,11 @@ Link intent 把各个发现阶段分开处理: 映射到必需的运行期 soname 要求,`capabilities` 映射到必需的运行期 能力要求。这些遗留字段都不创建提供者。 +`library_dirs` 有一个构建程序形态,用于一个只有 build.mcpp 才能定位的目录 +(一个 vcpkg 前缀的 `bin/`、一个 Qt SDK 的 `bin/`):`mcpp::runtime_library_dir(dir)` +(2026.9.27.1+,protocol 12;[30 —— 构建程序](30-build-mcpp.md)),并入的是 +同一个字段。 + `target///resolution.json` schema 2 存储 RuntimeBinding、 规范化后的要求/提供者/产物、LinkIntent、平台发现机制与链接后判定。 `mcpp why runtime` 是对最新存储文件的一个纯粹解读器:它既不会重新 diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index 16fab4d6..7e55266f 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -64,6 +64,7 @@ mcpp build # compiles + runs build.mcpp, then builds the project | `mcpp:windows-subsystem=:` *(2026.9.12.2+)* | 设置**本包**可执行目标 `` 的 PE 子系统(`console` 或 `windows`),与 `[targets.] windows_subsystem`(docs/04)是同一字段。只到达该目标的链接,不到达其他目标或消费者,在非 PE 目标上不产生任何标志。本包未以 `kind = "bin"` 声明该目标、取值不在集合内、取值与 mcpp.toml 的声明矛盾,这三种情形都在应用任何指令之前被拒绝 | | `mcpp:windows-entry=:` *(2026.9.12.2+)* | 设置可执行目标 `` 的入口函数(`main`、`wmain`、`WinMain` 或 `wWinMain`),与 `windows_entry` 是同一字段;作用域与拒绝条件同 `windows-subsystem` | | `mcpp:deploy=\t` *(2026.9.12.3+,protocol 11)* | 把本程序生成或选中的一个文件放到产物旁边的 ``(相对可执行文件所在目录)——`[runtime] deploy`(docs/04 §2.11)的构建程序形态。`` 可以是绝对路径(某个 action 自己声明的输出),也可以按包根解析;用 TAB 分隔,因为一个绝对的 Windows `` 本身含冒号。**到达消费者**,并入同一个被 `link-lib`/`link-search`/`link-flag` 喂入的 `LinkIntent`——见下 | +| `mcpp:runtime-library-dir=` *(2026.9.27.1+,protocol 12)* | 把 `` 加入启动期搜索路径——`[runtime] library_dirs`(docs/04 §2.11)的构建程序形态。相对路径按包根解析。**到达消费者**,并入清单键所填的同一个 `LinkIntent` 字段:在 ELF 与 Mach-O 上是 RUNPATH/rpath,绝不是 `-L`,并进入 `mcpp pack` 的闭包搜索——见下 | | `mcpp:link-script=` *(2026.8.19+)* | 用这个**链接脚本**链接(`-T`;相对路径按包根解析,发出的是绝对路径,因为链接是在构建目录里跑的)。与 `include-dir` 不同,它**到达消费者** —— 板子的内存布局恰恰是消费者写不出来的那一项 | | `mcpp:warning=` *(2026.8.21.2+)* | 对用户说一句话并**继续**。唯一一条不改变编译行、链接行与源码集的指令。它**穿过构建缓存** —— 见下 | | `mcpp:fact==` *(2026.9.5.2+)* | 陈述程序**测得的机器事实**(`cuda.driver=12.4`)。在编译任何东西之前与 floor 比较;见下 | @@ -123,6 +124,7 @@ int main() { | `mcpp::link_flag(s)` *(2026.9.6.5+)* | `mcpp:link-flag=` | | `mcpp::windows_subsystem(target, value)` / `mcpp::windows_entry(target, value)` *(2026.9.12.2+)* | `mcpp:windows-subsystem=` / `mcpp:windows-entry=` | | `mcpp::deploy(from, to)` *(2026.9.12.3+,protocol 11)* | `mcpp:deploy=\t` —— 见下 | +| `mcpp::runtime_library_dir(dir)` *(2026.9.27.1+,protocol 12)* | `mcpp:runtime-library-dir=` —— 见下 | | `mcpp::link_script(p)` *(2026.8.19+)* | `mcpp:link-script=` | | `mcpp::runner(tok)` *(2026.8.19.2+)* | `mcpp:runner=` —— 见下 | | `mcpp::xpkg_dir(ns, name)` / `mcpp::xpkg_dir(name)` *(2026.8.19+)* | `[xlings.workspace]` 里声明的包的载荷目录 —— 本 manifest 声明的,或编进本构建程序的某个依赖声明的(2026.9.6.6+);没声明或没安装时返回 `""`(见下) | @@ -559,6 +561,35 @@ int main() { | `.apk` | `assets/myapp.resources/` | | web | 静态目录里同一个相对路径,与 `.js` 放在一起。想把文件放进 `.data` 预加载的项目改用链接标志 `--preload-file @/`,那是一条普通的链接标志 | +### 一个启动期搜索目录:`runtime_library_dir`(2026.9.27.1+,protocol 12) + +`[runtime] library_dirs`(docs/04 §2.11)声明产物运行时要搜索的一个目录。 +它是一个固定的 TOML 数组,因此点不了一个只有 build.mcpp 才能发现的目录—— +一个 vcpkg 前缀的 `bin/`、一个 Qt SDK 的 `bin/`,或任何其它由构建期探测 +才能定位的预编译依赖布局。`mcpp::runtime_library_dir` 就是同一次声明,从 +构建程序里发出: + +```cpp +import mcpp; +#include + +int main() { + const std::string qtBin = locate_qt_prefix() + "/bin"; // 本包自己 + // 怎么找到它 + mcpp::link_search(qtBin.c_str()); + mcpp::runtime_library_dir(qtBin.c_str()); +} +``` + +- **并入清单键所填的同一个字段。** 一个由指令声明的目录,到达 + `[runtime] library_dirs` 的每一个消费者,与写在 `mcpp.toml` 里的完全一样: + `mcpp run` 的加载器路径、`mcpp pack` 的闭包搜索,以及 ELF 与 Mach-O 上的 + RUNPATH/rpath(绝不是 `-L`——一个启动期搜索目录不是一个链接库搜索路径)。 +- **`dir` 可以是绝对路径或按包根解析的相对路径。** 相对路径按包根解析, + 与其它每一个 `AbsPath` 指令(`include-dir`、`deploy` 的 `from`)一样。 +- **在缓存命中时被重放。** `runtime-library-dir` 指令与 `deploy`、`warning` + 一样进入构建缓存;一次缓存命中的重跑会像真正跑过一样把它恢复回来。 + ### 产出可分发物:`pack_format` 与 `stage_dir`(2026.9.11.1+) 一个 `.msi`、一个 `.deb`、一个 AppImage、一个签过名的 `.app`,都不是那四个 role 的 diff --git a/mcpp.toml b/mcpp.toml index 96c19fc0..425f9db9 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.9.26.1" +version = "2026.9.27.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/buildmcpp/src/directives.cppm b/modules/buildmcpp/src/directives.cppm index 2328774b..019e075e 100644 --- a/modules/buildmcpp/src/directives.cppm +++ b/modules/buildmcpp/src/directives.cppm @@ -164,6 +164,18 @@ enum class Slot : std::size_t { // splitting it into two slots would let a build with N deploy directives // pair them up wrong the moment N > 1. Deploy, + // A LAUNCH-TIME SEARCH DIRECTORY, THE BUILD-PROGRAM FORM OF `[runtime] + // library_dirs` (docs/04 §2.11). A dependency that brings a prebuilt + // shared library (a vcpkg prefix's `bin/`, a Qt SDK's `bin/`) knows where + // it lives only at build-program time, and the manifest key cannot be + // computed -- it is a fixed TOML array. `Transform::AbsPath`, not a new + // path shape: this is the SAME field the manifest key feeds + // (`RuntimeConfig::libraryDirs`), so every consumer of it -- `mcpp run`'s + // loader path, `mcpp pack`'s closure search, and the ELF/Mach-O + // `-Wl,-rpath` rendering -- sees a directive-declared entry exactly as it + // sees a TOML one, through the one merge in `plan.cppm` that already + // exists for the manifest key. No new downstream code path. + RuntimeLibraryDir, Count }; inline constexpr std::size_t kSlotCount = static_cast(Slot::Count); @@ -262,7 +274,7 @@ struct Def { int sinceProtocol; }; -inline constexpr std::array kTable{{ +inline constexpr std::array kTable{{ // wire tag slot scope transform must missingPrefix missingSuffix since {"cxxflag", "cxxflag", Slot::CxxFlags, Scope::PackagePrivate, Transform::Verbatim, false, "", "", 1}, {"cflag", "cflag", Slot::CFlags, Scope::PackagePrivate, Transform::Verbatim, false, "", "", 1}, @@ -413,6 +425,29 @@ inline constexpr std::array kTable{{ // before `apply` on both the run path and the cache-hit path, so a cached // replay refuses exactly what a fresh run would. {"deploy", "deploy", Slot::Deploy, Scope::LinkGlobal, Transform::Deploy, false, "", "", 11}, + // v12. The build-program form of `[runtime] library_dirs` (docs/04 + // §2.11): a launch-time search directory, for a dependency (a vcpkg + // prefix's `bin/`, a Qt SDK's `bin/`) whose location a build.mcpp learns + // rather than one an author can write into TOML. Scope::LinkGlobal, + // exactly as `deploy` above, because `apply` folds it into + // `RuntimeConfig::libraryDirs` -- the SAME field the manifest key + // populates -- so `resolve_runtime_contract`'s per-package merge + // (plan.cppm) carries it into a consumer's `linkIntent.runtimeSearchDirs` + // through the one path that already exists for the manifest key, rather + // than a second one this table would have to keep in sync. `mustExistAfterRun` + // is FALSE: a directory is not the declared-output contract's shape + // (`generated=`/`source=` name a file), and the directory need not even + // exist yet -- a payload not installed on this machine is the manifest + // key's own behaviour (`xpkg_dir()` returning empty configures no runner, + // silently and correctly). + // + // kCacheEpoch is NOT bumped, the same reasoning `warning`/`pack-format` + // state explicitly: an entry written before this row carries no + // `d runtime-library-dir` line, and the program that wrote it could not + // emit one -- so replaying it yields exactly what that program said, and + // the entry is still correct. An older engine reading a newer entry + // already discards the whole record through the unknown-tag path. + {"runtime-library-dir", "runtime-library-dir", Slot::RuntimeLibraryDir, Scope::LinkGlobal, Transform::AbsPath, false, "", "", 12}, }}; // ── Collected output of one run ──────────────────────────────────────────── @@ -945,6 +980,14 @@ void apply(mcpp::manifest::Manifest& m, const Directives& d) { for (auto const& p : d.at(Slot::IncludeDirsAfter)) bc.includeDirsAfter.emplace_back(p); + // `runtime-library-dir`: already absolute from the AbsPath + // transform, like IncludeDirs above. Joins `RuntimeConfig::libraryDirs`, + // the exact field `[runtime] library_dirs` populates from TOML -- so a + // consumer sees it exactly as it sees the manifest key, folded in by the + // SAME merge (plan.cppm's `resolve_runtime_contract`), never a second one. + for (auto const& p : d.at(Slot::RuntimeLibraryDir)) + m.runtimeConfig.libraryDirs.emplace_back(p); + // Claims join the runtime declarations the manifest could have carried // itself, so the version-floor check in prepare reads ONE list and never // learns which spelling a claim arrived in. diff --git a/modules/buildmcpp/src/program_protocol.cppm b/modules/buildmcpp/src/program_protocol.cppm index 8b1dbf82..2e476b25 100644 --- a/modules/buildmcpp/src/program_protocol.cppm +++ b/modules/buildmcpp/src/program_protocol.cppm @@ -81,7 +81,14 @@ export namespace mcpp::build::program_protocol { // at a path relative to the executable. Same cost as v5's: a package calling // `mcpp::deploy()` fails on an older engine at the build.mcpp COMPILE, because // that engine's bundled module has no such function. -inline constexpr int kProtocolVersion = 11; +// v12: adds `runtime-library-dir` -- the build-program form of `[runtime] +// library_dirs`: a launch-time search directory for a dependency (a vcpkg +// prefix's `bin/`, a Qt SDK's `bin/`) whose location a build.mcpp learns +// rather than one an author can write into TOML. Same cost as v5's: a +// package calling `mcpp::runtime_library_dir()` fails on an older engine at +// the build.mcpp COMPILE, because that engine's bundled module has no such +// function. +inline constexpr int kProtocolVersion = 12; // ── Cache-format epoch ───────────────────────────────────────────────────── // diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index 8723bc1f..c0700211 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.9.26.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.9.27.1"; } // namespace mcpp diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm index aecbf839..fc5696e7 100644 --- a/src/build/hostprogram.cppm +++ b/src/build/hostprogram.cppm @@ -183,6 +183,17 @@ inline void windows_entry(const char* target, const char* value) { inline void deploy(const char* from, const char* to) { std::printf("mcpp:deploy=%s\t%s\n", from, to); } +// The build-program form of `[runtime] library_dirs` (docs/04 §2.11): a +// directory to search at LAUNCH time, for a dependency (a vcpkg prefix's +// `bin/`, a Qt SDK's `bin/`) whose location this program learns rather than +// one an author can write into `mcpp.toml`. Reaches the consumer, joining the +// same `LinkIntent` `link_lib`/`link_search`/`deploy` feed, and gets the same +// treatment the manifest key does: RUNPATH/rpath on ELF and Mach-O, never +// `-L`, and `mcpp pack`'s closure search. Relative paths resolve against this +// package's root, like every other AbsPath directive. +inline void runtime_library_dir(const char* dir) { + std::printf("mcpp:runtime-library-dir=%s\n", dir); +} // ── Build-graph nodes (mcpp 2026.8.5.1+) ──────────────────────────────── // Declare WORK instead of doing it. A build program is a good place to decide // what the build looks like and a bad place to perform it: work done here is diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index e40927bf..c089c306 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -12389,6 +12389,10 @@ prepare_build(bool print_fingerprint, // snapshot. Anything past this index is a `mcpp::deploy()` residue // that needs the same mirror the flag/source tails get below. const auto rdeployN = m->runtimeConfig.linkIntent.deploy.size(); + // Same reason, one field wide: `mcpp::runtime_library_dir()` residue + // needs the same mirror `deploy` does, or `resolve_runtime_contract` + // (which reads `packages[0]`'s snapshot, not `*m`) never sees it. + const auto rlibDirN = m->runtimeConfig.libraryDirs.size(); // What the dependencies supplied as runners, before the root's program // speaks. The root's emissions are appended to the same slots, so a // name both supply becomes one argv joining the two (#634, §9 item 8, @@ -12503,6 +12507,14 @@ prepare_build(bool print_fingerprint, pkg0.manifest.runtimeConfig.linkIntent.deploy.end(), m->runtimeConfig.linkIntent.deploy.begin() + static_cast(rdeployN), m->runtimeConfig.linkIntent.deploy.end()); + // `mcpp::runtime_library_dir()` residue → `packages[0].manifest`, the + // same object and the same reason as the `deploy` mirror above: without + // it a directive-sourced entry lands in `*m` and `resolve_runtime_contract` + // never looks there. + pkg0.manifest.runtimeConfig.libraryDirs.insert( + pkg0.manifest.runtimeConfig.libraryDirs.end(), + m->runtimeConfig.libraryDirs.begin() + static_cast(rlibDirN), + m->runtimeConfig.libraryDirs.end()); } // ── Every device source must reach some action ───────────────────────── diff --git a/tests/e2e/779_a_build_program_declares_a_runtime_library_dir.sh b/tests/e2e/779_a_build_program_declares_a_runtime_library_dir.sh new file mode 100755 index 00000000..88907580 --- /dev/null +++ b/tests/e2e/779_a_build_program_declares_a_runtime_library_dir.sh @@ -0,0 +1,125 @@ +#!/usr/bin/env bash +# requires: pack gcc +# 779_a_build_program_declares_a_runtime_library_dir.sh -- +# `mcpp::runtime_library_dir(dir)`: the build-program form of `[runtime] +# library_dirs` (docs/04 §2.11). A dependency that brings a prebuilt shared +# library (a vcpkg prefix's `bin/`, a Qt SDK's `bin/`) knows where it lives +# only at build-program time, and the manifest key cannot be computed -- it is +# a fixed TOML array. This is the directive that closes that gap. +# +# What this holds, each with the wrong answer it excludes: +# +# 1. THE DIRECTIVE GETS THE SAME RUNPATH TREATMENT THE MANIFEST KEY DOES: +# `-Wl,-rpath,` in build.ninja, never `-L` -- a launch-time search +# directory is not a link-library search path (docs/04 §2.11's own +# table), read from the emitted graph rather than inferred from a green +# build, the same way 62_runtime_library_dirs.sh checks the manifest key. +# 2. `mcpp run` FINDS THE LIBRARY ONLY THROUGH THAT RUNPATH: the executable +# is linked against it (DT_NEEDED), and the directory is nowhere else on +# any search path this build would otherwise consult. +# 3. THE CACHE TAG IS NON-EMPTY, SO A REPLAY CARRIES THE ENTRY: rebuilding +# on a build.mcpp CACHE HIT (not a re-run) must keep the RUNPATH and keep +# `mcpp run` working -- the same replay criterion `mcpp:deploy=` is held +# to in 651_a_build_program_deploys_what_it_generated.sh. +# 4. `mcpp pack --format dir` STAGES IT into the bundle's `lib/` -- the +# same closure search `[runtime] library_dirs` already feeds. Checked +# LAST: `mcpp pack` builds under a different profile and would otherwise +# invalidate the `dev`-profile build.mcpp cache the replay check (3) +# depends on. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +export MCPP_HOME=${MCPP_HOME:-$HOME/.mcpp} + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +cd "$TMP" +mkdir -p app/src app/rtlib + +# The prebuilt-plugin shape a real build.mcpp would locate (a vcpkg/Qt-style +# `bin/`): compiled directly with the toolchain here, standing in for that +# discovery the same way 62_runtime_library_dirs.sh's precompiled .so stands +# in for the manifest key's case. +cat > app/rtlib/plugin.c <<'EOF' +int runtime_plugin_answer(void) { return 42; } +EOF +gcc -shared -fPIC app/rtlib/plugin.c -o app/rtlib/libruntime_plugin.so + +cat > app/src/main.cpp <<'EOF' +extern "C" int runtime_plugin_answer(); +int main() { return runtime_plugin_answer() == 42 ? 0 : 1; } +EOF + +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[toolchain] +linux = "gcc@16.1.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF + +# The directive under test. `link_lib`/`link_search` also come from the +# program, on purpose: the point is a dependency the build.mcpp DISCOVERS, +# not one written into mcpp.toml by hand -- the case `[runtime] library_dirs` +# cannot cover. +cat > app/build.mcpp <<'EOF' +import mcpp; +#include +int main() { + const std::string root = mcpp::manifest_dir(); + mcpp::link_lib("runtime_plugin"); + mcpp::link_search((root + "/rtlib").c_str()); + mcpp::runtime_library_dir((root + "/rtlib").c_str()); + return 0; +} +EOF + +MCPP="${MCPP:-mcpp}" +cd app +find_graph() { find target -name build.ninja | head -1; } + +# ── 1. a plain build gets the RUNPATH treatment, never -L ────────────────── +"$MCPP" build > b1.log 2>&1 || fail "build failed" b1.log +G=$(find_graph) +[ -n "$G" ] || fail "no build.ninja" b1.log +RTDIR=$(realpath rtlib) +grep -F -- "-Wl,-rpath,$RTDIR" "$G" >/dev/null \ + || fail "runtime-library-dir directive missing from RUNPATH intent" "$G" + +# ── 2. mcpp run finds the library only through that RUNPATH ──────────────── +"$MCPP" run > run1.log 2>&1 \ + || fail "run failed to find the plugin through the directive's RUNPATH" run1.log b1.log + +# ── 3. the replay criterion: a build.mcpp cache HIT keeps the directive ──── +# `mcpp pack` builds under a different profile and would invalidate this +# package's build.mcpp cache (a different ctx hash) before the replay is +# exercised, so the cache-hit rebuild happens BEFORE pack, on the same `dev` +# profile `mcpp build` and `mcpp run` already used. +touch src/main.cpp # past the whole-project no-op fast path, without + # touching build.mcpp itself +"$MCPP" build > b2.log 2>&1 || fail "second build failed" b2.log +grep -q "up to date (cached)" b2.log \ + || fail "the second build re-ran build.mcpp; the replay path was not exercised" b2.log +G2=$(find_graph) +[ -n "$G2" ] || fail "no build.ninja after the second build" b2.log +grep -F -- "-Wl,-rpath,$RTDIR" "$G2" >/dev/null \ + || fail "the RUNPATH directive did not survive a build.mcpp cache hit" "$G2" +"$MCPP" run > run2.log 2>&1 \ + || fail "run failed after the directive was replayed from a build.mcpp cache hit" run2.log b2.log + +# ── 4. mcpp pack --format dir stages it into the bundle's lib/ ───────────── +"$MCPP" pack --format dir > pack1.log 2>&1 || fail "pack --format dir failed" pack1.log +STAGED=$(find target/dist -name 'libruntime_plugin.so' | head -1) +[ -n "$STAGED" ] || fail "the runtime-library-dir closure was not staged by mcpp pack" pack1.log +case "$STAGED" in + */lib/libruntime_plugin.so) : ;; + *) fail "the staged library is not under the packed bundle's lib/" pack1.log ;; +esac + +echo "PASS: 779_a_build_program_declares_a_runtime_library_dir" diff --git a/tests/unit/test_build_directives.cpp b/tests/unit/test_build_directives.cpp index f1c2a04e..0fbdc709 100644 --- a/tests/unit/test_build_directives.cpp +++ b/tests/unit/test_build_directives.cpp @@ -895,13 +895,13 @@ TEST(BuildDirectives, DeployRowIsProtocolElevenWithLinkGlobalScopeAndATag) { EXPECT_EQ(def->scope, dirs::Scope::LinkGlobal); EXPECT_EQ(def->sinceProtocol, 11); EXPECT_FALSE(def->tag.empty()); - EXPECT_EQ(dirs::kProtocolVersion, 11); + EXPECT_EQ(dirs::kProtocolVersion, 12); } -TEST(BuildDirectives, ProtocolElevenIsAcceptedAndTwelveIsNot) { - auto ok = parse("mcpp:protocol=11\n"); +TEST(BuildDirectives, ProtocolTwelveIsAcceptedAndThirteenIsNot) { + auto ok = parse("mcpp:protocol=12\n"); EXPECT_FALSE(dirs::protocol_error(ok).has_value()); - auto no = parse("mcpp:protocol=12\n"); + auto no = parse("mcpp:protocol=13\n"); EXPECT_TRUE(dirs::protocol_error(no).has_value()); } @@ -977,3 +977,96 @@ TEST(BuildDirectives, DeployReachesOnlyTheRuntimeDeployListNotAnyFlagChannel) { EXPECT_TRUE(m.buildConfig.ldflags.empty()); EXPECT_TRUE(m.buildConfig.cxxflags.empty()); } + +// ── v12: `mcpp::runtime_library_dir(dir)` ─────────────────────────────────── +// +// The build-program form of `[runtime] library_dirs` (docs/04 §2.11): a +// launch-time search directory reached from a build.mcpp instead of TOML. +// What is asserted: it lands on the SAME manifest field the manifest key +// populates (`RuntimeConfig::libraryDirs`), which is what makes every existing +// consumer of that field (the plan merge, the RUNPATH rendering, `mcpp pack`'s +// closure search) see a directive-declared entry without any of them changing. + +TEST(BuildDirectives, RuntimeLibraryDirRowIsProtocolTwelveWithLinkGlobalScopeAndATag) { + auto def = dirs::find_by_wire("runtime-library-dir"); + ASSERT_NE(def, nullptr); + EXPECT_EQ(def->scope, dirs::Scope::LinkGlobal); + EXPECT_EQ(def->sinceProtocol, 12); + EXPECT_FALSE(def->tag.empty()); + EXPECT_EQ(def->transform, dirs::Transform::AbsPath); +} + +// The manifest key's own field, not a new one -- this is the whole point of +// the design: no downstream consumer has to learn a second field exists. +TEST(BuildDirectives, RuntimeLibraryDirJoinsTheSameFieldTheManifestKeyPopulates) { + auto d = parse("mcpp:runtime-library-dir=rtlib\n"); + mcpp::manifest::Manifest m; + dirs::apply(m, d); + ASSERT_EQ(m.runtimeConfig.libraryDirs.size(), 1u); + EXPECT_EQ(m.runtimeConfig.libraryDirs[0].string(), under_root("rtlib")); +} + +// Relative resolves against the package root, exactly as `include-dir`'s +// AbsPath case and `deploy`'s `from` do -- the directive never leaves a +// relative path for a later stage to guess the base of. +TEST(BuildDirectives, RuntimeLibraryDirRelativeToTheRootIsMadeAbsolute) { + auto d = parse("mcpp:runtime-library-dir=vendor/qt/bin\n"); + mcpp::manifest::Manifest m; + dirs::apply(m, d); + ASSERT_EQ(m.runtimeConfig.libraryDirs.size(), 1u); + EXPECT_EQ(m.runtimeConfig.libraryDirs[0].string(), under_root("vendor/qt/bin")); +} + +TEST(BuildDirectives, RuntimeLibraryDirAcceptsAnAlreadyAbsoluteValue) { + const std::string abs = under_root("prefix/lib"); + auto d = parse(std::format("mcpp:runtime-library-dir={}\n", abs)); + mcpp::manifest::Manifest m; + dirs::apply(m, d); + ASSERT_EQ(m.runtimeConfig.libraryDirs.size(), 1u); + EXPECT_EQ(m.runtimeConfig.libraryDirs[0].string(), abs); +} + +// Reaches only the run-time search field -- no compile or link flag channel +// gains anything, unlike `link-search`, which shares the manifest's -L/-L +// distinction (docs/04 §2.11's own table: `library_dirs` maps ONLY to +// runtime search). +TEST(BuildDirectives, RuntimeLibraryDirReachesNoFlagChannel) { + auto d = parse("mcpp:runtime-library-dir=rtlib\n"); + mcpp::manifest::Manifest m; + dirs::apply(m, d); + EXPECT_TRUE(m.buildConfig.ldflags.empty()); + EXPECT_TRUE(m.buildConfig.cxxflags.empty()); + EXPECT_TRUE(m.runtimeConfig.linkIntent.runtimeSearchDirs.empty()); + EXPECT_TRUE(m.runtimeConfig.linkIntent.linkLibraryDirs.empty()); +} + +// Multiple directives accumulate, in emission order, the same as every other +// repeated directive in this table. +TEST(BuildDirectives, RuntimeLibraryDirDirectivesAccumulate) { + auto d = parse("mcpp:runtime-library-dir=a\n" + "mcpp:runtime-library-dir=b\n"); + mcpp::manifest::Manifest m; + dirs::apply(m, d); + ASSERT_EQ(m.runtimeConfig.libraryDirs.size(), 2u); + EXPECT_EQ(m.runtimeConfig.libraryDirs[0].string(), under_root("a")); + EXPECT_EQ(m.runtimeConfig.libraryDirs[1].string(), under_root("b")); +} + +// Persisted like `deploy`, `warning` and `pack-format`: the cache tag is +// non-empty and round-trips through the same table-driven serialize/ +// accept_cache_record pair, so a build.mcpp CACHE HIT replays the directive +// rather than losing it on every build after the first. +TEST(BuildDirectives, RuntimeLibraryDirIsPersistedAndRoundTrips) { + auto d = parse("mcpp:runtime-library-dir=rtlib\n"); + std::ostringstream os; + dirs::serialize(os, d); + EXPECT_NE(os.str().find("d runtime-library-dir "), std::string::npos) << os.str(); + + dirs::Directives replayed; + ASSERT_TRUE(dirs::accept_cache_record(replayed, "runtime-library-dir", + under_root("rtlib"))); + mcpp::manifest::Manifest m; + dirs::apply(m, replayed); + ASSERT_EQ(m.runtimeConfig.libraryDirs.size(), 1u); + EXPECT_EQ(m.runtimeConfig.libraryDirs[0].string(), under_root("rtlib")); +} From c57c12f2b390aaf4be61abde80b24a2f274c6524 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Sat, 26 Sep 2026 06:08:01 +0800 Subject: [PATCH 02/26] A passing check moves a stamp it did not write past the inputs that changed --- docs/30-build-mcpp.md | 6 +- docs/zh/30-build-mcpp.md | 4 +- src/cli.cppm | 40 ++++++++-- ...ck_moves_its_stamp_past_a_changed_input.sh | 77 +++++++++++++++++++ 4 files changed, 120 insertions(+), 7 deletions(-) create mode 100755 tests/e2e/780_a_passing_check_moves_its_stamp_past_a_changed_input.sh diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index 1c42f724..bbe0de35 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -549,7 +549,11 @@ The verdict is the exit code; the stamp is bookkeeping the graph needs, and mcpp creates it when the command succeeds. Before this, every check needed a wrapper script to touch the file — and a command is an argv with no shell assumed, so that wrapper could not be written portably at all. A command that -already writes its own stamp is unaffected: an existing file is left alone. +writes its own stamp is unaffected: a stamp the command created or rewrote is +left as it is. A stamp the command did not write is created on the first pass +and has its modification time moved to the present on every later pass +(2026.9.27.1+), so after an input changes and the check passes again the stamp +is newer than that input and the check does not run on the next build. > A missing stamp does **not** fail the build. ninja leaves the declared output > absent and re-runs that edge on every build afterwards, which looks like a diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index 7e55266f..acdadd7f 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -468,7 +468,9 @@ mcpp 为那条边写出 `depfile =` 与 `deps = gcc`,ninja 读取该文件并 **check 的命令不必自己写 stamp**(mcpp 2026.8.29.1+)。判定是退出码,stamp 是**构建图** 需要的记账;命令成功时由 mcpp 创建它。在此之前每个 check 都需要一个包装脚本去 touch 那个文件 —— 而 command 是 argv、不假设有 shell,所以那个包装器**根本没法可移植地写出来**。 -已经自己写 stamp 的命令不受影响:已存在的文件不会被动。 +自己写 stamp 的命令不受影响:命令创建或改写过的 stamp 保持原样。命令没有写的 stamp, +第一次通过时创建,之后每次通过时把修改时间更新为当前时间(2026.9.27.1+);因此某个输入 +变化、check 再次通过之后,stamp 比该输入新,下一次构建不再运行这个 check。 > stamp 缺失**不会**让构建失败。ninja 只是留着那个声明的输出不存在,并在之后**每次构建 > 都重跑**那条边 —— 看起来像一个通过了的检查,实际上它从未被满足。 diff --git a/src/cli.cppm b/src/cli.cppm index 16d0ed29..e6449d60 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -989,9 +989,19 @@ int run(int argc, char** argv) { // platform mcpp runs on, so this needs no shell, no `touch`, and no // per-platform spelling. // - // A command that already creates its stamp is unaffected: existing files - // are left alone, so the pre-2026.8.29.1 wrapper scripts keep working - // byte-for-byte. + // A command that writes its own stamp is unaffected: a stamp the command + // created or rewrote is left as the command left it, so the + // pre-2026.8.29.1 wrapper scripts keep working byte-for-byte. + // + // A STAMP THE COMMAND DID NOT WRITE IS TOUCHED, NOT ONLY CREATED. The + // stamp is what ninja compares with the inputs, so after an input changes + // and the command passes again, its time has to move past that input. + // Until 2026.9.27.1 an existing stamp was left alone whatever the command + // did, and every check whose command writes nothing -- clang-tidy, an + // installer run through `mcpp-deps` -- re-ran on every build after its + // first input change, because its output stayed older than the input + // forever. The comparison is by the stamp's own time before and after the + // command, so a command that did write it is still left alone. if (std::string_view(argv[1]) == "__action-stamp") { std::vector stamps; int i = 2; @@ -1011,19 +1021,39 @@ int run(int argc, char** argv) { // `run_exec`: no shell, stdio inherited. The analyser's own output has // to reach the terminal unchanged — a check that fails is read by a // human, and capturing would either swallow it or reprint it wrapped. + // Each stamp's time before the command runs; empty when it is absent. + std::vector> before; + for (auto const& s : stamps) { + std::error_code ec; + const auto p = mcpp::platform::fs::extended_length(std::filesystem::path{s}); + const auto t = std::filesystem::last_write_time(p, ec); + before.push_back(ec ? std::nullopt : std::optional{t}); + } const int r = mcpp::platform::process::run_exec(cmd); // The stamps are written ONLY on success. Writing them anyway would // make ninja consider the edge satisfied, so the next build would skip // a check that had never passed. if (r != 0) return r; - for (auto const& s : stamps) { + for (std::size_t k = 0; k < stamps.size(); ++k) { + auto const& s = stamps[k]; std::error_code ec; // Relative to the build directory, which can be deep enough to // take the stamp past the Windows path limit (mcpp#641, item 3). const auto p = mcpp::platform::fs::extended_length(std::filesystem::path{s}); if (!p.parent_path().empty()) std::filesystem::create_directories(p.parent_path(), ec); - if (std::filesystem::exists(p, ec)) continue; + if (std::filesystem::exists(p, ec)) { + const auto now = std::filesystem::last_write_time(p, ec); + // Written by the command during this run: left as it is. + if (!ec && (!before[k] || now != *before[k])) continue; + std::filesystem::last_write_time( + p, std::filesystem::file_time_type::clock::now(), ec); + if (ec) { + std::println(stderr, "error: cannot update check stamp '{}': {}", s, ec.message()); + return 1; + } + continue; + } std::ofstream out(p, std::ios::trunc); if (!out) { std::println(stderr, "error: cannot write check stamp '{}'", s); diff --git a/tests/e2e/780_a_passing_check_moves_its_stamp_past_a_changed_input.sh b/tests/e2e/780_a_passing_check_moves_its_stamp_past_a_changed_input.sh new file mode 100755 index 00000000..c6abfc54 --- /dev/null +++ b/tests/e2e/780_a_passing_check_moves_its_stamp_past_a_changed_input.sh @@ -0,0 +1,77 @@ +#!/usr/bin/env bash +# 780_a_passing_check_moves_its_stamp_past_a_changed_input.sh — a `role = +# "check"` whose command writes no stamp, after one of its inputs changed. +# +# The engine writes such a check's stamp when the command succeeds (313). Until +# 2026.9.27.1 it only CREATED the stamp: an existing one was left alone. So once +# an input changed, the check ran and passed, and its stamp stayed older than +# that input -- and ninja ran it again on every build after, forever. Measured +# with mcpp-plugins' `deps-cmake`, whose check is a CMake build: every `mcpp +# build` after one edit re-ran configure, build and install. +# +# The criterion is the build AFTER the one that re-ran the check: it must not +# run the check again. Portable like 313: the command is the engine itself. +set -e + +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p src +cat > mcpp.toml <<'EOF' +[package] +name = "stampmove" +version = "0.1.0" +EOF +cat > src/main.cpp <<'EOF' +#include +int main() { std::printf("STAMP_MOVE_OK\n"); } +EOF +echo one > input.txt + +MCPP_HOST="$(host_path "$MCPP")" +INPUT_HOST="$(host_path "$TMP/input.txt")" +cat > build.mcpp < b1.log 2>&1 || { cat b1.log; echo "FAIL: build failed"; exit 1; } +ran b1.log || { cat b1.log; echo "FAIL: the first build did not run the check"; exit 1; } + +"$MCPP" build -v > b2.log 2>&1 || { cat b2.log; echo "FAIL: no-op rebuild failed"; exit 1; } +if ran b2.log; then cat b2.log; echo "FAIL: the check re-ran with nothing changed"; exit 1; fi + +# A changed input: the check runs once... +sleep 1 +echo two > input.txt +"$MCPP" build -v > b3.log 2>&1 || { cat b3.log; echo "FAIL: rebuild after the edit failed"; exit 1; } +ran b3.log || { cat b3.log; echo "FAIL: a changed input did not re-run the check"; exit 1; } + +# ...and not again: its stamp is now newer than the input. +"$MCPP" build -v > b4.log 2>&1 || { cat b4.log; echo "FAIL: rebuild after the check failed"; exit 1; } +if ran b4.log; then + cat b4.log + echo "FAIL: the check re-ran on the build after it passed -- its stamp stayed older than the input" + exit 1 +fi + +out="$("$MCPP" run 2>&1 | tail -1)" +[[ "$out" == *STAMP_MOVE_OK* ]] || { echo "FAIL: the program did not run: '$out'"; exit 1; } +echo "OK" From e2d0981b126621e138bdcd57c0261f3fe5a0b370 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Sat, 26 Sep 2026 07:52:50 +0800 Subject: [PATCH 03/26] docs: the compile database and #699/#701/#702 design record, and SPEC-007 draft 0.1 --- ...6-compile-database-and-issue-699-design.md | 904 ++++++++++++++++++ docs/specs/build-plugins.md | 181 ++++ 2 files changed, 1085 insertions(+) create mode 100644 .agents/docs/2026-09-26-compile-database-and-issue-699-design.md create mode 100644 docs/specs/build-plugins.md diff --git a/.agents/docs/2026-09-26-compile-database-and-issue-699-design.md b/.agents/docs/2026-09-26-compile-database-and-issue-699-design.md new file mode 100644 index 00000000..a65ccd22 --- /dev/null +++ b/.agents/docs/2026-09-26-compile-database-and-issue-699-design.md @@ -0,0 +1,904 @@ +--- +subject: design +status: active +--- + +# The compile database, `emit build-database`, and #701/#702: triage against the specifications, and one design + +- Sources: + - The report *compile_commands.json 对比分析:xmake vs mcpp* + (`/home/speak/test/mcpp/hello/cdb-report/README.md`, 2026-09-26), measured with mcpp + 2026.9.21.3 and clangd 22.1.8. Cited as "the report". Its xmake comparison is a reference + only; every verdict below is taken against a specification. + - mcpp-community/mcpp#699 (2026-09-25), *emit build-database for IDEs: one failing member + loses the whole database, and build programs cannot tell a plan pass*, from + Sunrisepeak/mcpp-language-server#23 and #24 on the GalTranslPP workspace. + - mcpp-community/mcpp#701 and pull request #702 (2026-09-25, head `c57c12f2`): a build + program declares a runtime library directory, and a passing check moves its stamp. Their + consumer is mcpp-plugins 0.13.0 (`deps-vcpkg`, `deps-cmake`, `rules-qt`), whose design + record (branch `feat/deps-vcpkg-rules-qt`, `.agents/docs/2026-09-26-deps-vcpkg-rules-qt-design.md`, + decision D8) defers #702 and builds on the released 2026.9.26.1 instead. + - Earlier records of C1: #397 item C-1 (2026-08-09, P0) and #677 item B1 (2026-09-20). +- Specifications: the JSON Compilation Database + (); S1 *C++ Build Database: IDE + Profile* 0.2.0 and S2 *discovery*, both in `Sunrisepeak/mcpp-language-server/docs/specs/`, + which SPEC-005 implements; SPEC-005; docs/50; docs/04 §2.11; docs/30. +- Consumers: **mcpp-language-server (mcppls) is the primary one.** It reads the S1 document + that `emit build-database` prints and drives clangd through a database it writes itself + (its `docs/90-architecture.md`). `compile_commands.json` serves the tools that read the JSON + format directly: clangd without mcppls, clang-tidy, and others. +- Basis: `origin/main` 3a6ac7f4 (mcpp 2026.9.26.1 and one records-only commit); #702 at + `c57c12f2`. Code citations are to those commits. +- Measurements: Linux x86_64 (Ubuntu 24.04); the released mcpp 2026.9.26.1 invoked by its xlings + store path; toolchains llvm@22.1.8, llvm@20.1.7, gcc@16.1.0; clangd and clang-tidy 22.1.8; CMake + 4.4.2. Every project is a copy or a fixture in a scratch directory. Appendix A holds the + readings. +- Status: accepted; implemented in #702 (§11). + - Revision 1 (2026-09-26): the triage and the first design. + - Revision 2 (2026-09-26): D1 and D3 accepted; mcppls named as the primary consumer; C3 + measured against the specification and three build systems; C5 rewritten around its four + questions; C1 and C2 replaced by one database per configuration (D6); #701 and #702 + reviewed (§5), with the defect F1 they led to; an overall review (§10). + - Revision 3 (2026-09-26): D2, D4, D5, D6, D7 and D8 accepted; F1 filed as #703; the + conformant design of #701/#702 (§5.4: `runtime_search_dir`, the stamp rule, the `prepare` + role, and W, the placement of a Windows program's DLLs); SPEC-007, the build plugin specification (`docs/specs/build-plugins.md`, §5.6); #702 + reused as the single pull request (§7). + - Revision 4 (2026-09-26): D9 to D12 accepted; the implementation plan, its task + dependencies and the cross-repository sequence (§11). + +--- + +## 0. Summary + +| | Finding | Source | Verdict | Repair | +|---|---|---|---|---| +| **C1** | A deleted `compile_commands.json` is not written again by the next `mcpp build` | report §3.6; #397 C-1; #677 B1 | **mcpp defect** | §3.2: the fast path restores the root database from the configuration's database | +| **C2** | The merge resolves `file` against the process's working directory, keeps other tools' entries, and mixes two toolchains in one file | report §3.1; the mixing is new | **mcpp defect** | §3.2: one database per configuration; the root file is replaced, never merged (D6) | +| **C3** | `directory` (and S1 `work-directory`) name the project root; the compiler runs in the output directory | report §3.2 | **mcpp defect.** The JSON format and S1-8-2 define the field as the directory the compiler runs in; CMake, ninja and xmake write that directory | §3.3 (D2) | +| **C4** | A module interface's language flag is missing from both databases | new | **mcpp defect**: clangd cannot handle a `.ixx` entry | §3.4 | +| **C5** | `compile_commands.json` lists no standard-library unit | report §3.3 | **mcpp gap**, contrary to S1-12-1 | §3.5 (D5) | +| **C6** | The database is written at the project root | report §3.5 | by design; the symlink redirect is undocumented | documentation | +| **E1** | One member's planning failure removes every member's sets | #699 item 1 | **mcpp gap** | §4.4 (D1, accepted) | +| **E2** | Under `emit`, a host tool whose build fails ends the requesting member's plan | #699 item 2 | **mcpp gap** | §4.5 | +| **E3** | A failing build program ends its member's plan | #699 item 2 | **mcpp gap** | §4.6 (D3, accepted) | +| **E4** | A signal that tells a build program it runs for a plan | #699 item 2 | **not adopted** | §4.7 (D4) | +| **R1** | #701/#702: `mcpp::runtime_library_dir(dir)` | #701 | the gap is real; as written it extends a field docs/04 retires | §5.4 R1': `runtime_search_dir` on `LinkIntent` (D7, accepted) | +| **R2** | #701/#702: a passing check moves its stamp | #701 | **mcpp defect**; the fix is correct | §5.4 R2: every stamp newer than every input (D10) | +| **F1** | `$ORIGIN` in a user link flag reaches the program as `/../lib` | the plugin record; measured here | **mcpp defect**, filed as #703 | §5.5: SPEC-004 §8 extends to link flags (D8, accepted) | +| **O1** | deps-cmake and deps-vcpkg run installation as `check` actions | the plugin record | design gap: no action role describes construction whose file names are unknown | §5.4 P: the `prepare` role (D9) | +| **O2** | on Windows, a program started by hand does not find DLLs from a runtime search directory | this review | framework gap | §5.4 W: placement after the link (D11) | +| **S7** | a contract for build plugins | this review | new specification | SPEC-007, draft 0.1 (§5.6, D12) | +| | xmake's database; GalTranslPP's incomplete Qt and its environment checks inside build programs | report §3.4; #699 | not mcpp | §3.7, §4.8 | + +Three statements: + +1. **A database states the build's facts, one configuration at a time.** `directory` is where + the compiler runs (C3), `arguments` carry the flags that decide how it reads its input (C4), + and one file holds one configuration (C1, C2). Where mcpp departs from this, it departs from + the JSON format, from S1 and from every build system measured. +2. **mcpp compiles nothing twice.** A second compile of `std` is a reader's, and it happens + because a BMI is readable only by the compiler that wrote it. The database's task is to name + both the provider, so any reader can compile, and the build's BMI, so a matching reader can + reuse it (C5). +3. **#699, #701 and #702 are one family:** a build program configures, actions construct and + verify, and a new engine surface joins the current model rather than a retiring one. + +## 1. Method + +Each claim was reproduced on the current release in a copy or a fixture, located in the code, +and classified against the contract that governs it. A repair is placed where the rule is +stated, with a criterion that fails on 2026.9.26.1. + +| Verdict | Meaning | +|---|---| +| mcpp defect | mcpp violates a contract it states or a specification it claims to follow | +| mcpp gap | mcpp does what its specification says; the specification omits a case its consumer needs | +| by design | deliberate, and the reasons still hold | +| usage / not mcpp | the effect belongs to a project, an environment or another program | + +## 2. Principles + +- **P1. One record, two documents.** `compile_commands.json`, `emit --spec compile-commands` and + the S1 document render one `UnitInvocation` per unit (`src/build/compile_commands.cppm:49-64`; + SPEC-005 R3.7). C3 and C4 change that record; no output is repaired on its own. +- **P2. A field states what the build does.** The JSON format defines `directory` as "the + working directory of the compilation" and `arguments` as the argument vector that "should run + the compilation step"; S1-8-2 defines `work-directory` as the "absolute path of the directory + the compiler runs in". +- **P3. The fast path replays a build.** Every product of the full path is produced by the fast + path or verified by it. +- **P4. A description names what it could not describe, per unit**: a member for a planning + failure, a package's directives for a build-program failure, nothing for a construction + failure. +- **P5. Configuration is not verification.** A build program configures; a `check` action + verifies. `mcpp build` runs both; `emit` runs the first. +- **P6. One program behaviour.** The plan `emit` describes is the plan + `mcpp build --configure-only` computes (R1.2). +- **P7. One database, one configuration.** A configuration is what the fingerprint names: + toolchain, target, profile, features. A file that mixes two configurations describes no build. +- **P8. A new surface joins the current model.** docs/04 §2.11 keeps `[runtime] library_dirs` + "for one compatibility train" and names its successor, `runtime_search_dirs`. +- **P9. The engine provides general mechanisms; a tool's knowledge stays in its plugin.** A gap a + plugin meets is closed in the engine only when it is general, and then by a mechanism that names + no tool (R1', R2, P and W below); vcpkg, CMake and Qt are never named by the engine. + +## 3. The compile database + +### 3.1 Readings on 2026.9.26.1 + +| Claim | Reading on 2026.9.26.1 | Finding | +|---|---|---| +| deleted database not rewritten (report: "not reproduced") | `mcpp build; rm compile_commands.json; mcpp build` prints `Finished dev in 0.00s` and leaves no file, on every attempt; `--no-cache` writes it | C1 | +| relative entries of another writer | kept from the project root (6 entries: `src/main.cpp` beside `/…/src/main.cpp`), pruned from `src/` (4 entries) | C2 | +| (new) two toolchains | after `mcpp test` (llvm) and `mcpp build --toolchain gcc@16.1.0`: three `g++` entries and one `clang++` entry | C2 | +| GCC entries cannot be replayed from `directory` | from `directory`: `test.cppm` and `main.cpp` fail and `gcm.cache/` appears in the project root; from the output directory: 3 of 3 write their objects; clang: 3 of 3 from either | C3 | +| omitted `-x c++-module` is harmless | a `.ixx` interface declared in `module_extensions`: clangd `[fe_expected_compiler_job]`; with the flag, 0 errors; GCC 16 compiles `.ixx` without it | C4 | +| no std unit | §3.5 | C5 | + +### 3.2 One database per configuration (C1, C2; D6) + +**What is wrong today.** `publish_compile_commands` merges the fresh plan into whatever the root +file holds (`src/build/compile_commands.cppm:272-319`): a prior entry survives when its `file`, +compared as written and probed against the process's working directory (`:288-311`), is absent +from the fresh plan and exists. The rule exists to keep the units of an earlier `mcpp test` +(`:69-79`), and it keeps every other entry too: another tool's, and mcpp's own from a previous +toolchain. Separately, the fast path never reaches the writer (`src/build/execute.cppm:1408-1530` +against `src/build/ninja_backend.cppm:3474`), so a deleted file stays deleted (C1); `git clean -fd` +reaches the same state, because the `.gitignore` that `mcpp new` writes lists `target/` and +`.mcpp/` only (`src/scaffold/create.cppm:407`). + +**The identity of a configuration.** mcpp already names every configuration: the output +directory `target//`, derived from the toolchain, target, profile and +features. `mcpp build` and `mcpp test` in one configuration share it (measured: after both, the +build's entries and the test entry name one fingerprint directory; the fingerprint deliberately +covers neither tests nor dev-dependencies, #407). After C3, it is the `directory` of every project +entry. A field of mcpp's own inside an entry is not an option: one unknown key makes clangd +report `Failed to load compilation database` and clang-tidy refuse the file (measured). S1 +consumers ignore unknown fields (S1-11.2-1); JSON-format readers do not. + +**Design.** + +1. **The configuration's database** is `target///compile_commands.json`. + Every command that plans in that configuration (`build`, `test`, `run`, `--configure-only`) + writes it: the fresh plan's entries, plus the entries it already holds whose `file`, + resolved against their `directory`, the fresh plan lacks and which still exist. Every entry + in it was written by mcpp in this configuration, so no ownership test is needed and none is + made. +2. **The root file is a copy of the current configuration's database**, replaced whole and never + merged, and left untouched when identical, so that clangd is not triggered for nothing. + Switching toolchain or profile therefore switches the whole file, and switching back restores + that configuration's entries, its test units included. A symlink at the root is written + through, as today. +3. **Another writer's entries are replaced, and said so.** When the replaced root file held + entries mcpp did not write, one warning states their number and that the file holds mcpp's + configuration. An entry is mcpp's when its `output` lies under this project's `target/` or + under the mcpp home, where the standard-library units of C5 write their objects; `directory` + cannot decide it, because those units run in the std cache. +4. **The fast path restores the root file (C1).** It knows the configuration's directory + (`match->outputDir`) and publishes the root file through the same function as the full path: + when the root file is missing or differs from the configuration's database, it is replaced, + with the same warning. No plan is needed, and P3 holds. +5. `emit build-database` writes neither file (R2.1), and prints its plan as today. +6. The scaffold's `.gitignore` also lists `compile_commands.json`: it names absolute paths of one + machine. + +This replaces the four-condition merge of revision 1. The answer to D6 is **replace across +configurations, merge only within one**, with the output directory as the identifier: a value +mcpp already computes, visible in a standard field, and stable across the commands that should +share a database. + +**Criteria.** (1) e2e: plain build; delete the root file; `mcpp build` restores it without a +plan (fails on 2026.9.26.1). (2) e2e: the root file replaced by another writer's entries; +`mcpp build` leaves only the configuration's entries and warns once. (3) e2e: `mcpp test`, then +`mcpp build`, same configuration: the test entries remain. (4) e2e: llvm, then gcc, then llvm: +the root holds one toolchain's entries each time, and the llvm test entries return with llvm. +(5) unit: the within-configuration merge resolves `file` against `directory`. + +### 3.3 C3: `directory` (D2) + +**The specifications.** The JSON format: "directory: The working directory of the compilation. +All paths specified in the command or file fields must be either absolute or relative to this +directory." S1-8-2: `work-directory` is the "absolute path of the directory the compiler runs +in", and S1-12-1 exports it as `directory`. + +**What mcpp does.** `unit_invocations` writes `plan.projectRoot` (`compile_commands.cppm:231`); +ninja runs every compile in the output directory (`ninja -C `: `execute.cppm:1193`, +`ninja_backend.cppm:3653`). The standard-library units already state their real directory, +recovered from the command mcpp runs (`src/build/build_database.cppm:315-372`). + +**What other producers do** (measured; the same two-directory project for CMake): + +| Producer | `directory` | Where the compiler runs | Replay from `directory` | +|---|---|---|---| +| CMake 4.4.2, Ninja generator | the build directory | the build directory | object written | +| CMake 4.4.2, Unix Makefiles | the target's build subdirectory (`/lib`) | that subdirectory | object written | +| `ninja -t compdb` over mcpp's own `build.ninja` | `target//` | the same | | +| xmake (the report's sample) | the project root | the project root: its `file` and `-o` are relative to it, and the report's replay passed 3 of 3 | objects written | +| mcpp 2026.9.26.1 | the project root | `target//` | GCC importers fail; `gcm.cache/` written into the source tree | + +Every producer names the directory its compiler runs in, whatever that directory is; mcpp is the +one that does not, and ninja reading mcpp's own graph names the directory mcpp should. + +**Repair.** `directory` and `work-directory` are the output directory, for every unit and every +toolchain. The directory exists whenever the database does: the full path writes `build.ninja` +into it, and `emit` plans into a work directory under the mcpp home (measured: it exists after +`emit`). + +**Consequence.** A tool that changes into `directory` needs it to exist: clang-tidy 22.1.8 aborts +with `LLVM ERROR: Cannot chdir into "…"!` when it does not, while clangd logs +`VFS: failed to set CWD` and continues (both measured). After `mcpp clean` the root file names a +directory that is gone until the next build; the module arguments of the same entries +(`-fmodule-file=`, `-fprebuilt-module-path=`) already point into it, so a cleaned tree already +fails for every module unit. With §3.2, the next build replaces the root file anyway. + +**Criteria.** (1) e2e, gcc and llvm rows: every entry replayed from its `directory` writes its +object, and no `gcm.cache/` appears under the project root (fails for gcc on 2026.9.26.1). +(2) unit: a project unit's `directory` is `plan.outputDir`. e2e 211 derives a sibling fixture +from `directory` (`tests/e2e/211_configure_only_cdb.sh:77-78`) and changes with it. + +### 3.4 C4: the interface language flag + +The build states a module interface's language explicitly (`BmiTraits::moduleInterfaceLangFlag`: +`-x c++-module` for clang, `-x c++` for GCC, `/interface /TP` for MSVC; +`modules/toolchain-model/src/model.cppm:456-683`), so that no driver infers it from an extension +(`modules/source-kind/src/source_kind.cppm:360-371`: "mcpp never lets it guess"). +`unit_invocations` omits it (`compile_commands.cppm:234-246`), and a reader infers after all: +clang does not know `.ixx` and hands the file to the linker, so clangd has no compiler job for it +(measured); GCC 16 compiles `.ixx` and `.cppm` without the flag. + +**Repair.** The record carries the flag at the position the build uses, before `-c `, +for every unit whose kind produces a BMI; in S1 it lands in the interface units' +`local-arguments`. `-MMD -MF` and `-fmodule-output=` stay omitted: they name side outputs and do +not change how the input is read. + +**Criteria.** (1) unit: an interface unit's arguments carry the dialect's flag before `-c`; an +implementation unit's do not. (2) e2e, llvm row, `module_extensions = [".ixx"]`: the interface +entry, replayed from its `directory`, writes an object. **Open:** whether clangd in clang-cl +mode accepts `/interface` is measured on Windows CI before the MSVC form is emitted. + +### 3.5 C5: the standard-library units (D5) + +**Does mcpp compile `std` twice?** No. mcpp builds the standard-library modules once per +toolchain, standard and flags into the shared std cache under the mcpp home, which every project +reuses; `emit` describes them and compiles nothing (R2.2). Listing a unit in a database compiles +nothing on mcpp's side. + +**Was leaving them out of `compile_commands.json` correct?** It was a choice (SPEC-005 R4.1), +and the wrong one. + +- The JSON format describes "one way a translation unit is compiled in the project". The build + compiles `std.cppm` and links its object into the program, so an entry for it states a fact of + the build. The format has no completeness rule, so the omission breaks no sentence of it. +- S1-12-1, the export rule of the profile mcpp implements, makes **every** translation unit of + the S1 document an entry, and mcpp's S1 document contains the std units (R3.10). R4.1 + contradicts it. +- The omission is what binds a reader to the toolchain's exact version: with no provider in the + database, clangd falls back to the prebuilt BMI the importers name, and fails on another + version (measured below). + +**Why does a reader compile `std` again, then?** Because a BMI is readable only by the compiler +that wrote it: clangd 22.1.8 on a BMI written by clang 20.1.7 reports `ast_file_version_too_old`, +and S1 §6 adds that "equal versions do not imply compatible BMIs" (`build-id` decides). S1-11.2-3 +therefore forbids a consumer to use build BMIs unless its engine matches the toolchain's +`family`, `version` and `build-id` exactly, the user has enabled an authoritative mode, and the +consumer can detect staleness. clangd's own policy (measured) is to build a module whose provider +the database lists, and to load a prebuilt file only when none is listed. The second compile is +the reader's own BMI, made for the reader's own compiler. + +**Measurement** (an interface importing `std`, an importer; clangd 22.1.8 `--check +--experimental-modules-support`): + +| Database | clangd errors | +|---|---| +| llvm@20.1.7, as written | 1: `Failed to build module std; due to Don't get the module unit for module std`, then `[ast_file_version_too_old]` | +| the same, with the two `mcpp:std` units appended | 0: `Built module std`, `Built module hello.greet` | +| gcc@16.1.0, as written | 1: `module 'std' not found` | +| the same, with GCC's `bits/std.cc` unit appended | 1: clang's scan of `bits/std.cc` under GCC's command fails | + +**The better answer: name both, and let the reader choose.** + +- **D5a.** `compile_commands.json` lists the standard-library units whenever the build imports + `std`, with the fields S1 gives them (R3.10, R3.11), and R4.1 follows S1-12-1. A reader of another + clang version can then build `std` (measured: clangd 22.1.8 on an llvm@20.1.7 database); the importers keep naming the build's BMI (P2), which a + reader of the same version may load. The cost falls on a matching plain clangd, which builds + `std` once per session instead of loading it (about 1.4 s in the measurement). +- **D5b.** The S1 document names the build's BMI for the standard-library units: `provides` + maps `std` and `std.compat` to the BMI paths in the shared std cache (S1-8-6: "the path of the + BMI the build writes"), which `emit` and the build share, and `ide.toolchains..build-id` + carries the compiler's build identity (S1 §6, MAY). An mcppls whose engine matches exactly, + in the authoritative mode S1-11.2-3 describes, then reuses mcpp's BMI and compiles nothing; + any other reader compiles, as S1 requires. Project modules keep `""` under `emit` (S1-8-6 + allows it), because the planning pass writes into its own work directory. +- For mcppls, D5a changes nothing: it exports S1 itself and receives the units already. D5b is + what removes its second compile. + +**Criteria.** (1) e2e, llvm row: the database holds an entry for the toolchain's `std.cppm` +whose `directory` is the std cache directory. (2) e2e: `emit --spec compile-commands` equals the +build's database apart from the work directory. (3) For D5b: `provides` of the `mcpp:std` units +names the std cache BMI, and `build-id` is present and stable across two runs. + +### 3.6 C6: the location + +By design: clangd finds `/compile_commands.json` by its upward search, without +configuration. A symlink at that path redirects the write (`compile_commands.cppm:333-354`); +docs/01 states neither. **Repair**: docs/01 and its zh counterpart state the location, the +redirect, and §3.2's rule. + +### 3.7 Not mcpp + +- **xmake** is a reference in the report, not a subject: its temporary mapper files and GCC-only + flags are its own. After §3.2 a shared root file holds mcpp's configuration alone, with a + warning. +- **clangd with a GCC database** cannot resolve `import std`: clang reads no GCC BMI and cannot + scan `bits/std.cc` under GCC's command (measured). An editor that wants clangd on a GCC-built + project asks for the clang view of the same plan (`emit build-database --toolchain llvm@…`). + +## 4. Issue #699 + +### 4.1 The report + +mcppls builds its model from `mcpp emit build-database --format json`. On the five-member +GalTranslPP workspace, the member GPPGUI requests the host tool `Updater` of `gpp.updater`, whose +build runs a `lupdate` check that failed because the machine's Qt lacked qtdeclarative. `emit` +answered `MCPP_BUILD_DATABASE_PLAN_FAILED` without `data`, the four members that planned lost +their sets, and mcppls fell back to a guessed model on which clangd crashed. The issue asks for +the planned members' sets with an error per failed member, and for one of: a plan-only signal, a +build-program failure reported per member, or no `check` actions in host-tool builds under +`emit`. It calls both behaviours specified (R5.2, R2.5) and files a request. + +### 4.2 Measurement + +| Invocation (fixture: `good` plans; `bad`'s build program exits 1; `user` requests host tool `t` of `tool`, whose blocking check fails) | Exit | `data` | Diagnostics | +|---|---|---|---| +| `emit --workspace` over `good`, `bad`, `user` | 1 | absent | `PLAN_FAILED`: `bad: build.mcpp exited with 1 (build aborted)`; `user` never planned | +| `emit` in `user` | 1 | absent | `PLAN_FAILED`: `building host tool 'tool:t' failed: build failed` | +| `mcpp build` in `user` | 2 | | the same failure, correct for a build; verbose, the inner build shows the check `… -- /bin/false` failing | +| `emit --workspace` over `good` | 0 | `good/good` | none | + +The member loop of `emit` stops at the first failure (`src/cli/cmd_build.cppm:352-380`), while +`mcpp build --workspace` continues past one ("continue-on-failure; first non-zero exit wins", +`:174-194`). + +### 4.3 Classification + +- **Not mcpp:** the environment (a Qt without qtdeclarative) and two project choices, a build + program that verifies the environment and a blocking check inside a host tool's build. With a + complete Qt the workspace plans all 227 units. +- **mcpp gap:** the unit of failure in `emit` is the whole command (R5.2), and a host tool's build + is part of planning (R2.5). + +### 4.4 E1: a member's failure costs that member (D1, accepted) + +`emit` plans every selected member. A member whose planning fails contributes no sets and one +`error` diagnostic with the code R5.2 assigns today (`MCPP_BUILD_DATABASE_PLAN_FAILED` or +`MCPP_OFFLINE_DOWNLOAD_REQUIRED`) and `path` naming its `mcpp.toml`, relative to the workspace +root. `data` is present when at least one member was planned, and its `watch` also lists each +failed member's `mcpp.toml` and `build.mcpp`. The exit status is 1 whenever an error is present. +A consumer reads three outcomes structurally: no `data`; `data` with errors (described, except +what each error names); `data` without errors. + +**S2 alignment.** S2-3.4-5 says `data` is present "when the command succeeded", and S2-3.4-11 +that a command without `data` has failed; S2 has no partial outcome. One sentence in S2 §3.4 +states it: `data` with `error` diagnostics is a document that describes everything except what +the errors name. `kindVersion` stays 1, which S2-3.4-3 requires. + +**Criteria.** e2e on the fixture: the sets name `good/good`, one error per failed member with +`path` `bad/mcpp.toml`, exit 1; a workspace in which every member fails omits `data`. + +### 4.5 E2: a host tool that does not build, under `emit` + +Under `emit`, a host tool whose build fails is a warning, `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT`, +naming the tool, its package and the first line of the failure. Planning continues, and the +build program receives the path the tool would be published at, which the store key fixes before +any build (`tool_store::bin_path`, `src/build/prepare.cppm:10342-10351`). A program that only +names the tool configures exactly as after a successful build; a program that runs it while +configuring (the pattern of docs/30's `dep_bin` example) fails, and E1 and E3 govern that +failure. The tool is still built, because a program may run it; its `check` actions still run, +because `blocking = true` is its author's statement, and a tool built without them would enter +the store unverified under the key a verified build uses. `mcpp build` is unchanged. + +**Criteria.** e2e, member `user`: `emit` exits 0 with the member's sets and one warning; `mcpp +build` still exits 2. + +### 4.6 E3: a build program that fails costs its own directives (D3, accepted) + +Under `emit`, a package whose build program fails (does not compile, exits non-zero, times out, +or prints output mcpp refuses) is described without that program's directives, with one error, +`MCPP_BUILD_DATABASE_PROGRAM_FAILED`, whose `path` names its `build.mcpp`. The manifest's part of +the configuration, the toolchain, the module graph and the standard-library units are described +as usual; a failure that follows from the missing directives fails the member, and E1 applies. +Directives from a failed run are never applied (`src/build/build_program.cppm:1601-1640`), so no +half-applied state exists. + +**Criteria.** e2e: a single package whose program exits 1, and one whose program does not +compile: `data` present with the package's sources, one error, exit 1. + +### 4.7 E4: the plan-only signal (D4) + +**What it is.** #699's first request is a signal, for example the environment variable +`MCPP_PLAN_ONLY=1` or a function `mcpp::plan_only()`, that tells a build program it runs for +`emit build-database` rather than for a build, so that it can skip checks and heavy work. **D4 +is the decision not to add it**, for three reasons. + +1. It gives a build program two behaviours, and the database describes one of them; nothing + checks that the other emits the same directives, so R1.2 would hold only by each author's + discipline (P6). +2. `--configure-only`, the reference R1.2 names, would have to run one of the two, and either + choice breaks an identity. +3. The separation already has a place: verification is a `check` action, which `emit` does not + run (P5), and R2 (§5.4) makes such a check cost nothing on a build whose inputs did not change. + +### 4.8 Usage side + +Environment verification is a `check` action, `blocking = true` where compilation must wait for +it; a program that cannot find something it would configure says so with `mcpp::warning` and +emits what it has. docs/30 states both. + +## 5. #701 and #702 + +### 5.1 The proposal + +#702 (2026.9.27.1, +494/-15, CI green except the two xcode-27 jobs of #669) adds: + +1. `mcpp::runtime_library_dir(dir)`, wire `mcpp:runtime-library-dir=`, protocol 12, "the + build-program form of `[runtime] library_dirs`": a new directive row + (`Scope::LinkGlobal`, `Transform::AbsPath`, persisted in the build-program cache) whose + `apply` appends to `RuntimeConfig::libraryDirs`, and the root package's residue mirrored into + the plan snapshot as `deploy`'s is. +2. `__action-stamp` records each stamp's modification time before the command and, on success, + moves a stamp the command did not write to the present. + +The pull request is reused as this round's single pull request (D7): it is retitled and +re-implemented to §5.4. + +### 5.2 Is the directive needed? + +**What exists.** `[runtime] runtime_search_dirs` (and the retiring `library_dirs`) are static +lists relative to the package root; `deploy` places named files at paths relative to the program. +The readers: the merge into `LinkIntent::runtimeSearchDirs` (`src/build/plan.cppm:911-948`), +RUNPATH/rpath on ELF and Mach-O (`src/build/flags.cppm:507`), `mcpp run`'s loader path +(`src/build/execute.cppm:553`), `mcpp pack`'s closure search (`src/pack/pipeline.cppm:194-195`, +`:463-464`) and runtime validation (`src/build/runtime_validation.cppm:475`). + +**The gap.** A directory whose location only a build program learns (a vcpkg install root, an SDK +found through `xpkg_dir`) and whose files do not exist when the program runs, because an +installation action creates them later. `deploy` needs the file names at plan time. The plugin's +route on 2026.9.26.1 is to deploy what exists at plan time, so the libraries a first installation +produces are placed by the next plan, which `mcpp run` performs (its record, D8, and +`deps/deps.cppm`). + +**Two observations narrow the need.** On Windows, placement beside the program is the platform's +convention and the only form that also serves a program started by hand, since PE has no run path; +a search directory serves `mcpp run` and `mcpp pack` only. And the consumer ships without the +directive. + +**Verdict.** The gap is real and general, a directory-level runtime search declared at +configuration time, and it is the build-program form of an existing key, as `include-dir` and +`deploy` are. + +### 5.3 #702 as written + +The directive follows the extension path the directive table defines: one row with protocol 12, +a non-empty cache tag replayed on a hit (the reasoning `warning` and `pack-format` state), the +root's residue mirrored, both languages of docs/04 and docs/30, unit and e2e tests. Four points do +not conform. + +1. **It extends a retiring field (P8).** `RuntimeConfig::libraryDirs` is one of "the four legacy + vectors … readable for one compatibility train" (`modules/manifest/src/types.cppm:1149-1150`); + docs/04 §2.11 says the same of `library_dirs`, and the fingerprint names it + `legacy-runtime-dir` (`src/build/prepare_inputs.cppm:723-725`). Its successor, + `LinkIntent::runtimeSearchDirs`, reaches the same readers through the same `absolute_from` + merge (`src/build/plan.cppm:911-920`) and accepts absolute paths, as `LinkIntent::deploy` + already does for the `deploy` directive's absolute `from`. #702's docs say the directive joins + `LinkIntent`; its code joins the legacy vector. +2. **The platform it is for has no criterion.** e2e 779 declares `# requires: pack gcc`, which + holds on Linux only; `mcpp run`'s `PATH` and `mcpp pack`'s PE closure on Windows, the cases the + issue names, are not exercised. +3. **The docs example** passes a Qt `bin/` to `link_search`; on Windows import libraries are in + `lib/` and DLLs in `bin/`. +4. **No CHANGELOG entry**, which every version pull request since 2026.9.21.3 carries. + +### 5.4 The conformant design + +Four changes, each stated as the rule it implements. SPEC-007 (§5.6) states the same rules from +the plugin author's side. + +**R1'. `mcpp::runtime_search_dir(dir)`: the build-program form of `runtime_search_dirs`.** + +- Wire `mcpp:runtime-search-dir=`, protocol 12: a directive row with `Scope::LinkGlobal`, + `Transform::AbsPath` and a non-empty cache tag. `apply` appends to + `LinkIntent::runtimeSearchDirs`, and the root package's residue is mirrored into the plan + snapshot as `deploy`'s is. +- The readers are the ones §5.2 lists, unchanged; a dependency's declaration reaches the + consumer's executable through the same merge. The directory need not exist when the program + runs, because a `prepare` action may populate it. +- `[runtime] library_dirs` keeps its legacy status and gains no directive. +- **Criteria.** Unit: the parse, the absolute-path transform, the protocol gate, the cache round + trip, and the field it lands in. e2e on Linux: RUNPATH in `build.ninja`, `mcpp run`, a replay on + a cache hit, `mcpp pack`. e2e on Windows: a DLL in a declared directory is found by `mcpp run` + through `PATH` and placed beside the program by `mcpp pack`. + +**R2. After a `check` or `prepare` command succeeds, every stamp is newer than every input.** +The engine creates each declared stamp that is missing and sets the modification time of each +existing one to the present, whether or not the command wrote it; on failure it writes nothing. +The rule needs no record of a stamp's time before the run. #702's form, which moves only a stamp +the command did not write, meets the same rule and is acceptable. A stamp feeds no compile or link +edge (a blocking check or a `prepare` action orders edges through an order-only edge), so touching +a stamp the command wrote changes no build. **Criterion:** e2e 780, which fails on 2026.9.26.1. + +**P. The `prepare` role (O1).** An action whose command populates a directory that the build reads +by directory, and whose file names are not known when the build program runs: installing a vcpkg +manifest or a CMake subproject into a prefix, unpacking an SDK. + +- **Outputs:** one or more stamps, which the engine writes as for `check` (R2), and one declared + directory, `a.output_dir(dir)`, which the command populates. When the command succeeds and the + directory does not exist, the engine writes no stamp and fails the edge, naming the directory. +- **R1.3 becomes a check.** A build program whose rerun inputs (`rerun_if_changed`, + `rerun_if_changed_glob`) lie inside a declared `prepare` directory reads a construction result + while it configures; the engine warns and names both. This is the pattern the plugins use today + to place a first installation's libraries on the next plan. +- **Ordering:** every compile edge and the link edge of the declaring package wait for it. It is + construction: a policy that concerns checks does not concern it, and the build's progress lines + label it `PREPARE`. +- **References to its products** are names fixed at configuration time: `include_dir`, + `link_search` and `runtime_search_dir` for directories, `link_flag` for a library's full path. + The rule `src/build/hostprogram.cppm:187-194` states ("Content may arrive later; names may not") + holds at the granularity of a directory. +- **Spelling:** `a.role = mcpp::roles::prepare;`. The constants + `mcpp::roles::{source, check, object, artifact, prepare}` exist from protocol 12. An older + engine's bundled module lacks them, so a build program that uses them fails to compile on that + engine and names the constant, instead of the older engine reading the string `"prepare"` as + `source`. +- **Unknown role strings are refused** from protocol 12, with the list of roles. Today + `decode_action` maps any unknown string to `source` + (`modules/buildmcpp/src/directives.cppm:1014-1018`) and `action_error` checks only the command + and the outputs, so a misspelt role changes an action's meaning without a word. +- **Why a role and not `check` with `blocking = true`.** A role is an input to engine decisions. + #699's third request, not adopted, was to skip the `check` actions of host-tool builds under + `emit`; had it been adopted, every installation written as a check would have been skipped. A + construction step labelled as verification is broken by the first decision made on its label. +- **Criteria.** Unit: the role decoded from the constant and from the string; an unknown string + refused with the list. e2e: a `prepare` action populates a directory whose file names the build + program does not know; a unit that includes a header from it compiles, and a program linked + against a library in it runs through `runtime_search_dir`, on the first build; a second build + runs nothing; after an input of the action changes, it runs once and not again. A `prepare` + action whose command creates nothing fails naming its directory, and a build program that + declares a rerun input inside that directory is warned. + +**W. A Windows program's runtime libraries are placed beside it (O2).** A PE program has no run +path, so a DLL in a runtime search directory serves `mcpp run` (through `PATH`) and `mcpp pack`, +and not a program started by hand from the build directory. The platform's own convention is +placement: vcpkg's integration copies a program's imported DLLs beside it after the link, and +CMake names the same set `$`. + +- **Rule.** After the link of a PE program whose plan has runtime search directories, an engine + edge places beside the program every DLL the program imports, directly or through another DLL, + that resolves in one of those directories. The resolution is `mcpp pack`'s: + `read_closure` (`src/pack/pack.cppm:1323`), the directory order of `pipeline.cppm:193-202`, and + the system rule that never copies an API set or a system DLL (`src/pack/binfmt.cppm:812-823`). +- **The edge** is engine-internal, as the check stamp's wrapper is: its inputs are the program and + the stamps of the graph's `prepare` actions, it reports the DLLs it resolved in a depfile (the + mechanism device compilers already use), so a later change to any of them runs it again, its + output is a stamp, and a copy is replaced only when the source differs. Two directories that offer one DLL name are resolved in search order, + and the choice is reported. +- **Nothing else changes:** ELF and Mach-O keep their run paths; `mcpp pack` is unchanged; a + plan without runtime search directories has no such edge. +- **Criteria.** e2e on Windows: a DLL in a declared runtime search directory, populated by a + `prepare` action, is found by the program started by hand from the build directory after the + first build; replacing the DLL in that directory replaces the copy on the next build; a system + DLL is never copied. + +### 5.5 F1: `$ORIGIN` in a user link flag (mcpp-community/mcpp#703) + +The plugin record reports that a build program's `link_flag` loses `$ORIGIN` through "ninja and +the shell". Measured on 2026.9.26.1 (gcc@16.1.0), for `mcpp::link_flag("-Wl,-rpath,$ORIGIN/../lib")` +and for `[build] ldflags = ["-Wl,-rpath,$ORIGIN/../lib"]` alike: + +``` +build.ninja: -Wl,-rpath,$$ORIGIN/../lib +program: (RPATH) [::/../lib:] +``` + +**Cause.** SPEC-004 §8 states how an element of `cflags`, `cxxflags` and `asmflags` (and of +their directives) becomes words, forbids an implementation to interpret `$` (rule 7), and requires +every word to reach the compiler verbatim whatever the host's command-line reader. `ldflags` and +the link directives have no such statement, and `normalize_ldflag` +(`src/build/flags.cppm:367-385`) escapes an element for ninja only, so the `sh` that runs the +command expands `$ORIGIN` to nothing. The engine's own run path is rendered +`-Wl,-rpath,'$$ORIGIN'` (`src/build/plan.cppm:745`, `:2125`), quoted for ninja and for the shell. +The result is worse than a lost entry: `/../lib` is the host's `/lib`, a host directory in the +program's run path, the class #696 closed for links. + +**Repair.** SPEC-004 §8's reading applies to `ldflags` and to the link directives: an element is +read into words, and each word reaches the linker verbatim, escaped for ninja and quoted for the +host. The same reading splits an element that packs several tokens, which the rendering of +link-unit flags guards against case by case today (`src/build/ninja_backend.cppm:342-370`). +**Compatibility:** an element written for the shell or for ninja by hand (`\$ORIGIN`, +`'$$ORIGIN'`) changes meaning; the index is searched for such spellings before release. Windows is +unaffected (no shell). **Criterion:** e2e, `$ORIGIN` from the manifest and from `link_flag` +reaches the program's run path verbatim (fails on 2026.9.26.1). Filed as #703. + +### 5.6 SPEC-007: the build plugin specification + +`docs/specs/build-plugins.md` (SPEC-007, draft 0.1, written in Chinese as SPEC-001 to SPEC-006 +are) is the contract for rule packages, dependency adapters and distribution members. Each rule +carries the implementation state the spec index defines, and a rule no engine check enforces is +marked as an author's obligation. + +| § | Content | +|---|---| +| 1 | Three kinds of work and the mechanism of each: configuration (directives), construction (actions), verification (`check`). Construction is never done while the build program runs; environment incompleteness is a warning, never a non-zero exit; configuration depends only on declared inputs and never on construction results; one behaviour under planning and building | +| 2 | Which directive expresses which configuration (including `runtime_search_dir`); package-relative paths; no host system directories; no retiring fields; link flags read as words (#703) | +| 3 | Actions as argv with the tool as an input; outputs named at submission; the five roles with `prepare`; `check` only for verification; the stamp rule R2; role constants; network access belongs to installation, and an offline build (`MCPP_OFFLINE=1`, inherited by actions, `src/cli.cppm:167-172`) does not reach the network | +| 4 | Runtime search: `runtime_search_dir` for a directory, `deploy` for a file known at configuration time, what Windows provides, no run path through `link_flag` | +| 5 | Obligations under planning: E2 and E3 from the plugin's side | +| 6 | Payloads declared where the lookup happens; no probing of host paths | +| 7 | The engine release a plugin needs, stated; the index CI pin moves and `min_mcpp` does not | +| 8 | A criterion on every platform claimed; a criterion that fails before its change; a planning criterion | + +**What it asks of mcpp-plugins 0.13** (the feedback for the plugin side): + +| Plugin | Today (its design record, D8) | Under SPEC-007 | +|---|---|---| +| `deps-vcpkg`, `deps-cmake` | the installation is a blocking `check` | a `prepare` action (R3.3, R3.4) | +| `deps-vcpkg`, `deps-cmake` | the prefix's shared libraries are enumerated at plan time and deployed one by one, so a first installation's libraries are placed by the next plan | `runtime_search_dir` for the prefix's `bin/` (Windows) or `lib/` (R4.1); no enumeration of construction results (R1.3, now warned); on Windows, W places the DLLs beside the program after the first build | +| `deps-vcpkg` | vcpkg may download during the build | stated in its documentation; an offline build completes from vcpkg's caches or fails naming what is missing (R3.7) | +| `rules-qt` | Windows: the DLLs of the linked modules and of the modules they depend on, deployed beside the program | conforms as it is: the SDK is a payload installed before the build, so these files are known at configuration time (R1.3, R4.2). After W, `runtime_search_dir` for the SDK's `bin/` gives the same result without the plugin computing the module closure | +| `rules-qt` | Linux: glib, zstd and zlib linked by full path with `--no-as-needed` | unchanged: QtCore's own `DT_RUNPATH` stops the program's run path from applying to QtCore's dependencies, a loader rule rather than an engine gap; `$ORIGIN` in link flags is usable after #703 (R2.4) | +| all | engine floors written as versions | the first mcpp release that speaks protocol 12, stated in each plugin's documentation (R7.1) | + +**O2** (a program started by hand on Windows) is designed in §5.4 as W, a general mechanism +(P9), and SPEC-007 R4.3 states it. + +### 5.7 Disposition + +#702 is reused as this round's single pull request (D7): retitled, and re-implemented to R1', R2, +P and W, with F1 and the rest of §7 in the same pull request. mcpp-plugins adopts SPEC-007 after the +release that carries protocol 12. + +## 6. Specifications and documentation + +| Document | Change | +|---|---| +| SPEC-005, to v1.3 | R2.5: a host tool that does not build under `emit` is a warning (E2). R3.7: `work-directory` is the directory the compiler runs in; `arguments` include the interface language flag (C3, C4). R3.8: `provides` of the standard-library units names the std cache BMI (D5b). R4.1: the compile-commands document includes the standard-library units (C5, S1-12-1). R5.2: member containment and the program-failure description (E1, E3). | +| SPEC-004 §8 | the element reading extends to `ldflags` and the link directives (F1, #703) | +| SPEC-007, new (draft 0.1) | `docs/specs/build-plugins.md`, and its row in `docs/specs/README.md` | +| docs/50 §8, and zh | the failure paragraph; `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT` (warning), `MCPP_BUILD_DATABASE_PROGRAM_FAILED` (error); `path` on member errors | +| docs/01, and zh | the root file and the configuration's database (§3.2), the symlink redirect, the standard-library units | +| docs/30, and zh | configuration, construction and verification (§4.8, SPEC-007 §1); `dep_bin` under `emit`; `runtime_search_dir`; the `prepare` role and the role constants; the stamp rule | +| docs/31, and zh | the role table gains `prepare`; a pointer to SPEC-007 | +| docs/04 §2.11, and zh | the build-program form of `runtime_search_dirs`; the PE column of the link-intent table gains W's placement | +| S2 §3.4 (mcppls repository) | one sentence: `data` with error diagnostics (E1) | +| CHANGELOG | one entry per change | + +## 7. Plan + +Every item lands in #702, retitled. + +| # | Change | Where | Tests | +|---|---|---|---| +| W1 | §3.2: the configuration's database, the root copy, the warning, the fast-path restore, the `.gitignore` | `src/build/compile_commands.cppm`, `src/build/ninja_backend.cppm`, `src/build/execute.cppm`, `src/scaffold/create.cppm` | e2e (four), unit | +| W2 | C3 and C4 in `UnitInvocation` | `src/build/compile_commands.cppm` | unit; e2e replay (gcc, llvm); e2e 211 | +| W3 | C5: the plan carries the standard-library description; both databases render it; D5b's `provides` and `build-id` | `src/build/plan.cppm`, `src/build/prepare.cppm`, `src/build/compile_commands.cppm`, `src/build/build_database.cppm` | e2e | +| W4 | E1 | `src/cli/cmd_build.cppm`, `src/build/build_database.cppm` | e2e | +| W5 | E2 | `src/build/prepare.cppm` (host-tool branch, `plan_only` only) | e2e | +| W6 | E3 | `src/build/prepare.cppm` (both `run_build_program` sites, `plan_only` only); a severity on `PlanNote` | e2e (two) | +| W7 | R1' and R2 | `modules/buildmcpp/src/directives.cppm`, `modules/buildmcpp/src/program_protocol.cppm`, `src/build/hostprogram.cppm`, `src/build/prepare.cppm`, `src/cli.cppm` | unit; e2e on Linux and Windows; e2e 780 | +| W8 | F1 (#703): SPEC-004 §8's reading for link flags | `src/build/flags.cppm`, `docs/specs/manifest-semantics.md` | unit, e2e | +| W9 | P: the `prepare` role with `output_dir`, the role constants, the refusal of unknown roles, the R1.3 warning | `modules/buildmcpp/src/directives.cppm`, `src/build/hostprogram.cppm`, `src/build/ninja_backend.cppm` (stamp wrapper, directory post-condition, ordering including the link edge, the #534 self-check, the progress label), `src/build/prepare.cppm` (the warning) | unit, e2e | +| W10 | W: runtime DLL placement after a PE link | `src/build/ninja_backend.cppm` (the edge), `src/cli.cppm` (its engine-internal command), `src/pack/pack.cppm` (`read_closure` reused) | e2e on Windows | +| W11 | §6, SPEC-007 included | specifications, docs, CHANGELOG; S2 in the mcppls repository | docs checks | + +W7 and W9 share one protocol bump, to 12; W10 follows W7 and W9, whose directories it reads. W1 and W2 are tested together (W1's criteria read +`directory`). The version is the next date version at release time. After the release, mcppls +reads `data` with errors and may use D5b's reuse, and mcpp-plugins moves to SPEC-007. + +**Cross-platform.** C3 on Windows is the native spelling of `plan.outputDir`. W1 copies rather +than links the root file, since a symlink needs a privilege on Windows. W7 carries a Windows leg. +W9 is platform-independent: an action is an argv on every host. W10 exists on Windows only, and +its criterion runs there. C4's MSVC form and C5 on MSVC are +open measurements. F1 is POSIX-only. E1 to E3 do not depend on the platform. + +## 8. Decisions + +| | Decision | State | +|---|---|---| +| D1 | E1 as the default | **accepted** | +| D2 | C3 for every toolchain | **accepted** | +| D3 | E3 as the default | **accepted** | +| D4 | no plan-only signal | **accepted** | +| D5 | D5a: standard-library units in `compile_commands.json`; D5b: their BMI in S1 `provides`, and `build-id` | **accepted** | +| D6 | one database per configuration | **accepted** | +| D7 | #702 reused as the single pull request and re-implemented to §5.4 | **accepted** | +| D8 | F1 repaired by extending SPEC-004 §8 to link flags; filed as #703 | **accepted** | +| D9 | P: the `prepare` role with its declared directory, the role constants, the refusal of unknown roles, and the R1.3 warning, in this round | **accepted** | +| D10 | R2 in its simple form (every stamp touched on success) | **accepted** | +| D11 | W (O2): the engine places a Windows program's imported DLLs beside it after the link | **accepted**, with the Windows e2e as its criterion | +| D12 | SPEC-007 draft 0.1 as the contract relayed to mcpp-plugins | **accepted** | + +## 9. Risks and open measurements + +- C4 and C5 on MSVC (Windows CI): `/interface` in a clang-cl-mode clangd; an MSVC STL `std.ixx` + unit. +- C5's cost for a matching plain clangd: one `std` build per session. +- §3.2 replaces a root file another tool shares with mcpp, including through a root symlink into + that tool's directory; the warning makes it visible. +- E3: code after a build program in `prepare_build` may assume its directives were applied; each + such failure fails the member under E1, and the e2e set includes one. +- E2 retries a failing tool build on every `emit`, as today's failure does. +- F1 changes the meaning of a link flag escaped for the shell or for ninja by hand. +- P on an engine older than protocol 12: a build program that spells the role with a constant + fails to compile and names it; one that writes the string `"prepare"` is read as `source` by + that engine. SPEC-007 R3.6 requires the constant. +- deps-vcpkg's downloads at build time (SPEC-007 R3.7) depend on how vcpkg's caches are populated; + the plugin states its answer. +- W copies DLLs into the build directory on every PE build that has runtime search directories; + a copy is replaced only when its source differs, and its criterion runs on Windows CI only, + where it is the first measurement of the closure walk outside `mcpp pack`. +- The R1.3 warning appears on the first build after the upgrade for the plugins that use the + second-plan route today; SPEC-007 tells them what replaces it. +- E1 depends on one sentence in S2. + +## 10. Overall review + +- **Architecture.** One record serves both databases (P1). One directory names a configuration, + and it is the directory the compiler runs in (C3, §3.2), so the identifier the merge needs and + the fact the specifications ask for are one value. Configuration, construction and verification + each have one mechanism (SPEC-007 §1), and the one gap in that table, construction whose file + names are unknown, gets a role instead of a borrowed one (P). New surfaces join the current model + (R1', P8). Every engine change is general and names no tool (P9); what the plugins knew and + worked around, the engine now states once (R1', P, W). +- **Consistency with the specifications.** C3 meets the JSON format and S1-8-2; C5 meets S1-12-1; + D5b meets S1-8-6 and S1-11.2-3; E1 needs one sentence in S2; F1 extends SPEC-004 §8 to the one + flag list it did not cover. No field outside the JSON format is written into + `compile_commands.json`, which its readers reject (measured). +- **Compatibility.** Behaviour a user can see changes in eight places: the root database is + replaced rather than merged, with a warning when another writer's entries go; `directory` names + the output directory; interface entries carry their language flag; standard-library entries + appear; `emit` returns partial documents with errors; `$ORIGIN` in a link flag reaches the linker; + an unknown action role is refused instead of read as `source`; on Windows, a program's DLLs from + its runtime search directories are placed beside it. Each follows a specification or an + accepted decision. The `emit` envelope keeps `kindVersion` 1. Protocol 12 is additive: a build + program that uses none of its surfaces behaves as before, and one that uses them fails to compile + on an older engine and names what it lacks. +- **Stability.** The fast path gains a copy, not a plan. No compile changes. W1 to W6 change + descriptions. A link changes where W8 corrects a run path that is wrong today, and where a build + program declares a runtime search directory. R2 changes a stamp's time. P adds an ordering that + the declaring package asked for. +- **User experience.** An editor keeps every member that plans (E1), a package whose program fails + (E3), and a member whose host tool does not build (E2). A deleted database returns on the next + build. A dependency installed by a plugin is usable on the first build, without the second plan + the plugin needs today, and on Windows a program started by hand finds its DLLs (W). Verification in `check` actions costs nothing when its inputs did not + change (R2). +- **Test coverage.** Each item has a criterion that fails on 2026.9.26.1 (C1, C3 for gcc, C4, E1, + E2, E3, R2, F1) or states a new property (§3.2's four e2e, C5, D5b, R1' on Linux and Windows, P). + W has its criterion on Windows. SPEC-007 §8 asks the same of every plugin, including a planning + criterion. +- **Measured and not measured.** Measured: every reading in §3, §4.2, §5.5 and Appendix A. Not + measured: MSVC (C4, C5), macOS, clangd 23.1 itself (its mechanism is reproduced with llvm@20.1.7 + and clangd 22.1.8), Meson (its binary on the measuring host does not run), and W, whose criterion + runs on Windows CI. +- **Corrections made while writing.** C1 had been recorded twice before (#397 C-1, #677 B1). + Listing the standard-library units was first expected not to help; clangd builds from a listed + provider. GCC was first expected to hand `.ixx` to the linker; GCC 16 compiles it. Revision 1's + four-condition merge kept a merge across configurations; §3.2 removes it. The first draft of the + plugin migration table moved `rules-qt` on Linux to `runtime_search_dir`; QtCore's own + `DT_RUNPATH` makes the program's run path irrelevant to QtCore's dependencies, so that row stays + as the plugin has it. + +## 11. Implementation + +### 11.1 Tasks + +| Task | Items | Branch | Owns | e2e | +|---|---|---|---|---| +| T1 the compile database | W1, W2, W3 | `feat/702-cdb` | `compile_commands.cppm`; the fast path in `execute.cppm`; `scaffold/create.cppm`; the standard-library units, `provides` and `build-id` in `build_database.cppm`; the writer's call site in `ninja_backend.cppm` | 781-786 | +| T2 `emit` | W4, W5, W6 | `feat/702-emit` | the member loop in `cmd_build.cppm`; member diagnostics in `build_database.cppm`; the host-tool and build-program branches of `prepare.cppm` under `plan_only` | 787-789 | +| T3 build-program surfaces | W7, W9 | `feat/702-actions` | `directives.cppm`, `program_protocol.cppm`, `hostprogram.cppm`; `__action-stamp` in `cli.cppm`; the action edges in `ninja_backend.cppm`; the residue mirror and the R1.3 warning in `prepare.cppm` | 779, 780, 790-794 | +| T4 link flags | W8 | `feat/702-link` | `normalize_ldflag` in `flags.cppm`; the link-flag rendering in `ninja_backend.cppm`; SPEC-004 §8 | 795-796 | +| T5 DLL placement | W10 | `feat/702-link` | the placement edge in `ninja_backend.cppm`; its internal command in `cli.cppm`; the closure walk shared with `mcpp pack` | 797-798 | +| T6 documents and release | W11 | `feat/runtime-library-dir` | specifications, docs and their zh pairs, CHANGELOG, the version, the xlings pin | | + +### 11.2 Dependencies + +- T1 to T4 are independent and proceed in parallel, each in a worktree from #702's head. +- T5 reads `LinkIntent::runtimeSearchDirs`, which `[runtime] runtime_search_dirs` already + feeds, so it proceeds in parallel against the manifest key; T3's `prepare` stamps join its + inputs at integration. +- The shared files are `prepare.cppm` (T1, T2, T3), `ninja_backend.cppm` (T1, T3, T4, T5), + `cli.cppm` (T3, T5) and `build_database.cppm` (T1, T2); each task changes its own functions. + Integration merges T3 first (it moves the protocol), then T4 and T5, then T1, then T2. +- T6 follows all five: each task reports its specification and documentation text. + +### 11.3 Cross-repository sequence + +1. mcpp #702 carries T1 to T6, the version and the xlings pin; every CI leg is green except the + xcode-27 legs of #669. +2. Before the release, the index and mcpp-plugins are searched for action roles outside the + five (P refuses them) and for link flags that escape `$` by hand (F1 changes their meaning). +3. The release: `release.yml`; each archive is uploaded to GitCode with the local gtc as it + appears; the xim-pkgindex bump is merged by a maintainer; the index's `latest` is read back. +4. The ecosystem: the published mcpp is verified in an xlings sandbox with the CN mirror + (`xlings subos use --sandbox --cmd ...`): installation, `new`/`build`/`run`, the compile + database, `emit` over a workspace with a failing member, a build program with + `runtime_search_dir` and a `prepare` action, and index packages including the rules plugins. +5. mcppls: S2 §3.4's sentence on `data` with error diagnostics, in its repository. +6. mcpp-plugins: SPEC-007 is relayed by the maintainer, and the plugins move after the release. +7. #699, #701 and #703 are closed with the release's readings. + +### 11.4 Review axes + +| Axis | What holds it | +|---|---| +| Architecture | one record for both databases (P1); one directory names a configuration; one mechanism per kind of work (SPEC-007 §1); engine changes name no tool (P9) | +| Stability | the fast path gains a copy and no plan; no compile command changes; links change only where F1 corrects a run path or a program declares a runtime search directory | +| Simplicity | one function publishes the root file on both paths; one stamp rule for both stamped roles; one role table decodes constants and strings | +| User experience | an editor keeps every member that plans; a deleted database returns; a first installation is usable on the first build; a Windows program started by hand finds its DLLs | +| Compatibility and upgrade | protocol 12 is additive; `kindVersion` stays 1; cache entries of older programs replay unchanged; a foreign database and a rerun input inside a `prepare` directory are warnings; the one new refusal, an unknown role, is searched for in the index before the release | +| Cross-platform | C3 in native spelling; the root file copied, never linked; R1' with a Windows leg; W on Windows CI; C4 and C5 measured on MSVC before their MSVC form is emitted | +| Consistency | SPEC-004, SPEC-005, SPEC-007, docs/01, docs/04, docs/30, docs/31, docs/50 and their zh pairs change in the same pull request as the code | +| Test coverage | each item has a criterion that fails on 2026.9.26.1 or states a new property; the full e2e suite runs on the integration branch before the pull request is pushed | + +## Appendix A. Measurement record + +All runs use `~/.xlings/data/xpkgs/xim-x-mcpp/2026.9.26.1/bin/mcpp`, clangd and clang-tidy from +`~/.xlings/data/xpkgs/xim-x-llvm-tools/22.1.8/bin`, and scratch directories. + +### A.1 The compile database (a copy of `hello`) + +``` +report 3.6: rm compile_commands.json; mcpp build + last line: Finished dev in 0.00s; compile_commands.json NOT regenerated; after --no-cache: present +report 3.1: xmake's sample moved to the copy, touch src/main.cpp, mcpp build + from the root: 6 entries (g++ bits/std.cc; clang++ greet.cppm, main.cpp, test.cppm; g++ src/greet.cppm, src/main.cpp) + from src/: 4 entries (the two relative ones pruned) +report 3.2: replay with -o redirected, interfaces first + gcc from the output directory: 3 of 3 rc=0, objects written + gcc from directory: greet.cppm rc=0; test.cppm rc=1; main.cpp rc=1; gcm.cache/ written into the project root + llvm from either: 3 of 3 rc=0 +two toolchains: mcpp test (llvm), then mcpp build --toolchain gcc@16.1.0 + g++ greet.cppm | g++ main.cpp | g++ test.cppm | clang++ tests/test_smoke.cpp +configurations: target/x86_64-linux-gnu/ holds two fingerprint directories after build, test and + --no-cache runs with one toolchain and a build with the other; after `mcpp build` and `mcpp test` + in one configuration, the three build entries and the test entry all name one directory (6413abdf...) +``` + +### A.2 clangd and clang-tidy + +``` +.ixx without -x c++-module: E [fe_expected_compiler_job] ... expected exactly one compiler job +.ixx with the flag: All checks completed, 0 errors +llvm@20.1.7 database: Failed to build module std; due to Don't get the module unit for module std + [ast_file_version_too_old] ... std.pcm; 1 error ++ mcpp:std units: Built module std; Built module hello.greet; 0 errors +gcc@16.1.0 database: module 'std' not found; 1 error (unchanged with bits/std.cc appended) +missing directory: clangd: VFS: failed to set CWD ...; 0 errors + clang-tidy: LLVM ERROR: Cannot chdir into "..."! (exit 134) +one unknown key in an entry: clangd: Failed to load compilation database; clang-tidy: cannot load it +GCC 16 on .ixx: g++ -std=c++23 -fmodules -c m.ixx: rc=0, object written +``` + +### A.3 #699 + +``` +emit --workspace (good, bad, user): rc=1, no data, PLAN_FAILED bad: build.mcpp exited with 1 (build aborted) +emit in user: rc=1, no data, PLAN_FAILED building host tool 'tool:t' failed: build failed +mcpp build in user (verbose): [1/5] mcpp __action-stamp .../lupdate.stamp -- /bin/false; FAILED +emit --workspace (good): rc=0, sets ['good/good'] +``` + +### A.4 `directory` in other producers + +``` +CMake 4.4.2, Ninja: directory ; file /lib/lib.cpp; replay: object written +CMake 4.4.2, Unix Makefiles: directory /lib; file /lib/lib.cpp; replay: object written +ninja -t compdb, mcpp graph: directory /target/x86_64-linux-gnu/ +``` + +### A.5 F1 + +``` +mcpp::link_flag("-Wl,-rpath,$ORIGIN/../lib") and [build] ldflags, gcc@16.1.0: + build.ninja: -Wl,-rpath,$$ORIGIN/../lib + program RPATH: [.../xim-x-glibc/2.44/lib64:.../xim-x-gcc/16.1.0/lib64:/../lib:.../subos/default/lib] +``` diff --git a/docs/specs/build-plugins.md b/docs/specs/build-plugins.md new file mode 100644 index 00000000..605a544e --- /dev/null +++ b/docs/specs/build-plugins.md @@ -0,0 +1,181 @@ +# SPEC-007:构建插件:配置、施工与校验的分工,运行时与规划期的义务 + +| 项 | 值 | +|---|---| +| 规范编号 | SPEC-007 | +| 标题 | 构建插件:配置、施工与校验的分工,运行时与规划期的义务 | +| 状态 | 草案 v0.1 | +| 版本 | 0.1 | +| 最后修改 | 2026-09-26 | +| 对应实现 | 逐条标注。标「已实现」的条款对应 mcpp >= 2026.9.26.1;标「未实现」的条款随 mcpp#702 落地 | +| 相关设计文档 | `.agents/docs/2026-09-26-compile-database-and-issue-699-design.md`(§5) | +| 相关 issue | mcpp#699、mcpp#701、mcpp#702、mcpp#703 | +| 使用文档 | [docs/30 - build.mcpp](../30-build-mcpp.md)、[docs/31 - 编写规则包](../31-authoring-a-rule-package.md) | + +本规范规定构建插件对引擎和对消费方承担的义务,以及引擎为此提供的机制。docs/31 说明怎样编写 +插件,本规范规定插件必须满足什么。 + +规范用语与实现状态标记见 [规范索引](README.md)。只约束插件作者、引擎不做检查的条款标注 +「作者义务」。 + +## 0. 适用范围 + +构建插件是为其他包的构建贡献工作的包,以及它导入消费方构建程序的模块(`rule_module`、 +`host-module`)。 + +| 类别 | 例 | 贡献 | +|---|---|---| +| 规则包 | `rules-qt`、`rules-spirv`、`rules-cuda` | 代码生成,设备语言的编译 | +| 依赖适配包 | `deps-vcpkg`、`deps-cmake` | 把外部包管理器或外部构建系统的产物接入构建 | +| 分发成员 | `dist-wix`、`dist-apk` | 由链接产物生成可安装的分发物 | + +项目自己的 `build.mcpp` 中的同类代码同样适用本规范。 + +**分工。** mcpp 只提供通用机制:指令、action 的角色、stamp、运行时库的搜索与放置。某一个 +工具(vcpkg、CMake、Qt)的知识只属于它的插件。插件遇到的缺口若是通用的,由 mcpp 以通用机制 +补上(第 3.3 节的 `prepare`、第 4.1 节的 `runtime_search_dir`、第 4.3 节的 DLL 放置),而不 +由插件绕过。 + +## 1. 三类工作 + +插件所做的每一件事属于且只属于下表的一类。 + +| 类 | 定义 | 发生在 | 机制 | +|---|---|---|---| +| 配置 | 决定构建的形状:编译哪些源、用哪些选项、链接什么、运行时在哪里找库 | 构建程序运行时,即规划期 | 指令(第 2 节) | +| 施工 | 产生构建读取的文件:生成源码、编译设备代码、安装依赖前缀、打包 | 构建期 | `mcpp::action`(第 3 节) | +| 校验 | 判断环境或产物是否满足要求,不产生构建读取的文件 | 构建期 | `role = "check"` 的 action | + +- **R1.1** 施工**必须**声明为 action,**禁止**在构建程序运行期间进行。构建程序可以运行工具 + 来**查询**配置所需的答案(版本、选项、路径),**禁止**借此产生构建读取的文件。(作者义务; + action 自 2026.8.5.1 **已实现**) +- **R1.2** 校验**必须**是 `check` action。构建程序发现环境不完整(缺少一个 SDK 模块、缺少 + 一个库)时,**必须**用 `mcpp::warning` 报告,并输出它能确定的全部配置;**禁止**因环境不完整 + 以非零状态退出。编译或链接会在缺失处失败,位于该警告之后。(作者义务) +- **R1.3** 配置**必须**只取决于构建程序声明的输入:包的清单、声明的载荷,以及 + `rerun_if_changed`、`rerun_if_changed_glob`、`rerun_if_env_changed` 列出的文件、glob 与 + 环境变量。配置**禁止**依赖施工的结果,例如枚举一个安装前缀中由 action 产生的文件:规划 + (`mcpp emit build-database`)与第一次构建都发生在施工之前,依赖施工结果的配置在第一次与 + 第二次构建之间不同。(作者义务;构建程序的重新运行依据落在一个 `prepare` action 声明的目录内 + 时,引擎给出一条警告,**未实现**,mcpp#702) +- **R1.4** 构建程序在规划与构建中**必须**行为相同。引擎不提供「正在规划」的信号,因为规划 + 所描述的计划与 `mcpp build --configure-only` 计算的计划相同(SPEC-005 R1.2)。(**已实现**) + +## 2. 配置:指令 + +- **R2.1** 下表中的配置**必须**用对应的指令表达;**禁止**用 `link_flag`、`cxxflag` 拼写表中 + 已有指令所表达的内容(例如以 `-Wl,-rpath,` 代替运行时搜索目录,以 `-I` 代替头文件目录)。 + 指令由引擎按平台渲染、去重,并进入缓存与构建数据库。(作者义务) + + | 配置 | C++ 接口 | 线格式 | 自 | + |---|---|---|---| + | 头文件目录 | `include_dir`、`include_dir_after` | `include-dir`、`include-dir-after` | 协议 1 | + | 编译选项 | `cxxflag`、`cflag` | `cxxflag`、`cflag` | 协议 1 | + | 宏 | `define` | `cfg` | 协议 1 | + | 链接库与库目录 | `link_lib`、`link_search` | `link-lib`、`link-search` | 协议 1 | + | 其他链接选项(含库的完整路径) | `link_flag` | `link-flag` | 协议 8 | + | 运行时搜索目录 | `runtime_search_dir` | `runtime-search-dir` | 协议 12,**未实现**(mcpp#702) | + | 放到程序旁的文件 | `deploy` | `deploy` | 协议 11 | + | 生成的源 | `generated`、`source` 与 `role = "source"` 的 action | `generated`、`source`、`action` | 协议 1 | + | 重新运行构建程序的依据 | `rerun_if_changed`、`rerun_if_changed_glob`、`rerun_if_env_changed` | 同名 | 协议 1、2 | + | 报告与探测 | `warning`;`fact`、`floor` | 同名 | 协议 5、7 | + +- **R2.2** 相对路径按声明它的包的根目录解析。插件**禁止**把宿主系统目录(`/usr/include`、 + `/usr/lib`、`/lib`、`C:\Windows\System32` 及同类)声明为头文件、链接或运行时搜索目录;SDK + 与工具的路径取自声明的载荷(`mcpp::xpkg_dir`)或依赖边(`mcpp::dep_dir`、`mcpp::dep_bin`)。 + (作者义务;对图目标的链接,引擎检查 `-L`,mcpp#696 **已实现**) +- **R2.3** 插件**必须**使用现行字段的指令,**禁止**依赖只为兼容而保留的字段 + (`[runtime] library_dirs`,docs/04 §2.11)。(作者义务) +- **R2.4** 链接选项的一个元素按 SPEC-004 §8 读成词,每个词原样到达链接器。插件**禁止**依赖 + 引擎内部的转义拼写(例如 `'$$ORIGIN'`)。(**未实现**,mcpp#703:当前 `ldflags` 与 + `link_flag` 中的 `$` 会被宿主 shell 展开) + +## 3. 施工:action + +- **R3.1** action 的命令是 argv,**禁止**假定 shell。命令调用的工具**必须**列为输入;命令 + 自己发现的输入用 depfile 报告。(argv 与 depfile **已实现**;工具作为输入是作者义务) +- **R3.2** action **必须**在提交时命名它的输出文件,因为引擎在规划期确定源集合、指纹与模块 + 图。输出文件名在施工前无法得知的工作(安装一个前缀、解开一个 SDK)使用 `prepare` 角色。 + (输出命名 **已实现**;`prepare` **未实现**) +- **R3.3** 角色: + + | `role` | 输出 | 顺序 | + |---|---|---| + | `source` | 可编译的输出加入声明包的编译集 | 声明包的每条编译边等待它 | + | `object` | 加入链接集 | 链接边消费它 | + | `artifact` | 新文件,输入是链接产物 | 链接之后 | + | `check` | 引擎写的 stamp | 与编译并行;`blocking = true` 时声明包的编译边等待它 | + | `prepare` | 引擎写的 stamp;命令填充它用 `output_dir` 声明的目录,构建按目录引用其内容 | 声明包的每条编译边与链接边等待它 | + + `prepare`(**未实现**,mcpp#702)是施工,不是校验,任何只针对校验的策略都不作用于它。它的 + 产物由配置以目录为单位引用(`include_dir`、`link_search`、`runtime_search_dir`),或以 + `link_flag` 中的完整路径引用;这些名字在配置时确定,内容在施工时到达。一个 `prepare` action + **必须**用 `output_dir` 声明它填充的目录;命令成功而该目录不存在时,引擎不写 stamp,并以 + 指出该目录的消息使这条边失败。 +- **R3.4** `check` 只用于校验,**禁止**用来表示施工。(作者义务) +- **R3.5** `check` 与 `prepare` 的命令成功后,引擎创建或更新它们的 stamp,使 stamp 新于该 + action 的每个输入;命令失败时不写 stamp。命令无需自己写 stamp。(创建自 2026.8.29.1 + **已实现**;更新**未实现**,mcpp#702:当前一个已存在的 stamp 不被更新,输入改变一次后该 + action 在此后每次构建中都会重新运行) +- **R3.6** 角色**应当**以常量书写(`mcpp::roles::prepare` 等),使不认识该角色的旧引擎在编译 + 构建程序时拒绝它;引擎拒绝未知的角色字符串,并列出可用的角色。(**未实现**,mcpp#702: + 当前引擎把未知的角色字符串当作 `source`) +- **R3.7** 构建期**不应**访问网络:下载属于安装期(载荷的安装、依赖的解析)。一个必须在构建期 + 下载的 action(例如由包管理器取得源码)**必须**在其说明中写明,并在离线构建中 + (`--offline` 或 `MCPP_OFFLINE=1`;前者在进程环境中设置后者,action 继承之)不访问网络: + 从缓存完成,或以指出缺失内容的消息失败。(作者义务;环境传递 **已实现**) + +## 4. 运行时:程序如何找到它依赖的共享库 + +- **R4.1** 一个目录中的共享库由施工产生(`prepare`)或文件名不定时,插件**必须**用 + `runtime_search_dir` 声明该目录。引擎把它用于 ELF 与 Mach-O 的运行路径(RUNPATH/rpath, + 从不作为 `-L`)、`mcpp run` 的加载路径、`mcpp pack` 的闭包搜索与运行时校验,并把依赖包的 + 声明传到消费方的可执行文件。(**未实现**,mcpp#702) +- **R4.2** 一个在配置时已知的文件需要位于程序旁的某个相对位置时(Qt 的平台插件、Vulkan 的 + ICD 清单),插件用 `deploy`。(**已实现**,协议 11) +- **R4.3** Windows 的可执行文件没有运行路径。`mcpp run` 通过 `PATH` 使用运行时搜索目录, + `mcpp pack` 把闭包需要的 DLL 放到程序旁(**已实现**)。链接之后,引擎把程序直接或间接导入的、 + 位于其运行时搜索目录中的非系统 DLL 放到程序旁,使从构建目录直接启动的程序同样能找到它们; + 闭包的求解与 `mcpp pack` 相同。(**未实现**,mcpp#702) +- **R4.4** 插件**禁止**在 `link_flag` 中写运行路径(`-Wl,-rpath,...`),**必须**使用 R4.1。 + (作者义务) + +## 5. 规划期的义务(`mcpp emit build-database`) + +- **R5.1** 规划运行构建程序,不运行任何 action(SPEC-005 R2.2)。插件遵守 R1.3 时,规划得出 + 的配置与构建相同。(**已实现**) +- **R5.2** 一个包的构建程序在规划中失败时,该包只按其清单描述,并得到一条错误诊断;成员的 + 其余部分照常描述。插件遵守 R1.2 时,环境不完整不会使构建程序失败。(**未实现**,mcpp#702) +- **R5.3** 插件所需的宿主工具在规划中构建失败时,规划继续,构建程序收到该工具将被发布的路径, + 并产生一条警告。插件**应当**在 action 中运行宿主工具,而不是在构建程序中运行,使规划不依赖 + 工具能否构建。(引擎部分 **未实现**,mcpp#702;「应当」为作者义务) + +## 6. 环境与载荷 + +- **R6.1** 插件驱动的工具与 SDK **必须**声明为载荷(`[xlings]` 或 `[feature-xlings]`),需要 + 时以目标轴选择器门控,并在构建程序中用 `mcpp::xpkg_dir` 取得路径。声明**必须**放在查询发生 + 的包上:`xpkg_dir` 为正在构建的包回答;`host-module` 的声明对编入它的每个构建程序可见 + (docs/31)。(**已实现**) +- **R6.2** 插件**禁止**探测宿主路径来寻找工具或 SDK;未声明的依赖不可复现。(作者义务) + +## 7. 版本与兼容 + +- **R7.1** 使用协议 N 的指令或角色的插件,**必须**在其文档中写明第一个支持协议 N 的 mcpp + 版本。索引测量该插件时,CI 所用的 mcpp 版本移到该版本;索引的 `min_mcpp` 不因此改变。旧引擎 + 编译该构建程序时因缺少函数或常量而失败,并指出其名称。(编译期失败 **已实现**) +- **R7.2** 插件**禁止**依赖引擎内部的拼写与未写入文档的行为(R2.3、R2.4)。(作者义务) + +## 8. 判据 + +- **R8.1** 插件**必须**在它声明支持的每个平台上有一个在该平台运行的判据。`# requires: gcc` + 只在 Linux 成立,不能作为 Windows 或 macOS 的判据。(作者义务) +- **R8.2** 每个判据**必须**在它所验证的改动之前失败。(作者义务) +- **R8.3** 插件**应当**有一个规划判据:它的一个消费方工程,在没有安装其载荷的机器上运行 + `mcpp emit build-database`,得到一份文档,其中该插件只贡献警告,或只使声明它的包缺少 + 构建程序的指令(R5.2),而不是整次失败。(作者义务) + +## 9. 变更记录 + +| 版本 | 日期 | 变更 | +|---|---|---| +| 0.1 | 2026-09-26 | 首版草案(mcpp#699、#701、#702、#703)。 | From 8601a386bfb9c053c1e15dfe0ae46abcd8d4b129 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Sat, 26 Sep 2026 08:18:56 +0800 Subject: [PATCH 04/26] Link flags are read as words and reach the linker as written (#703) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit SPEC-004 §8's element reading extends to ldflags and the link directives: each element is read into words, and each word reaches the linker quoted for the host and escaped for ninja. A $ORIGIN in [build] ldflags, in mcpp::link_flag or in a dependency's ldflags reaches the program's run path instead of /../lib; a search directory whose path holds a space is one argument. Dependency flags propagate word by word, the link directives spell engine-built paths as one word, the build-program cache epoch moves to 3, and the first plan names an element whose words differ from what 2026.9.26.1 passed. --- docs/04-mcpp-toml.md | 23 ++- docs/specs/manifest-semantics.md | 37 ++-- docs/zh/04-mcpp-toml.md | 16 +- modules/buildmcpp/src/directives.cppm | 16 +- modules/buildmcpp/src/program_protocol.cppm | 5 +- src/build/flags.cppm | 30 ++- src/build/prepare.cppm | 66 +++++-- src/build/runtime_validation.cppm | 5 +- ...link_flag_reaches_the_linker_as_written.sh | 176 ++++++++++++++++++ 9 files changed, 321 insertions(+), 53 deletions(-) create mode 100755 tests/e2e/795_a_link_flag_reaches_the_linker_as_written.sh diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index 6737e9be..a651bf79 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -422,10 +422,12 @@ bmi_schedule = "auto" # Module-edge scheduling: auto (= off) | on | #### Compile-flag syntax *(mcpp 2026.9.17.1+)* An element of `cflags`, `cxxflags` or `asmflags` stands for one or more compiler -arguments ("words"). The syntax is the same on every host, wherever the list is -written: `[build]`, `[targets.]`, a `flags` glob entry, a feature, a -`[target..build]` section, an xpkg descriptor, and the `mcpp:cflag=` / -`mcpp:cxxflag=` directives of a build program. +arguments ("words"), and an element of `ldflags` for one or more linker +arguments (mcpp 2026.9.27.1+). The syntax is the same on every host, wherever +the list is written: `[build]`, `[targets.]`, a `flags` glob entry, a +feature, a `[target..build]` section, an xpkg descriptor, and the +`mcpp:cflag=` / `mcpp:cxxflag=` / `mcpp:link-flag=` directives of a build +program. | Written | Words the compiler receives | |---|---| @@ -437,6 +439,7 @@ written: `[build]`, `[targets.]`, a `flags` glob entry, a feature, a | `"-I/opt/my\\ dir/include"` | `-I/opt/my dir/include` | | `"-IC:\\sdk\\include"` | `-IC:\sdk\include` | | `"-DNAME=a$b"` | `-DNAME=a$b` | +| `"-Wl,-rpath,$ORIGIN/../lib"` (in `ldflags`) | `-Wl,-rpath,$ORIGIN/../lib` | The rules, stated on the element's text (after TOML or Lua has removed its own escapes): @@ -452,8 +455,16 @@ escapes): taken verbatim, as in every earlier release. A `defines` entry is one value and is not read by this syntax: `defines = -["NAME=\"text\""]` passes the single word `-DNAME="text"`. `ldflags`, -`dialect_cxxflags` and `std-module-flags` are not covered by this section. +["NAME=\"text\""]` passes the single word `-DNAME="text"`. `dialect_cxxflags` +and `std-module-flags` are not covered by this section. + +`ldflags` follows the syntax from mcpp 2026.9.27.1 (#703). Before, a link-flag +element was escaped for ninja and not quoted for the shell, so on Linux and +macOS a `$ORIGIN` reached the program's run path as `/../lib`. A `-L` or +`-Wl,-rpath,` word with a package-relative path resolves against the package +root, and a dependency's `ldflags` reach its consumer word by word. An element +escaped for ninja or the shell by hand (`\$ORIGIN`, `'$$ORIGIN'`) now reads as +written; the first plan names such an element under `build/flag-words`. `compile_commands.json` and `mcpp emit build-database` list the same words in `arguments`, ready to execute without a shell. diff --git a/docs/specs/manifest-semantics.md b/docs/specs/manifest-semantics.md index 79f409fa..ea9cfd31 100644 --- a/docs/specs/manifest-semantics.md +++ b/docs/specs/manifest-semantics.md @@ -5,8 +5,8 @@ | **规范编号** | SPEC-004 | | **标题** | `mcpp.toml` 的平面划分、条件化形状、解析轴与命名规约 | | **状态** | **草案(Draft)** | -| **版本** | 1.6 | -| **最后修改** | 2026-09-25 | +| **版本** | 1.7 | +| **最后修改** | 2026-09-26 | | **最低实现版本** | 条件化形状:mcpp **2026.8.29.1**(`[target..build-dependencies]` 起齐备);目标轴:mcpp **2026.9.6.4** | | **作者/维护** | mcpp-community | | **相关设计文档** | `.agents/docs/2026-09-07-mcpp-toml-unified-semantics-design.md`
`.agents/docs/2026-06-04-manifest-schema-ownership.md`
`.agents/docs/2026-09-03-xlings-workspace-as-the-one-table.md`
`.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md` | @@ -353,14 +353,18 @@ feature-deps feature-xlings ← 限定词是门 客户端上都能构建;两次发布的归档逐字节相同;缺少 `version` 的兄弟 `path` 边被拒绝且报错给出 应写的一行;无需修改的包的归档与此前逐字节相同 (`tests/e2e/772_a_published_member_is_self_contained.sh`)。 +15. §8 对链接 flag 的判据:`[build] ldflags` 与构建程序的 `mcpp::link_flag` 中写出的 + `-Wl,-rpath,$ORIGIN/../lib` 原样到达程序的运行路径,不出现 `/../lib`;依赖传播的同一 + 元素同样原样到达;含空格的 `link_search` 目录是一个参数 + (`tests/e2e/795_a_link_flag_reaches_the_linker_as_written.sh`)。 -## 8. 编译 flag 列表的元素 +## 8. flag 列表的元素 -`cflags`、`cxxflags` 与 `asmflags` 的一个元素是一段文本,代表零个或多个词;编译器收到的 -参数就是这些词,按列表顺序排列。本节在元素的文本上陈述(TOML 或 Lua 先去掉自己的转义)。 -该读法对这三个键的每一个出现位置相同:`[build]`、`[targets.]`、`flags` 的 glob 条目、 -feature、`[profile.]`、`[target..build]`、xpkg 描述符,以及构建程序的 -`mcpp:cflag=` 与 `mcpp:cxxflag=` 指令。 +`cflags`、`cxxflags`、`asmflags` 与 `ldflags` 的一个元素是一段文本,代表零个或多个词; +编译器或链接器收到的参数就是这些词,按列表顺序排列。本节在元素的文本上陈述(TOML 或 Lua +先去掉自己的转义)。该读法对这四个键的每一个出现位置相同:`[build]`、`[targets.]`、 +`flags` 的 glob 条目、feature、`[profile.]`、`[target..build]`、xpkg 描述符, +以及构建程序的 `mcpp:cflag=`、`mcpp:cxxflag=` 与 `mcpp:link-flag=` 指令。 1. 未加引号的空格与制表符分隔词,连续的分隔符等同于一个。 2. `'` 开启一段单引号区域,区域内的字符按字面取到下一个 `'` 为止。 @@ -382,14 +386,20 @@ feature、`[profile.]`、`[target..build]`、xpkg 描述符,以及 `[build]`、各个命中的 `[target..build]`。实现**必须**满足:同一宏名的后一个 条目在原位替换前一个;条目 `!NAME` 移除宏名 `NAME`;一个包的每个编译单元对每个宏名 至多收到一个 `-D` 词;`defines` 条目取代同一个包的 `cflags`、`cxxflags` 中读作单个词 -且宏名相同的 `-D` 词。实现向这三个列表插入一个词 `w` 时,**必须**使用一个按上述规则读回恰为 `w` 的拼写。 +且宏名相同的 `-D` 词。实现向这四个列表插入一个词 `w` 时,**必须**使用一个按上述规则读回恰为 `w` 的拼写; +构建程序的 `mcpp:link-lib=`、`mcpp:link-search=` 与 `mcpp:link-script=` 指令由名字或路径构成的 +值属于这种插入。 -实现**必须**把每个词原样交给编译器,与宿主的命令行读取规则(POSIX `sh`、MSVCRT)无关; -`compile_commands.json` 与构建数据库(SPEC-005 R3.7)列出的参数**必须**是这些词。 +实现**必须**把每个词原样交给编译器或链接器,与宿主的命令行读取规则(POSIX `sh`、MSVCRT) +无关;`compile_commands.json` 与构建数据库(SPEC-005 R3.7)列出的参数**必须**是这些词。 +依赖的 `ldflags` 传播给消费者时按词传播,包内相对的搜索路径按词解析为绝对路径。 +规则 7 对链接 flag 的一个推论:`$ORIGIN` 等加载器记号原样到达链接器。为 shell 或 ninja +手工转义的写法(`\$ORIGIN`、`'$$ORIGIN'`)按上述规则读取,不再是转义。 -`ldflags`、`dialect_cxxflags` 与 `std-module-flags` 不在本节范围内。 +`dialect_cxxflags` 与 `std-module-flags` 不在本节范围内。 -**状态:已实现(mcpp 2026.9.17.1;`defines` 的集合语义 mcpp 2026.9.25.1)。** +**状态:已实现(mcpp 2026.9.17.1;`defines` 的集合语义 mcpp 2026.9.25.1;`ldflags` 与链接指令 +mcpp 2026.9.27.1,#703)。** ## 9. 工作空间继承与构建需求的作用域 @@ -425,3 +435,4 @@ feature、`[profile.]`、`[target..build]`、xpkg 描述符,以及 | 1.4 | 2026-09-15 | 库目标的默认链接形态 `linkage`(mcpp 2026.9.15.2):§3.1.1 补默认值的语义、优先顺序与拒绝条件;§7 补第 9 条判据。 | | 1.5 | 2026-09-17 | 编译 flag 列表元素的读法(mcpp 2026.9.17.1,#655):新增 §8 与 §7 第 10 条判据。 | | 1.6 | 2026-09-25 | 工作空间继承与构建需求的作用域(mcpp 2026.9.25.1,#690):§8 补 `defines` 的集合语义;新增 §9 与 §7 第 11 至 14 条判据。 | +| 1.7 | 2026-09-26 | §8 的读法扩展到 `ldflags` 与构建程序的链接指令(mcpp 2026.9.27.1,#703):`$ORIGIN` 原样到达链接器;§7 补第 15 条判据。 | diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 70fc7adb..cf8e2ee1 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -424,10 +424,11 @@ bmi_schedule = "auto" # Module-edge scheduling: auto (= off) | on | #### 编译 flag 的写法 *(mcpp 2026.9.17.1+)* `cflags`、`cxxflags` 或 `asmflags` 里的一个元素代表一个或多个编译器 -参数(「词」)。这套语法在每个宿主上都相同,无论写在哪张表里: +参数(「词」),`ldflags` 里的一个元素代表一个或多个链接器参数 +(mcpp 2026.9.27.1+)。这套语法在每个宿主上都相同,无论写在哪张表里: `[build]`、`[targets.]`、`flags` glob 条目、feature、 `[target..build]` 小节、xpkg 描述符,以及构建程序的 -`mcpp:cflag=` / `mcpp:cxxflag=` 指令。 +`mcpp:cflag=` / `mcpp:cxxflag=` / `mcpp:link-flag=` 指令。 | 写法 | 编译器收到的词 | |---|---| @@ -439,6 +440,7 @@ bmi_schedule = "auto" # Module-edge scheduling: auto (= off) | on | | `"-I/opt/my\\ dir/include"` | `-I/opt/my dir/include` | | `"-IC:\\sdk\\include"` | `-IC:\sdk\include` | | `"-DNAME=a$b"` | `-DNAME=a$b` | +| `"-Wl,-rpath,$ORIGIN/../lib"`(在 `ldflags` 中) | `-Wl,-rpath,$ORIGIN/../lib` | 以下规则施加于元素的文本上(TOML 或 Lua 已经去掉了它自己的转义之后): @@ -453,8 +455,14 @@ bmi_schedule = "auto" # Module-edge scheduling: auto (= off) | on | 与以往每一个版本相同。 一条 `defines` 条目是一个值,不受这套语法解析:`defines = -["NAME=\"text\""]` 传出单独一个词 `-DNAME="text"`。`ldflags`、 -`dialect_cxxflags` 与 `std-module-flags` 不受本节约束。 +["NAME=\"text\""]` 传出单独一个词 `-DNAME="text"`。`dialect_cxxflags` 与 +`std-module-flags` 不受本节约束。 + +`ldflags` 自 mcpp 2026.9.27.1 起遵循这套语法(#703)。此前链接 flag 的元素只为 +ninja 转义、没有为 shell 加引号,所以在 Linux 与 macOS 上 `$ORIGIN` 以 `/../lib` +进入程序的运行路径。带包内相对路径的 `-L` 或 `-Wl,-rpath,` 词相对包根解析,依赖的 +`ldflags` 按词传给消费者。为 ninja 或 shell 手工转义的元素(`\$ORIGIN`、 +`'$$ORIGIN'`)现在按写法读取;首次 plan 会在 `build/flag-words` 下点名这样的元素。 `compile_commands.json` 与 `mcpp emit build-database` 在 `arguments` 里列出同样的词,可以不经 shell 直接执行。 diff --git a/modules/buildmcpp/src/directives.cppm b/modules/buildmcpp/src/directives.cppm index 019e075e..26a2aceb 100644 --- a/modules/buildmcpp/src/directives.cppm +++ b/modules/buildmcpp/src/directives.cppm @@ -717,9 +717,15 @@ std::string transformed(const Def& def, std::string_view raw, const fs::path& root) { switch (def.transform) { case Transform::Verbatim: return std::string(raw); - case Transform::LibFlag: return mcpp::toolchain::lib_flag_for(dial, raw); - case Transform::LibSearchPath: return std::string(dial.libSearchPrefix) - + abs_against(root, raw); + // A link-flag element is read into words (SPEC-004 §8, #703), so a + // value the engine builds from a name or a path is spelled to read + // back as one word: a library or a directory with a space in its path + // stays one argument. + case Transform::LibFlag: + return mcpp::manifest::flag_element(mcpp::toolchain::lib_flag_for(dial, raw)); + case Transform::LibSearchPath: + return mcpp::manifest::flag_element(std::string(dial.libSearchPrefix) + + abs_against(root, raw)); // A define is one word of a compile-flag list, whatever it contains. case Transform::DefinePrefix: return mcpp::manifest::flag_element(std::string(dial.definePrefix) + std::string(raw)); @@ -727,7 +733,9 @@ std::string transformed(const Def& def, std::string_view raw, // Absolute on purpose: the link runs in the build directory, so a // relative script path resolves against the wrong root and lld // answers "cannot find linker script link.ld" — measured. - case Transform::LinkerScript: return "-T " + abs_against(root, raw); + // Two words, `-T` and the path, the second spelled as one word. + case Transform::LinkerScript: + return "-T " + mcpp::manifest::flag_element(abs_against(root, raw)); // Resolve `from` NOW, while `root` (this build.mcpp's package root) is // still in hand -- `apply` is never given it. `to` is left untouched: // it is a destination relative to an executable this package has not diff --git a/modules/buildmcpp/src/program_protocol.cppm b/modules/buildmcpp/src/program_protocol.cppm index 2e476b25..847ad4a1 100644 --- a/modules/buildmcpp/src/program_protocol.cppm +++ b/modules/buildmcpp/src/program_protocol.cppm @@ -101,7 +101,10 @@ inline constexpr int kProtocolVersion = 12; // Epoch 2 (#359): entries gained `glob` records. An engine that does not know // them would replay a strict subset of the declared inputs and call a stale // build fresh, which is exactly the silent-wrong-answer this guard exists for. -inline constexpr int kCacheEpoch = 2; +// Epoch 3 (#703): an `ldflag` value is read into words (SPEC-004 §8), and the +// link directives spell a path as one word. A value an earlier engine cached, +// `-L/opt/my sdk/lib` for instance, would now read as two words. +inline constexpr int kCacheEpoch = 3; // ── Run bound ────────────────────────────────────────────────────────────── // diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 90673ed4..9b21fdcc 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -364,24 +364,30 @@ std::string escape_path(const std::filesystem::path& p) { return escape_ninja_chars(p.string()); } -std::string normalize_ldflag(const std::filesystem::path& root, const std::string& flag) { +// One WORD of the link-flag list, with a package-relative search path made +// absolute against the package root. The result is the word the linker +// receives, not ninja text: the caller quotes it for the host and escapes it +// for ninja (`ninja_command_word`). Until #703 this function escaped the +// element for ninja only, so the `sh` that runs a POSIX link expanded a +// `$ORIGIN` the author wrote, and the program's run path held `/../lib`. +std::string normalize_ldflag(const std::filesystem::path& root, const std::string& word) { auto absolute_path = [&](std::string_view raw) { std::filesystem::path p{std::string(raw)}; if (p.is_absolute() || is_loader_relative_search_path(raw)) return p; return root / p; }; - if (flag.starts_with("-L") && flag.size() > 2) { - return "-L" + escape_path(absolute_path(std::string_view(flag).substr(2))); + if (word.starts_with("-L") && word.size() > 2) { + return "-L" + absolute_path(std::string_view(word).substr(2)).string(); } constexpr std::string_view rpathPrefix = "-Wl,-rpath,"; - if (flag.starts_with(rpathPrefix) && flag.size() > rpathPrefix.size()) { + if (word.starts_with(rpathPrefix) && word.size() > rpathPrefix.size()) { return std::string(rpathPrefix) - + escape_path(absolute_path(std::string_view(flag).substr(rpathPrefix.size()))); + + absolute_path(std::string_view(word).substr(rpathPrefix.size())).string(); } - return flag; + return word; } } // namespace @@ -1046,11 +1052,17 @@ CompileFlags compute_flags(const BuildPlan& plan) { plan.manifest.buildConfig.cxxRuntime))); } - // User link flags + // User link flags: `[build] ldflags`, the `link_flag` and `link_lib` + // directives, and what the dependencies propagate. SPEC-004 §8's reading + // applies to them as to the compile flags (#703): each element is read + // into words, and each word reaches the linker verbatim, quoted for the + // host and then escaped for ninja. A `$ORIGIN` therefore reaches the + // linker as written, and an element that packs several tokens is several + // words on every host. std::string user_ldflags; - for (auto const& flag : plan.manifest.buildConfig.ldflags) { + for (auto const& word : mcpp::manifest::flag_words(plan.manifest.buildConfig.ldflags)) { user_ldflags += ' '; - user_ldflags += normalize_ldflag(plan.projectRoot, flag); + user_ldflags += ninja_command_word(normalize_ldflag(plan.projectRoot, word)); } // C standard. The file-level `$cflags` carries the engine default, a diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index c089c306..864f2b4b 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -587,6 +587,18 @@ std::vector previous_release_words(std::string element, bool define } void report_flag_words_changes(const mcpp::manifest::Manifest& m) { + auto show = [](const std::vector& words) { + std::string out = "["; + for (auto const& w : words) + out += std::format("{}'{}'", out.size() > 1 ? ", " : "", w); + return out + "]"; + }; + auto note_change = [&](std::string what, std::string hint) { + auto note = std::pair{std::move(what), std::move(hint)}; + auto& notes = pending_flag_words_notes(); + if (std::ranges::find(notes, note) == notes.end()) notes.push_back(std::move(note)); + }; + auto const who = m.package.name.empty() ? std::string("(root)") : m.package.name; auto check = [&](std::string_view where, const std::vector& list, bool define) { for (auto const& e : list) { @@ -594,27 +606,45 @@ void report_flag_words_changes(const mcpp::manifest::Manifest& m) { : mcpp::manifest::flag_words(e); auto before = previous_release_words(e, define); if (now == before) continue; - auto show = [](const std::vector& words) { - std::string out = "["; - for (auto const& w : words) - out += std::format("{}'{}'", out.size() > 1 ? ", " : "", w); - return out + "]"; - }; - auto note = std::pair{std::format( + note_change(std::format( "{}: {} element '{}' reaches the compiler as {}; mcpp before " "2026.9.17.1 passed {} on this host", - m.package.name.empty() ? std::string("(root)") : m.package.name, - where, e, show(now), show(before)), + who, where, e, show(now), show(before)), std::string( "a compile-flag element is read by one syntax on every host, and a " "`defines` entry is one value (docs/04-mcpp-toml.md, " "\"Compile-flag syntax\"); spell the element so that it reads as the " - "words meant")}; - auto& notes = pending_flag_words_notes(); - if (std::ranges::find(notes, note) == notes.end()) notes.push_back(std::move(note)); + "words meant")); + } + }; + // THE SAME QUESTION FOR THE LINK FLAGS, which take the reading from + // 2026.9.27.1 (#703). Before, a `-L` or `-Wl,-rpath,` element was escaped + // for ninja, so its text reached the host's reader as written, and any + // other element was pasted into the ninja rule, so ninja replaced its `$` + // sequences first. `$ORIGIN` written plainly reads the same under both + // models, because neither reproduces the shell's expansion that lost it; + // an element escaped for ninja or for the shell by hand is what differs. + auto check_link = [&](std::string_view where, const std::vector& list) { + for (auto const& e : list) { + auto now = mcpp::manifest::flag_words(e); + auto before = e.starts_with("-L") || e.starts_with("-Wl,-rpath,") + ? mcpp::manifest::host_command_words(e, mcpp::platform::is_windows) + : previous_release_words(e, false); + if (now == before) continue; + note_change(std::format( + "{}: {} element '{}' reaches the linker as {}; mcpp before " + "2026.9.27.1 passed {} on this host", + who, where, e, show(now), show(before)), + std::string( + "a link-flag element is read by the compile-flag syntax, so `$ORIGIN` " + "reaches the linker as written and an element escaped for ninja or the " + "shell by hand is no longer unescaped (docs/04-mcpp-toml.md, " + "\"Compile-flag syntax\"); spell the element so that it reads as the " + "words meant")); } }; auto const& bc = m.buildConfig; + check_link("[build] ldflags", bc.ldflags); check("[build] cflags", bc.cflags, false); check("[build] cxxflags", bc.cxxflags, false); check("[build] defines", bc.defines, true); @@ -7103,9 +7133,14 @@ prepare_build(bool print_fingerprint, const mcpp::manifest::Manifest& depManifest) -> std::vector { + // Word by word (SPEC-004 §8, #703): a search path is made absolute + // per word, and each word is written back as an element that reads as + // exactly that word, so the consumer's renderer reads the dependency's + // flags with the same reading its own flags receive, and an element + // that packs several tokens is several words on both sides. std::vector added; - for (auto const& flag : depManifest.buildConfig.ldflags) { - auto normalized = normalizeDepLdflag(depRoot, flag); + for (auto const& word : mcpp::manifest::flag_words(depManifest.buildConfig.ldflags)) { + auto normalized = mcpp::manifest::flag_element(normalizeDepLdflag(depRoot, word)); m->buildConfig.ldflags.push_back(normalized); added.push_back(std::move(normalized)); } @@ -12111,7 +12146,8 @@ prepare_build(bool print_fingerprint, facts.hasSources = !mcpp::modgraph::package_source_files( packages[i].root, pkg).empty(); facts.carriesForeignLinkInputs = - lf::carries_foreign_link_inputs(pkg.buildConfig.ldflags); + lf::carries_foreign_link_inputs( + mcpp::manifest::flag_words(pkg.buildConfig.ldflags)); facts.isDistribution = mcpp::pack::is_distribution_package(pkg); for (auto const& artifact : pkg.runtimeConfig.artifacts) { if (artifact.role == "static-library") facts.shipsStatic = true; diff --git a/src/build/runtime_validation.cppm b/src/build/runtime_validation.cppm index d96d2015..3110f2b4 100644 --- a/src/build/runtime_validation.cppm +++ b/src/build/runtime_validation.cppm @@ -916,7 +916,10 @@ check_symbol_provision(const mcpp::build::BuildPlan& plan, // The flags this build hands the linker, as ONE vector, because the // question is whether ANY of them took the export decision away from // mcpp. Per-unit flags join below; these are the whole-build ones. - std::vector globalFlags = plan.manifest.buildConfig.ldflags; + // Read as words (SPEC-004 §8, #703), which is what the linker receives: + // an element that packs `-Wl,-E` with another token is still a request. + std::vector globalFlags = + mcpp::manifest::flag_words(plan.manifest.buildConfig.ldflags); auto searchDirs = runtime_search_dirs(plan); diff --git a/tests/e2e/795_a_link_flag_reaches_the_linker_as_written.sh b/tests/e2e/795_a_link_flag_reaches_the_linker_as_written.sh new file mode 100755 index 00000000..642fef5d --- /dev/null +++ b/tests/e2e/795_a_link_flag_reaches_the_linker_as_written.sh @@ -0,0 +1,176 @@ +#!/usr/bin/env bash +# requires: elf unix-shell +# mcpp#703: a link flag reaches the linker as written. +# +# SPEC-004 §8 reads an element of a flag list into words, and each word +# reaches the tool verbatim whatever the host's command-line reader. Until +# 2026.9.27.1 that reading covered the compile flags only. A link-flag element +# was escaped for ninja and not quoted for the shell, so the `sh` that runs a +# POSIX link expanded a `$ORIGIN` the author wrote, and the program's run path +# held `/../lib`, which is the host's `/lib`. +# +# Five legs: +# A. `[build] ldflags`: `$ORIGIN` reaches the program's run path. +# B. a build program's `mcpp::link_flag`: the same. +# C. a dependency's `ldflags`, propagated to the consumer: the same. +# D. `mcpp::link_search` with a directory whose path holds a space: one +# argument. Before, the shell split it, and the linker read the second +# half as an input file. +# E. an element escaped for ninja and the shell by hand is read as written, +# and the first plan says what the linker receives now. +set -e + +MCPP="${MCPP:-mcpp}" +work="$(mktemp -d)" +trap 'rm -rf "$work"' EXIT + +fail() { [ -n "$2" ] && cat "$2"; echo "FAIL: $1"; exit 1; } + +program() { find target -path '*/bin/*' -name "$1" -type f | head -1; } + +# The run-path entries of a program, one per line. +run_path() { + readelf -d "$1" | sed -n 's/.*(R[UN]*PATH).*\[\(.*\)\]/\1/p' | tr ':' '\n' +} + +# The run path names `$ORIGIN/` and no entry that ends in `/../` +# without it. +expect_origin() { + local bin="$1" rest="$2" log="$3" entries + entries="$(run_path "$bin")" + printf '%s\n' "$entries" | grep -qxF "\$ORIGIN/$rest" \ + || { printf 'run path:\n%s\n' "$entries"; fail "\$ORIGIN/$rest is not in the run path" "$log"; } + if printf '%s\n' "$entries" | grep -qxF "/$rest"; then + printf 'run path:\n%s\n' "$entries" + fail "the run path holds /$rest: \$ORIGIN was expanded by the shell" "$log" + fi +} + +write_main() { + mkdir -p "$1/src" + cat > "$1/src/main.cpp" <<'CPP' +#include +int main() { std::printf("LINK_FLAG_OK\n"); } +CPP +} + +# ── A. [build] ldflags ────────────────────────────────────────────────────── +mkdir -p "$work/a" +write_main "$work/a" +cat > "$work/a/mcpp.toml" <<'TOML' +[package] +name = "ldorigin" +version = "0.1.0" + +[build] +ldflags = ["-Wl,-rpath,$ORIGIN/../lib"] +TOML +cd "$work/a" +"$MCPP" build > a.log 2>&1 || fail "the build failed" a.log +bin="$(program ldorigin)" +[ -n "$bin" ] || fail "no program was produced" a.log +expect_origin "$bin" "../lib" a.log +echo " ok: \$ORIGIN in [build] ldflags reaches the run path" + +# ── B. mcpp::link_flag ────────────────────────────────────────────────────── +mkdir -p "$work/b" +write_main "$work/b" +cat > "$work/b/mcpp.toml" <<'TOML' +[package] +name = "linkflagorigin" +version = "0.1.0" +TOML +cat > "$work/b/build.mcpp" <<'CPP' +import mcpp; +int main() { + mcpp::link_flag("-Wl,-rpath,$ORIGIN/../plugins"); +} +CPP +cd "$work/b" +"$MCPP" build > b.log 2>&1 || fail "the build failed" b.log +bin="$(program linkflagorigin)" +[ -n "$bin" ] || fail "no program was produced" b.log +expect_origin "$bin" "../plugins" b.log +echo " ok: \$ORIGIN in mcpp::link_flag reaches the run path" + +# ── C. a dependency's ldflags ─────────────────────────────────────────────── +mkdir -p "$work/c/dep/src" "$work/c/app" +cat > "$work/c/dep/mcpp.toml" <<'TOML' +[package] +name = "originlib" +version = "0.1.0" + +[build] +ldflags = ["-Wl,-rpath,$ORIGIN/../dep"] + +[targets.originlib] +kind = "lib" +TOML +cat > "$work/c/dep/src/originlib.cppm" <<'CPP' +export module originlib; +export int originlib_value() { return 7; } +CPP +write_main "$work/c/app" +cat > "$work/c/app/src/main.cpp" <<'CPP' +#include +import originlib; +int main() { std::printf("LINK_FLAG_OK %d\n", originlib_value()); } +CPP +cat > "$work/c/app/mcpp.toml" <<'TOML' +[package] +name = "originapp" +version = "0.1.0" + +[dependencies] +originlib = { path = "../dep" } +TOML +cd "$work/c/app" +"$MCPP" build > c.log 2>&1 || fail "the build failed" c.log +bin="$(program originapp)" +[ -n "$bin" ] || fail "no program was produced" c.log +expect_origin "$bin" "../dep" c.log +echo " ok: a dependency's \$ORIGIN reaches the consumer's run path" + +# ── D. a search directory with a space ────────────────────────────────────── +mkdir -p "$work/d/my libs" +write_main "$work/d" +cat > "$work/d/mcpp.toml" <<'TOML' +[package] +name = "spacedsearch" +version = "0.1.0" +TOML +cat > "$work/d/build.mcpp" <<'CPP' +import mcpp; +int main() { + mcpp::link_search("my libs"); +} +CPP +cd "$work/d" +"$MCPP" build > d.log 2>&1 \ + || fail "a search directory whose path holds a space broke the link" d.log +out="$("$MCPP" run 2>&1)" || fail "the program did not run: $out" +printf '%s\n' "$out" | grep -q LINK_FLAG_OK || fail "unexpected output: $out" +echo " ok: a search directory whose path holds a space is one argument" + +# ── E. an element escaped by hand ─────────────────────────────────────────── +mkdir -p "$work/e" +write_main "$work/e" +cat > "$work/e/mcpp.toml" <<'TOML' +[package] +name = "handescaped" +version = "0.1.0" + +[build] +ldflags = ["-Wl,-rpath='$$ORIGIN/../x'"] +TOML +cd "$work/e" +"$MCPP" build > e.log 2>&1 || fail "the build failed" e.log +grep -q "reaches the linker as \['-Wl,-rpath=\$\$ORIGIN/../x'\]" e.log \ + || fail "the first plan does not say what the linker receives now" e.log +"$MCPP" build > e2.log 2>&1 || fail "the second build failed" e2.log +if grep -q 'reaches the linker' e2.log; then + fail "a build that repeats the plan repeated the note" e2.log +fi +echo " ok: an element escaped by hand reads as written, and the first plan says so" + +echo "PASS: a link flag reaches the linker as written" From c5c0b52bd46b7190be4c04f628e2b72c563c4f33 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Sat, 26 Sep 2026 08:18:56 +0800 Subject: [PATCH 05/26] A Windows program's runtime DLLs are placed beside it after its link A PE image has no run path, so a DLL in a runtime search directory served mcpp run and mcpp pack and not a program started by hand. A PE program whose plan has runtime search directories now gets an edge after its link, mcpp place-dlls, which reads the program's import closure with mcpp pack's read_closure and system rule and publishes each DLL it resolves in those directories beside the program. Its depfile names the DLLs it placed, so a DLL replaced in its directory is placed again. Deployed DLLs become order-only inputs of the link, since the linker reads the import library. --- src/build/ninja_backend.cppm | 54 ++++++- src/cli.cppm | 6 + src/cli/cmd_publish.cppm | 60 ++++++++ src/pack/pack.cppm | 86 +++++++++++ ...indows_program_finds_its_dlls_beside_it.sh | 134 ++++++++++++++++++ tests/unit/test_ninja_backend.cpp | 62 ++++++++ 6 files changed, 396 insertions(+), 6 deletions(-) create mode 100755 tests/e2e/797_a_windows_program_finds_its_dlls_beside_it.sh diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 2d260ee7..b3c9633d 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -1887,6 +1887,33 @@ std::string emit_ninja_string(const BuildPlan& plan) { append(" command = $mcpp coff-def --output $out --name $def_name $in\n"); append(" description = DEF $out\n\n"); + // A WINDOWS PROGRAM'S RUNTIME DLLS, PLACED AFTER ITS LINK (SPEC-007 R4.3). + // A PE image has no run path, so a DLL in a runtime search directory + // serves `mcpp run` (through `PATH`) and `mcpp pack`, and not a program + // started by hand from the build directory. `mcpp place-dlls` reads the + // program's import closure as `mcpp pack` does and publishes each DLL it + // resolves in those directories beside the program. The names are not + // known when this graph is written, because an action may populate the + // directory, so the edge's output is a stamp and its depfile names the + // DLLs it placed: a DLL replaced in its directory is placed again on the + // next build. ELF and Mach-O keep their run paths, and a plan without + // runtime search directories has no such edge. + const bool placeRuntimeDlls = [&] { + if (plan.linkIntent.runtimeSearchDirs.empty()) return false; + const auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); + return t ? t->is_pe() : bool(mcpp::platform::is_windows); + }(); + if (placeRuntimeDlls) { + std::string dirs; + for (auto const& d : plan.linkIntent.runtimeSearchDirs) + dirs += " " + ninja_command_word(d.string()); + append("rule place_dlls\n"); + append(" command = $mcpp place-dlls --output $out --depfile $out.d $in" + dirs + "\n"); + append(" depfile = $out.d\n"); + append(" deps = gcc\n"); + append(" description = DLLS $in\n\n"); + } + append("rule runtime_alias\n"); if constexpr (mcpp::platform::is_windows) { // PE has no soname symlink, so the alias is a copy — and a copy of a @@ -2681,13 +2708,19 @@ std::string emit_ninja_string(const BuildPlan& plan) { for (auto& input : lu.implicitInputs) { implicit += " " + escape_ninja_path(input); } - // Windows runtime-DLL deployment: an executable takes an implicit + // Windows runtime-DLL deployment: an executable takes an ORDER-ONLY // dependency on each staged dep DLL (bin/), so ninja copies them - // beside the .exe before the build is considered done. Empty on RPATH - // platforms (no *.dll deps), so other targets are unaffected. + // beside the .exe whenever the .exe is built. Order-only rather than + // implicit, because the linker never reads a deployed DLL: it links + // the import library. As an implicit input, a DLL that changed, or a + // deploy entry that a later plan added (a runtime search directory an + // action populated after the first plan), relinked a program whose + // link inputs had not changed. Empty on RPATH platforms (no *.dll + // deps), so other targets are unaffected. + std::string orderOnly; if (lu.kind == LinkUnit::Binary || lu.kind == LinkUnit::TestBinary) { for (auto const& d : deployFiles) - implicit += " " + escape_ninja_path(d.dest); + orderOnly += " " + escape_ninja_path(d.dest); } // The import library is a SECOND output of this edge, declared as an @@ -2740,9 +2773,10 @@ std::string emit_ninja_string(const BuildPlan& plan) { if (!lu.defFile.empty()) implicit += " " + escape_ninja_path(lu.defFile); - std::string out_line = std::format("build {}{} : {}{}{}\n", + std::string out_line = std::format("build {}{} : {}{}{}{}\n", escape_ninja_path(lu.output), implicitOut, rule, ins, - implicit.empty() ? std::string{} : " |" + implicit); + implicit.empty() ? std::string{} : " |" + implicit, + orderOnly.empty() ? std::string{} : " ||" + orderOnly); if (auto flag = shared_soname_flag(lu, plan); !flag.empty()) out_line += " soname_flag = " + flag + "\n"; if (auto flag = shared_soname_default(lu, plan); !flag.empty()) @@ -2863,6 +2897,14 @@ std::string emit_ninja_string(const BuildPlan& plan) { append("default " + elf + ".bin\n\n"); } + // The placement edge (see `place_dlls` above), after the link it reads. + if (placeRuntimeDlls + && (lu.kind == LinkUnit::Binary || lu.kind == LinkUnit::TestBinary)) { + const auto exe = escape_ninja_path(lu.output); + append("build " + exe + ".dlls: place_dlls " + exe + "\n"); + append("default " + exe + ".dlls\n\n"); + } + for (auto const& alias : lu.runtimeAliases) { append(std::format("build {} : runtime_alias {}\n", escape_ninja_path(alias), diff --git a/src/cli.cppm b/src/cli.cppm index e6449d60..3702acf2 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -929,6 +929,11 @@ int run(int argc, char** argv) { .option(cl::Option("verify").takes_value().value_name("MODE") .help("How an existing destination is judged up to date: content (default) | size")) .action(wrap_rc(cmd_stage))) + .subcommand(cl::App("place-dlls") + .description("(internal: invoked by ninja) Place beside a Windows program the DLLs it imports from its runtime search directories") + .option(cl::Option("output").takes_value().value_name("PATH").help("the stamp to write")) + .option(cl::Option("depfile").takes_value().value_name("PATH").help("the depfile naming every DLL placed")) + .action(wrap_rc(cmd_place_dlls))) .subcommand(cl::App("coff-def") .description("(internal: invoked by ninja) Write a .def of every exportable symbol in the given COFF objects") .option(cl::Option("output").takes_value().value_name("PATH").help("the .def to write")) @@ -1152,6 +1157,7 @@ int run(int argc, char** argv) { "update", "search", "publish", "pack", "emit", "xpkg", "toolchain", "cache", "index", "self", "explain", "version", "dyndep", "why", "resolve", "stage", "bmi-equal", "coff-def", + "place-dlls", "bmi-compile", "bmi-supervise", "bmi-await", }); bool ok = false; diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index e304d57d..5737e953 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -36,6 +36,66 @@ export int cmd_publish(const mcpplibs::cmdline::ParsedArgs& parsed) { parsed.is_flag_set("dry-run"), parsed.is_flag_set("allow-dirty")); } +// `mcpp place-dlls --output --depfile ...` -- the +// edge that follows a Windows program's link when its plan has runtime search +// directories (mcpp.pack's `place_runtime_dlls`, SPEC-007 R4.3). Internal: +// only a generated build.ninja names it, and it runs on whatever host builds, +// because it reads the program's import table rather than asking a loader. +// +// The depfile names every DLL placed, so ninja runs the edge again when one of +// them changes in its directory; the stamp is the edge's only declared output, +// because the DLL names are not known when the graph is written. +export int cmd_place_dlls(const mcpplibs::cmdline::ParsedArgs& parsed) { + const std::filesystem::path stamp{parsed.option_or_empty("output").value()}; + const std::filesystem::path depfile{parsed.option_or_empty("depfile").value()}; + if (stamp.empty() || depfile.empty() || parsed.positional_count() < 1) { + std::println(stderr, "error: place-dlls requires --output, --depfile and a program"); + return 2; + } + const std::filesystem::path program{parsed.positional(0)}; + std::vector dirs; + for (std::size_t i = 1; i < parsed.positional_count(); ++i) + dirs.emplace_back(parsed.positional(i)); + + auto placed = mcpp::pack::place_runtime_dlls(program, dirs); + if (!placed) { + std::println(stderr, "error: {}", placed.error().message); + return 1; + } + for (auto const& n : placed->notes) std::println("note: {}", n); + + // The depfile syntax ninja reads (`deps = gcc`): a space and `#` are + // escaped with a backslash, and `$` is doubled. + auto dep_word = [](const std::filesystem::path& p) { + std::string out; + for (char c : p.generic_string()) { + if (c == ' ' || c == '#') out.push_back('\\'); + if (c == '$') out.push_back('$'); + out.push_back(c); + } + return out; + }; + std::error_code ec; + if (depfile.has_parent_path()) + std::filesystem::create_directories(depfile.parent_path(), ec); + { + std::ofstream d(depfile, std::ios::trunc); + d << dep_word(stamp) << ':'; + for (auto const& src : placed->sources) d << " \\\n " << dep_word(src); + d << '\n'; + if (!d) { + std::println(stderr, "error: cannot write '{}'", depfile.string()); + return 1; + } + } + std::ofstream st(stamp, std::ios::trunc); + if (!st) { + std::println(stderr, "error: cannot write '{}'", stamp.string()); + return 1; + } + return 0; +} + namespace { int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index c9c2c83b..a2ed8173 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -38,6 +38,7 @@ export module mcpp.pack; import std; import mcpp.build.loader_contract; +import mcpp.build.stage; // place_runtime_dlls publishes through the staging primitive import mcpp.config; import mcpp.pack.binfmt; import mcpp.pack.host_requirements; @@ -358,6 +359,34 @@ struct ClosureRead { // exists. ClosureRead read_closure(const ClosureReadInput& in); +// WHAT A WINDOWS PROGRAM NEEDS BESIDE IT AFTER ITS LINK (SPEC-007 R4.3). +// +// A PE image has no run path: the loader searches the program's directory, +// the system directories and `PATH`, and nothing the program carries. A DLL in +// a runtime search directory therefore serves `mcpp run`, which puts those +// directories on `PATH`, and `mcpp pack`, which stages the closure; a program +// started by hand from the build directory does not find it. The platform's +// own answer is placement: vcpkg's integration copies a program's imported +// DLLs beside it after the link, and CMake names the same set +// `$`. +// +// This is that placement, as one engine mechanism that names no tool. The +// closure is `mcpp pack`'s (`read_closure` under `ClosureRule::Pe`, with its +// system rule), searched in the program's own directory first and then in the +// runtime search directories, in their order. Every resolved DLL outside the +// program's directory is published beside the program through the staging +// primitive, which writes only when the bytes differ. `sources` lists what was +// resolved, for the edge's depfile, so that a DLL replaced in its directory is +// placed again on the next build; `notes` names each DLL that more than one +// directory offers, with the one the search order chose. +struct RuntimeDllPlacement { + std::vector sources; + std::vector notes; +}; +std::expected +place_runtime_dlls(const std::filesystem::path& program, + const std::vector& searchDirs); + // Build a Plan from already-resolved inputs. Caller is expected to have // already run `mcpp build` (or equivalent) and pass the resulting // binary path in. @@ -1320,6 +1349,63 @@ make_tarball(const std::filesystem::path& stagingRoot, } // namespace detail +std::expected +place_runtime_dlls(const std::filesystem::path& program, + const std::vector& searchDirs) +{ + const auto programDir = program.parent_path(); + auto same_dir = [](const std::filesystem::path& a, const std::filesystem::path& b) { + std::error_code ec; + if (std::filesystem::equivalent(a, b, ec)) return true; + return a.lexically_normal() == b.lexically_normal(); + }; + + ClosureReadInput in; + in.object = program; + in.rule = ClosureRule::Pe; + in.searchDirs.push_back(programDir.empty() ? std::filesystem::path(".") : programDir); + for (auto const& d : searchDirs) + if (!same_dir(d, in.searchDirs.front())) in.searchDirs.push_back(d); + const auto read = read_closure(in); + + // The program itself is the one object the caller chose, so a program that + // cannot be read is an error; `read_closure` reports it as unresolved + // under the program's own name. + for (auto const& u : read.unresolved) + if (u.name == program.filename().string()) + return std::unexpected(Error{std::format( + "cannot read the imports of '{}': {}", program.string(), u.why)}); + + RuntimeDllPlacement out; + for (auto const& m : read.members) { + if (same_dir(m.source.parent_path(), in.searchDirs.front())) continue; + auto staged = mcpp::build::stage::stage_file(m.source, programDir / m.name); + if (!staged) + return std::unexpected(Error{std::format( + "cannot place '{}' beside '{}': {}", m.source.string(), + program.filename().string(), staged.error().message)}); + out.sources.push_back(m.source); + // Search order decides between two directories that offer one name, + // and the decision is stated, because the other copy may be the one + // the author meant. + std::vector offering; + for (std::size_t i = 1; i < in.searchDirs.size(); ++i) { + std::error_code ec; + if (std::filesystem::is_regular_file(in.searchDirs[i] / m.name, ec)) + offering.push_back(in.searchDirs[i]); + } + if (offering.size() > 1) { + std::string others; + for (std::size_t i = 1; i < offering.size(); ++i) + others += (others.empty() ? "" : ", ") + offering[i].string(); + out.notes.push_back(std::format( + "'{}' is offered by {} and by {}; the program receives the first, in " + "runtime search order", m.name, offering.front().string(), others)); + } + } + return out; +} + ClosureRead read_closure(const ClosureReadInput& in) { namespace bf = mcpp::pack::binfmt; diff --git a/tests/e2e/797_a_windows_program_finds_its_dlls_beside_it.sh b/tests/e2e/797_a_windows_program_finds_its_dlls_beside_it.sh new file mode 100755 index 00000000..02395270 --- /dev/null +++ b/tests/e2e/797_a_windows_program_finds_its_dlls_beside_it.sh @@ -0,0 +1,134 @@ +#!/usr/bin/env bash +# requires: mingw-cross wine +# SPEC-007 R4.3: a Windows program's runtime DLLs are placed beside it after +# its link. +# +# A PE image has no run path. A DLL in a runtime search directory therefore +# served `mcpp run`, which puts the directory on PATH, and `mcpp pack`, which +# stages the closure, and not a program started by hand from the build +# directory. The engine now follows the link with an edge that reads the +# program's import closure, as `mcpp pack` does, and publishes each DLL it +# resolves in a runtime search directory beside the program. +# +# The directory here is populated by a `prepare` action, so the DLL does not +# exist when the build program runs and no plan-time listing can name it: the +# first build has to place it after the link. Four properties: +# +# 1. after the FIRST build, the program started from the build directory, +# with nothing on PATH, finds the DLL (wine is the loader); +# 2. a build with nothing changed neither relinks nor places again; +# 3. a DLL replaced in its directory replaces the copy on the next build; +# 4. no system DLL is ever copied. +# +# WINE IS EVIDENCE, NOT PROOF, as in 257: what it gives is a PE loader actually +# resolving the DLL from the program's directory. +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +TRIPLE=x86_64-windows-gnu + +fail() { [ -n "$2" ] && cat "$2"; echo "FAIL: $1"; exit 1; } + +# ── The DLL, built once per answer, kept outside the consumer ────────────── +mkdir -p mathkit/src store implib +cat > mathkit/src/mathkit.cppm <<'EOF' +export module mathkit; +export extern "C" int mk_answer(); +EOF +cat > mathkit/mcpp.toml <<'EOF' +[package] +name = "mathkit" +version = "0.1.0" +[build] +sources = ["src/*.cppm", "src/*.cpp"] +[targets.mathkit] +kind = "shared" +EOF +build_dll() { + cat > mathkit/src/impl.cpp < build.log 2>&1) \ + || fail "the DLL did not build" mathkit/build.log + cp "$(find mathkit/target -name libmathkit.dll | head -1)" store/libmathkit.dll + cp "$(find mathkit/target -name libmathkit.dll.a | head -1)" implib/libmathkit.dll.a +} +build_dll 42 + +# ── The consumer: a prepare action fills rt/, declared a runtime search dir ─ +mkdir -p app/src +cat > app/src/main.cpp <<'EOF' +#include +extern "C" int mk_answer(); +int main() { std::printf("ANSWER %d\n", mk_answer()); return 0; } +EOF +IMPLIB="$(host_path "$TMP/implib/libmathkit.dll.a")" +cat > app/mcpp.toml < app/build.mcpp <&1 | tr -d '\r'); } + +# ── 1. the first build places the DLL ────────────────────────────────────── +"$MCPP" build --target "$TRIPLE" -v > b1.log 2>&1 || fail "the first build failed" b1.log +[ -f "$(bindir)/libmathkit.dll" ] || { ls "$(bindir)"; fail "libmathkit.dll is not beside the program after the first build" b1.log; } +out="$(run_by_hand)" +printf '%s\n' "$out" | grep -qx 'ANSWER 42' || fail "the program started by hand did not load its DLL: $out" +echo " ok: after the first build the program started by hand finds its DLL" + +# ── 2. nothing changed: no relink, no placement ─────────────────────────── +"$MCPP" build --target "$TRIPLE" -v > b2.log 2>&1 || fail "the second build failed" b2.log +"$MCPP" build --target "$TRIPLE" -v > b3.log 2>&1 || fail "the third build failed" b3.log +for log in b2.log b3.log; do + if grep -q 'place-dlls' "$log"; then fail "a build with nothing changed placed the DLLs again" "$log"; fi + if grep -q -- '-o bin/dllapp.exe' "$log"; then fail "a build with nothing changed relinked the program" "$log"; fi +done +echo " ok: a build with nothing changed neither relinks nor places again" + +# ── 3. a replaced DLL is placed again ───────────────────────────────────── +cd "$TMP" +build_dll 43 +cd app +"$MCPP" build --target "$TRIPLE" -v > b4.log 2>&1 || fail "the build after the DLL changed failed" b4.log +grep -q 'place-dlls' b4.log || fail "the placement did not run after the DLL changed" b4.log +out="$(run_by_hand)" +printf '%s\n' "$out" | grep -qx 'ANSWER 43' || fail "the copy beside the program was not replaced: $out" +echo " ok: a DLL replaced in its directory replaces the copy" + +# ── 4. no system DLL is copied ──────────────────────────────────────────── +if ls "$(bindir)" | grep -qiE '^(kernel32|msvcrt|ucrtbase|user32|api-ms-|ext-ms-)'; then + ls "$(bindir)" + fail "a system DLL was copied beside the program" +fi +echo " ok: no system DLL is copied" + +echo "PASS: a Windows program finds its runtime DLLs beside it" diff --git a/tests/unit/test_ninja_backend.cpp b/tests/unit/test_ninja_backend.cpp index e4a49690..8ca03473 100644 --- a/tests/unit/test_ninja_backend.cpp +++ b/tests/unit/test_ninja_backend.cpp @@ -1118,6 +1118,68 @@ TEST(NinjaBackend, StdArtifactsAndRuntimeDllsUseTheStageRule) { EXPECT_EQ(afterDll.find("verify"), std::string::npos) << afterDll; } +// ── SPEC-007 R4.3: a Windows program's runtime DLLs are placed after its link ── + +namespace { + +BuildPlan program_plan(std::string_view triple, bool withRuntimeDirs) { + auto plan = minimal_plan(); + plan.toolchain.targetTriple = std::string(triple); + plan.linkUnits.push_back({ + .targetName = "app", + .kind = mcpp::build::LinkUnit::Binary, + .objects = {"obj/main.o"}, + .output = "bin/app.exe", + .entryMain = "src/main.cpp", + }); + if (withRuntimeDirs) + plan.linkIntent.runtimeSearchDirs.push_back("/sdk/my bin"); + return plan; +} + +} // namespace + +TEST(NinjaBackend, PeProgramWithRuntimeSearchDirsGetsAPlacementEdge) { + auto ninja = emit_ninja_string(program_plan("x86_64-w64-windows-gnu", true)); + // One rule, carrying the directories in search order, each one word. + EXPECT_NE(ninja.find("rule place_dlls\n"), std::string::npos) << ninja; + EXPECT_NE(ninja.find("place-dlls --output $out --depfile $out.d $in "), + std::string::npos) << ninja; + EXPECT_NE(ninja.find("my bin"), std::string::npos) << ninja; + EXPECT_NE(ninja.find(" deps = gcc\n"), std::string::npos) << ninja; + // One edge per program, after the link it reads, and built by default. + EXPECT_NE(ninja.find("build bin/app.exe.dlls: place_dlls bin/app.exe\n"), + std::string::npos) << ninja; + EXPECT_NE(ninja.find("default bin/app.exe.dlls\n"), std::string::npos) << ninja; +} + +TEST(NinjaBackend, NoPlacementEdgeWithoutRuntimeSearchDirsOrOffPe) { + // A PE program with no runtime search directory has nothing to place. + auto pe = emit_ninja_string(program_plan("x86_64-w64-windows-gnu", false)); + EXPECT_EQ(pe.find("place_dlls"), std::string::npos) << pe; + // ELF keeps its run path: the directories are RUNPATH entries, not copies. + auto elf = emit_ninja_string(program_plan("x86_64-linux-gnu", true)); + EXPECT_EQ(elf.find("place_dlls"), std::string::npos) << elf; +} + +TEST(NinjaBackend, DeployedDllsAreOrderOnlyInputsOfTheLink) { + // The linker reads the import library, never the deployed DLL, so a DLL + // that changes (or a deploy entry a later plan adds) must not relink. + auto plan = program_plan("x86_64-w64-windows-gnu", false); + plan.runtimeDeployFiles.push_back({"/pkg/lib/libfoo.dll", "bin/libfoo.dll"}); + auto ninja = emit_ninja_string(plan); + auto link = ninja.find("build bin/app.exe"); + ASSERT_NE(link, std::string::npos) << ninja; + auto line = ninja.substr(link, ninja.find('\n', link) - link); + auto oo = line.find(" || "); + ASSERT_NE(oo, std::string::npos) << line; + EXPECT_NE(line.find("bin/libfoo.dll", oo), std::string::npos) << line; + auto implicitBar = line.find(" | "); + EXPECT_TRUE(implicitBar == std::string::npos || implicitBar >= oo + || line.substr(implicitBar, oo - implicitBar).find("libfoo.dll") + == std::string::npos) << line; +} + // ── #311: staging failures must stay readable through the output filter ── TEST(NinjaBackend, FilterKeepsStagingDiagnosticsAndFailedTarget) { From d4b3d8aaab0850d9063d6c7b097f61fd5f289e60 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Sat, 26 Sep 2026 08:19:54 +0800 Subject: [PATCH 06/26] ci: the mingw-cross job runs the DLL placement e2e and asserts each property --- .github/workflows/cross-build-test.yml | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index 0fabc96e..6a16b6f9 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -370,6 +370,26 @@ jobs: export MCPP_VENDORED_XLINGS="$XLINGS_BIN" bash tests/e2e/248_pack_library_fat_pe_leg.sh + # SPEC-007 R4.3: the engine places a Windows program's runtime DLLs + # beside it after its link, so a program started by hand finds them. + # Named here for the reason the steps above are: the Linux shards skip + # `# requires: mingw-cross wine`. Each property is held to the line the + # script prints only when that property was checked. + - name: "e2e: a Windows program finds its runtime DLLs beside it" + run: | + set -o pipefail + export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + log="$RUNNER_TEMP/797.log" + bash tests/e2e/797_a_windows_program_finds_its_dlls_beside_it.sh 2>&1 | tee "$log" + for line in \ + " ok: after the first build the program started by hand finds its DLL" \ + " ok: a build with nothing changed neither relinks nor places again" \ + " ok: a DLL replaced in its directory replaces the copy" \ + " ok: no system DLL is copied" \ + "PASS: a Windows program finds its runtime DLLs beside it"; do + grep -qxF "$line" "$log" || { echo "::error::797 did not print: $line"; exit 1; } + done + # ── windows → linux ─────────────────────────────────────────────────────── # The mirror of mingw-cross-wine. Two jobs because a Windows runner cannot # execute the ELF it produces; the artefact is handed to a Linux job and From f75c4c6a147087c82aec4c213ede2ed5990b3c67 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Sat, 26 Sep 2026 08:40:20 +0800 Subject: [PATCH 07/26] emit build-database plans every member, and a failing tool or program is a warning MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #699 item 1 (E1). `emit build-database --workspace` stopped its whole member loop at the first planning failure, discarding the sets of every member that had already planned; `mcpp build --workspace` never had this defect (continue-on-failure). The loop now plans every selected member on its own: a member whose planning fails contributes no set and one `error` diagnostic (`MCPP_BUILD_DATABASE_PLAN_FAILED` or `MCPP_OFFLINE_DOWNLOAD_REQUIRED`, the refusal code read right where that member's planning ends so an earlier member's recovery cannot relabel a later one's reason) whose `path` names its `mcpp.toml`, relative to the workspace root. `data` is present when at least one member planned; a workspace in which every member fails still omits it, since S2 has no partial outcome. The exit status is 1 whenever any diagnostic is an error, independent of `--format`. #699 item 2 (E2). Under `emit` (`plan_only`), a host tool that fails to build no longer costs the requesting member's plan: it is a warning, `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT`, naming the tool, its package and the first line of the failure. The tool build itself and its `check` actions are unchanged, and the requesting build program still receives the path the tool would have been published at (the store key already fixes it before the build runs). `mcpp build` is unaffected: the same failure still ends the build. #699 item 2 (E3). Under `emit`, a package whose build program fails (does not compile, exits non-zero, times out, or is refused) is described without that program's directives instead of costing its member: the manifest's own configuration, the toolchain, the module graph and the standard-library units are described as usual, with one error, `MCPP_BUILD_DATABASE_PROGRAM_FAILED`, whose `path` names its `build.mcpp`. Nothing from a failed run was ever applied (`run_build_program` returns before parsing directives on every failure path); a later failure that follows from the gap fails the member under E1's rule. Both `run_build_program` sites (a dependency's and the root's) carry the same branch, gated on `overrides.plan_only`. `PlanNote` gains a severity (default warning, so every existing note is unchanged) and a `path` (absolute when recorded, rewritten to the workspace-relative form `render()` uses for everything else). `emit`'s per-member diagnostics and `build_database::render`'s note handling and `watch` assembly (a failed member's `mcpp.toml` and `build.mcpp`, when present, join `watch` exactly as a planned member's do) change together. Design: .agents/docs/2026-09-26-compile-database-and-issue-699-design.md §4. --- src/build/build_database.cppm | 37 +++- src/build/prepare.cppm | 329 ++++++++++++++++++++++------------ src/cli/cmd_build.cppm | 120 ++++++++----- 3 files changed, 328 insertions(+), 158 deletions(-) diff --git a/src/build/build_database.cppm b/src/build/build_database.cppm index 429595aa..edc30d93 100644 --- a/src/build/build_database.cppm +++ b/src/build/build_database.cppm @@ -66,15 +66,23 @@ struct Rendered { nlohmann::json compileCommands = nlohmann::json::array(); std::vector watch; std::string inputsFingerprint; - // Conditions found while rendering; reported as warnings by the command. + // Conditions found while rendering, at the severity the command reports + // them: most are warnings the document is still complete despite, and a + // failed build program (#699 item 2, E3) is an error whose `path` names + // its `build.mcpp`, rewritten here to the workspace-relative form. std::vector notes; }; // The S1 document for `members`, and the inputs whose change changes it. +// `failedMemberRoots` is every selected workspace member whose planning +// failed (#699 item 1, E1): it contributes no set, but its `mcpp.toml` and +// `build.mcpp` (when present) still join `watch`, exactly as a planned +// member's do — fixing either file is what should make a consumer ask again. // `workspaceRoot` anchors the relative `watch` patterns; `selector` is the // command's own selection (target, toolchain, profile, members), which enters // the fingerprint because the same files answer differently under another one. Rendered render(std::span members, + std::span failedMemberRoots, const std::filesystem::path& workspaceRoot, std::string_view selector); @@ -372,6 +380,7 @@ std::optional recover_invocation(const std::vector& com } Rendered render(std::span members, + std::span failedMemberRoots, const std::filesystem::path& workspaceRoot, std::string_view selector) { Rendered r; @@ -544,7 +553,31 @@ Rendered render(std::span members, for (auto const& f : declared.files) watch_file(f); for (auto const& g : declared.globs) watch_glob(declared.root, g); } - for (auto const& note : ctx.planNotes) r.notes.push_back(note); + for (auto note : ctx.planNotes) { + // `note.path`, when set, is the absolute path a `plan_only` + // branch recorded (#699 item 2, E3); every other `path` in this + // document is relative to the workspace root, and a note's is + // rewritten to match before it reaches the command. + if (!note.path.empty()) { + const std::filesystem::path abs{note.path}; + if (auto rel = relative_to(abs, workspaceRoot); rel && !rel->empty()) + note.path = *rel; + else + note.path = native_string(abs.lexically_normal()); + } + r.notes.push_back(std::move(note)); + } + } + + // Every member whose planning failed still has its likely cause watched + // (#699 item 1, E1): editing its `mcpp.toml`, or the `build.mcpp` that may + // have failed, is what should make a consumer ask again for the members + // that could not be described this time. + for (auto const& failedRoot : failedMemberRoots) { + watch_file(failedRoot / "mcpp.toml"); + std::error_code ec; + if (std::filesystem::exists(failedRoot / "build.mcpp", ec)) + watch_file(failedRoot / "build.mcpp"); } r.database = nlohmann::json{ diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index c089c306..4bfefa69 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -100,6 +100,7 @@ import mcpp.pm.lock_io; import mcpp.version_req; import mcpp.ui; import mcpp.log; +import mcpp.wire; // Severity, for PlanNote (#699 item 2, E3) import mcpp.fallback.install_integrity; import mcpp.bmi_cache; import mcpp.project; @@ -1089,11 +1090,21 @@ export std::string_view cache_mode_name(CacheMode m) { } // A condition a planning pass reports instead of acting on (plan_only): the -// code is stable and the message is for people. Emitted as warning diagnostics -// by the command that asked for the plan. +// code is stable and the message is for people. Emitted as diagnostics by the +// command that asked for the plan, at the severity carried here — most notes +// are warnings the document is still complete despite (a lock that would +// change, a generated file left unmaterialized); a build program whose run +// failed under `plan_only` (#699 item 2, E3) is an error, because the sets it +// would have shaped are described without its directives. export struct PlanNote { std::string code; std::string message; + mcpp::wire::Severity severity = mcpp::wire::Severity::Warning; + // The absolute, native path of the file the condition is about (a + // package's `build.mcpp`), empty when the note names no file. + // `mcpp.build.build_database::render` rewrites it to the workspace- + // relative form every other `path` in the document uses. + std::string path; }; export struct BuildContext { @@ -10351,6 +10362,26 @@ prepare_build(bool print_fingerprint, continue; } + // #699 item 2 (E2): under `emit build-database` + // (`plan_only`), a host tool that fails to build is a + // warning, not a refusal that costs the whole plan — the + // requesting member is still worth describing, and its + // build program receives the path the tool would have + // been published at (`binOut`, fixed above before any of + // this runs). `mcpp build` is unchanged below: it still + // returns `std::unexpected` and the target fails. + auto host_tool_unbuilt = [&](std::string_view failure) { + // The first line only: a nested build's message can + // run to several, and the warning names the tool and + // its package, not the whole log. + const auto first = failure.substr(0, failure.find('\n')); + planNotes.push_back({"MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT", + std::format("host tool '{}' of package '{}' did not " + "build: {}", toolName, depName, first), + mcpp::wire::Severity::Warning}); + record(binOut); + }; + mcpp::ui::status("Building", std::format( "host tool {}:{} from {} v{} (once per package source and " "host toolchain)", depName, toolName, depName, @@ -10440,6 +10471,10 @@ prepare_build(bool print_fingerprint, /*includeDevDeps=*/false, /*extraTargets=*/{}, sub); if (!subCtx) { + if (overrides.plan_only) { + host_tool_unbuilt(subCtx.error()); + continue; + } return std::unexpected(std::format( "building host tool '{}:{}' failed: {}{}", depName, toolName, subCtx.error(), subContext())); @@ -10453,6 +10488,12 @@ prepare_build(bool print_fingerprint, if (lu.targetName == toolName) { goal = lu.output; break; } } if (goal.empty()) { + if (overrides.plan_only) { + host_tool_unbuilt("produced no link unit — its " + "required_features may not be satisfiable on " + "this platform"); + continue; + } return std::unexpected(std::format( "host tool '{}:{}' produced no link unit — its " "required_features may not be satisfiable on this " @@ -10471,6 +10512,10 @@ prepare_build(bool print_fingerprint, bopt.verbose = true; auto br = be->build(subCtx->plan, bopt); if (!br) { + if (overrides.plan_only) { + host_tool_unbuilt(br.error().message); + continue; + } auto diag = br.error().diagnosticOutput; if (diag.empty()) diag = "(the inner build produced no diagnostic " @@ -10481,6 +10526,11 @@ prepare_build(bool print_fingerprint, subContext(), diag)); } if (br->exitCode != 0) { + if (overrides.plan_only) { + host_tool_unbuilt(std::format( + "build exited with {}", br->exitCode)); + continue; + } return std::unexpected(std::format( "building host tool '{}:{}' failed (exit {}){}", depName, toolName, br->exitCode, subContext())); @@ -10492,6 +10542,11 @@ prepare_build(bool print_fingerprint, std::error_code cpEc; auto produced = subCtx->plan.outputDir / goal; if (!std::filesystem::exists(produced, cpEc)) { + if (overrides.plan_only) { + host_tool_unbuilt(std::format( + "built but '{}' is missing", produced.string())); + continue; + } return std::unexpected(std::format( "host tool '{}:{}' built but '{}' is missing", depName, toolName, produced.string())); @@ -10503,6 +10558,11 @@ prepare_build(bool print_fingerprint, std::filesystem::copy_file(produced, tmp, std::filesystem::copy_options::overwrite_existing, cpEc); if (cpEc) { + if (overrides.plan_only) { + host_tool_unbuilt(std::format( + "staging failed: {}", cpEc.message())); + continue; + } return std::unexpected(std::format( "staging host tool '{}:{}' failed: {}", depName, toolName, cpEc.message())); @@ -10514,6 +10574,11 @@ prepare_build(bool print_fingerprint, std::filesystem::perm_options::add, cpEc); std::filesystem::rename(tmp, binOut, cpEc); if (cpEc) { + if (overrides.plan_only) { + host_tool_unbuilt(std::format( + "publishing failed: {}", cpEc.message())); + continue; + } return std::unexpected(std::format( "publishing host tool '{}:{}' failed: {}", depName, toolName, cpEc.message())); @@ -10610,6 +10675,24 @@ prepare_build(bool print_fingerprint, pkg.manifest, pkg.root, host->first, host->second, pkg.manifest.cppStandard, bpEnv); !r) { + // #699 item 2 (E3): under `emit build-database` (`plan_only`), + // a failing build program describes its package without that + // program's directives, instead of costing the whole plan — + // the manifest's own configuration, the toolchain and the + // module graph are still worth describing. Nothing is applied + // either way: `run_build_program` returns before + // `Directives::apply` on every failure path. A later failure + // that follows from the missing directives (a source the + // program would have added, say) fails the member under the + // ordinary rule (E1). + if (overrides.plan_only) { + planNotes.push_back({"MCPP_BUILD_DATABASE_PROGRAM_FAILED", + std::format("dependency '{}': {}", + pkg.manifest.package.name, r.error()), + mcpp::wire::Severity::Error, + (pkg.root / "build.mcpp").string()}); + continue; + } return std::unexpected(std::format( "dependency '{}': {}", pkg.manifest.package.name, r.error())); } @@ -12399,122 +12482,136 @@ prepare_build(bool print_fingerprint, // measured: `run-A.sh run-B.sh `). const auto runnerBeforeRoot = bcRoot.runner; const auto namedBeforeRoot = bcRoot.namedRunners; - if (auto bp = mcpp::build::run_build_program( - *m, *root, host->first, host->second, - m->cppStandard, bpEnv); - !bp) { + auto bp = mcpp::build::run_build_program( + *m, *root, host->first, host->second, + m->cppStandard, bpEnv); + if (!bp && !overrides.plan_only) { return std::unexpected(bp.error()); } - // THE SAME RULE THE DEPENDENCIES ARE HELD TO, WITH THE ROOT AS A PARTY. - // Two suppliers of one runner are refused naming both, and the - // manifest is the way to choose: a `[target.]` runner the - // project writes outranks every supplied one where the runner is - // looked up, so a name the manifest declares is not refused here. - { - const auto rowKey = [&]() -> std::string { - if (!tc) return {}; - auto t = mcpp::toolchain::triple::parse(tc->targetTriple); - return t ? t->str() : tc->targetTriple; - }(); - const auto row = m->targetOverrides.find(rowKey); - const auto manifestNames = [&](std::string_view name) { - if (row == m->targetOverrides.end()) return false; - if (name.empty()) return !row->second.runner.empty(); - return row->second.namedRunners.contains(std::string(name)); - }; - if (!runnerProvider.empty() && !runnerBeforeRoot.empty() - && bcRoot.runner.size() > runnerBeforeRoot.size() - && !manifestNames({})) { - return std::unexpected(std::format( - "the dependency '{}' and this project's build program both " - "supply the runner for this target, and the two would be " - "joined into one argv.\n" - " Drop one of them, or state the runner in " - "[target.{}].runner.", - runnerProvider, rowKey)); - } - for (auto const& [name, nr] : bcRoot.namedRunners) { - auto before = namedBeforeRoot.find(name); - auto who = namedRunnerProvider.find(name); - if (before == namedBeforeRoot.end() || before->second.argv.empty() - || who == namedRunnerProvider.end() || who->second.empty()) - continue; - if (nr.argv.size() <= before->second.argv.size()) continue; - if (manifestNames(name)) continue; - return std::unexpected(std::format( - "the dependency '{}' and this project's build program both " - "supply a runner named '{}' for this target, and the two " - "would be joined into one argv.\n" - " Drop one of them, or state it in " - "[target.{}.runners].{}.", - who->second, name, rowKey, name)); - } - } - auto& pkg0 = packages[0]; - // Compile-visible tail → privateBuild: the shared fold (same owner - // as the dep loop; the root's TUs read privateBuild). - foldDirectiveTailIntoPrivateBuild(pkg0, *m, mark); - // Before the source residues are mirrored below: adopting an action's - // outputs APPENDS to bcRoot.sources, and those appends must be inside - // the tail that gets copied into the packages[0] snapshot the scan reads. - adoptActionOutputs(*m, *root, ractN); - // The root's build program has spoken; a floor it stated is checked - // now, with the facts every package (it included) established. - if (auto err = checkVersionFloors(); err) return std::unexpected(*err); - // Root residues — apply() mutated *m, but packages[0].manifest is a - // value-copy snapshot taken at makePackageRoot, so everything the - // scan/fingerprint read from the snapshot needs the tail mirrored: - // sources → the scan walks packages[0].manifest, not *m. - pkg0.manifest.buildConfig.sources.insert( - pkg0.manifest.buildConfig.sources.end(), - bcRoot.sources.begin() + rsrcN, bcRoot.sources.end()); - pkg0.manifest.modules.sources.insert( - pkg0.manifest.modules.sources.end(), - m->modules.sources.begin() + rmodN, m->modules.sources.end()); - // Fingerprint metadata (canonical_package_build_metadata folds - // packages[].manifest.buildConfig) — mirror the flag/include tails, - // as the old pre-snapshot ordering implicitly did. - pkg0.manifest.buildConfig.cflags.insert( - pkg0.manifest.buildConfig.cflags.end(), - bcRoot.cflags.begin() + static_cast(mark.cflags), - bcRoot.cflags.end()); - pkg0.manifest.buildConfig.cxxflags.insert( - pkg0.manifest.buildConfig.cxxflags.end(), - bcRoot.cxxflags.begin() + static_cast(mark.cxxflags), - bcRoot.cxxflags.end()); - pkg0.manifest.buildConfig.includeDirs.insert( - pkg0.manifest.buildConfig.includeDirs.end(), - bcRoot.includeDirs.begin() + static_cast(mark.includeDirs), - bcRoot.includeDirs.end()); - pkg0.manifest.buildConfig.includeDirsAfter.insert( - pkg0.manifest.buildConfig.includeDirsAfter.end(), - bcRoot.includeDirsAfter.begin() - + static_cast(mark.includeDirsAfter), - bcRoot.includeDirsAfter.end()); - // Link flags → the final link reads *m (already applied); keep the - // linkUsage snapshot and fingerprint metadata equivalent too. - pkg0.linkUsage.ldflags.insert(pkg0.linkUsage.ldflags.end(), - bcRoot.ldflags.begin() + rldN, bcRoot.ldflags.end()); - pkg0.manifest.buildConfig.ldflags.insert( - pkg0.manifest.buildConfig.ldflags.end(), - bcRoot.ldflags.begin() + rldN, bcRoot.ldflags.end()); - // #622 A4: `mcpp::deploy()` residue → `packages[0].manifest`, the - // object `resolve_runtime_contract` (plan.cppm) actually reads. - // Without this mirror a directive-sourced deploy entry lands in `*m` - // and nowhere the planner looks — the same gap this block already - // closes for sources/flags, one more field wide. - pkg0.manifest.runtimeConfig.linkIntent.deploy.insert( - pkg0.manifest.runtimeConfig.linkIntent.deploy.end(), - m->runtimeConfig.linkIntent.deploy.begin() + static_cast(rdeployN), - m->runtimeConfig.linkIntent.deploy.end()); - // `mcpp::runtime_library_dir()` residue → `packages[0].manifest`, the - // same object and the same reason as the `deploy` mirror above: without - // it a directive-sourced entry lands in `*m` and `resolve_runtime_contract` - // never looks there. - pkg0.manifest.runtimeConfig.libraryDirs.insert( - pkg0.manifest.runtimeConfig.libraryDirs.end(), - m->runtimeConfig.libraryDirs.begin() + static_cast(rlibDirN), - m->runtimeConfig.libraryDirs.end()); + // #699 item 2 (E3): under `emit build-database` (`plan_only`), a + // failing root build program describes the package without its + // directives rather than costing the whole plan. Every mirror below + // reads what the program would have added to `*m`, so skipping + // straight past it (nothing runs on this path) is what "without its + // directives" means; a later failure that follows from the gap + // fails the member under the ordinary rule (E1). + if (!bp) { + planNotes.push_back({"MCPP_BUILD_DATABASE_PROGRAM_FAILED", + bp.error(), mcpp::wire::Severity::Error, + (*root / "build.mcpp").string()}); + } + if (bp) { + // THE SAME RULE THE DEPENDENCIES ARE HELD TO, WITH THE ROOT AS A PARTY. + // Two suppliers of one runner are refused naming both, and the + // manifest is the way to choose: a `[target.]` runner the + // project writes outranks every supplied one where the runner is + // looked up, so a name the manifest declares is not refused here. + { + const auto rowKey = [&]() -> std::string { + if (!tc) return {}; + auto t = mcpp::toolchain::triple::parse(tc->targetTriple); + return t ? t->str() : tc->targetTriple; + }(); + const auto row = m->targetOverrides.find(rowKey); + const auto manifestNames = [&](std::string_view name) { + if (row == m->targetOverrides.end()) return false; + if (name.empty()) return !row->second.runner.empty(); + return row->second.namedRunners.contains(std::string(name)); + }; + if (!runnerProvider.empty() && !runnerBeforeRoot.empty() + && bcRoot.runner.size() > runnerBeforeRoot.size() + && !manifestNames({})) { + return std::unexpected(std::format( + "the dependency '{}' and this project's build program both " + "supply the runner for this target, and the two would be " + "joined into one argv.\n" + " Drop one of them, or state the runner in " + "[target.{}].runner.", + runnerProvider, rowKey)); + } + for (auto const& [name, nr] : bcRoot.namedRunners) { + auto before = namedBeforeRoot.find(name); + auto who = namedRunnerProvider.find(name); + if (before == namedBeforeRoot.end() || before->second.argv.empty() + || who == namedRunnerProvider.end() || who->second.empty()) + continue; + if (nr.argv.size() <= before->second.argv.size()) continue; + if (manifestNames(name)) continue; + return std::unexpected(std::format( + "the dependency '{}' and this project's build program both " + "supply a runner named '{}' for this target, and the two " + "would be joined into one argv.\n" + " Drop one of them, or state it in " + "[target.{}.runners].{}.", + who->second, name, rowKey, name)); + } + } + auto& pkg0 = packages[0]; + // Compile-visible tail → privateBuild: the shared fold (same owner + // as the dep loop; the root's TUs read privateBuild). + foldDirectiveTailIntoPrivateBuild(pkg0, *m, mark); + // Before the source residues are mirrored below: adopting an action's + // outputs APPENDS to bcRoot.sources, and those appends must be inside + // the tail that gets copied into the packages[0] snapshot the scan reads. + adoptActionOutputs(*m, *root, ractN); + // The root's build program has spoken; a floor it stated is checked + // now, with the facts every package (it included) established. + if (auto err = checkVersionFloors(); err) return std::unexpected(*err); + // Root residues — apply() mutated *m, but packages[0].manifest is a + // value-copy snapshot taken at makePackageRoot, so everything the + // scan/fingerprint read from the snapshot needs the tail mirrored: + // sources → the scan walks packages[0].manifest, not *m. + pkg0.manifest.buildConfig.sources.insert( + pkg0.manifest.buildConfig.sources.end(), + bcRoot.sources.begin() + rsrcN, bcRoot.sources.end()); + pkg0.manifest.modules.sources.insert( + pkg0.manifest.modules.sources.end(), + m->modules.sources.begin() + rmodN, m->modules.sources.end()); + // Fingerprint metadata (canonical_package_build_metadata folds + // packages[].manifest.buildConfig) — mirror the flag/include tails, + // as the old pre-snapshot ordering implicitly did. + pkg0.manifest.buildConfig.cflags.insert( + pkg0.manifest.buildConfig.cflags.end(), + bcRoot.cflags.begin() + static_cast(mark.cflags), + bcRoot.cflags.end()); + pkg0.manifest.buildConfig.cxxflags.insert( + pkg0.manifest.buildConfig.cxxflags.end(), + bcRoot.cxxflags.begin() + static_cast(mark.cxxflags), + bcRoot.cxxflags.end()); + pkg0.manifest.buildConfig.includeDirs.insert( + pkg0.manifest.buildConfig.includeDirs.end(), + bcRoot.includeDirs.begin() + static_cast(mark.includeDirs), + bcRoot.includeDirs.end()); + pkg0.manifest.buildConfig.includeDirsAfter.insert( + pkg0.manifest.buildConfig.includeDirsAfter.end(), + bcRoot.includeDirsAfter.begin() + + static_cast(mark.includeDirsAfter), + bcRoot.includeDirsAfter.end()); + // Link flags → the final link reads *m (already applied); keep the + // linkUsage snapshot and fingerprint metadata equivalent too. + pkg0.linkUsage.ldflags.insert(pkg0.linkUsage.ldflags.end(), + bcRoot.ldflags.begin() + rldN, bcRoot.ldflags.end()); + pkg0.manifest.buildConfig.ldflags.insert( + pkg0.manifest.buildConfig.ldflags.end(), + bcRoot.ldflags.begin() + rldN, bcRoot.ldflags.end()); + // #622 A4: `mcpp::deploy()` residue → `packages[0].manifest`, the + // object `resolve_runtime_contract` (plan.cppm) actually reads. + // Without this mirror a directive-sourced deploy entry lands in `*m` + // and nowhere the planner looks — the same gap this block already + // closes for sources/flags, one more field wide. + pkg0.manifest.runtimeConfig.linkIntent.deploy.insert( + pkg0.manifest.runtimeConfig.linkIntent.deploy.end(), + m->runtimeConfig.linkIntent.deploy.begin() + static_cast(rdeployN), + m->runtimeConfig.linkIntent.deploy.end()); + // `mcpp::runtime_library_dir()` residue → `packages[0].manifest`, the + // same object and the same reason as the `deploy` mirror above: without + // it a directive-sourced entry lands in `*m` and `resolve_runtime_contract` + // never looks there. + pkg0.manifest.runtimeConfig.libraryDirs.insert( + pkg0.manifest.runtimeConfig.libraryDirs.end(), + m->runtimeConfig.libraryDirs.begin() + static_cast(rlibDirN), + m->runtimeConfig.libraryDirs.end()); + } } // ── Every device source must reach some action ───────────────────────── diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index c9589c05..f88ed578 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -303,10 +303,14 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) } return 0; }; - // A failure is one envelope with diagnostics and no `data` (S2 0.2.0 §3.4: - // a command without data has failed), and exit 1. - auto failed = [&](std::string code, std::string message) -> int { - diagnostics.push_back({std::move(code), Severity::Error, std::move(message)}); + // A failure with nothing to describe is one envelope with diagnostics and + // no `data` (S2 0.2.0 §3.4: a command without data has failed), and exit + // 1. `fail_no_data` finishes with whatever `diagnostics` already holds — + // used once a single diagnostic is pushed onto it (`failed`, below, for a + // usage error decided before any member is tried) and once every + // selected member's own planning has failed in turn (#699 item 1, E1: a + // workspace where nothing planned still names each member's own reason). + auto fail_no_data = [&]() -> int { if (!envelope) { for (auto const& d : diagnostics) std::println(stderr, "{}: {}", @@ -322,6 +326,10 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) (void)publish(text); return 1; }; + auto failed = [&](std::string code, std::string message) -> int { + diagnostics.push_back({std::move(code), Severity::Error, std::move(message)}); + return fail_no_data(); + }; auto root = mcpp::project::find_manifest_root(std::filesystem::current_path()); if (!root) @@ -340,21 +348,46 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) requests.emplace_back(std::string{}, ov); } + // An offline plan that needs a download is not a defect of the project, and + // a client that plans offline by default (an editor) has to tell the two + // apart without reading the message (#648 A1). The code is taken right + // where a member's planning ends: `take()` clears the sink, so a refusal + // recorded by a member that failed cannot relabel a later member's own + // reason (the per-member analogue of the rule this used to apply once). + auto plan_failure_code = [&] { + const bool offline = mcpp::platform::env::offline_mode() + || mcpp::platform::env::no_auto_install(); + const bool wasOfflineDownload = mcpp::build::refusal::take() + == mcpp::build::refusal::Code::OfflineDownloadRequired; + return std::string(wasOfflineDownload && offline + ? "MCPP_OFFLINE_DOWNLOAD_REQUIRED" : "MCPP_BUILD_DATABASE_PLAN_FAILED"); + }; + std::vector contexts; std::vector workDirs; std::vector prefixes; std::vector>> testDiscovery; - std::optional planError; + // The root of every member whose planning failed: `mcpp.toml` and + // `build.mcpp` (when it exists) join `watch` exactly as a planned + // member's do (render(), below), so an edit that might fix the failure is + // what wakes a consumer to ask again. + std::vector failedMemberRoots; { // Planning narrates on stdout and may start programs that inherit it; // the document is printed after this scope, alone. mcpp::platform::terminal::StdoutToStderr narration; for (auto& [member, mo] : requests) { + const auto memberRoot = member.empty() ? *root : *root / member; + const auto memberPath = member.empty() ? std::string("mcpp.toml") + : member + "/mcpp.toml"; auto discovered = mcpp::build::discover_test_targets(*root, mo.package_filter); if (!discovered) { - planError = member.empty() ? discovered.error() - : std::format("{}: {}", member, discovered.error()); - break; + diagnostics.push_back({plan_failure_code(), Severity::Error, + member.empty() ? discovered.error() + : std::format("{}: {}", member, discovered.error()), + memberPath}); + failedMemberRoots.push_back(memberRoot); + continue; } // As `--configure-only`: tests and dev-dependencies are part of the // surface an editor needs. @@ -368,9 +401,12 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) includeDevDeps, std::move(discovered->targets), mo); if (!ctx) { - planError = member.empty() ? ctx.error() - : std::format("{}: {}", member, ctx.error()); - break; + diagnostics.push_back({plan_failure_code(), Severity::Error, + member.empty() ? ctx.error() + : std::format("{}: {}", member, ctx.error()), + memberPath}); + failedMemberRoots.push_back(memberRoot); + continue; } contexts.push_back(std::move(*ctx)); workDirs.push_back(mo.work_dir); @@ -378,20 +414,12 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) testDiscovery.push_back(std::move(discovery)); } } - // An offline plan that needs a download is not a defect of the project, and - // a client that plans offline by default (an editor) has to tell the two - // apart without reading the message (#648 A1). The code is taken only while - // the run is offline, so a refusal recorded on a path that recovered cannot - // relabel an unrelated failure. - if (planError) { - const bool offline = mcpp::platform::env::offline_mode() - || mcpp::platform::env::no_auto_install(); - if (mcpp::build::refusal::take() - == mcpp::build::refusal::Code::OfflineDownloadRequired - && offline) - return failed("MCPP_OFFLINE_DOWNLOAD_REQUIRED", *planError); - return failed("MCPP_BUILD_DATABASE_PLAN_FAILED", *planError); - } + // Every selected member was planned independently (#699 item 1, E1): one + // that failed contributed its own diagnostic above and nothing else. + // Only when none of them planned is there nothing left to describe — S2 + // has no partial outcome, so `data` is present or it is not. + if (contexts.empty()) + return fail_no_data(); // The lock this planning produced, against the project's. The project's is // never written; a difference is reported. @@ -430,35 +458,47 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) spec, ov.target_triple, mcpp::platform::env::get("MCPP_TOOLCHAIN").value_or(""), ov.profile, ov.features, ov.capabilities, ov.accel, ov.force_static, ov.package_filter, parsed.is_flag_set("workspace")); - auto rendered = mcpp::build::database::render(members, *root, selector); + auto rendered = mcpp::build::database::render(members, failedMemberRoots, + *root, selector); + // A note's severity is its own (E3's program-failure note is an error; + // every other note today is a warning) and its `path`, when set, already + // names the file relative to the workspace root — render() rewrote it. for (auto& note : rendered.notes) - diagnostics.push_back({std::move(note.code), Severity::Warning, - std::move(note.message)}); + diagnostics.push_back({std::move(note.code), note.severity, + std::move(note.message), std::move(note.path)}); auto document = spec == "s1" ? std::move(rendered.database) : std::move(rendered.compileCommands); + // The exit status is 1 whenever an error is present (#699 item 1, E1; + // item 2, E3) even though `data` is: a workspace member's own failure, or + // a build program's, is still a failure this command reports through its + // exit code, only not by withholding the sets that DID plan. + const bool hasError = std::ranges::any_of(diagnostics, + [](const Diagnostic& d) { return d.severity == Severity::Error; }); if (!envelope) { for (auto const& d : diagnostics) std::println(stderr, "{}: {}", mcpp::wire::severity_name(d.severity), d.message); - return publish(document.dump(2) + "\n"); + if (const auto rc = publish(document.dump(2) + "\n"); rc != 0) return rc; + return hasError ? 1 : 0; } std::vector effects{Effect::ReadProject, Effect::WriteGlobalCache}; if (ranBuildPrograms) effects.push_back(Effect::ExecBuildScript); nlohmann::json specJson{{"name", spec}}; if (spec == "s1") specJson["version"] = std::string(mcpp::build::database::kProfileVersion); - return publish(mcpp::wire::to_json(mcpp::wire::Envelope{ - .kind = "mcpp.build-database", - .effects = std::move(effects), - .data = nlohmann::json{ - {"spec", std::move(specJson)}, - {"database", std::move(document)}, - {"watch", std::move(rendered.watch)}, - {"inputs-fingerprint", std::move(rendered.inputsFingerprint)}, - }, - .diagnostics = diagnostics, - }).dump(2) + "\n"); + if (const auto rc = publish(mcpp::wire::to_json(mcpp::wire::Envelope{ + .kind = "mcpp.build-database", + .effects = std::move(effects), + .data = nlohmann::json{ + {"spec", std::move(specJson)}, + {"database", std::move(document)}, + {"watch", std::move(rendered.watch)}, + {"inputs-fingerprint", std::move(rendered.inputsFingerprint)}, + }, + .diagnostics = diagnostics, + }).dump(2) + "\n"); rc != 0) return rc; + return hasError ? 1 : 0; } export int cmd_run(const mcpplibs::cmdline::ParsedArgs& parsed, From e1d4361c31262068db1271e58a415e6b9140d097 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Sat, 26 Sep 2026 08:40:27 +0800 Subject: [PATCH 08/26] docs: emit build-database's per-member failures and the two new codes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit docs/50 §8 and its zh counterpart now describe the shape #699's fix gives `mcpp.build-database`: `emit` plans every selected member on its own, a failed member's diagnostic carries `path`, `data` is present whenever one member planned and absent only when none did, and the exit status is 1 whenever any diagnostic is an error even though `data` is present. The two new codes join the table: `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT` (warning) and `MCPP_BUILD_DATABASE_PROGRAM_FAILED` (error). SPEC-005 R2.5 states the host-tool warning and that its `check` actions and `mcpp build`'s own failure are unchanged; R5.2 states member containment, the per-member `path`, the program-failure description and the exit-code rule. Header, version line and R3.7/R3.8/R4.1 are untouched (T1's). --- docs/50-machine-output.md | 56 ++++++++++++++++++++++++++---------- docs/specs/build-database.md | 21 ++++++++++---- docs/zh/50-machine-output.md | 39 +++++++++++++++++++------ 3 files changed, 87 insertions(+), 29 deletions(-) diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index 48a3e61a..4c7f2040 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -463,21 +463,47 @@ whatever it would print to `` instead. The content of the document, the no-write guarantee and the `watch` rules are [SPEC-005](specs/build-database.md). -A failure omits `data` and exits 1, with the diagnostic code -`MCPP_BUILD_DATABASE_NO_PROJECT` outside a project, -`MCPP_OFFLINE_DOWNLOAD_REQUIRED` when an offline plan (`--offline`, -`MCPP_OFFLINE`, `MCPP_NO_AUTO_INSTALL`) needs something that has to be -downloaded (a toolchain, a package, a git revision or the package index; the -message names the first one), or `MCPP_BUILD_DATABASE_PLAN_FAILED` when planning -fails for any other reason. The first of the three is not a defect of the -project: one run without `--offline` removes it. Warnings leave the -document in place: - -| code | | -|---|---| -| `MCPP_LOCK_WOULD_CHANGE` | the resolution differs from the project's `mcpp.lock`, which the command does not write | -| `MCPP_GENERATED_FILE_NOT_MATERIALIZED` | a root `[build] generated_files` entry is missing or stale on disk, and the command does not write it | -| `MCPP_BUILD_DATABASE_STD_UNIT_UNDESCRIBED` | no standard-library build command names its module source, so that unit is not listed | +`emit` plans every selected member on its own (#699 item 1): one member's +planning failure does not cost its siblings'. Outside a project, or when +every selected member fails to plan, the envelope omits `data` and exits 1, +with one diagnostic per failed member: `MCPP_BUILD_DATABASE_NO_PROJECT` +outside a project; `MCPP_OFFLINE_DOWNLOAD_REQUIRED` when an offline plan +(`--offline`, `MCPP_OFFLINE`, `MCPP_NO_AUTO_INSTALL`) needs something that has +to be downloaded (a toolchain, a package, a git revision or the package +index; the message names the first one); or `MCPP_BUILD_DATABASE_PLAN_FAILED` +when planning fails for any other reason. The offline code is not a defect +of the project: one run without `--offline` removes it. Each member's +diagnostic carries `path`, that member's `mcpp.toml` relative to the +workspace root. + +When at least one selected member planned, `data` is present and describes +every member that did: a member that failed contributes no set, one `error` +diagnostic as above, and its `mcpp.toml` and `build.mcpp` (when present) join +`watch`. The exit status is still 1 whenever any diagnostic is an error, so a +consumer reads three outcomes structurally — no `data`; `data` with `error` +diagnostics, describing everything except what they name; `data` with none — +without parsing a message. + +A package whose build program fails (#699 item 2) is described without that +program's directives: the manifest's own configuration, the toolchain, the +module graph and the standard-library units are described as usual, and one +`error` diagnostic, `MCPP_BUILD_DATABASE_PROGRAM_FAILED`, names it, with +`path` naming its `build.mcpp`. A later failure that follows from the missing +directives fails the whole member instead, under the rule above. A host tool +a package requested that fails to build is a warning instead, +`MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT`, naming the tool, its package and the +first line of the failure; planning continues, and a build program that only +names the tool configures as it would after a successful build. `mcpp build` +is unaffected by either: a build program or a host tool that fails there +still fails the build. + +| code | severity | | +|---|---|---| +| `MCPP_LOCK_WOULD_CHANGE` | warning | the resolution differs from the project's `mcpp.lock`, which the command does not write | +| `MCPP_GENERATED_FILE_NOT_MATERIALIZED` | warning | a root `[build] generated_files` entry is missing or stale on disk, and the command does not write it | +| `MCPP_BUILD_DATABASE_STD_UNIT_UNDESCRIBED` | warning | no standard-library build command names its module source, so that unit is not listed | +| `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT` | warning | a requested host tool failed to build; the tool is still built and its `check` actions still run | +| `MCPP_BUILD_DATABASE_PROGRAM_FAILED` | error | a build program failed; its package is described without its directives | `--protocol-version` declares `init-mcpp-home`, `read-project`, `network`, `write-global-cache` and `exec-build-script` for the command, and never diff --git a/docs/specs/build-database.md b/docs/specs/build-database.md index 60b42f3e..3987cbbd 100644 --- a/docs/specs/build-database.md +++ b/docs/specs/build-database.md @@ -53,7 +53,10 @@ Database 定义,本规范不重复它们的字段定义,只规定 mcpp 作为生 每个输出一条警告 `MCPP_GENERATED_FILE_NOT_MATERIALIZED`。**已实现** - **R2.5** 构建程序照常运行,工作目录为包根,与 `mcpp build` 相同;构建程序在 `MCPP_OUT_DIR` 之外写入的内容不在本保证之内。依赖提供的宿主工具照常构建到全局 - 工具库。**已实现** + 工具库,它声明的 `check` 动作照常运行。构建失败的宿主工具在本命令下降级为 + 警告 `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT`,消息点名工具、其所属包与失败信息 + 的第一行;规划继续,请求该工具的构建程序收到的是该工具本应发布到的路径。 + `mcpp build` 不受影响,宿主工具构建失败在其中仍使目标失败。**已实现** - **R2.6** `mcpp --protocol-version` 为这条命令声明 `init-mcpp-home`、`read-project`、 `network`、`write-global-cache` 与 `exec-build-script`,不声明 `write-project`。 **已实现** @@ -145,11 +148,19 @@ mcpp 输出的 S1 文档满足 S1 等级 2,不输出 `ide.options`。等级 3 - **R5.1** `kind` 为 `mcpp.build-database`,`kindVersion` 为 1。`data` 含 `spec` (`{"name": "s1", "version": "0.2.0"}` 或 `{"name": "compile-commands"}`)、 `database`、`watch` 与 `inputs-fingerprint`。**已实现** -- **R5.2** 失败时信封不含 `data`,`diagnostics` 至少含一条 `error`,退出码为 1:不在 - 工程中为 `MCPP_BUILD_DATABASE_NO_PROJECT`;离线运行而规划需要下载时为 +- **R5.2** 命令独立规划每一个被选中的成员:一个成员规划失败只影响它自己,不影响 + 其余成员的集合(#699 第 1 项)。规划失败的成员不贡献任何集合,只贡献一条 `error` + 诊断,`path` 为该成员的 `mcpp.toml`,相对工作区根目录;诊断码为:不在工程中时 + `MCPP_BUILD_DATABASE_NO_PROJECT`;该成员的规划因离线而需要下载时 `MCPP_OFFLINE_DOWNLOAD_REQUIRED`,消息指出需要下载的第一项;其他规划失败为 - `MCPP_BUILD_DATABASE_PLAN_FAILED`。工作区中消息指出成员;任一成员规划失败,整次命令 - 失败。**已实现**(离线诊断码:mcpp >= 2026.9.16.1) + `MCPP_BUILD_DATABASE_PLAN_FAILED`。`data` 在至少一个被选中的成员规划成功时出现, + 并描述每一个规划成功的成员;被选中的成员全部规划失败时,信封不含 `data`。规划成功 + 的成员中,构建程序失败的包被描述为不含该程序产生的指令(清单自身的配置、工具链、 + 模块图与标准库单元仍照常描述),`diagnostics` 另有一条 `error`, + `MCPP_BUILD_DATABASE_PROGRAM_FAILED`,`path` 为该包的 `build.mcpp`;后续失败若是 + 由缺失的指令引起,则按前一条规则使整个成员失败。只要 `diagnostics` 中有一条 + `error`,退出码就是 1,无论 `data` 是否出现。**已实现**(离线诊断码: + mcpp >= 2026.9.16.1;成员独立规划、`path` 与构建程序失败的描述:mcpp >= 2026.9.27.1) - **R5.3** 信封的 `effects` 为 `read-project` 与 `write-global-cache`,运行了构建程序时 另有 `exec-build-script`,本次运行启动过网络子进程(索引刷新、安装、git 远程操作, 失败或超时的也算)时另有 `network`。**已实现**(`network`:mcpp >= 2026.9.16.1) diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index bc41c37a..20631659 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -432,18 +432,39 @@ mcpp emit build-database [--spec s1|compile-commands] --format json 写入 ``。文档的内容、不写入项目目录这条保证,以及 `watch` 的规则,见 [SPEC-005](../specs/build-database.md)。 -失败时省略 `data` 并以 1 退出,诊断码为:不在项目中时是 +`emit` 独立规划每一个被选中的成员(#699 第 1 项):一个成员的规划失败不会 +连累它的兄弟成员。不在项目中,或者被选中的成员全部规划失败时,信封省略 +`data` 并以 1 退出,每个失败的成员各带一条诊断:不在项目中是 `MCPP_BUILD_DATABASE_NO_PROJECT`;离线规划(`--offline`、`MCPP_OFFLINE`、 `MCPP_NO_AUTO_INSTALL`)需要下载某样东西(工具链、包、git 修订,或包索引, 消息会指出第一个)时是 `MCPP_OFFLINE_DOWNLOAD_REQUIRED`;因其他原因规划失败 -时是 `MCPP_BUILD_DATABASE_PLAN_FAILED`。三者中的第一种不是项目的缺陷:不带 -`--offline` 再运行一次即可消除它。警告不影响文档本身: - -| 诊断码 | | -|---|---| -| `MCPP_LOCK_WOULD_CHANGE` | 解析结果与项目的 `mcpp.lock` 不一致,命令不写这个文件 | -| `MCPP_GENERATED_FILE_NOT_MATERIALIZED` | 根包 `[build] generated_files` 中的某个文件缺失或内容已过期,命令不写这个文件 | -| `MCPP_BUILD_DATABASE_STD_UNIT_UNDESCRIBED` | 没有任何标准库构建命令点名它的模块源文件,该单元因此不被列出 | +时是 `MCPP_BUILD_DATABASE_PLAN_FAILED`。离线这一种不是项目的缺陷:不带 +`--offline` 再运行一次即可消除它。每个成员的诊断都带 `path`,即该成员的 +`mcpp.toml`,相对工作区根目录。 + +只要有一个被选中的成员规划成功,`data` 就会出现,并描述每一个规划成功的 +成员:规划失败的成员不贡献任何集合,只贡献上面那样一条 `error` 诊断,它的 +`mcpp.toml` 与存在时的 `build.mcpp` 一并加入 `watch`。只要诊断里有一条是 +`error`,退出码依然是 1——因此消费方靠结构就能读出三种结果:没有 `data`; +`data` 伴随若干 `error` 诊断,描述了诊断所指之外的一切;`data` 且没有 +`error`,不需要解析消息文本。 + +构建程序失败的包(#699 第 2 项)会被描述为不含该程序产生的指令:清单自身 +的那部分配置、工具链、模块图与标准库单元仍照常描述,另附一条 `error` 诊断 +`MCPP_BUILD_DATABASE_PROGRAM_FAILED` 点名它,`path` 为它的 `build.mcpp`。若 +后续失败是由缺失的指令引起的,则按上面的规则使整个成员失败。包请求的宿主 +工具构建失败则降级为警告 `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT`,点名工具、 +所属包与失败信息的第一行;规划继续进行,只点名该工具而不运行它的构建程序 +会像该工具构建成功时一样完成配置。这两者都不影响 `mcpp build`:构建程序或 +宿主工具在其中失败仍然会使构建失败。 + +| 诊断码 | 严重级别 | | +|---|---|---| +| `MCPP_LOCK_WOULD_CHANGE` | 警告 | 解析结果与项目的 `mcpp.lock` 不一致,命令不写这个文件 | +| `MCPP_GENERATED_FILE_NOT_MATERIALIZED` | 警告 | 根包 `[build] generated_files` 中的某个文件缺失或内容已过期,命令不写这个文件 | +| `MCPP_BUILD_DATABASE_STD_UNIT_UNDESCRIBED` | 警告 | 没有任何标准库构建命令点名它的模块源文件,该单元因此不被列出 | +| `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT` | 警告 | 被请求的宿主工具构建失败;该工具仍会被构建,它的 `check` 动作仍会运行 | +| `MCPP_BUILD_DATABASE_PROGRAM_FAILED` | 错误 | 构建程序失败;它所属的包被描述为不含它产生的指令 | `--protocol-version` 为这条命令声明 `init-mcpp-home`、`read-project`、 `network`、`write-global-cache` 与 `exec-build-script`,从不声明 From aa0e088b896498289dfc6b0ded40fc709df8df6e Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Sat, 26 Sep 2026 08:40:36 +0800 Subject: [PATCH 09/26] e2e: emit build-database plans every member past a failing one (#699) 787: a workspace member whose planning fails (an unresolvable dependency, not a build program) does not cost its sibling's set; the failed member's `path` and its containment in `watch`; a workspace in which every member fails omits `data`; the exit code is 1 with and without `--format`. 788: a host tool whose build carries a failing blocking `check` (the e2e 315 fixture) is a warning under `emit`, not a lost plan; `mcpp build` still fails on the same check. 789: a build program that exits 1, and one that does not compile, are each described without their directives, `path` naming `build.mcpp`, and nothing a failed run printed before exiting reaches the described unit (the exit-code check precedes directive parsing). A third case, a library target with no sources of its own relying entirely on directives, shows the member failing under E1 when the missing directives leave nothing to link. All three fail on the released 2026.9.26.1 (measured) and pass on this branch's build. --- .../787_emit_plans_every_member_on_its_own.sh | 121 ++++++++++++++++ ...788_emit_host_tool_unbuilt_is_a_warning.sh | 102 +++++++++++++ ...es_a_member_past_a_failed_build_program.sh | 136 ++++++++++++++++++ 3 files changed, 359 insertions(+) create mode 100755 tests/e2e/787_emit_plans_every_member_on_its_own.sh create mode 100755 tests/e2e/788_emit_host_tool_unbuilt_is_a_warning.sh create mode 100755 tests/e2e/789_emit_describes_a_member_past_a_failed_build_program.sh diff --git a/tests/e2e/787_emit_plans_every_member_on_its_own.sh b/tests/e2e/787_emit_plans_every_member_on_its_own.sh new file mode 100755 index 00000000..a7e338f3 --- /dev/null +++ b/tests/e2e/787_emit_plans_every_member_on_its_own.sh @@ -0,0 +1,121 @@ +#!/usr/bin/env bash +# requires: python3 +# 787 -- `emit build-database` plans every selected workspace member on its +# own; one member's planning failure does not cost its siblings' sets +# (mcpp-community/mcpp#699 item 1, design 2026-09-26 §4.4, D1). +# +# Before this fix the member loop stopped at the first failure +# (src/cli/cmd_build.cppm), the same way `mcpp emit build-database --workspace` +# over `good` and `bad` used to report `MCPP_BUILD_DATABASE_PLAN_FAILED` with +# no `data` at all, discarding `good`'s sets along with `bad`'s. `mcpp build +# --workspace` never had this defect (continue-on-failure, +# src/cli/cmd_build.cppm cmd_build) -- this script is `emit`'s analogue. +# +# `bad`'s failure is a genuine PLANNING failure (an unresolvable dependency), +# deliberately NOT a build-program failure: a failing build.mcpp is its own, +# narrower case (E3, #699 item 2 -- see e2e 789), where the member is still +# described. Criteria: +# A. `emit --workspace --format json` over `good` and `bad`: exit 1, `data` +# present, its one set is `good/good`, `diagnostics` holds exactly one +# `error` with `path` "bad/mcpp.toml" and a message naming `bad`, and +# `watch` lists `bad/mcpp.toml` alongside `good`'s own inputs. +# B. Without `--format`, the same run prints the document (not empty) and +# still exits 1 -- the exit code is not conditioned on `--format`. +# C. A workspace in which every member fails omits `data` altogether, with +# one diagnostic per member. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +PY=python3 + +mkdir -p "$TMP/ws/good/src" "$TMP/ws/bad/src" +cd "$TMP/ws" +cat > mcpp.toml <<'EOF' +[workspace] +members = ["good", "bad"] +EOF +cat > good/mcpp.toml <<'EOF' +[package] +name = "good" +version = "0.1.0" +EOF +echo 'int main() { return 0; }' > good/src/main.cpp +# `bad` names a `path` dependency that does not exist: a plan failure with +# nothing to do with build programs, the same shape #699's own report used +# for the member whose plan never reaches a build.mcpp at all. +cat > bad/mcpp.toml <<'EOF' +[package] +name = "bad" +version = "0.1.0" + +[dependencies] +missing = { path = "../does-not-exist" } +EOF +echo 'int main() { return 0; }' > bad/src/main.cpp + +# ── A ────────────────────────────────────────────────────────────────────── +set +e +"$MCPP" emit build-database --workspace --format json > a.json 2> a.err +rc=$? +set -e +[ "$rc" = 1 ] || fail "A: exit status $rc, expected 1" a.err a.json +"$PY" - a.json <<'EOF' || fail "A: the envelope" a.json +import json, sys +e = json.load(open(sys.argv[1])) +d = e["data"] +sets = [s["name"] for s in d["database"]["sets"]] +assert sets == ["good/good"], sets +diags = e["diagnostics"] +assert len(diags) == 1, diags +diag = diags[0] +assert diag["code"] == "MCPP_BUILD_DATABASE_PLAN_FAILED", diag +assert diag["severity"] == "error", diag +assert diag["path"] == "bad/mcpp.toml", diag +assert "bad" in diag["message"], diag["message"] +assert "bad/mcpp.toml" in d["watch"], d["watch"] +assert any(w.startswith("good/") for w in d["watch"]), d["watch"] +EOF +echo "ok: A, one failed member's diagnostic and path, the other member's set kept" + +# ── B ────────────────────────────────────────────────────────────────────── +set +e +"$MCPP" emit build-database --workspace > b.out 2> b.err +rc=$? +set -e +[ "$rc" = 1 ] || fail "B: bare invocation exit status $rc, expected 1" b.out b.err +[ -s b.out ] || fail "B: the bare document is empty" b.err +"$PY" -c 'import json,sys; d=json.load(open(sys.argv[1])); assert d["sets"][0]["name"]=="good/good", d["sets"]' b.out \ + || fail "B: the bare document's set" b.out +grep -q "bad" b.err || fail "B: the failed member's reason is not on stderr" b.err +echo "ok: B, the exit code is 1 without --format too, and the document still prints" + +# ── C: every member fails ─────────────────────────────────────────────────── +cat > good/mcpp.toml <<'EOF' +[package] +name = "good" +version = "0.1.0" + +[dependencies] +missing2 = { path = "../also-does-not-exist" } +EOF +set +e +"$MCPP" emit build-database --workspace --format json > c.json 2> c.err +rc=$? +set -e +[ "$rc" = 1 ] || fail "C: exit status $rc, expected 1" c.err c.json +"$PY" - c.json <<'EOF' || fail "C: the all-failed envelope" c.json +import json, sys +e = json.load(open(sys.argv[1])) +assert "data" not in e, e +diags = e["diagnostics"] +assert len(diags) == 2, diags +paths = sorted(d["path"] for d in diags) +assert paths == ["bad/mcpp.toml", "good/mcpp.toml"], paths +assert all(d["code"] == "MCPP_BUILD_DATABASE_PLAN_FAILED" and d["severity"] == "error" + for d in diags), diags +EOF +echo "ok: C, a workspace in which every member fails omits data" + +echo "PASS: 787_emit_plans_every_member_on_its_own" diff --git a/tests/e2e/788_emit_host_tool_unbuilt_is_a_warning.sh b/tests/e2e/788_emit_host_tool_unbuilt_is_a_warning.sh new file mode 100755 index 00000000..d589b271 --- /dev/null +++ b/tests/e2e/788_emit_host_tool_unbuilt_is_a_warning.sh @@ -0,0 +1,102 @@ +#!/usr/bin/env bash +# requires: gcc python3 +# 788 -- under `emit build-database`, a host tool that fails to build is a +# warning, not a refusal that costs the plan (mcpp-community/mcpp#699 item 2, +# design 2026-09-26 §4.5). +# +# `user` requests the host tool `t` of package `tool`, whose build carries a +# blocking `check` action that always fails (docs/30's `dep_bin` pattern, and +# e2e 315's fixture for a blocking check). Before this fix `emit` in `user` +# reported `MCPP_BUILD_DATABASE_PLAN_FAILED` with no `data` at all (measured +# in the design record, Appendix A.3) -- the same failure that correctly ends +# `mcpp build`, which does not plan around missing tools. Criteria: +# A. `emit --format json` in `user`: exit 0, `data` present with `user`'s +# set, and exactly one warning `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT` +# naming the tool, its package and the first line of the failure. +# B. `mcpp build` in `user` still exits non-zero: the tool's build itself, +# and its blocking check, are unchanged. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +PY=python3 + +mkdir -p "$TMP/tool/src" "$TMP/user/src" + +cat > "$TMP/tool/mcpp.toml" <<'EOF' +[package] +name = "tool" +version = "0.1.0" + +[targets.t] +kind = "bin" +main = "src/main.cpp" +EOF +echo 'int main() { return 0; }' > "$TMP/tool/src/main.cpp" +cat > "$TMP/tool/check.sh" <<'EOF' +#!/usr/bin/env bash +echo "the check says no" >&2 +exit 1 +EOF +chmod +x "$TMP/tool/check.sh" +# Same shape as e2e 315's blocking-check fixture: a `check` action, marked +# `blocking = true`, that always fails. +cat > "$TMP/tool/build.mcpp" <<'EOF' +#include +import mcpp; +int main() { + const std::string root = mcpp::manifest_dir(); + mcpp::action a; + a.id = "gate"; + a.role = "check"; + a.blocking = true; + a.arg((root + "/check.sh").c_str()) + .output("${mcpp.out_dir}/gate.stamp") + .submit(); +} +EOF + +cat > "$TMP/user/mcpp.toml" <<'EOF' +[package] +name = "user" +version = "0.1.0" + +[dependencies] +tool = { path = "../tool", tools = ["t"] } +EOF +echo 'int main() { return 0; }' > "$TMP/user/src/main.cpp" + +cd "$TMP/user" + +# ── A ────────────────────────────────────────────────────────────────────── +set +e +"$MCPP" emit build-database --format json > a.json 2> a.err +rc=$? +set -e +[ "$rc" = 0 ] || fail "A: emit exited $rc, expected 0" a.err a.json +"$PY" - a.json <<'EOF' || fail "A: the envelope" a.json +import json, sys +e = json.load(open(sys.argv[1])) +d = e["data"] +sets = [s["name"] for s in d["database"]["sets"]] +assert sets == ["user"], sets +diags = e["diagnostics"] +assert len(diags) == 1, diags +diag = diags[0] +assert diag["code"] == "MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT", diag +assert diag["severity"] == "warning", diag +assert "t" in diag["message"] and "tool" in diag["message"], diag["message"] +EOF +echo "ok: A, emit succeeds with user's set and one host-tool warning" + +# ── B ────────────────────────────────────────────────────────────────────── +set +e +"$MCPP" build > build.log 2>&1 +rc=$? +set -e +[ "$rc" != 0 ] || fail "B: mcpp build succeeded despite the failing blocking check" build.log +grep -q "the check says no" build.log || fail "B: the check's own failure is not on the build's output" build.log +echo "ok: B, mcpp build still fails on the same blocking check" + +echo "PASS: 788_emit_host_tool_unbuilt_is_a_warning" diff --git a/tests/e2e/789_emit_describes_a_member_past_a_failed_build_program.sh b/tests/e2e/789_emit_describes_a_member_past_a_failed_build_program.sh new file mode 100755 index 00000000..7899ea41 --- /dev/null +++ b/tests/e2e/789_emit_describes_a_member_past_a_failed_build_program.sh @@ -0,0 +1,136 @@ +#!/usr/bin/env bash +# requires: gcc python3 +# 789 -- under `emit build-database`, a package whose build program fails is +# described without that program's directives, instead of costing its whole +# member (mcpp-community/mcpp#699 item 2, design 2026-09-26 §4.6, D3). +# +# Three packages, each its own single-package project (no workspace: the +# member-containment half of this rule is e2e 787's): +# - `exits1`: build.mcpp compiles and exits 1. +# - `nocompile`: build.mcpp does not compile at all. +# - `libonly`: build.mcpp exits 1, and its only target is a library with no +# sources of its own -- everything it would link comes from directives +# the failed run never emitted, so THIS member fails as a whole (E1 +# applies, mcpp-community/mcpp#699 item 1) rather than being described +# with an empty set. +# Criteria: +# A. `exits1`: exit 1, `data` present with its source described, one error +# `MCPP_BUILD_DATABASE_PROGRAM_FAILED` whose `path` is "build.mcpp", and +# no directive the failed run printed before exiting reaches the unit's +# arguments (build_program.cppm checks the exit code before parsing any +# output at all, so nothing from a failed run is ever applied). +# B. `nocompile`: the same shape; the message names a compiler diagnostic, +# not a bare exit code. +# C. `libonly`: `data` is absent and the one diagnostic is +# `MCPP_BUILD_DATABASE_PLAN_FAILED` with `path` "mcpp.toml" -- the +# missing directives left the target with nothing to link, which fails +# the member under the ordinary rule, not under E3's own code. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +PY=python3 + +# ── A ────────────────────────────────────────────────────────────────────── +mkdir -p "$TMP/exits1/src" && cd "$TMP/exits1" +cat > mcpp.toml <<'EOF' +[package] +name = "exits1" +version = "0.1.0" +EOF +echo 'int main() { return 0; }' > src/main.cpp +# Prints a directive before failing: build_program.cppm checks the exit code +# BEFORE it parses stdout for directives at all, so this must never reach the +# described unit's arguments. +cat > build.mcpp <<'EOF' +#include +int main() { std::printf("mcpp:cxxflag=-DSHOULD_NOT_APPEAR\n"); return 1; } +EOF +set +e +"$MCPP" emit build-database --format json > a.json 2> a.err +rc=$? +set -e +[ "$rc" = 1 ] || fail "A: exit status $rc, expected 1" a.err a.json +"$PY" - a.json <<'EOF' || fail "A: the envelope" a.json +import json, sys +e = json.load(open(sys.argv[1])) +d = e["data"] +sets = {s["name"]: s for s in d["database"]["sets"]} +assert "exits1" in sets, sets +unit = sets["exits1"]["translation-units"][0] +assert unit["source"].endswith("main.cpp"), unit +assert "-DSHOULD_NOT_APPEAR" not in unit["arguments"], unit["arguments"] +diags = e["diagnostics"] +assert len(diags) == 1, diags +diag = diags[0] +assert diag["code"] == "MCPP_BUILD_DATABASE_PROGRAM_FAILED", diag +assert diag["severity"] == "error", diag +assert diag["path"] == "build.mcpp", diag +EOF +echo "ok: A, the package is described past its program's failure, and nothing it printed leaked in" + +# ── B ────────────────────────────────────────────────────────────────────── +mkdir -p "$TMP/nocompile/src" && cd "$TMP/nocompile" +cat > mcpp.toml <<'EOF' +[package] +name = "nocompile" +version = "0.1.0" +EOF +echo 'int main() { return 0; }' > src/main.cpp +cat > build.mcpp <<'EOF' +int main() { this is not valid c++ } +EOF +set +e +"$MCPP" emit build-database --format json > b.json 2> b.err +rc=$? +set -e +[ "$rc" = 1 ] || fail "B: exit status $rc, expected 1" b.err b.json +"$PY" - b.json <<'EOF' || fail "B: the envelope" b.json +import json, sys +e = json.load(open(sys.argv[1])) +d = e["data"] +sets = {s["name"]: s for s in d["database"]["sets"]} +assert "nocompile" in sets, sets +diags = e["diagnostics"] +assert len(diags) == 1, diags +diag = diags[0] +assert diag["code"] == "MCPP_BUILD_DATABASE_PROGRAM_FAILED", diag +assert diag["path"] == "build.mcpp", diag +# Not a bare exit-code message: an actual compiler diagnostic reached it. +assert "error" in diag["message"], diag["message"] +EOF +echo "ok: B, a build program that does not compile is described the same way" + +# ── C ────────────────────────────────────────────────────────────────────── +mkdir -p "$TMP/libonly" && cd "$TMP/libonly" +cat > mcpp.toml <<'EOF' +[package] +name = "libonly" +version = "0.1.0" +standard = "c++23" + +[targets.lib] +kind = "lib" +EOF +cat > build.mcpp <<'EOF' +int main() { return 1; } +EOF +set +e +"$MCPP" emit build-database --format json > c.json 2> c.err +rc=$? +set -e +[ "$rc" = 1 ] || fail "C: exit status $rc, expected 1" c.err c.json +"$PY" - c.json <<'EOF' || fail "C: the envelope" c.json +import json, sys +e = json.load(open(sys.argv[1])) +assert "data" not in e, e +diags = e["diagnostics"] +assert len(diags) == 1, diags +diag = diags[0] +assert diag["code"] == "MCPP_BUILD_DATABASE_PLAN_FAILED", diag +assert diag["path"] == "mcpp.toml", diag +EOF +echo "ok: C, missing directives that leave a target with nothing to link fail the whole member" + +echo "PASS: 789_emit_describes_a_member_past_a_failed_build_program" From a25d3856ceb2d7d30ae693e1929869a45bee48ba Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Sat, 26 Sep 2026 08:42:15 +0800 Subject: [PATCH 10/26] emit: the comments cite S2 0.3.0's partial answer --- src/cli/cmd_build.cppm | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index f88ed578..e26940cb 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -304,8 +304,7 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) return 0; }; // A failure with nothing to describe is one envelope with diagnostics and - // no `data` (S2 0.2.0 §3.4: a command without data has failed), and exit - // 1. `fail_no_data` finishes with whatever `diagnostics` already holds — + // no `data` (S2 §3.4: a command without data has failed), and exit 1. `fail_no_data` finishes with whatever `diagnostics` already holds — // used once a single diagnostic is pushed onto it (`failed`, below, for a // usage error decided before any member is tried) and once every // selected member's own planning has failed in turn (#699 item 1, E1: a @@ -416,8 +415,9 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) } // Every selected member was planned independently (#699 item 1, E1): one // that failed contributed its own diagnostic above and nothing else. - // Only when none of them planned is there nothing left to describe — S2 - // has no partial outcome, so `data` is present or it is not. + // Only when none of them planned is there nothing left to describe. A + // document with `data` and error diagnostics is S2 0.3.0's partial answer + // (S2-3.4-12, S2-3.4-13): it describes everything the errors do not name. if (contexts.empty()) return fail_no_data(); From 6f0cf594400161c022829ce43118fd935c22e50e Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Sat, 26 Sep 2026 08:59:52 +0800 Subject: [PATCH 11/26] One compile database per configuration, the output directory, and the standard-library units MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Implements T1 (W1, W2, W3) of the 2026-09-26 compile-database design record (.agents/docs/2026-09-26-compile-database-and-issue-699-design.md), against issues #397, #677, #699 and the xmake-comparison report it triages. W1 (design §3.2). compile_commands.json is now two files with one rule each. The CONFIGURATION's database, target///compile_commands.json, holds the fresh plan's entries merged with the entries it already had whose `file`, resolved against `directory`, the fresh plan lacks and which still exist -- unchanged in spirit from before, but now scoped to one output directory instead of the project root, and keyed by the resolved path rather than the raw string. The ROOT file is a copy of that database: replaced whole, never merged, left untouched when byte-identical. When the replaced root file held entries mcpp did not write (another tool's, or a stale mix of toolchains), one warning states how many; an entry is mcpp's own when its `output`, resolved against `directory`, lies under this project's `target/` tree or under the mcpp home (the standard-library units' shared cache). The fast path (execute.cppm, try_fast_build and try_fast_run) publishes the root file through the same function the full path uses, reading the configuration database already on disk, so a deleted root file returns on the next build without a plan (C1). `emit build-database` still writes neither file. The scaffold's .gitignore lists compile_commands.json. W2 (design §3.3, §3.4). `directory` is the output directory the compiler actually runs in, for every project unit and every toolchain, matching the JSON Compilation Database format, S1-8-2 and what CMake, ninja and xmake already write (C3); before this, GCC's importers failed when replayed from `directory` and wrote gcm.cache/ into the project root. A unit that provides a module also carries its dialect's language flag (BmiTraits:: moduleInterfaceLangFlag) immediately before `-c`, so a reader does not infer it from the extension the way clang's driver cannot for `.ixx` (C4); the MSVC form stays out until a Windows measurement of clang-cl-mode clangd answers whether it accepts `/interface`. W3 (design §3.5). The standard-library units a build compiles are recovered once, in prepare.cppm, from the same derivation `ensure_built` and `describe_std_module` both read, and carried on the plan (BuildPlan::stdModuleUnits) rather than recovered again by each renderer: compile_commands.json, `emit --spec compile-commands` and the S1 document (mcpp.build.build_database) now render the identical record for these units, which follows S1-12-1 and could not previously disagree because it did not exist in the JSON-format documents at all (D5a). The S1 document's `provides` for `std`/`std.compat` now names the BMI path in the shared std cache instead of an empty string (D5b), and `ide.toolchains..build-id` carries the compiler's build identity -- the same value already computed as one field of the toolchain fingerprint (the driver's normalized --version banner, or a hash of the driver binary) -- stable across two runs of one toolchain. Project modules keep "" in `provides` under `emit`, unaffected. `mcpp.build.plan` gains `recover_invocation`/`split_command_words`, moved down from `mcpp.build.build_database` (which re-exports them under their established names for its own tests) so both `mcpp.build.compile_commands` and `mcpp.build.build_database` -- neither of which may import the other -- can render from the same recovered record without a dependency cycle. Tests: unit (test_compile_commands.cpp: the within-configuration merge resolves `file` against `directory`; a project unit's `directory` is the output directory; the language flag before `-c` for an interface unit and its absence for an implementation unit and for MSVC); e2e 781-786 (new, against 2026.9.26.1: 781, 784 and 785 fail as designed); e2e 211 and 47 updated for `directory` now naming the output directory; e2e 688 updated for the standard-library units now appearing in both the S1 document's compile- commands export and the real build's compile_commands.json. --- src/build/build_database.cppm | 198 +++---- src/build/compile_commands.cppm | 515 +++++++++++++----- src/build/execute.cppm | 40 +- src/build/ninja_backend.cppm | 8 + src/build/plan.cppm | 149 +++++ src/build/prepare.cppm | 65 ++- src/scaffold/create.cppm | 8 +- tests/e2e/211_configure_only_cdb.sh | 14 +- tests/e2e/47_cdb_prebuilt_module_path_abs.sh | 20 +- tests/e2e/688_emit_build_database.sh | 26 +- tests/e2e/781_deleted_root_cdb_is_restored.sh | 67 +++ ...2_foreign_root_cdb_entries_are_replaced.sh | 76 +++ ...b_switches_whole_with_the_configuration.sh | 72 +++ .../e2e/784_cdb_replays_from_its_directory.sh | 81 +++ ...85_cdb_interface_flag_module_extensions.sh | 72 +++ ...6_std_unit_in_the_database_and_build_id.sh | 107 ++++ tests/unit/test_compile_commands.cpp | 119 +++- 17 files changed, 1341 insertions(+), 296 deletions(-) create mode 100755 tests/e2e/781_deleted_root_cdb_is_restored.sh create mode 100755 tests/e2e/782_foreign_root_cdb_entries_are_replaced.sh create mode 100755 tests/e2e/783_cdb_switches_whole_with_the_configuration.sh create mode 100755 tests/e2e/784_cdb_replays_from_its_directory.sh create mode 100755 tests/e2e/785_cdb_interface_flag_module_extensions.sh create mode 100755 tests/e2e/786_std_unit_in_the_database_and_build_id.sh diff --git a/src/build/build_database.cppm b/src/build/build_database.cppm index 429595aa..82c57527 100644 --- a/src/build/build_database.cppm +++ b/src/build/build_database.cppm @@ -92,26 +92,15 @@ std::string_view stdlib_name(std::string_view stdlibId); std::string toolchain_id(const mcpp::toolchain::Toolchain& tc, std::string_view compilerTriple); -// Splits a command string mcpp rendered for the host shell into words, undoing -// its quoting: POSIX `sh` rules, or the Microsoft C runtime's rules on Windows. -// The reader is mcpp::manifest::host_command_words; this name is kept for the -// standard library units' recovery below. -std::vector split_command_words(std::string_view command, bool windows); - -// The working directory and argument vector of the command in `commands` whose -// words name `source`, recovered from the rendering: a leading `cd`, an `env` -// word and environment assignments, and redirections are removed. A driver -// path that the rendering left unquoted despite a space is rejoined. Empty when -// no command names the source. -struct Invocation { - std::filesystem::path workDirectory; - std::vector arguments; -}; -std::optional recover_invocation(const std::vector& commands, - const std::filesystem::path& source, - const std::filesystem::path& driver, - const std::filesystem::path& defaultDirectory, - bool windows); +// The standard-library units' recovery (mcpp.build.plan::recover_invocation) +// now runs once, in prepare.cppm, onto BuildPlan::stdModuleUnits (design +// 2026-09-26 §3.5, D5a/D5b: the plan carries the standard-library +// description, so this document and compile_commands.json render the exact +// same record and cannot disagree, P1). Re-exported here under the names +// this file's tests use. +using Invocation = mcpp::build::RecoveredInvocation; +using mcpp::build::recover_invocation; +using mcpp::build::split_command_words; } // namespace mcpp::build::database @@ -131,29 +120,6 @@ std::string qualified_name(const mcpp::manifest::Manifest& m) { : m.package.namespace_ + "." + m.package.name; } -bool is_assignment(std::string_view w) { - auto eq = w.find('='); - if (eq == std::string_view::npos || eq == 0) return false; - if (!(std::isalpha(static_cast(w[0])) || w[0] == '_')) return false; - for (std::size_t i = 1; i < eq; ++i) { - const unsigned char c = static_cast(w[i]); - if (!(std::isalnum(c) || c == '_')) return false; - } - return true; -} - -// `2>&1`, `>file`, `nul`, ``, `2>`, `<`, -// `>>`: the target is the next word. -enum class Redirect { None, Attached, Detached }; -Redirect redirect_kind(std::string_view w) { - std::size_t i = 0; - while (i < w.size() && std::isdigit(static_cast(w[i]))) ++i; - if (i >= w.size() || (w[i] != '>' && w[i] != '<')) return Redirect::None; - std::size_t j = i + 1; - if (j < w.size() && w[j] == w[i]) ++j; // `>>` - return j == w.size() ? Redirect::Detached : Redirect::Attached; -} - bool names_path(std::string_view word, const std::filesystem::path& path, const std::filesystem::path& cwd) { std::filesystem::path w{std::string(word)}; @@ -234,6 +200,23 @@ void split_baseline(nlohmann::json& set) { set["baseline-arguments"] = std::move(baseline); } +// D5b, S1 §6: "the compiler's build revision ... equal versions do not imply +// compatible BMIs" (S1-11.2-3 gates the authoritative reuse of a build BMI on +// this value matching exactly). mcpp already computes it, as one field of the +// toolchain fingerprint (mcpp.toolchain.fingerprint::compute_fingerprint, +// field 3): the compiler's normalized `--version` banner when the toolchain +// probe recorded one (`Toolchain::driverIdent`), which changes with a build +// commit even when the reported version string does not, and a hash of the +// driver binary otherwise. Deriving build-id from the SAME computation, +// rather than a second one, is what keeps the two stable together and each +// stable across two runs of one toolchain. +std::string build_id(const mcpp::toolchain::Toolchain& tc) { + return !tc.driverIdent.empty() + ? mcpp::toolchain::hash_string(tc.driverIdent) + : (tc.binaryPath.empty() ? std::string{} + : mcpp::toolchain::hash_file(tc.binaryPath)); +} + nlohmann::json toolchain_json(const mcpp::toolchain::Toolchain& tc, std::string_view compilerTriple, const std::vector& invocations) { @@ -250,6 +233,7 @@ nlohmann::json toolchain_json(const mcpp::toolchain::Toolchain& tc, if (!tc.stdlibVersion.empty()) stdlib["version"] = tc.stdlibVersion; j["stdlib"] = std::move(stdlib); } + if (auto id = build_id(tc); !id.empty()) j["build-id"] = id; return j; } @@ -308,74 +292,10 @@ std::string toolchain_id(const mcpp::toolchain::Toolchain& tc, return std::format("{}-{}-{}", tc.compiler_family(), tc.version, compilerTriple); } -std::vector split_command_words(std::string_view s, bool windows) { - return mcpp::manifest::host_command_words(s, windows); -} - -std::optional recover_invocation(const std::vector& commands, - const std::filesystem::path& source, - const std::filesystem::path& driver, - const std::filesystem::path& defaultDirectory, - bool windows) { - const std::string driverText = driver.string(); - for (auto const& command : commands) { - auto words = split_command_words(command, windows); - std::vector> segments(1); - for (auto& w : words) { - if (w == "&&") segments.emplace_back(); - else segments.back().push_back(std::move(w)); - } - std::filesystem::path cwd; - for (auto& seg : segments) { - if (seg.empty()) continue; - if (seg.front() == "cd") { - std::size_t k = 1; - if (k < seg.size() && (seg[k] == "/d" || seg[k] == "/D")) ++k; - if (k < seg.size()) cwd = std::filesystem::path{seg[k]}; - continue; - } - std::size_t b = 0; - if (b < seg.size() && seg[b] == "env") ++b; - while (b < seg.size() && is_assignment(seg[b])) ++b; - std::vector argv; - for (std::size_t k = b; k < seg.size(); ++k) { - switch (redirect_kind(seg[k])) { - case Redirect::Attached: continue; - case Redirect::Detached: ++k; continue; - case Redirect::None: argv.push_back(seg[k]); - } - } - if (argv.empty()) continue; - // A driver path rendered without quotes splits at its spaces. - if (argv.front() != driverText - && driverText.find(' ') != std::string::npos) { - std::string joined = argv.front(); - std::size_t k = 1; - while (k < argv.size() && joined.size() < driverText.size()) { - joined += ' '; - joined += argv[k]; - ++k; - } - if (joined == driverText) { - argv.erase(argv.begin() + 1, argv.begin() + static_cast(k)); - argv.front() = driverText; - } - } - const bool named = std::ranges::any_of(argv, [&](const std::string& w) { - return names_path(w, source, cwd.empty() ? defaultDirectory : cwd); - }); - if (!named) continue; - return Invocation{cwd.empty() ? defaultDirectory : cwd, std::move(argv)}; - } - } - return std::nullopt; -} - Rendered render(std::span members, const std::filesystem::path& workspaceRoot, std::string_view selector) { Rendered r; - const bool windows = mcpp::platform::is_windows; nlohmann::json toolchains = nlohmann::json::object(); nlohmann::json sets = nlohmann::json::array(); @@ -471,40 +391,48 @@ Rendered render(std::span members, }); } - if (ctx.stdModule) { - const auto& sm = *ctx.stdModule; - auto add_std = [&](const std::filesystem::path& source, - const std::vector& commands, - const std::filesystem::path& object, - std::string_view module, - std::vector requires_) { - if (source.empty() || commands.empty()) return; - auto inv = recover_invocation(commands, source, ctx.tc.binaryPath, - sm.cacheDir, windows); - if (!inv) { - r.notes.push_back({"MCPP_BUILD_DATABASE_STD_UNIT_UNDESCRIBED", - std::format("no command that builds the {} module names its " - "source '{}'; the unit is not listed", - module, source.string())}); - return; - } - auto& set = set_for(member.setPrefix + std::string(kStdSetName), - std::string(kStdSetName), "library"); + // D5a/D5b (design 2026-09-26 §3.5): the standard-library units, + // already recovered onto the plan (prepare.cppm, from the same + // derivation `ensure_built` and `describe_std_module` both read), so + // this rendering cannot list a command or a BMI path that + // compile_commands.json (mcpp.build.compile_commands) disagrees with + // (P1). S1-12-1: they are translation units of the build like any + // other, so `--spec compile-commands` lists them too. + if (!ctx.plan.stdModuleUnits.empty()) { + auto& set = set_for(member.setPrefix + std::string(kStdSetName), + std::string(kStdSetName), "library"); + for (auto const& unit : ctx.plan.stdModuleUnits) { + const auto sourceStr = native_string(unit.source); + const auto objectStr = native_string(unit.object); + const auto workDirStr = native_string(unit.workDirectory); + r.compileCommands.push_back(nlohmann::json{ + {"directory", workDirStr}, + {"file", sourceStr}, + {"arguments", unit.arguments}, + {"output", objectStr}, + }); + nlohmann::json requires_ = nlohmann::json::array(); + for (auto const& name : unit.requiresModules) requires_.push_back(name); set.units.push_back(nlohmann::json{ - {"source", native_string(source)}, - {"work-directory", native_string(inv->workDirectory)}, - {"arguments", std::move(inv->arguments)}, - {"object", native_string(object)}, + {"source", sourceStr}, + {"work-directory", workDirStr}, + {"arguments", unit.arguments}, + {"object", objectStr}, {"private", false}, - {"provides", {{std::string(module), ""}}}, + // D5b, S1-8-6: the path of the BMI the build writes, in + // the shared std cache that `emit` and the build share. + // The planning pass never compiles (SPEC-005 R2.2), but + // this path is the cache key's, not a compiler output it + // would have to run to learn. + {"provides", {{unit.module, native_string(unit.bmi)}}}, {"requires", std::move(requires_)}, {"ide", {{"role", "module-interface"}}}, }); - }; - add_std(ctx.tc.stdModuleSource, sm.stdCommands, sm.objectPath, "std", {}); - add_std(ctx.tc.stdCompatSource, sm.compatCommands, sm.compatObjectPath, - "std.compat", {"std"}); + } } + // A recovery failure is now reported where the recovery runs + // (prepare.cppm, onto BuildContext::planNotes) and reaches `r.notes` + // through the unconditional copy below, with every other plan note. for (auto const& name : order) { auto& set = groups.at(name); diff --git a/src/build/compile_commands.cppm b/src/build/compile_commands.cppm index 93afe61c..7e52861f 100644 --- a/src/build/compile_commands.cppm +++ b/src/build/compile_commands.cppm @@ -7,10 +7,33 @@ // Uses the `arguments` array format (preferred over `command` string // per clangd docs). // -// Output location: /compile_commands.json so clangd finds -// it via its default upward directory walk — zero configuration needed. +// TWO FILES, ONE RULE EACH (design 2026-09-26, .agents/docs/2026-09-26- +// compile-database-and-issue-699-design.md §3.2). // -// See .agents/docs/2026-05-12-compile-commands-design.md. +// - The CONFIGURATION's database, `/compile_commands.json`, is +// the merged record of every command that has planned in this exact +// output directory (`build`, `test`, `run`, `--configure-only`): the +// fresh plan's entries, plus the entries it already holds whose `file`, +// resolved against `directory`, the fresh plan lacks and which still +// exist. Every entry in it was written by mcpp in this one configuration, +// so no ownership test is needed and none is made. +// - The ROOT file, `/compile_commands.json` (the path clangd's +// own upward search finds, or `plan.compileDbPath` where a symlink there +// is written through), is a COPY of the current configuration's database: +// replaced whole, never merged, and left untouched when byte-identical so +// clangd is not triggered for nothing. Switching toolchain or profile +// switches the whole file; switching back restores that configuration's +// entries, its test units included. +// +// `write_compile_commands` performs both steps for a freshly rendered plan +// (the full build path, ninja_backend.cppm). `publish_root_compile_commands` +// is exported on its own because the FAST path (execute.cppm) restores the +// root file from a configuration database already on disk, with no plan at +// all (C1: a deleted root file used to stay deleted forever, because nothing +// on the fast path ever reached a writer). +// +// See .agents/docs/2026-05-12-compile-commands-design.md and the 2026-09-26 +// design record above. export module mcpp.build.compile_commands; @@ -18,17 +41,19 @@ import std; import mcpp.source_kind; import mcpp.build.plan; import mcpp.build.flags; +import mcpp.home; import mcpp.libs.json; import mcpp.platform.fs; import mcpp.platform; import mcpp.manifest.flag_words; +import mcpp.toolchain.model; export namespace mcpp::build { // The words the compiler receives from one flag string the engine rendered // for a ninja `command =` line: ninja's `$` escapes are undone, then the -// host's command-line reader splits the text (POSIX `sh`, or the MSVCRT rules -// on Windows; mcpp::manifest::host_command_words). +// host's command-line reader splits the text (POSIX `sh`, or the MSVCRT +// rules on Windows; mcpp::manifest::host_command_words). // // It serves the rendered strings only, the global `$cflags`/`$cxxflags`/ // `$asmflags`. A unit's own flag lists are never rendered and re-read: the @@ -67,16 +92,24 @@ std::vector unit_invocations(const BuildPlan& plan, std::string emit_compile_commands(const BuildPlan& plan, const CompileFlags& flags); // Merge freshly-emitted CDB text (`fresh`, from the current build plan) with a -// prior CDB on disk (`existing`). A prior entry is preserved ONLY when its -// `file` is absent from `fresh` AND still exists on disk (per `fileExists`); +// prior CDB of the SAME CONFIGURATION on disk (`existing`). A prior entry is +// preserved ONLY when its `file`, resolved against its `directory` when not +// already absolute, is absent from `fresh` (under the same resolution) AND +// still exists on disk (per `fileExists`, probed against the resolved path); // everything else comes from `fresh`. Result is sorted by `file` for stable // output. A malformed `existing` is ignored (falls back to `fresh`). // -// Rationale: `mcpp build` regenerates the CDB from a plan that lacks test files -// / dev-deps, while `mcpp test` writes them in. Without merging, whichever ran -// last wins and clangd loses coverage for tests/ (no completion). Merging makes -// the CDB the union of every command's real plan — offline-safe, no extra -// dependency resolution. See .agents/docs/2026-06-25-cdb-test-coverage-design.md. +// Rationale: `mcpp build` regenerates the database from a plan that lacks +// test files / dev-deps, while `mcpp test` writes them in; both plan into the +// same output directory, so without merging, whichever ran last wins and +// clangd loses coverage for tests/ (no completion). Merging makes the +// configuration's database the union of every command's real plan — +// offline-safe, no extra dependency resolution. `directory` enters the +// identity because a unit's own `directory` need not be the process's +// working directory (the standard-library units always name the shared std +// cache; every project unit now names the output directory, §3.3). See +// .agents/docs/2026-06-25-cdb-test-coverage-design.md and the 2026-09-26 +// design record. std::string merge_compile_commands( std::string_view fresh, std::string_view existing, @@ -85,6 +118,11 @@ std::string merge_compile_commands( struct CompileCommandsWriteResult { bool changed = false; std::size_t commandCount = 0; + // Set only by publish_root_compile_commands (and by write_compile_commands, + // which calls it): the number of entries the REPLACED root file held that + // were not mcpp's own. Zero when the root was not replaced, held nothing + // foreign, or is being written for the first time. + std::size_t foreignEntries = 0; }; struct CompileCommandsWriteError { @@ -95,6 +133,11 @@ using ReplaceFile = std::function; +// Writes the CONFIGURATION's database at `path` (normally +// `plan.outputDir / "compile_commands.json"`): merges `fresh` with whatever +// `path` already holds (a symlink there is followed, as at the root), +// leaves the file untouched when the result is byte-identical, and replaces +// it atomically otherwise. std::expected publish_compile_commands( const std::filesystem::path& path, @@ -102,6 +145,32 @@ publish_compile_commands( const std::function& fileExists, ReplaceFile replaceFile = mcpp::platform::fs::replace_file); +// Copies the configuration database's CURRENT CONTENT on disk at +// `configDbPath` over the root file at `rootPath` (following a symlink +// there, as today): replaced WHOLE, never merged, and left untouched when +// byte-identical. This is the one function both the full build (right after +// it writes the configuration database) and the fast path (which has no +// plan, and reads the configuration database instead of holding it in +// memory) call to publish the root file — design §3.2 items 2 to 4. +// +// `targetRoot` and `mcppHome` decide which of the root file's PRIOR entries +// (by `output`, resolved against `directory`) are mcpp's own: under +// `targetRoot` (this project's whole `target/` tree — a toolchain or +// profile switch replaces the root file too, and that is not a foreign +// write) or under `mcppHome` (the shared std cache the standard-library +// units of §3.5 write into). Every other prior entry is counted in the +// result's `foreignEntries`, so the caller can warn once. +std::expected +publish_root_compile_commands( + const std::filesystem::path& configDbPath, + const std::filesystem::path& rootPath, + const std::filesystem::path& targetRoot, + const std::filesystem::path& mcppHome, + ReplaceFile replaceFile = mcpp::platform::fs::replace_file); + +// Writes the configuration's database for `plan`, then publishes the root +// copy from it. `commandCount` is the configuration database's entry count; +// `foreignEntries` is the root publish's (see publish_root_compile_commands). std::expected write_compile_commands(const BuildPlan& plan, const CompileFlags& flags); @@ -109,10 +178,6 @@ write_compile_commands(const BuildPlan& plan, const CompileFlags& flags); namespace mcpp::build { -namespace { - -} // namespace - // The order is the one ninja and the host apply: ninja replaces `$ `, `$:` and // `$$` while it builds the command line, and only then does the host read the // line into words. flags.cppm escapes for ninja before it quotes for the host, @@ -194,6 +259,160 @@ CompileCommandsWriteError write_error(std::string message) { return CompileCommandsWriteError{std::move(message)}; } +// `file` (or, for the ownership test below, `output`) resolved against +// `directory` when it is not already absolute: the identity the within- +// configuration merge and the root's ownership test both use (design §3.2 +// items 1 and 3). Every path mcpp itself writes into either field is already +// absolute (a unit's source and its output are both absolute; see +// plan.cppm), so this only matters for an entry another tool wrote, or for a +// future producer that follows the JSON format's licence to write a relative +// one. +std::filesystem::path resolve_against_directory(const nlohmann::json& entry, + std::string_view field) { + const std::string key(field); + if (!entry.is_object() || !entry.contains(key) || !entry.at(key).is_string()) + return {}; + std::filesystem::path p(entry.at(key).get()); + if (p.is_absolute()) return p.lexically_normal(); + if (!entry.contains("directory") || !entry.at("directory").is_string()) return p; + return (std::filesystem::path(entry.at("directory").get()) / p).lexically_normal(); +} + +// A key for the merge's dedup set: the resolved path, in one spelling. Not +// just `.string()` — a prior CDB written before the mixed-separator fix +// (#390) carries `root\generated/modules\x.cppm` entries that are the SAME +// file as a fresh `root\generated\modules\x.cppm` one, and a literal string +// comparison would keep both. Normalizing makes the merge self-healing. +std::string dedup_key(const std::filesystem::path& resolved) { + auto p = resolved.lexically_normal(); + p.make_preferred(); + return p.string(); +} + +// §3.2 item 3: an entry is mcpp's own when its `output`, resolved against its +// `directory`, lies under this project's `target/` (every configuration, not +// only the current one — switching toolchain or profile replaces the root +// file too, and that is not a foreign write) or under the mcpp home (the +// shared std cache the standard-library units of §3.5 write their objects +// into, outside `target/`). An entry with no `output` at all — the shape a +// hand-written or another tool's database uses — is never mcpp's: mcpp +// always writes one. +bool is_mcpps_entry(const nlohmann::json& entry, + const std::filesystem::path& targetRoot, + const std::filesystem::path& mcppHome) { + auto output = resolve_against_directory(entry, "output"); + if (output.empty()) return false; + auto under = [&](const std::filesystem::path& base) { + if (base.empty()) return false; + auto rel = output.lexically_relative(base.lexically_normal()); + if (rel.empty() || rel == std::filesystem::path(".")) return false; + return *rel.begin() != std::filesystem::path(".."); + }; + return under(targetRoot) || under(mcppHome); +} + +// Reads `path` through one symlink hop (as the root file's redirect always +// has) and returns the real file to publish to, plus its current content +// when it exists. A first build has nothing to resolve: symlink_status +// reports the missing path with type()==not_found on every standard library +// (the error code category differs — generic ENOENT vs system +// ERROR_FILE_NOT_FOUND — so that case is not an error). +struct ExistingDocument { + std::filesystem::path publishPath; + std::optional content; +}; + +std::expected +read_existing_document(const std::filesystem::path& path) { + std::filesystem::path publishPath = path; + std::error_code statusEc; + const auto linkStatus = std::filesystem::symlink_status(path, statusEc); + if (statusEc && linkStatus.type() != std::filesystem::file_type::not_found) { + return std::unexpected(write_error(std::format( + "cannot inspect compile database '{}': {}", path.string(), + statusEc.message()))); + } + if (linkStatus.type() == std::filesystem::file_type::symlink) { + auto target = std::filesystem::read_symlink(path, statusEc); + if (statusEc) { + return std::unexpected(write_error(std::format( + "cannot resolve compile database link '{}': {}", path.string(), + statusEc.message()))); + } + publishPath = target.is_absolute() ? target : path.parent_path() / target; + } + + std::optional existing; + std::ifstream input(publishPath, std::ios::binary); + if (input) { + std::stringstream ss; + ss << input.rdbuf(); + if (input.bad()) { + return std::unexpected(write_error(std::format( + "cannot read existing compile database '{}'", publishPath.string()))); + } + existing = ss.str(); + } else { + std::error_code existsEc; + auto exists = std::filesystem::exists(publishPath, existsEc); + if (existsEc) { + return std::unexpected(write_error(std::format( + "cannot inspect existing compile database '{}': {}", + publishPath.string(), existsEc.message()))); + } + if (exists) { + return std::unexpected(write_error(std::format( + "cannot read existing compile database '{}'", publishPath.string()))); + } + } + return ExistingDocument{std::move(publishPath), std::move(existing)}; +} + +std::expected +atomic_replace_document(const std::filesystem::path& publishPath, + const std::string& content, const ReplaceFile& replaceFile) { + static std::atomic sequence{0}; + const auto nonce = std::random_device{}(); + // 临时文件和链接目标同目录,避免 rename 跨文件系统;随机量降低跨进程碰撞概率。 + auto temp = publishPath.parent_path() + / std::format(".{}.tmp.{}.{}.{}", publishPath.filename().string(), + std::chrono::steady_clock::now().time_since_epoch().count(), + static_cast(nonce), + sequence.fetch_add(1, std::memory_order_relaxed)); + auto cleanup_temp = [&] { + std::error_code cleanupEc; + std::filesystem::remove(temp, cleanupEc); + }; + + std::ofstream output(temp, std::ios::binary | std::ios::trunc); + if (!output) { + return std::unexpected(write_error(std::format( + "cannot open temporary compile database '{}'", temp.string()))); + } + output << content; + output.flush(); + if (!output) { + output.close(); + cleanup_temp(); + return std::unexpected(write_error(std::format( + "cannot write temporary compile database '{}'", temp.string()))); + } + output.close(); + if (!output) { + cleanup_temp(); + return std::unexpected(write_error(std::format( + "cannot close temporary compile database '{}'", temp.string()))); + } + + std::error_code ec; + if (!replaceFile(temp, publishPath, ec)) { + cleanup_temp(); + return std::unexpected(write_error(std::format( + "cannot replace '{}': {}", publishPath.string(), ec.message()))); + } + return {}; +} + } // namespace std::vector unit_asm_flags(const CompileUnit& cu) { @@ -211,6 +430,14 @@ std::vector unit_invocations(const BuildPlan& plan, std::vector out; out.reserve(plan.compileUnits.size()); + // C4: the driver's own suffix→language table is private and version- + // dependent (BmiTraits::moduleInterfaceLangFlag's note on why the build + // states it unconditionally instead of trusting a driver to infer it); + // this record states the same flag at the same position, so a reader + // that replays `arguments` verbatim reads a module-extension source the + // way the build did rather than guessing from its name. + const auto traits = mcpp::toolchain::bmi_traits(plan.toolchain); + for (auto& cu : plan.compileUnits) { // NASM units carry a command line no CDB consumer (clangd, …) can // interpret — a bogus entry actively harms LSP diagnostics, so they @@ -228,7 +455,12 @@ std::vector unit_invocations(const BuildPlan& plan, UnitInvocation inv; inv.unit = &cu; inv.output = native_string(plan.outputDir / cu.object); - inv.directory = native_string(plan.projectRoot); + // C3: every project unit runs in the output directory — the directory + // ninja actually invokes the compiler in — never the project root. + // The standard-library units keep their own `directory` (the shared + // std cache), which unit_invocations never touches: they are + // synthesised separately (mcpp.build.build_database::render). + inv.directory = native_string(plan.outputDir); inv.file = native_string(cu.source); // Build arguments array. @@ -240,6 +472,18 @@ std::vector unit_invocations(const BuildPlan& plan, for (auto& f : isGasSource ? mcpp::manifest::flag_words(unit_asm_flags(cu)) : package_flag_args(cu, isCSource)) inv.arguments.push_back(std::move(f)); + // C4: immediately before `-c`, since the GNU dialects read the flag + // positionally (BmiTraits::moduleInterfaceLangFlag). Only a unit that + // actually provides a module needs it — an implementation unit (a + // plain source, or `module M;` inside a module-extension file) is + // not the "which language is this extension" ambiguity the flag + // removes. MSVC's form (`/interface /TP`) waits for a Windows + // measurement of clang-cl-mode clangd (design §3.4, "Open"). + if (!cu.providesModule.empty() + && plan.toolchain.compiler != mcpp::toolchain::CompilerId::MSVC) { + for (auto& w : split_flags(traits.moduleInterfaceLangFlag)) + inv.arguments.push_back(std::move(w)); + } inv.arguments.push_back("-c"); inv.arguments.push_back(inv.file); inv.arguments.push_back("-o"); @@ -266,6 +510,24 @@ std::string emit_compile_commands(const BuildPlan& plan, const CompileFlags& fla entries.push_back(std::move(entry)); } + // D5a, S1-12-1: the standard-library units are translation units of this + // build like any other, recovered once onto the plan (prepare.cppm, + // BuildPlan::stdModuleUnits) from the SAME command + // mcpp.build.build_database's S1 rendering uses, so the two documents + // cannot list different arguments for the same unit (P1). + for (auto const& unit : plan.stdModuleUnits) { + nlohmann::json args = nlohmann::json::array(); + for (auto const& a : unit.arguments) args.push_back(a); + + nlohmann::json entry; + entry["directory"] = native_string(unit.workDirectory); + entry["file"] = native_string(unit.source); + entry["arguments"] = std::move(args); + entry["output"] = native_string(unit.object); + + entries.push_back(std::move(entry)); + } + return entries.dump(2) + "\n"; } @@ -277,38 +539,29 @@ std::string merge_compile_commands( if (freshJ.is_discarded() || !freshJ.is_array()) return std::string(fresh); - // Dedup key = the file's PATH, spelled the way a fresh plan spells it - // (native separators). A prior CDB written before the mixed-separator - // fix (#390) carries `root\generated/modules\x.cppm` entries that are - // the SAME file as the fresh `root\generated\modules\x.cppm` — a literal - // string comparison would keep both and the user's upgrade would not - // visibly fix anything. Normalizing makes the merge self-healing: the - // stale mixed entry is skipped on the first `mcpp build` after upgrade. - // fileExists still probes the raw spelling — Windows accepts both. - auto norm_key = [](std::string_view f) { - auto p = std::filesystem::path(std::string(f)).lexically_normal(); - p.make_preferred(); - return p.string(); - }; - // Files the current plan already covers — those entries are authoritative. + // The key resolves `file` against `directory` (resolve_against_directory): + // every path mcpp writes is already absolute, so this is a no-op for a + // fresh entry, but it is what makes an EXISTING entry from before this + // resolution rule (or, in principle, one with a relative `file`) compare + // correctly against it. std::set freshFiles; for (auto const& e : freshJ) { - if (e.contains("file") && e["file"].is_string()) - freshFiles.insert(norm_key(e["file"].get())); + auto resolved = resolve_against_directory(e, "file"); + if (!resolved.empty()) freshFiles.insert(dedup_key(resolved)); } // Keep fresh order, then append still-valid prior entries the plan doesn't // cover (e.g. tests/ from a previous `mcpp test`). Drop entries for files - // that no longer exist so the CDB never accrues dead references. + // that no longer exist so the database never accrues dead references. nlohmann::json merged = freshJ; auto existingJ = nlohmann::json::parse(existing, nullptr, /*allow_exceptions=*/false); if (!existingJ.is_discarded() && existingJ.is_array()) { for (auto const& e : existingJ) { - if (!e.contains("file") || !e["file"].is_string()) continue; - auto f = e["file"].get(); - if (freshFiles.contains(norm_key(f))) continue; // fresh wins - if (!fileExists(std::filesystem::path(f))) continue; // pruned + auto resolved = resolve_against_directory(e, "file"); + if (resolved.empty()) continue; + if (freshFiles.contains(dedup_key(resolved))) continue; // fresh wins + if (!fileExists(resolved)) continue; // pruned merged.push_back(e); } } @@ -330,64 +583,17 @@ publish_compile_commands( "fresh compile database for '{}' is not a JSON array", path.string()))); } - std::filesystem::path publishPath = path; - std::error_code statusEc; - const auto linkStatus = std::filesystem::symlink_status(path, statusEc); - // A first build has no prior CDB to inspect. symlink_status reports the - // missing path with type()==not_found on libstdc++/libc++/MSVC (the error - // code category differs: generic ENOENT vs system ERROR_FILE_NOT_FOUND), - // so that is the normal fresh-workspace case, not an error. - if (statusEc && linkStatus.type() != std::filesystem::file_type::not_found) { - return std::unexpected(write_error(std::format( - "cannot inspect compile database '{}': {}", path.string(), - statusEc.message()))); - } - const bool isLink = linkStatus.type() == std::filesystem::file_type::symlink; - if (isLink) { - auto target = std::filesystem::read_symlink(path, statusEc); - if (statusEc) { - return std::unexpected(write_error(std::format( - "cannot resolve compile database link '{}': {}", path.string(), - statusEc.message()))); - } - publishPath = target.is_absolute() ? target : path.parent_path() / target; - } - - std::optional existing; - { - std::ifstream input(publishPath, std::ios::binary); - if (input) { - std::stringstream ss; - ss << input.rdbuf(); - if (input.bad()) { - return std::unexpected(write_error(std::format( - "cannot read existing compile database '{}'", publishPath.string()))); - } - existing = ss.str(); - } else { - std::error_code existsEc; - auto exists = std::filesystem::exists(publishPath, existsEc); - if (existsEc) { - return std::unexpected(write_error(std::format( - "cannot inspect existing compile database '{}': {}", - publishPath.string(), existsEc.message()))); - } - if (exists) { - return std::unexpected(write_error(std::format( - "cannot read existing compile database '{}'", publishPath.string()))); - } - } - } // input closed before the atomic replace: Windows cannot replace an open file. + auto doc = read_existing_document(path); + if (!doc) return std::unexpected(doc.error()); // 完全相同的有效输入不重写文件,避免 clangd 因 mtime 变化重复索引。 - if (existing && *existing == fresh) { - return CompileCommandsWriteResult{false, freshJson.size()}; - } + if (doc->content && *doc->content == fresh) + return CompileCommandsWriteResult{false, freshJson.size(), 0}; std::string content(fresh); - if (existing) { + if (doc->content) { // 保留仍存在但当前 plan 未覆盖的条目,主要是之前 test 生成的 TU。 - content = merge_compile_commands(content, *existing, fileExists); + content = merge_compile_commands(content, *doc->content, fileExists); } auto finalJson = nlohmann::json::parse(content, nullptr, /*allow_exceptions=*/false); @@ -398,58 +604,75 @@ publish_compile_commands( sort_entries_by_file(finalJson); content = finalJson.dump(2) + "\n"; - if (existing && *existing == content) { - return CompileCommandsWriteResult{false, finalJson.size()}; - } + if (doc->content && *doc->content == content) + return CompileCommandsWriteResult{false, finalJson.size(), 0}; - static std::atomic sequence{0}; - const auto nonce = std::random_device{}(); - // 临时文件和链接目标同目录,避免 rename 跨文件系统;随机量降低跨进程碰撞概率。 - auto temp = publishPath.parent_path() - / std::format(".{}.tmp.{}.{}.{}", publishPath.filename().string(), - std::chrono::steady_clock::now().time_since_epoch().count(), - static_cast(nonce), - sequence.fetch_add(1, std::memory_order_relaxed)); - auto cleanup_temp = [&] { - std::error_code cleanupEc; - std::filesystem::remove(temp, cleanupEc); - }; + if (auto r = atomic_replace_document(doc->publishPath, content, replaceFile); !r) + return std::unexpected(r.error()); - std::ofstream output(temp, std::ios::binary | std::ios::trunc); - if (!output) { - return std::unexpected(write_error(std::format( - "cannot open temporary compile database '{}'", temp.string()))); - } - output << content; - output.flush(); - if (!output) { - output.close(); - cleanup_temp(); - return std::unexpected(write_error(std::format( - "cannot write temporary compile database '{}'", temp.string()))); + return CompileCommandsWriteResult{true, finalJson.size(), 0}; +} + +std::expected +publish_root_compile_commands( + const std::filesystem::path& configDbPath, + const std::filesystem::path& rootPath, + const std::filesystem::path& targetRoot, + const std::filesystem::path& mcppHome, + ReplaceFile replaceFile) { + std::string content; + { + std::ifstream input(configDbPath, std::ios::binary); + if (!input) { + return std::unexpected(write_error(std::format( + "cannot read configuration compile database '{}'", configDbPath.string()))); + } + std::stringstream ss; + ss << input.rdbuf(); + if (input.bad()) { + return std::unexpected(write_error(std::format( + "cannot read configuration compile database '{}'", configDbPath.string()))); + } + content = ss.str(); } - output.close(); - if (!output) { - cleanup_temp(); + + auto configJson = nlohmann::json::parse(content, nullptr, /*allow_exceptions=*/false); + if (configJson.is_discarded() || !configJson.is_array()) { return std::unexpected(write_error(std::format( - "cannot close temporary compile database '{}'", temp.string()))); + "configuration compile database '{}' is not a JSON array", + configDbPath.string()))); } - std::error_code ec; - if (!replaceFile(temp, publishPath, ec)) { - cleanup_temp(); - return std::unexpected(write_error(std::format( - "cannot replace '{}': {}", publishPath.string(), ec.message()))); + auto doc = read_existing_document(rootPath); + if (!doc) return std::unexpected(doc.error()); + + // §3.2 item 2: identical → untouched, so clangd is not triggered for + // nothing (and the fast path costs one read, not one write, on a hit). + if (doc->content && *doc->content == content) + return CompileCommandsWriteResult{false, configJson.size(), 0}; + + // §3.2 item 3: count the replaced file's foreign entries BEFORE replacing + // it, so the caller can warn with a number. A prior file this parser + // cannot read as a JSON array at all (another tool's own format, or one + // corrupted) is replaced the same way, silently — there is no entry + // count to report for a document that was never a list of entries. + std::size_t foreign = 0; + if (doc->content) { + auto priorJson = nlohmann::json::parse(*doc->content, nullptr, false); + if (!priorJson.is_discarded() && priorJson.is_array()) { + for (auto const& e : priorJson) + if (!is_mcpps_entry(e, targetRoot, mcppHome)) ++foreign; + } } - return CompileCommandsWriteResult{true, finalJson.size()}; + if (auto r = atomic_replace_document(doc->publishPath, content, replaceFile); !r) + return std::unexpected(r.error()); + + return CompileCommandsWriteResult{true, configJson.size(), foreign}; } std::expected write_compile_commands(const BuildPlan& plan, const CompileFlags& flags) { - auto path = plan.compileDbPath.empty() - ? plan.projectRoot / "compile_commands.json" - : plan.compileDbPath; // A JSON document holds UTF-8 text only, and the serialiser throws on any // other byte (`type_error.316`). The entry points refuse or skip a path // with no UTF-8 spelling (#693), so a string reaching this point in another @@ -464,11 +687,31 @@ write_compile_commands(const BuildPlan& plan, const CompileFlags& flags) { "it cannot be written as JSON, which holds UTF-8 text only ({})", e.what()))); } - return publish_compile_commands( - path, fresh, - [](const std::filesystem::path& candidate) { - return std::filesystem::exists(candidate); - }); + + auto fileExists = [](const std::filesystem::path& candidate) { + return std::filesystem::exists(candidate); + }; + + // §3.2 item 1: the configuration's own database, merged within itself. + auto configPath = plan.outputDir / "compile_commands.json"; + auto configResult = publish_compile_commands(configPath, fresh, fileExists); + if (!configResult) return configResult; + + // §3.2 items 2-3: the root is a copy of it, replaced whole. `emit` + // (`mcpp emit build-database`) never reaches this function (SPEC-005 + // R2.1: it writes nothing into the project), so in practice + // `plan.compileDbPath` is always the project root's file here; the + // fallback exists because the field itself is more general (plan.cppm). + auto rootPath = plan.compileDbPath.empty() + ? plan.projectRoot / "compile_commands.json" + : plan.compileDbPath; + auto targetRoot = plan.outputDir.parent_path().parent_path(); + auto rootResult = publish_root_compile_commands( + configPath, rootPath, targetRoot, mcpp::home::root()); + if (!rootResult) return rootResult; + + return CompileCommandsWriteResult{ + rootResult->changed, configResult->commandCount, rootResult->foreignEntries}; } } // namespace mcpp::build diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 6694eec6..00ca62ca 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -11,6 +11,7 @@ export module mcpp.build.execute; import std; import mcpp.build.build_program; // #359 glob inputs the mtime sweep cannot see +import mcpp.build.compile_commands; // C1: the fast path restores a deleted root CDB import mcpp.build.prepare; import mcpp.pack; // #622 A10: mcpp::pack::Options / Format import mcpp.pack.pipeline; // #622 A10: build_and_pack, for `run --format` @@ -1405,6 +1406,37 @@ export int list_runners(const std::string& package_filter, return 0; } +// C1: a deleted root compile_commands.json used to stay deleted forever on +// the fast path, because nothing on it ever reached a writer (the fast path +// is defined as "skip preparation" — see the hooksActive check below, and +// design .agents/docs/2026-09-26-compile-database-and-issue-699-design.md +// §3.2 item 4). This restores it from the configuration's own database, +// already on disk at `outputDir` — no plan is built, so P3 (the fast path +// replays a build) holds: the root file is a copy, never a fresh plan. +// Errors are reported as warnings and never fail the fast build itself: a +// permission problem here is exactly what a normal build would already warn +// about (`write_compile_commands`), not a reason to fall back to the full +// path. +void restore_root_compile_commands(const std::filesystem::path& projectRoot, + const std::filesystem::path& outputDir) { + auto configPath = outputDir / "compile_commands.json"; + std::error_code ec; + if (!std::filesystem::exists(configPath, ec) || ec) return; + auto rootPath = projectRoot / "compile_commands.json"; + auto targetRoot = outputDir.parent_path().parent_path(); + auto result = mcpp::build::publish_root_compile_commands( + configPath, rootPath, targetRoot, mcpp::home::root()); + if (!result) { + mcpp::ui::warning(std::format( + "compile_commands.json was not updated: {}", result.error().message)); + } else if (result->foreignEntries > 0) { + mcpp::ui::warning(std::format( + "compile_commands.json held {} entr{} mcpp did not write; " + "the file now holds mcpp's configuration", + result->foreignEntries, result->foreignEntries == 1 ? "y" : "ies")); + } +} + export std::optional try_fast_build(const std::filesystem::path& projectRoot, bool verbose, bool no_cache, std::string_view currentTarget = "") { @@ -1519,6 +1551,11 @@ export std::optional try_fast_build(const std::filesystem::path& projectRoo if (!validatedBefore) return std::nullopt; // All inputs are older than build.ninja → fast-path: just run ninja. + // C1: this configuration is confirmed current, so the root database is + // restored here rather than left to whatever a NEXT full build happens to + // do — a project that never edits a source again would otherwise never + // see it back. + restore_root_compile_commands(projectRoot, outputDir); std::chrono::milliseconds elapsed{}; auto rc = run_ninja_fast(ninjaProgram, outputDir, ninjaPath, verbose, runtimeEnvKey, runtimeEnvValue, &elapsed); @@ -1667,7 +1704,8 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, if (!validatedBefore) return std::nullopt; // Fresh → run ninja (picks up any incremental object/link work) then - // exec the cached exe path directly. + // exec the cached exe path directly. C1, same reason as try_fast_build's. + restore_root_compile_commands(projectRoot, outputDir); auto rc = run_ninja_fast(ninjaProgram, outputDir, ninjaPath, /*verbose=*/false, match->runtimeEnvKey, match->runtimeEnvValue); if (!rc) return std::nullopt; diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 2d260ee7..1538f76c 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -3484,6 +3484,14 @@ std::expected NinjaBackend::build(const BuildPlan& plan } mcpp::ui::warning(std::format( "compile_commands.json was not updated: {}", cdb.error().message)); + } else if (cdb->foreignEntries > 0) { + // §3.2 item 3: the root file is replaced whole, never merged; when it + // held another writer's entries, say so once rather than silently + // discard them. + mcpp::ui::warning(std::format( + "compile_commands.json held {} entr{} mcpp did not write; " + "the file now holds mcpp's configuration", + cdb->foreignEntries, cdb->foreignEntries == 1 ? "y" : "ies")); } // A SHARED LIBRARY ON A TARGET WHOSE LINK IS DRIVEN BY THE LINKER, SAID diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 85d82d25..f9f0f68b 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -224,6 +224,49 @@ StaticPlacement place_static_packages( const std::map& sharedImages, const std::set& boundaries); +// The standard-library units this configuration's build compiles, when it +// imports `std` (design 2026-09-26 §3.5, D5a/D5b). Populated once, in +// prepare.cppm, by recovering the invocation from the command mcpp actually +// ran to build the module into the shared std cache +// (mcpp.toolchain.stdmod::StdModuleDescription) — recovered here, on the +// plan, rather than separately by each renderer, so compile_commands.json, +// `emit --spec compile-commands` and the S1 document cannot list different +// arguments for the same unit (P1, mcpp.build.compile_commands's header). +// Zero, one (`std`) or two (`std`, `std.compat`) entries. +struct StdModuleUnit { + std::filesystem::path source; + std::filesystem::path workDirectory; // the shared std cache, not outputDir + std::vector arguments; // driver first, as recovered + std::filesystem::path object; + std::filesystem::path bmi; // the BMI the build writes (S1 `provides`) + std::string module; // "std" | "std.compat" + std::vector requiresModules; // {} for std, {"std"} for std.compat +}; + +// Splits a command string mcpp rendered for the host shell into words, +// undoing its quoting: POSIX `sh` rules, or the Microsoft C runtime's rules +// on Windows. The reader is mcpp::manifest::host_command_words; this name is +// kept for the standard-library units' recovery below. +std::vector split_command_words(std::string_view command, bool windows); + +// The working directory and argument vector of the command in `commands` +// whose words name `source`, recovered from the rendering: a leading `cd`, an +// `env` word and environment assignments, and redirections are removed. A +// driver path that the rendering left unquoted despite a space is rejoined. +// Empty when no command names the source. Used to populate +// StdModuleUnit::workDirectory/arguments (prepare.cppm) from +// mcpp.toolchain.stdmod::StdModuleDescription's recorded commands. +struct RecoveredInvocation { + std::filesystem::path workDirectory; + std::vector arguments; +}; +std::optional recover_invocation( + const std::vector& commands, + const std::filesystem::path& source, + const std::filesystem::path& driver, + const std::filesystem::path& defaultDirectory, + bool windows); + struct BuildPlan { mcpp::manifest::Manifest manifest; // Packages whose declared `[build] c_standard` the compiler does not apply, @@ -307,6 +350,8 @@ struct BuildPlan { // (possibly read-only) registry directory, and deriving the path would put // an IDE database there. Empty → projectRoot, the historical default. std::filesystem::path compileDbPath; + // See StdModuleUnit above. Empty when the build does not import `std`. + std::vector stdModuleUnits; // GCC only: a specs file that replaces the pristine `*link:`, so the // payload's own (patched by every home that ever installed against it) // cannot inject rpath entries into this build's artifacts. Empty for @@ -2410,4 +2455,108 @@ package_link_objects(const BuildPlan& plan, std::string_view packageName) { return objects; } +namespace { + +// `V=x`: an environment assignment word, the shape mcpp's own `env` prefix +// renders before a driver's argv. Recognized so it can be skipped rather +// than mistaken for the driver or an input path. +bool std_command_is_assignment(std::string_view w) { + auto eq = w.find('='); + if (eq == std::string_view::npos || eq == 0) return false; + if (!(std::isalpha(static_cast(w[0])) || w[0] == '_')) return false; + for (std::size_t i = 1; i < eq; ++i) { + const unsigned char c = static_cast(w[i]); + if (!(std::isalnum(c) || c == '_')) return false; + } + return true; +} + +// `2>&1`, `>file`, `nul`, ``, `2>`, `<`, +// `>>`: the target is the next word. +enum class StdCommandRedirect { None, Attached, Detached }; +StdCommandRedirect std_command_redirect_kind(std::string_view w) { + std::size_t i = 0; + while (i < w.size() && std::isdigit(static_cast(w[i]))) ++i; + if (i >= w.size() || (w[i] != '>' && w[i] != '<')) return StdCommandRedirect::None; + std::size_t j = i + 1; + if (j < w.size() && w[j] == w[i]) ++j; // `>>` + return j == w.size() ? StdCommandRedirect::Detached : StdCommandRedirect::Attached; +} + +bool std_command_names_path(std::string_view word, const std::filesystem::path& path, + const std::filesystem::path& cwd) { + std::filesystem::path w{std::string(word)}; + const auto want = path.lexically_normal(); + if (w.lexically_normal() == want) return true; + if (!w.is_absolute() && !cwd.empty() && (cwd / w).lexically_normal() == want) + return true; + return false; +} + +} // namespace + +std::vector split_command_words(std::string_view s, bool windows) { + return mcpp::manifest::host_command_words(s, windows); +} + +std::optional recover_invocation( + const std::vector& commands, + const std::filesystem::path& source, + const std::filesystem::path& driver, + const std::filesystem::path& defaultDirectory, + bool windows) { + const std::string driverText = driver.string(); + for (auto const& command : commands) { + auto words = split_command_words(command, windows); + std::vector> segments(1); + for (auto& w : words) { + if (w == "&&") segments.emplace_back(); + else segments.back().push_back(std::move(w)); + } + std::filesystem::path cwd; + for (auto& seg : segments) { + if (seg.empty()) continue; + if (seg.front() == "cd") { + std::size_t k = 1; + if (k < seg.size() && (seg[k] == "/d" || seg[k] == "/D")) ++k; + if (k < seg.size()) cwd = std::filesystem::path{seg[k]}; + continue; + } + std::size_t b = 0; + if (b < seg.size() && seg[b] == "env") ++b; + while (b < seg.size() && std_command_is_assignment(seg[b])) ++b; + std::vector argv; + for (std::size_t k = b; k < seg.size(); ++k) { + switch (std_command_redirect_kind(seg[k])) { + case StdCommandRedirect::Attached: continue; + case StdCommandRedirect::Detached: ++k; continue; + case StdCommandRedirect::None: argv.push_back(seg[k]); + } + } + if (argv.empty()) continue; + // A driver path rendered without quotes splits at its spaces. + if (argv.front() != driverText + && driverText.find(' ') != std::string::npos) { + std::string joined = argv.front(); + std::size_t k = 1; + while (k < argv.size() && joined.size() < driverText.size()) { + joined += ' '; + joined += argv[k]; + ++k; + } + if (joined == driverText) { + argv.erase(argv.begin() + 1, argv.begin() + static_cast(k)); + argv.front() = driverText; + } + } + const bool named = std::ranges::any_of(argv, [&](const std::string& w) { + return std_command_names_path(w, source, cwd.empty() ? defaultDirectory : cwd); + }); + if (!named) continue; + return RecoveredInvocation{cwd.empty() ? defaultDirectory : cwd, std::move(argv)}; + } + } + return std::nullopt; +} + } // namespace mcpp::build diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index c089c306..427f58f3 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -13331,6 +13331,22 @@ prepare_build(bool print_fingerprint, stdObjectPath = sm->objectPath; stdCompatBmiPath = sm->compatBmiPath; stdCompatObjectPath = sm->compatObjectPath; + // C5 / D5a (design 2026-09-26 §3.5): compile_commands.json and the + // S1 document list the standard-library units too, so the plan + // needs the commands mcpp ran to build them (§13396 below), not + // only their output paths. `describe_std_module` is the pure + // derivation `ensure_built` itself reads before running anything + // (mcpp.toolchain.stdmod's header); calling it again here starts + // no process and cannot name a different command or directory. + // A failure here is not this build's failure -- `ensure_built` + // above already succeeded with the same inputs -- so it only + // means the description is unavailable for the plan, silently. + auto described = mcpp::toolchain::describe_std_module( + *tc, m->package.standard, stdFlagAndDialect, + mcpp::platform::macos::deployment_target( + stdTargetIsMacos, m->buildConfig.macosDeploymentTarget), + mcpp::toolchain::default_cache_root(), stdCrt); + if (described) describedStdModule = std::move(*described); } } @@ -13367,7 +13383,12 @@ prepare_build(bool print_fingerprint, } ctx.stdBmi = stdBmiPath; ctx.stdObject = stdObjectPath; - ctx.stdModule = std::move(describedStdModule); + // Copied, not moved: `describedStdModule` is read again once `ctx.plan` + // exists (below), to recover the standard-library units' commands onto + // it (StdModuleUnit, C5 / D5a-b). A `std::optional` move leaves the + // source engaged with a moved-from value, so a plain move here would + // hand build_database.cppm's render() a value and the plan an empty one. + ctx.stdModule = describedStdModule; // Every directory a package payload may legitimately have been INSTALLED // into. There is more than one: the global registry, plus the two // project-local data roots a custom git index installs into @@ -13623,6 +13644,48 @@ prepare_build(bool print_fingerprint, // plan, and every reader of `compute_flags` runs after this line. ctx.plan.targetSide = resolvedTargetSide; + // C5 / D5a-b (design 2026-09-26 §3.5): the standard-library units this + // configuration's build compiles, when it imports `std`. Recovered here, + // once, from the SAME command derivation `ensure_built` and + // `describe_std_module` both read (mcpp.toolchain.stdmod), and carried on + // the plan (BuildPlan::stdModuleUnits) so compile_commands.json, + // `emit --spec compile-commands` and the S1 document render the exact + // same record and cannot disagree (P1, mcpp.build.compile_commands). + if (describedStdModule) { + const auto& sm = *describedStdModule; + auto add_std_unit = [&](const std::filesystem::path& source, + const std::vector& commands, + const std::filesystem::path& object, + const std::filesystem::path& bmi, + std::string_view module, + std::vector requiresModules) { + if (source.empty() || commands.empty()) return; + auto inv = mcpp::build::recover_invocation( + commands, source, tc->binaryPath, sm.cacheDir, + mcpp::platform::is_windows); + if (!inv) { + planNotes.push_back({"MCPP_BUILD_DATABASE_STD_UNIT_UNDESCRIBED", + std::format("no command that builds the {} module names its " + "source '{}'; the unit is not listed", + module, source.string())}); + return; + } + ctx.plan.stdModuleUnits.push_back(mcpp::build::StdModuleUnit{ + .source = source, + .workDirectory = std::move(inv->workDirectory), + .arguments = std::move(inv->arguments), + .object = object, + .bmi = bmi, + .module = std::string(module), + .requiresModules = std::move(requiresModules), + }); + }; + add_std_unit(tc->stdModuleSource, sm.stdCommands, sm.objectPath, + sm.bmiPath, "std", {}); + add_std_unit(tc->stdCompatSource, sm.compatCommands, sm.compatObjectPath, + sm.compatBmiPath, "std.compat", {"std"}); + } + // A DEPENDENCY'S C++ SHARED LIBRARY OVER A C++ RUNTIME THAT IS A PACKAGE // (#641, item 5). // diff --git a/src/scaffold/create.cppm b/src/scaffold/create.cppm index f83151ea..9d7d4f7d 100644 --- a/src/scaffold/create.cppm +++ b/src/scaffold/create.cppm @@ -403,8 +403,14 @@ int main() { { // `.mcpp/` is the per-project xlings sandbox (and, when no MCPP_HOME // can be resolved, the local BMI cache) — build state, never sources. + // `compile_commands.json` at the root names absolute paths of one + // machine's build (design 2026-09-26 §3.2): it is a copy of the + // current configuration's own database under `target/`, replaced + // whole on the next build, so a checkout gains nothing from shipping + // it and loses a warning the first time another machine's copy + // disagrees with this one. if (auto written = mcpp::scaffold::write_text_file( - root / ".gitignore", "target/\n.mcpp/\n"); !written) { + root / ".gitignore", "target/\n.mcpp/\ncompile_commands.json\n"); !written) { mcpp::ui::error(written.error()); return 1; } diff --git a/tests/e2e/211_configure_only_cdb.sh b/tests/e2e/211_configure_only_cdb.sh index 17f884cb..fc39e19f 100755 --- a/tests/e2e/211_configure_only_cdb.sh +++ b/tests/e2e/211_configure_only_cdb.sh @@ -66,23 +66,25 @@ grep -q 'tests[\/][\/]*test_smoke.cpp' compile_commands.json || { exit 1 } if command -v python3 >/dev/null 2>&1; then - python3 - compile_commands.json <<'PY' + # The devkit path is passed in directly rather than derived from the CDB's + # `directory`: since C3 (design 2026-09-26 §3.3), `directory` is the + # OUTPUT directory the compiler runs in, not the project root, so a + # sibling fixture's path can no longer be recovered from it. + python3 - compile_commands.json "$TMP/devkit/include" <<'PY' import json, sys entries = json.load(open(sys.argv[1], encoding="utf-8")) +devkit_include = sys.argv[2] normal = lambda p: p.replace("\\", "/").rstrip("/") test = next(e for e in entries if normal(e["file"]).endswith("/tests/test_smoke.cpp")) main = next(e for e in entries if normal(e["file"]).endswith("/src/main.cpp")) args = test["arguments"] -# Windows 原生进程与 MSYS 可能用不同根路径表示同一临时目录, -# 因此从 CDB 的 directory 字段推导相邻 devkit 路径。 -fixture = normal(test["directory"]).rsplit("/", 1)[0] -expected_include = f"{fixture}/devkit/include".casefold() +expected_include = normal(devkit_include).casefold() include_args = { normal(a[2:]).casefold() for a in args if a[:2].casefold() == "-i" } -assert expected_include in include_args, args +assert expected_include in include_args, (expected_include, args) assert any("MCPP_CONFIGURE_ONLY_TEST_FLAG=1" in a for a in args), args assert not any("MCPP_CONFIGURE_ONLY_TEST_FLAG=1" in a for a in main["arguments"]), main PY diff --git a/tests/e2e/47_cdb_prebuilt_module_path_abs.sh b/tests/e2e/47_cdb_prebuilt_module_path_abs.sh index 4b0cf2fb..8a9d5843 100755 --- a/tests/e2e/47_cdb_prebuilt_module_path_abs.sh +++ b/tests/e2e/47_cdb_prebuilt_module_path_abs.sh @@ -3,12 +3,14 @@ # 47_cdb_prebuilt_module_path_abs.sh — `-fprebuilt-module-path` in # compile_commands.json must be an ABSOLUTE path, NOT a bare `pcm.cache`, # AND must not carry ninja-escape artefacts like `C$:` on Windows. -# Reason: CDB `directory` is the project root and clangd does `cd -# directory` before running the args, so a bare relative path points at -# `/pcm.cache` (missing) and a `C$:` prefix is treated as a -# literal string, not a Windows drive letter. Both modes silently break -# clangd's module resolution while `mcpp build` itself keeps working -# (ninja runs from outputDir AND unescapes its own escape sequences). +# Reason: since design 2026-09-26 §3.3 (C3), CDB `directory` IS the output +# directory the compiler runs in, so a bare relative `pcm.cache` would in +# fact resolve there too — but the flag is still rendered from the same +# absolute path the build's own command line uses (flags.cppm), and a +# `C$:` prefix would still be treated as a literal string, not a Windows +# drive letter, wherever `directory` points. Both modes would silently +# break clangd's module resolution while `mcpp build` itself keeps working +# (ninja unescapes its own escape sequences). set -e TMP=$(mktemp -d) @@ -89,9 +91,9 @@ while IFS= read -r v; do : else echo "FAIL: value is relative: '$v'" - echo " CDB 'directory' is the project root, but the BMI cache" - echo " lives under target/// — clangd resolves to" - echo " the wrong location and module imports fail." + echo " the flag must carry the same absolute BMI cache path" + echo " the build's own command line uses, whatever 'directory'" + echo " resolves to." fail=1 fi diff --git a/tests/e2e/688_emit_build_database.sh b/tests/e2e/688_emit_build_database.sh index 0ccffd86..70a9a59e 100755 --- a/tests/e2e/688_emit_build_database.sh +++ b/tests/e2e/688_emit_build_database.sh @@ -200,17 +200,22 @@ EOF echo "ok: J, baseline and local arguments, private, config-files; K, the test's imports" # ── F (the S1 and compile-commands renderings agree) ────────────────────── +# D5a: the standard-library units are translation units of the build like any +# other (S1-12-1), so they are in BOTH renderings now, not just the S1 one -- +# no `mcpp:std` exclusion on the S1 side any more. "$MCPP" emit build-database --spec compile-commands > "$OUT/cc.json" 2> "$OUT/cc.err" \ || fail "F: --spec compile-commands exited non-zero" "$OUT/cc.err" "$PY" - "$OUT/env.json" "$OUT/cc.json" <<'EOF' || fail "F: the two renderings disagree" "$OUT/cc.json" import json, sys db = json.load(open(sys.argv[1]))["data"]["database"] cc = json.load(open(sys.argv[2])) -s1 = {u["source"]: u["arguments"] for s in db["sets"] if s["name"] != "mcpp:std" for u in s["translation-units"]} +s1 = {u["source"]: u["arguments"] for s in db["sets"] for u in s["translation-units"]} cdb = {e["file"]: e["arguments"] for e in cc} assert s1 == cdb, (sorted(s1), sorted(cdb)) +std_files = {u["source"] for s in db["sets"] if s["name"] == "mcpp:std" for u in s["translation-units"]} +assert std_files and std_files <= set(cdb), (std_files, sorted(cdb)) EOF -echo "ok: F, the S1 units and the compile-commands entries carry the same arguments" +echo "ok: F, the S1 units and the compile-commands entries carry the same arguments, std included" # ── G ────────────────────────────────────────────────────────────────────── set +e @@ -305,14 +310,23 @@ import json, sys emitted = json.load(open(sys.argv[1])) written = json.load(open(sys.argv[2])) # The one difference by construction is where the build writes: the planning -# pass writes under its work directory, configure-only under the project. +# pass writes under its work directory, configure-only under the project. The +# standard-library units' `output` names the shared std cache instead (D5a), +# outside either root, so the mapping is learned from a PROJECT entry -- +# `compile_commands.json` is sorted by `file` and a std source's absolute +# path may sort before a project one, so index 0 is not reliably a project +# entry any more. def slash(text): # One spelling for the comparison: a Windows argument may name a path with # either separator, and the mapping below is textual. return text.replace("\\", "/") -def write_root(entry): - return slash(entry["output"]).split("/target/")[0] -work, project = write_root(emitted[0]), write_root(written[0]) +def write_root(entries): + for e in entries: + s = slash(e["output"]) + if "/target/" in s: + return s.split("/target/")[0] + raise AssertionError("no project entry (an /target/ output) found") +work, project = write_root(emitted), write_root(written) def mapped(args): return [slash(a).replace(work, project) for a in args] e = {slash(x["file"]): mapped(x["arguments"]) for x in emitted} diff --git a/tests/e2e/781_deleted_root_cdb_is_restored.sh b/tests/e2e/781_deleted_root_cdb_is_restored.sh new file mode 100755 index 00000000..b61030b0 --- /dev/null +++ b/tests/e2e/781_deleted_root_cdb_is_restored.sh @@ -0,0 +1,67 @@ +#!/usr/bin/env bash +# requires: +# 781_deleted_root_cdb_is_restored.sh — C1 (design 2026-09-26 +# .agents/docs/2026-09-26-compile-database-and-issue-699-design.md §3.2 item +# 4): a deleted root compile_commands.json is restored by the very next +# `mcpp build`, through the FAST PATH -- no plan at all, since nothing on this +# tree changed. Before the fix (2026.9.26.1 and earlier), the fast path never +# reached a compile-database writer, so a deleted database stayed deleted +# forever (#397 C-1, #677 B1, report §3.6). +set -euo pipefail + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT + +cd "$TMP" +"$MCPP" new app > /dev/null +cd app +# Pinned rather than left to the machine's default: the property under test +# does not depend on which toolchain is used, and pinning keeps the test's +# reading independent of shared state outside this tree. +TCFLAG=(--toolchain gcc@16.1.0) + +"$MCPP" build "${TCFLAG[@]}" > build1.log 2>&1 || { cat build1.log; echo "FAIL: the first build failed"; exit 1; } +[[ -s compile_commands.json ]] || { echo "FAIL: no compile_commands.json after the first build"; exit 1; } + +config_cdb=$(find target -name compile_commands.json | head -1) +[[ -n "$config_cdb" ]] || { echo "FAIL: no configuration database under target/"; exit 1; } +before=$(cat "$config_cdb") + +rm compile_commands.json + +"$MCPP" build "${TCFLAG[@]}" > build2.log 2>&1 || { + cat build2.log + echo "FAIL: the build after deleting the root database failed" + exit 1 +} + +[[ -f compile_commands.json ]] || { + echo "FAIL: compile_commands.json was not restored" + cat build2.log + exit 1 +} + +# "Without a plan": nothing in the tree changed since the first build, so a +# real prepare pass has nothing to compile -- the restore must not trigger +# one. A plan prints a "Compiling " line; the fast path prints only +# "Finished ... in