diff --git a/.agents/docs/2026-09-26-compile-database-and-issue-699-design.md b/.agents/docs/2026-09-26-compile-database-and-issue-699-design.md new file mode 100644 index 000000000..cc1796c4f --- /dev/null +++ b/.agents/docs/2026-09-26-compile-database-and-issue-699-design.md @@ -0,0 +1,948 @@ +--- +subject: design +status: landed +--- + +# The compile database, `emit build-database`, and #701/#702: triage against the specifications, and one design + +- Sources: + - The report *compile_commands.json 对比分析：xmake vs mcpp* + (`/home/speak/test/mcpp/hello/cdb-report/README.md`, 2026-09-26), measured with mcpp + 2026.9.21.3 and clangd 22.1.8. Cited as "the report". Its xmake comparison is a reference + only; every verdict below is taken against a specification. + - mcpp-community/mcpp#699 (2026-09-25), *emit build-database for IDEs: one failing member + loses the whole database, and build programs cannot tell a plan pass*, from + Sunrisepeak/mcpp-language-server#23 and #24 on the GalTranslPP workspace. + - mcpp-community/mcpp#701 and pull request #702 (2026-09-25, head `c57c12f2`): a build + program declares a runtime library directory, and a passing check moves its stamp. Their + consumer is mcpp-plugins 0.13.0 (`deps-vcpkg`, `deps-cmake`, `rules-qt`), whose design + record (branch `feat/deps-vcpkg-rules-qt`, `.agents/docs/2026-09-26-deps-vcpkg-rules-qt-design.md`, + decision D8) defers #702 and builds on the released 2026.9.26.1 instead. + - Earlier records of C1: #397 item C-1 (2026-08-09, P0) and #677 item B1 (2026-09-20). +- Specifications: the JSON Compilation Database + (); S1 *C++ Build Database: IDE + Profile* 0.2.0 and S2 *discovery*, both in `Sunrisepeak/mcpp-language-server/docs/specs/`, + which SPEC-005 implements; SPEC-005; docs/50; docs/04 §2.11; docs/30. +- Consumers: **mcpp-language-server (mcppls) is the primary one.** It reads the S1 document + that `emit build-database` prints and drives clangd through a database it writes itself + (its `docs/90-architecture.md`). `compile_commands.json` serves the tools that read the JSON + format directly: clangd without mcppls, clang-tidy, and others. +- Basis: `origin/main` 3a6ac7f4 (mcpp 2026.9.26.1 and one records-only commit); #702 at + `c57c12f2`. Code citations are to those commits. +- Measurements: Linux x86_64 (Ubuntu 24.04); the released mcpp 2026.9.26.1 invoked by its xlings + store path; toolchains llvm@22.1.8, llvm@20.1.7, gcc@16.1.0; clangd and clang-tidy 22.1.8; CMake + 4.4.2. Every project is a copy or a fixture in a scratch directory. Appendix A holds the + readings. +- Status: landed in mcpp 2026.9.26.2 through #702 (§11, §12). + - Revision 1 (2026-09-26): the triage and the first design. + - Revision 2 (2026-09-26): D1 and D3 accepted; mcppls named as the primary consumer; C3 + measured against the specification and three build systems; C5 rewritten around its four + questions; C1 and C2 replaced by one database per configuration (D6); #701 and #702 + reviewed (§5), with the defect F1 they led to; an overall review (§10). + - Revision 3 (2026-09-26): D2, D4, D5, D6, D7 and D8 accepted; F1 filed as #703; the + conformant design of #701/#702 (§5.4: `runtime_search_dir`, the stamp rule, the `prepare` + role, and W, the placement of a Windows program's DLLs); SPEC-007, the build plugin specification (`docs/specs/build-plugins.md`, §5.6); #702 + reused as the single pull request (§7). + - Revision 4 (2026-09-26): D9 to D12 accepted; the implementation plan, its task + dependencies and the cross-repository sequence (§11). + - Revision 5 (2026-09-26): implemented in #702 as mcpp 2026.9.26.2; what the implementation + changed or added to the design, and the tests (§12). + +--- + +## 0. Summary + +| | Finding | Source | Verdict | Repair | +|---|---|---|---|---| +| **C1** | A deleted `compile_commands.json` is not written again by the next `mcpp build` | report §3.6; #397 C-1; #677 B1 | **mcpp defect** | §3.2: the fast path restores the root database from the configuration's database | +| **C2** | The merge resolves `file` against the process's working directory, keeps other tools' entries, and mixes two toolchains in one file | report §3.1; the mixing is new | **mcpp defect** | §3.2: one database per configuration; the root file is replaced, never merged (D6) | +| **C3** | `directory` (and S1 `work-directory`) name the project root; the compiler runs in the output directory | report §3.2 | **mcpp defect.** The JSON format and S1-8-2 define the field as the directory the compiler runs in; CMake, ninja and xmake write that directory | §3.3 (D2) | +| **C4** | A module interface's language flag is missing from both databases | new | **mcpp defect**: clangd cannot handle a `.ixx` entry | §3.4 | +| **C5** | `compile_commands.json` lists no standard-library unit | report §3.3 | **mcpp gap**, contrary to S1-12-1 | §3.5 (D5) | +| **C6** | The database is written at the project root | report §3.5 | by design; the symlink redirect is undocumented | documentation | +| **E1** | One member's planning failure removes every member's sets | #699 item 1 | **mcpp gap** | §4.4 (D1, accepted) | +| **E2** | Under `emit`, a host tool whose build fails ends the requesting member's plan | #699 item 2 | **mcpp gap** | §4.5 | +| **E3** | A failing build program ends its member's plan | #699 item 2 | **mcpp gap** | §4.6 (D3, accepted) | +| **E4** | A signal that tells a build program it runs for a plan | #699 item 2 | **not adopted** | §4.7 (D4) | +| **R1** | #701/#702: `mcpp::runtime_library_dir(dir)` | #701 | the gap is real; as written it extends a field docs/04 retires | §5.4 R1': `runtime_search_dir` on `LinkIntent` (D7, accepted) | +| **R2** | #701/#702: a passing check moves its stamp | #701 | **mcpp defect**; the fix is correct | §5.4 R2: every stamp newer than every input (D10) | +| **F1** | `$ORIGIN` in a user link flag reaches the program as `/../lib` | the plugin record; measured here | **mcpp defect**, filed as #703 | §5.5: SPEC-004 §8 extends to link flags (D8, accepted) | +| **O1** | deps-cmake and deps-vcpkg run installation as `check` actions | the plugin record | design gap: no action role describes construction whose file names are unknown | §5.4 P: the `prepare` role (D9) | +| **O2** | on Windows, a program started by hand does not find DLLs from a runtime search directory | this review | framework gap | §5.4 W: placement after the link (D11) | +| **S7** | a contract for build plugins | this review | new specification | SPEC-007, draft 0.1 (§5.6, D12) | +| | xmake's database; GalTranslPP's incomplete Qt and its environment checks inside build programs | report §3.4; #699 | not mcpp | §3.7, §4.8 | + +Three statements: + +1. **A database states the build's facts, one configuration at a time.** `directory` is where + the compiler runs (C3), `arguments` carry the flags that decide how it reads its input (C4), + and one file holds one configuration (C1, C2). Where mcpp departs from this, it departs from + the JSON format, from S1 and from every build system measured. +2. **mcpp compiles nothing twice.** A second compile of `std` is a reader's, and it happens + because a BMI is readable only by the compiler that wrote it. The database's task is to name + both the provider, so any reader can compile, and the build's BMI, so a matching reader can + reuse it (C5). +3. **#699, #701 and #702 are one family:** a build program configures, actions construct and + verify, and a new engine surface joins the current model rather than a retiring one. + +## 1. Method + +Each claim was reproduced on the current release in a copy or a fixture, located in the code, +and classified against the contract that governs it. A repair is placed where the rule is +stated, with a criterion that fails on 2026.9.26.1. + +| Verdict | Meaning | +|---|---| +| mcpp defect | mcpp violates a contract it states or a specification it claims to follow | +| mcpp gap | mcpp does what its specification says; the specification omits a case its consumer needs | +| by design | deliberate, and the reasons still hold | +| usage / not mcpp | the effect belongs to a project, an environment or another program | + +## 2. Principles + +- **P1. One record, two documents.** `compile_commands.json`, `emit --spec compile-commands` and + the S1 document render one `UnitInvocation` per unit (`src/build/compile_commands.cppm:49-64`; + SPEC-005 R3.7). C3 and C4 change that record; no output is repaired on its own. +- **P2. A field states what the build does.** The JSON format defines `directory` as "the + working directory of the compilation" and `arguments` as the argument vector that "should run + the compilation step"; S1-8-2 defines `work-directory` as the "absolute path of the directory + the compiler runs in". +- **P3. The fast path replays a build.** Every product of the full path is produced by the fast + path or verified by it. +- **P4. A description names what it could not describe, per unit**: a member for a planning + failure, a package's directives for a build-program failure, nothing for a construction + failure. +- **P5. Configuration is not verification.** A build program configures; a `check` action + verifies. `mcpp build` runs both; `emit` runs the first. +- **P6. One program behaviour.** The plan `emit` describes is the plan + `mcpp build --configure-only` computes (R1.2). +- **P7. One database, one configuration.** A configuration is what the fingerprint names: + toolchain, target, profile, features. A file that mixes two configurations describes no build. +- **P8. A new surface joins the current model.** docs/04 §2.11 keeps `[runtime] library_dirs` + "for one compatibility train" and names its successor, `runtime_search_dirs`. +- **P9. The engine provides general mechanisms; a tool's knowledge stays in its plugin.** A gap a + plugin meets is closed in the engine only when it is general, and then by a mechanism that names + no tool (R1', R2, P and W below); vcpkg, CMake and Qt are never named by the engine. + +## 3. The compile database + +### 3.1 Readings on 2026.9.26.1 + +| Claim | Reading on 2026.9.26.1 | Finding | +|---|---|---| +| deleted database not rewritten (report: "not reproduced") | `mcpp build; rm compile_commands.json; mcpp build` prints `Finished dev in 0.00s` and leaves no file, on every attempt; `--no-cache` writes it | C1 | +| relative entries of another writer | kept from the project root (6 entries: `src/main.cpp` beside `/…/src/main.cpp`), pruned from `src/` (4 entries) | C2 | +| (new) two toolchains | after `mcpp test` (llvm) and `mcpp build --toolchain gcc@16.1.0`: three `g++` entries and one `clang++` entry | C2 | +| GCC entries cannot be replayed from `directory` | from `directory`: `test.cppm` and `main.cpp` fail and `gcm.cache/` appears in the project root; from the output directory: 3 of 3 write their objects; clang: 3 of 3 from either | C3 | +| omitted `-x c++-module` is harmless | a `.ixx` interface declared in `module_extensions`: clangd `[fe_expected_compiler_job]`; with the flag, 0 errors; GCC 16 compiles `.ixx` without it | C4 | +| no std unit | §3.5 | C5 | + +### 3.2 One database per configuration (C1, C2; D6) + +**What is wrong today.** `publish_compile_commands` merges the fresh plan into whatever the root +file holds (`src/build/compile_commands.cppm:272-319`): a prior entry survives when its `file`, +compared as written and probed against the process's working directory (`:288-311`), is absent +from the fresh plan and exists. The rule exists to keep the units of an earlier `mcpp test` +(`:69-79`), and it keeps every other entry too: another tool's, and mcpp's own from a previous +toolchain. Separately, the fast path never reaches the writer (`src/build/execute.cppm:1408-1530` +against `src/build/ninja_backend.cppm:3474`), so a deleted file stays deleted (C1); `git clean -fd` +reaches the same state, because the `.gitignore` that `mcpp new` writes lists `target/` and +`.mcpp/` only (`src/scaffold/create.cppm:407`). + +**The identity of a configuration.** mcpp already names every configuration: the output +directory `target//`, derived from the toolchain, target, profile and +features. `mcpp build` and `mcpp test` in one configuration share it (measured: after both, the +build's entries and the test entry name one fingerprint directory; the fingerprint deliberately +covers neither tests nor dev-dependencies, #407). After C3, it is the `directory` of every project +entry. A field of mcpp's own inside an entry is not an option: one unknown key makes clangd +report `Failed to load compilation database` and clang-tidy refuse the file (measured). S1 +consumers ignore unknown fields (S1-11.2-1); JSON-format readers do not. + +**Design.** + +1. **The configuration's database** is `target///compile_commands.json`. + Every command that plans in that configuration (`build`, `test`, `run`, `--configure-only`) + writes it: the fresh plan's entries, plus the entries it already holds whose `file`, + resolved against their `directory`, the fresh plan lacks and which still exist. Every entry + in it was written by mcpp in this configuration, so no ownership test is needed and none is + made. +2. **The root file is a copy of the current configuration's database**, replaced whole and never + merged, and left untouched when identical, so that clangd is not triggered for nothing. + Switching toolchain or profile therefore switches the whole file, and switching back restores + that configuration's entries, its test units included. A symlink at the root is written + through, as today. +3. **Another writer's entries are replaced, and said so.** When the replaced root file held + entries mcpp did not write, one warning states their number and that the file holds mcpp's + configuration. An entry is mcpp's when its `output` lies under this project's `target/` or + under the mcpp home, where the standard-library units of C5 write their objects; `directory` + cannot decide it, because those units run in the std cache. +4. **The fast path restores the root file (C1).** It knows the configuration's directory + (`match->outputDir`) and publishes the root file through the same function as the full path: + when the root file is missing or differs from the configuration's database, it is replaced, + with the same warning. No plan is needed, and P3 holds. +5. `emit build-database` writes neither file (R2.1), and prints its plan as today. +6. The scaffold's `.gitignore` also lists `compile_commands.json`: it names absolute paths of one + machine. + +This replaces the four-condition merge of revision 1. The answer to D6 is **replace across +configurations, merge only within one**, with the output directory as the identifier: a value +mcpp already computes, visible in a standard field, and stable across the commands that should +share a database. + +**Criteria.** (1) e2e: plain build; delete the root file; `mcpp build` restores it without a +plan (fails on 2026.9.26.1). (2) e2e: the root file replaced by another writer's entries; +`mcpp build` leaves only the configuration's entries and warns once. (3) e2e: `mcpp test`, then +`mcpp build`, same configuration: the test entries remain. (4) e2e: llvm, then gcc, then llvm: +the root holds one toolchain's entries each time, and the llvm test entries return with llvm. +(5) unit: the within-configuration merge resolves `file` against `directory`. + +### 3.3 C3: `directory` (D2) + +**The specifications.** The JSON format: "directory: The working directory of the compilation. +All paths specified in the command or file fields must be either absolute or relative to this +directory." S1-8-2: `work-directory` is the "absolute path of the directory the compiler runs +in", and S1-12-1 exports it as `directory`. + +**What mcpp does.** `unit_invocations` writes `plan.projectRoot` (`compile_commands.cppm:231`); +ninja runs every compile in the output directory (`ninja -C `: `execute.cppm:1193`, +`ninja_backend.cppm:3653`). The standard-library units already state their real directory, +recovered from the command mcpp runs (`src/build/build_database.cppm:315-372`). + +**What other producers do** (measured; the same two-directory project for CMake): + +| Producer | `directory` | Where the compiler runs | Replay from `directory` | +|---|---|---|---| +| CMake 4.4.2, Ninja generator | the build directory | the build directory | object written | +| CMake 4.4.2, Unix Makefiles | the target's build subdirectory (`/lib`) | that subdirectory | object written | +| `ninja -t compdb` over mcpp's own `build.ninja` | `target//` | the same | | +| xmake (the report's sample) | the project root | the project root: its `file` and `-o` are relative to it, and the report's replay passed 3 of 3 | objects written | +| mcpp 2026.9.26.1 | the project root | `target//` | GCC importers fail; `gcm.cache/` written into the source tree | + +Every producer names the directory its compiler runs in, whatever that directory is; mcpp is the +one that does not, and ninja reading mcpp's own graph names the directory mcpp should. + +**Repair.** `directory` and `work-directory` are the output directory, for every unit and every +toolchain. The directory exists whenever the database does: the full path writes `build.ninja` +into it, and `emit` plans into a work directory under the mcpp home (measured: it exists after +`emit`). + +**Consequence.** A tool that changes into `directory` needs it to exist: clang-tidy 22.1.8 aborts +with `LLVM ERROR: Cannot chdir into "…"!` when it does not, while clangd logs +`VFS: failed to set CWD` and continues (both measured). After `mcpp clean` the root file names a +directory that is gone until the next build; the module arguments of the same entries +(`-fmodule-file=`, `-fprebuilt-module-path=`) already point into it, so a cleaned tree already +fails for every module unit. With §3.2, the next build replaces the root file anyway. + +**Criteria.** (1) e2e, gcc and llvm rows: every entry replayed from its `directory` writes its +object, and no `gcm.cache/` appears under the project root (fails for gcc on 2026.9.26.1). +(2) unit: a project unit's `directory` is `plan.outputDir`. e2e 211 derives a sibling fixture +from `directory` (`tests/e2e/211_configure_only_cdb.sh:77-78`) and changes with it. + +### 3.4 C4: the interface language flag + +The build states a module interface's language explicitly (`BmiTraits::moduleInterfaceLangFlag`: +`-x c++-module` for clang, `-x c++` for GCC, `/interface /TP` for MSVC; +`modules/toolchain-model/src/model.cppm:456-683`), so that no driver infers it from an extension +(`modules/source-kind/src/source_kind.cppm:360-371`: "mcpp never lets it guess"). +`unit_invocations` omits it (`compile_commands.cppm:234-246`), and a reader infers after all: +clang does not know `.ixx` and hands the file to the linker, so clangd has no compiler job for it +(measured); GCC 16 compiles `.ixx` and `.cppm` without the flag. + +**Repair.** The record carries the flag at the position the build uses, before `-c `, +for every unit whose kind produces a BMI; in S1 it lands in the interface units' +`local-arguments`. `-MMD -MF` and `-fmodule-output=` stay omitted: they name side outputs and do +not change how the input is read. + +**Criteria.** (1) unit: an interface unit's arguments carry the dialect's flag before `-c`; an +implementation unit's do not. (2) e2e, llvm row, `module_extensions = [".ixx"]`: the interface +entry, replayed from its `directory`, writes an object. **Open:** whether clangd in clang-cl +mode accepts `/interface` is measured on Windows CI before the MSVC form is emitted. + +### 3.5 C5: the standard-library units (D5) + +**Does mcpp compile `std` twice?** No. mcpp builds the standard-library modules once per +toolchain, standard and flags into the shared std cache under the mcpp home, which every project +reuses; `emit` describes them and compiles nothing (R2.2). Listing a unit in a database compiles +nothing on mcpp's side. + +**Was leaving them out of `compile_commands.json` correct?** It was a choice (SPEC-005 R4.1), +and the wrong one. + +- The JSON format describes "one way a translation unit is compiled in the project". The build + compiles `std.cppm` and links its object into the program, so an entry for it states a fact of + the build. The format has no completeness rule, so the omission breaks no sentence of it. +- S1-12-1, the export rule of the profile mcpp implements, makes **every** translation unit of + the S1 document an entry, and mcpp's S1 document contains the std units (R3.10). R4.1 + contradicts it. +- The omission is what binds a reader to the toolchain's exact version: with no provider in the + database, clangd falls back to the prebuilt BMI the importers name, and fails on another + version (measured below). + +**Why does a reader compile `std` again, then?** Because a BMI is readable only by the compiler +that wrote it: clangd 22.1.8 on a BMI written by clang 20.1.7 reports `ast_file_version_too_old`, +and S1 §6 adds that "equal versions do not imply compatible BMIs" (`build-id` decides). S1-11.2-3 +therefore forbids a consumer to use build BMIs unless its engine matches the toolchain's +`family`, `version` and `build-id` exactly, the user has enabled an authoritative mode, and the +consumer can detect staleness. clangd's own policy (measured) is to build a module whose provider +the database lists, and to load a prebuilt file only when none is listed. The second compile is +the reader's own BMI, made for the reader's own compiler. + +**Measurement** (an interface importing `std`, an importer; clangd 22.1.8 `--check +--experimental-modules-support`): + +| Database | clangd errors | +|---|---| +| llvm@20.1.7, as written | 1: `Failed to build module std; due to Don't get the module unit for module std`, then `[ast_file_version_too_old]` | +| the same, with the two `mcpp:std` units appended | 0: `Built module std`, `Built module hello.greet` | +| gcc@16.1.0, as written | 1: `module 'std' not found` | +| the same, with GCC's `bits/std.cc` unit appended | 1: clang's scan of `bits/std.cc` under GCC's command fails | + +**The better answer: name both, and let the reader choose.** + +- **D5a.** `compile_commands.json` lists the standard-library units whenever the build imports + `std`, with the fields S1 gives them (R3.10, R3.11), and R4.1 follows S1-12-1. A reader of another + clang version can then build `std` (measured: clangd 22.1.8 on an llvm@20.1.7 database); the importers keep naming the build's BMI (P2), which a + reader of the same version may load. The cost falls on a matching plain clangd, which builds + `std` once per session instead of loading it (about 1.4 s in the measurement). +- **D5b.** The S1 document names the build's BMI for the standard-library units: `provides` + maps `std` and `std.compat` to the BMI paths in the shared std cache (S1-8-6: "the path of the + BMI the build writes"), which `emit` and the build share, and `ide.toolchains..build-id` + carries the compiler's build identity (S1 §6, MAY). An mcppls whose engine matches exactly, + in the authoritative mode S1-11.2-3 describes, then reuses mcpp's BMI and compiles nothing; + any other reader compiles, as S1 requires. Project modules keep `""` under `emit` (S1-8-6 + allows it), because the planning pass writes into its own work directory. +- For mcppls, D5a changes nothing: it exports S1 itself and receives the units already. D5b is + what removes its second compile. + +**Criteria.** (1) e2e, llvm row: the database holds an entry for the toolchain's `std.cppm` +whose `directory` is the std cache directory. (2) e2e: `emit --spec compile-commands` equals the +build's database apart from the work directory. (3) For D5b: `provides` of the `mcpp:std` units +names the std cache BMI, and `build-id` is present and stable across two runs. + +### 3.6 C6: the location + +By design: clangd finds `/compile_commands.json` by its upward search, without +configuration. A symlink at that path redirects the write (`compile_commands.cppm:333-354`); +docs/01 states neither. **Repair**: docs/01 and its zh counterpart state the location, the +redirect, and §3.2's rule. + +### 3.7 Not mcpp + +- **xmake** is a reference in the report, not a subject: its temporary mapper files and GCC-only + flags are its own. After §3.2 a shared root file holds mcpp's configuration alone, with a + warning. +- **clangd with a GCC database** cannot resolve `import std`: clang reads no GCC BMI and cannot + scan `bits/std.cc` under GCC's command (measured). An editor that wants clangd on a GCC-built + project asks for the clang view of the same plan (`emit build-database --toolchain llvm@…`). + +## 4. Issue #699 + +### 4.1 The report + +mcppls builds its model from `mcpp emit build-database --format json`. On the five-member +GalTranslPP workspace, the member GPPGUI requests the host tool `Updater` of `gpp.updater`, whose +build runs a `lupdate` check that failed because the machine's Qt lacked qtdeclarative. `emit` +answered `MCPP_BUILD_DATABASE_PLAN_FAILED` without `data`, the four members that planned lost +their sets, and mcppls fell back to a guessed model on which clangd crashed. The issue asks for +the planned members' sets with an error per failed member, and for one of: a plan-only signal, a +build-program failure reported per member, or no `check` actions in host-tool builds under +`emit`. It calls both behaviours specified (R5.2, R2.5) and files a request. + +### 4.2 Measurement + +| Invocation (fixture: `good` plans; `bad`'s build program exits 1; `user` requests host tool `t` of `tool`, whose blocking check fails) | Exit | `data` | Diagnostics | +|---|---|---|---| +| `emit --workspace` over `good`, `bad`, `user` | 1 | absent | `PLAN_FAILED`: `bad: build.mcpp exited with 1 (build aborted)`; `user` never planned | +| `emit` in `user` | 1 | absent | `PLAN_FAILED`: `building host tool 'tool:t' failed: build failed` | +| `mcpp build` in `user` | 2 | | the same failure, correct for a build; verbose, the inner build shows the check `… -- /bin/false` failing | +| `emit --workspace` over `good` | 0 | `good/good` | none | + +The member loop of `emit` stops at the first failure (`src/cli/cmd_build.cppm:352-380`), while +`mcpp build --workspace` continues past one ("continue-on-failure; first non-zero exit wins", +`:174-194`). + +### 4.3 Classification + +- **Not mcpp:** the environment (a Qt without qtdeclarative) and two project choices, a build + program that verifies the environment and a blocking check inside a host tool's build. With a + complete Qt the workspace plans all 227 units. +- **mcpp gap:** the unit of failure in `emit` is the whole command (R5.2), and a host tool's build + is part of planning (R2.5). + +### 4.4 E1: a member's failure costs that member (D1, accepted) + +`emit` plans every selected member. A member whose planning fails contributes no sets and one +`error` diagnostic with the code R5.2 assigns today (`MCPP_BUILD_DATABASE_PLAN_FAILED` or +`MCPP_OFFLINE_DOWNLOAD_REQUIRED`) and `path` naming its `mcpp.toml`, relative to the workspace +root. `data` is present when at least one member was planned, and its `watch` also lists each +failed member's `mcpp.toml` and `build.mcpp`. The exit status is 1 whenever an error is present. +A consumer reads three outcomes structurally: no `data`; `data` with errors (described, except +what each error names); `data` without errors. + +**S2 alignment.** S2-3.4-5 says `data` is present "when the command succeeded", and S2-3.4-11 +that a command without `data` has failed; S2 has no partial outcome. One sentence in S2 §3.4 +states it: `data` with `error` diagnostics is a document that describes everything except what +the errors name. `kindVersion` stays 1, which S2-3.4-3 requires. + +**Criteria.** e2e on the fixture: the sets name `good/good`, one error per failed member with +`path` `bad/mcpp.toml`, exit 1; a workspace in which every member fails omits `data`. + +### 4.5 E2: a host tool that does not build, under `emit` + +Under `emit`, a host tool whose build fails is a warning, `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT`, +naming the tool, its package and the first line of the failure. Planning continues, and the +build program receives the path the tool would be published at, which the store key fixes before +any build (`tool_store::bin_path`, `src/build/prepare.cppm:10342-10351`). A program that only +names the tool configures exactly as after a successful build; a program that runs it while +configuring (the pattern of docs/30's `dep_bin` example) fails, and E1 and E3 govern that +failure. The tool is still built, because a program may run it; its `check` actions still run, +because `blocking = true` is its author's statement, and a tool built without them would enter +the store unverified under the key a verified build uses. `mcpp build` is unchanged. + +**Criteria.** e2e, member `user`: `emit` exits 0 with the member's sets and one warning; `mcpp +build` still exits 2. + +### 4.6 E3: a build program that fails costs its own directives (D3, accepted) + +Under `emit`, a package whose build program fails (does not compile, exits non-zero, times out, +or prints output mcpp refuses) is described without that program's directives, with one error, +`MCPP_BUILD_DATABASE_PROGRAM_FAILED`, whose `path` names its `build.mcpp`. The manifest's part of +the configuration, the toolchain, the module graph and the standard-library units are described +as usual; a failure that follows from the missing directives fails the member, and E1 applies. +Directives from a failed run are never applied (`src/build/build_program.cppm:1601-1640`), so no +half-applied state exists. + +**Criteria.** e2e: a single package whose program exits 1, and one whose program does not +compile: `data` present with the package's sources, one error, exit 1. + +### 4.7 E4: the plan-only signal (D4) + +**What it is.** #699's first request is a signal, for example the environment variable +`MCPP_PLAN_ONLY=1` or a function `mcpp::plan_only()`, that tells a build program it runs for +`emit build-database` rather than for a build, so that it can skip checks and heavy work. **D4 +is the decision not to add it**, for three reasons. + +1. It gives a build program two behaviours, and the database describes one of them; nothing + checks that the other emits the same directives, so R1.2 would hold only by each author's + discipline (P6). +2. `--configure-only`, the reference R1.2 names, would have to run one of the two, and either + choice breaks an identity. +3. The separation already has a place: verification is a `check` action, which `emit` does not + run (P5), and R2 (§5.4) makes such a check cost nothing on a build whose inputs did not change. + +### 4.8 Usage side + +Environment verification is a `check` action, `blocking = true` where compilation must wait for +it; a program that cannot find something it would configure says so with `mcpp::warning` and +emits what it has. docs/30 states both. + +## 5. #701 and #702 + +### 5.1 The proposal + +#702 (2026.9.27.1, +494/-15, CI green except the two xcode-27 jobs of #669) adds: + +1. `mcpp::runtime_library_dir(dir)`, wire `mcpp:runtime-library-dir=`, protocol 12, "the + build-program form of `[runtime] library_dirs`": a new directive row + (`Scope::LinkGlobal`, `Transform::AbsPath`, persisted in the build-program cache) whose + `apply` appends to `RuntimeConfig::libraryDirs`, and the root package's residue mirrored into + the plan snapshot as `deploy`'s is. +2. `__action-stamp` records each stamp's modification time before the command and, on success, + moves a stamp the command did not write to the present. + +The pull request is reused as this round's single pull request (D7): it is retitled and +re-implemented to §5.4. + +### 5.2 Is the directive needed? + +**What exists.** `[runtime] runtime_search_dirs` (and the retiring `library_dirs`) are static +lists relative to the package root; `deploy` places named files at paths relative to the program. +The readers: the merge into `LinkIntent::runtimeSearchDirs` (`src/build/plan.cppm:911-948`), +RUNPATH/rpath on ELF and Mach-O (`src/build/flags.cppm:507`), `mcpp run`'s loader path +(`src/build/execute.cppm:553`), `mcpp pack`'s closure search (`src/pack/pipeline.cppm:194-195`, +`:463-464`) and runtime validation (`src/build/runtime_validation.cppm:475`). + +**The gap.** A directory whose location only a build program learns (a vcpkg install root, an SDK +found through `xpkg_dir`) and whose files do not exist when the program runs, because an +installation action creates them later. `deploy` needs the file names at plan time. The plugin's +route on 2026.9.26.1 is to deploy what exists at plan time, so the libraries a first installation +produces are placed by the next plan, which `mcpp run` performs (its record, D8, and +`deps/deps.cppm`). + +**Two observations narrow the need.** On Windows, placement beside the program is the platform's +convention and the only form that also serves a program started by hand, since PE has no run path; +a search directory serves `mcpp run` and `mcpp pack` only. And the consumer ships without the +directive. + +**Verdict.** The gap is real and general, a directory-level runtime search declared at +configuration time, and it is the build-program form of an existing key, as `include-dir` and +`deploy` are. + +### 5.3 #702 as written + +The directive follows the extension path the directive table defines: one row with protocol 12, +a non-empty cache tag replayed on a hit (the reasoning `warning` and `pack-format` state), the +root's residue mirrored, both languages of docs/04 and docs/30, unit and e2e tests. Four points do +not conform. + +1. **It extends a retiring field (P8).** `RuntimeConfig::libraryDirs` is one of "the four legacy + vectors … readable for one compatibility train" (`modules/manifest/src/types.cppm:1149-1150`); + docs/04 §2.11 says the same of `library_dirs`, and the fingerprint names it + `legacy-runtime-dir` (`src/build/prepare_inputs.cppm:723-725`). Its successor, + `LinkIntent::runtimeSearchDirs`, reaches the same readers through the same `absolute_from` + merge (`src/build/plan.cppm:911-920`) and accepts absolute paths, as `LinkIntent::deploy` + already does for the `deploy` directive's absolute `from`. #702's docs say the directive joins + `LinkIntent`; its code joins the legacy vector. +2. **The platform it is for has no criterion.** e2e 779 declares `# requires: pack gcc`, which + holds on Linux only; `mcpp run`'s `PATH` and `mcpp pack`'s PE closure on Windows, the cases the + issue names, are not exercised. +3. **The docs example** passes a Qt `bin/` to `link_search`; on Windows import libraries are in + `lib/` and DLLs in `bin/`. +4. **No CHANGELOG entry**, which every version pull request since 2026.9.21.3 carries. + +### 5.4 The conformant design + +Four changes, each stated as the rule it implements. SPEC-007 (§5.6) states the same rules from +the plugin author's side. + +**R1'. `mcpp::runtime_search_dir(dir)`: the build-program form of `runtime_search_dirs`.** + +- Wire `mcpp:runtime-search-dir=`, protocol 12: a directive row with `Scope::LinkGlobal`, + `Transform::AbsPath` and a non-empty cache tag. `apply` appends to + `LinkIntent::runtimeSearchDirs`, and the root package's residue is mirrored into the plan + snapshot as `deploy`'s is. +- The readers are the ones §5.2 lists, unchanged; a dependency's declaration reaches the + consumer's executable through the same merge. The directory need not exist when the program + runs, because a `prepare` action may populate it. +- `[runtime] library_dirs` keeps its legacy status and gains no directive. +- **Criteria.** Unit: the parse, the absolute-path transform, the protocol gate, the cache round + trip, and the field it lands in. e2e on Linux: RUNPATH in `build.ninja`, `mcpp run`, a replay on + a cache hit, `mcpp pack`. e2e on Windows: a DLL in a declared directory is found by `mcpp run` + through `PATH` and placed beside the program by `mcpp pack`. + +**R2. After a `check` or `prepare` command succeeds, every stamp is newer than every input.** +The engine creates each declared stamp that is missing and sets the modification time of each +existing one to the present, whether or not the command wrote it; on failure it writes nothing. +The rule needs no record of a stamp's time before the run. #702's form, which moves only a stamp +the command did not write, meets the same rule and is acceptable. A stamp feeds no compile or link +edge (a blocking check or a `prepare` action orders edges through an order-only edge), so touching +a stamp the command wrote changes no build. **Criterion:** e2e 780, which fails on 2026.9.26.1. + +**P. The `prepare` role (O1).** An action whose command populates a directory that the build reads +by directory, and whose file names are not known when the build program runs: installing a vcpkg +manifest or a CMake subproject into a prefix, unpacking an SDK. + +- **Outputs:** one or more stamps, which the engine writes as for `check` (R2), and one declared + directory, `a.output_dir(dir)`, which the command populates. When the command succeeds and the + directory does not exist, the engine writes no stamp and fails the edge, naming the directory. +- **R1.3 becomes a check.** A build program whose rerun inputs (`rerun_if_changed`, + `rerun_if_changed_glob`) lie inside a declared `prepare` directory reads a construction result + while it configures; the engine warns and names both. This is the pattern the plugins use today + to place a first installation's libraries on the next plan. +- **Ordering:** every compile edge and the link edge of the declaring package wait for it. It is + construction: a policy that concerns checks does not concern it, and the build's progress lines + label it `PREPARE`. +- **References to its products** are names fixed at configuration time: `include_dir`, + `link_search` and `runtime_search_dir` for directories, `link_flag` for a library's full path. + The rule `src/build/hostprogram.cppm:187-194` states ("Content may arrive later; names may not") + holds at the granularity of a directory. +- **Spelling:** `a.role = mcpp::roles::prepare;`. The constants + `mcpp::roles::{source, check, object, artifact, prepare}` exist from protocol 12. An older + engine's bundled module lacks them, so a build program that uses them fails to compile on that + engine and names the constant, instead of the older engine reading the string `"prepare"` as + `source`. +- **Unknown role strings are refused** from protocol 12, with the list of roles. Today + `decode_action` maps any unknown string to `source` + (`modules/buildmcpp/src/directives.cppm:1014-1018`) and `action_error` checks only the command + and the outputs, so a misspelt role changes an action's meaning without a word. +- **Why a role and not `check` with `blocking = true`.** A role is an input to engine decisions. + #699's third request, not adopted, was to skip the `check` actions of host-tool builds under + `emit`; had it been adopted, every installation written as a check would have been skipped. A + construction step labelled as verification is broken by the first decision made on its label. +- **Criteria.** Unit: the role decoded from the constant and from the string; an unknown string + refused with the list. e2e: a `prepare` action populates a directory whose file names the build + program does not know; a unit that includes a header from it compiles, and a program linked + against a library in it runs through `runtime_search_dir`, on the first build; a second build + runs nothing; after an input of the action changes, it runs once and not again. A `prepare` + action whose command creates nothing fails naming its directory, and a build program that + declares a rerun input inside that directory is warned. + +**W. A Windows program's runtime libraries are placed beside it (O2).** A PE program has no run +path, so a DLL in a runtime search directory serves `mcpp run` (through `PATH`) and `mcpp pack`, +and not a program started by hand from the build directory. The platform's own convention is +placement: vcpkg's integration copies a program's imported DLLs beside it after the link, and +CMake names the same set `$`. + +- **Rule.** After the link of a PE program whose plan has runtime search directories, an engine + edge places beside the program every DLL the program imports, directly or through another DLL, + that resolves in one of those directories. The resolution is `mcpp pack`'s: + `read_closure` (`src/pack/pack.cppm:1323`), the directory order of `pipeline.cppm:193-202`, and + the system rule that never copies an API set or a system DLL (`src/pack/binfmt.cppm:812-823`). +- **The edge** is engine-internal, as the check stamp's wrapper is: its inputs are the program and + the stamps of the graph's `prepare` actions, it reports the DLLs it resolved in a depfile (the + mechanism device compilers already use), so a later change to any of them runs it again, its + output is a stamp, and a copy is replaced only when the source differs. Two directories that offer one DLL name are resolved in search order, + and the choice is reported. +- **Nothing else changes:** ELF and Mach-O keep their run paths; `mcpp pack` is unchanged; a + plan without runtime search directories has no such edge. +- **Criteria.** e2e on Windows: a DLL in a declared runtime search directory, populated by a + `prepare` action, is found by the program started by hand from the build directory after the + first build; replacing the DLL in that directory replaces the copy on the next build; a system + DLL is never copied. + +### 5.5 F1: `$ORIGIN` in a user link flag (mcpp-community/mcpp#703) + +The plugin record reports that a build program's `link_flag` loses `$ORIGIN` through "ninja and +the shell". Measured on 2026.9.26.1 (gcc@16.1.0), for `mcpp::link_flag("-Wl,-rpath,$ORIGIN/../lib")` +and for `[build] ldflags = ["-Wl,-rpath,$ORIGIN/../lib"]` alike: + +``` +build.ninja: -Wl,-rpath,$$ORIGIN/../lib +program: (RPATH) [::/../lib:] +``` + +**Cause.** SPEC-004 §8 states how an element of `cflags`, `cxxflags` and `asmflags` (and of +their directives) becomes words, forbids an implementation to interpret `$` (rule 7), and requires +every word to reach the compiler verbatim whatever the host's command-line reader. `ldflags` and +the link directives have no such statement, and `normalize_ldflag` +(`src/build/flags.cppm:367-385`) escapes an element for ninja only, so the `sh` that runs the +command expands `$ORIGIN` to nothing. The engine's own run path is rendered +`-Wl,-rpath,'$$ORIGIN'` (`src/build/plan.cppm:745`, `:2125`), quoted for ninja and for the shell. +The result is worse than a lost entry: `/../lib` is the host's `/lib`, a host directory in the +program's run path, the class #696 closed for links. + +**Repair.** SPEC-004 §8's reading applies to `ldflags` and to the link directives: an element is +read into words, and each word reaches the linker verbatim, escaped for ninja and quoted for the +host. The same reading splits an element that packs several tokens, which the rendering of +link-unit flags guards against case by case today (`src/build/ninja_backend.cppm:342-370`). +**Compatibility:** an element written for the shell or for ninja by hand (`\$ORIGIN`, +`'$$ORIGIN'`) changes meaning; the index is searched for such spellings before release. Windows is +unaffected (no shell). **Criterion:** e2e, `$ORIGIN` from the manifest and from `link_flag` +reaches the program's run path verbatim (fails on 2026.9.26.1). Filed as #703. + +### 5.6 SPEC-007: the build plugin specification + +`docs/specs/build-plugins.md` (SPEC-007, draft 0.1, written in Chinese as SPEC-001 to SPEC-006 +are) is the contract for rule packages, dependency adapters and distribution members. Each rule +carries the implementation state the spec index defines, and a rule no engine check enforces is +marked as an author's obligation. + +| § | Content | +|---|---| +| 1 | Three kinds of work and the mechanism of each: configuration (directives), construction (actions), verification (`check`). Construction is never done while the build program runs; environment incompleteness is a warning, never a non-zero exit; configuration depends only on declared inputs and never on construction results; one behaviour under planning and building | +| 2 | Which directive expresses which configuration (including `runtime_search_dir`); package-relative paths; no host system directories; no retiring fields; link flags read as words (#703) | +| 3 | Actions as argv with the tool as an input; outputs named at submission; the five roles with `prepare`; `check` only for verification; the stamp rule R2; role constants; network access belongs to installation, and an offline build (`MCPP_OFFLINE=1`, inherited by actions, `src/cli.cppm:167-172`) does not reach the network | +| 4 | Runtime search: `runtime_search_dir` for a directory, `deploy` for a file known at configuration time, what Windows provides, no run path through `link_flag` | +| 5 | Obligations under planning: E2 and E3 from the plugin's side | +| 6 | Payloads declared where the lookup happens; no probing of host paths | +| 7 | The engine release a plugin needs, stated; the index CI pin moves and `min_mcpp` does not | +| 8 | A criterion on every platform claimed; a criterion that fails before its change; a planning criterion | + +**What it asks of mcpp-plugins 0.13** (the feedback for the plugin side): + +| Plugin | Today (its design record, D8) | Under SPEC-007 | +|---|---|---| +| `deps-vcpkg`, `deps-cmake` | the installation is a blocking `check` | a `prepare` action (R3.3, R3.4) | +| `deps-vcpkg`, `deps-cmake` | the prefix's shared libraries are enumerated at plan time and deployed one by one, so a first installation's libraries are placed by the next plan | `runtime_search_dir` for the prefix's `bin/` (Windows) or `lib/` (R4.1); no enumeration of construction results (R1.3, now warned); on Windows, W places the DLLs beside the program after the first build | +| `deps-vcpkg` | vcpkg may download during the build | stated in its documentation; an offline build completes from vcpkg's caches or fails naming what is missing (R3.7) | +| `rules-qt` | Windows: the DLLs of the linked modules and of the modules they depend on, deployed beside the program | conforms as it is: the SDK is a payload installed before the build, so these files are known at configuration time (R1.3, R4.2). After W, `runtime_search_dir` for the SDK's `bin/` gives the same result without the plugin computing the module closure | +| `rules-qt` | Linux: glib, zstd and zlib linked by full path with `--no-as-needed` | unchanged: QtCore's own `DT_RUNPATH` stops the program's run path from applying to QtCore's dependencies, a loader rule rather than an engine gap; `$ORIGIN` in link flags is usable after #703 (R2.4) | +| all | engine floors written as versions | the first mcpp release that speaks protocol 12, stated in each plugin's documentation (R7.1) | + +**O2** (a program started by hand on Windows) is designed in §5.4 as W, a general mechanism +(P9), and SPEC-007 R4.3 states it. + +### 5.7 Disposition + +#702 is reused as this round's single pull request (D7): retitled, and re-implemented to R1', R2, +P and W, with F1 and the rest of §7 in the same pull request. mcpp-plugins adopts SPEC-007 after the +release that carries protocol 12. + +## 6. Specifications and documentation + +| Document | Change | +|---|---| +| SPEC-005, to v1.3 | R2.5: a host tool that does not build under `emit` is a warning (E2). R3.7: `work-directory` is the directory the compiler runs in; `arguments` include the interface language flag (C3, C4). R3.8: `provides` of the standard-library units names the std cache BMI (D5b). R4.1: the compile-commands document includes the standard-library units (C5, S1-12-1). R5.2: member containment and the program-failure description (E1, E3). | +| SPEC-004 §8 | the element reading extends to `ldflags` and the link directives (F1, #703) | +| SPEC-007, new (draft 0.1) | `docs/specs/build-plugins.md`, and its row in `docs/specs/README.md` | +| docs/50 §8, and zh | the failure paragraph; `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT` (warning), `MCPP_BUILD_DATABASE_PROGRAM_FAILED` (error); `path` on member errors | +| docs/01, and zh | the root file and the configuration's database (§3.2), the symlink redirect, the standard-library units | +| docs/30, and zh | configuration, construction and verification (§4.8, SPEC-007 §1); `dep_bin` under `emit`; `runtime_search_dir`; the `prepare` role and the role constants; the stamp rule | +| docs/31, and zh | the role table gains `prepare`; a pointer to SPEC-007 | +| docs/04 §2.11, and zh | the build-program form of `runtime_search_dirs`; the PE column of the link-intent table gains W's placement | +| S2 §3.4 (mcppls repository) | one sentence: `data` with error diagnostics (E1) | +| CHANGELOG | one entry per change | + +## 7. Plan + +Every item lands in #702, retitled. + +| # | Change | Where | Tests | +|---|---|---|---| +| W1 | §3.2: the configuration's database, the root copy, the warning, the fast-path restore, the `.gitignore` | `src/build/compile_commands.cppm`, `src/build/ninja_backend.cppm`, `src/build/execute.cppm`, `src/scaffold/create.cppm` | e2e (four), unit | +| W2 | C3 and C4 in `UnitInvocation` | `src/build/compile_commands.cppm` | unit; e2e replay (gcc, llvm); e2e 211 | +| W3 | C5: the plan carries the standard-library description; both databases render it; D5b's `provides` and `build-id` | `src/build/plan.cppm`, `src/build/prepare.cppm`, `src/build/compile_commands.cppm`, `src/build/build_database.cppm` | e2e | +| W4 | E1 | `src/cli/cmd_build.cppm`, `src/build/build_database.cppm` | e2e | +| W5 | E2 | `src/build/prepare.cppm` (host-tool branch, `plan_only` only) | e2e | +| W6 | E3 | `src/build/prepare.cppm` (both `run_build_program` sites, `plan_only` only); a severity on `PlanNote` | e2e (two) | +| W7 | R1' and R2 | `modules/buildmcpp/src/directives.cppm`, `modules/buildmcpp/src/program_protocol.cppm`, `src/build/hostprogram.cppm`, `src/build/prepare.cppm`, `src/cli.cppm` | unit; e2e on Linux and Windows; e2e 780 | +| W8 | F1 (#703): SPEC-004 §8's reading for link flags | `src/build/flags.cppm`, `docs/specs/manifest-semantics.md` | unit, e2e | +| W9 | P: the `prepare` role with `output_dir`, the role constants, the refusal of unknown roles, the R1.3 warning | `modules/buildmcpp/src/directives.cppm`, `src/build/hostprogram.cppm`, `src/build/ninja_backend.cppm` (stamp wrapper, directory post-condition, ordering including the link edge, the #534 self-check, the progress label), `src/build/prepare.cppm` (the warning) | unit, e2e | +| W10 | W: runtime DLL placement after a PE link | `src/build/ninja_backend.cppm` (the edge), `src/cli.cppm` (its engine-internal command), `src/pack/pack.cppm` (`read_closure` reused) | e2e on Windows | +| W11 | §6, SPEC-007 included | specifications, docs, CHANGELOG; S2 in the mcppls repository | docs checks | + +W7 and W9 share one protocol bump, to 12; W10 follows W7 and W9, whose directories it reads. W1 and W2 are tested together (W1's criteria read +`directory`). The version is the next date version at release time. After the release, mcppls +reads `data` with errors and may use D5b's reuse, and mcpp-plugins moves to SPEC-007. + +**Cross-platform.** C3 on Windows is the native spelling of `plan.outputDir`. W1 copies rather +than links the root file, since a symlink needs a privilege on Windows. W7 carries a Windows leg. +W9 is platform-independent: an action is an argv on every host. W10 exists on Windows only, and +its criterion runs there. C4's MSVC form and C5 on MSVC are +open measurements. F1 is POSIX-only. E1 to E3 do not depend on the platform. + +## 8. Decisions + +| | Decision | State | +|---|---|---| +| D1 | E1 as the default | **accepted** | +| D2 | C3 for every toolchain | **accepted** | +| D3 | E3 as the default | **accepted** | +| D4 | no plan-only signal | **accepted** | +| D5 | D5a: standard-library units in `compile_commands.json`; D5b: their BMI in S1 `provides`, and `build-id` | **accepted** | +| D6 | one database per configuration | **accepted** | +| D7 | #702 reused as the single pull request and re-implemented to §5.4 | **accepted** | +| D8 | F1 repaired by extending SPEC-004 §8 to link flags; filed as #703 | **accepted** | +| D9 | P: the `prepare` role with its declared directory, the role constants, the refusal of unknown roles, and the R1.3 warning, in this round | **accepted** | +| D10 | R2 in its simple form (every stamp touched on success) | **accepted** | +| D11 | W (O2): the engine places a Windows program's imported DLLs beside it after the link | **accepted**, with the Windows e2e as its criterion | +| D12 | SPEC-007 draft 0.1 as the contract relayed to mcpp-plugins | **accepted** | + +## 9. Risks and open measurements + +- C4 and C5 on MSVC (Windows CI): `/interface` in a clang-cl-mode clangd; an MSVC STL `std.ixx` + unit. +- C5's cost for a matching plain clangd: one `std` build per session. +- §3.2 replaces a root file another tool shares with mcpp, including through a root symlink into + that tool's directory; the warning makes it visible. +- E3: code after a build program in `prepare_build` may assume its directives were applied; each + such failure fails the member under E1, and the e2e set includes one. +- E2 retries a failing tool build on every `emit`, as today's failure does. +- F1 changes the meaning of a link flag escaped for the shell or for ninja by hand. +- P on an engine older than protocol 12: a build program that spells the role with a constant + fails to compile and names it; one that writes the string `"prepare"` is read as `source` by + that engine. SPEC-007 R3.6 requires the constant. +- deps-vcpkg's downloads at build time (SPEC-007 R3.7) depend on how vcpkg's caches are populated; + the plugin states its answer. +- W copies DLLs into the build directory on every PE build that has runtime search directories; + a copy is replaced only when its source differs, and its criterion runs on Windows CI only, + where it is the first measurement of the closure walk outside `mcpp pack`. +- The R1.3 warning appears on the first build after the upgrade for the plugins that use the + second-plan route today; SPEC-007 tells them what replaces it. +- E1 depends on one sentence in S2. + +## 10. Overall review + +- **Architecture.** One record serves both databases (P1). One directory names a configuration, + and it is the directory the compiler runs in (C3, §3.2), so the identifier the merge needs and + the fact the specifications ask for are one value. Configuration, construction and verification + each have one mechanism (SPEC-007 §1), and the one gap in that table, construction whose file + names are unknown, gets a role instead of a borrowed one (P). New surfaces join the current model + (R1', P8). Every engine change is general and names no tool (P9); what the plugins knew and + worked around, the engine now states once (R1', P, W). +- **Consistency with the specifications.** C3 meets the JSON format and S1-8-2; C5 meets S1-12-1; + D5b meets S1-8-6 and S1-11.2-3; E1 needs one sentence in S2; F1 extends SPEC-004 §8 to the one + flag list it did not cover. No field outside the JSON format is written into + `compile_commands.json`, which its readers reject (measured). +- **Compatibility.** Behaviour a user can see changes in eight places: the root database is + replaced rather than merged, with a warning when another writer's entries go; `directory` names + the output directory; interface entries carry their language flag; standard-library entries + appear; `emit` returns partial documents with errors; `$ORIGIN` in a link flag reaches the linker; + an unknown action role is refused instead of read as `source`; on Windows, a program's DLLs from + its runtime search directories are placed beside it. Each follows a specification or an + accepted decision. The `emit` envelope keeps `kindVersion` 1. Protocol 12 is additive: a build + program that uses none of its surfaces behaves as before, and one that uses them fails to compile + on an older engine and names what it lacks. +- **Stability.** The fast path gains a copy, not a plan. No compile changes. W1 to W6 change + descriptions. A link changes where W8 corrects a run path that is wrong today, and where a build + program declares a runtime search directory. R2 changes a stamp's time. P adds an ordering that + the declaring package asked for. +- **User experience.** An editor keeps every member that plans (E1), a package whose program fails + (E3), and a member whose host tool does not build (E2). A deleted database returns on the next + build. A dependency installed by a plugin is usable on the first build, without the second plan + the plugin needs today, and on Windows a program started by hand finds its DLLs (W). Verification in `check` actions costs nothing when its inputs did not + change (R2). +- **Test coverage.** Each item has a criterion that fails on 2026.9.26.1 (C1, C3 for gcc, C4, E1, + E2, E3, R2, F1) or states a new property (§3.2's four e2e, C5, D5b, R1' on Linux and Windows, P). + W has its criterion on Windows. SPEC-007 §8 asks the same of every plugin, including a planning + criterion. +- **Measured and not measured.** Measured: every reading in §3, §4.2, §5.5 and Appendix A. Not + measured: MSVC (C4, C5), macOS, clangd 23.1 itself (its mechanism is reproduced with llvm@20.1.7 + and clangd 22.1.8), Meson (its binary on the measuring host does not run), and W, whose criterion + runs on Windows CI. +- **Corrections made while writing.** C1 had been recorded twice before (#397 C-1, #677 B1). + Listing the standard-library units was first expected not to help; clangd builds from a listed + provider. GCC was first expected to hand `.ixx` to the linker; GCC 16 compiles it. Revision 1's + four-condition merge kept a merge across configurations; §3.2 removes it. The first draft of the + plugin migration table moved `rules-qt` on Linux to `runtime_search_dir`; QtCore's own + `DT_RUNPATH` makes the program's run path irrelevant to QtCore's dependencies, so that row stays + as the plugin has it. + +## 11. Implementation + +### 11.1 Tasks + +| Task | Items | Branch | Owns | e2e | +|---|---|---|---|---| +| T1 the compile database | W1, W2, W3 | `feat/702-cdb` | `compile_commands.cppm`; the fast path in `execute.cppm`; `scaffold/create.cppm`; the standard-library units, `provides` and `build-id` in `build_database.cppm`; the writer's call site in `ninja_backend.cppm` | 781-786 | +| T2 `emit` | W4, W5, W6 | `feat/702-emit` | the member loop in `cmd_build.cppm`; member diagnostics in `build_database.cppm`; the host-tool and build-program branches of `prepare.cppm` under `plan_only` | 787-789 | +| T3 build-program surfaces | W7, W9 | `feat/702-actions` | `directives.cppm`, `program_protocol.cppm`, `hostprogram.cppm`; `__action-stamp` in `cli.cppm`; the action edges in `ninja_backend.cppm`; the residue mirror and the R1.3 warning in `prepare.cppm` | 779, 780, 790-794 | +| T4 link flags | W8 | `feat/702-link` | `normalize_ldflag` in `flags.cppm`; the link-flag rendering in `ninja_backend.cppm`; SPEC-004 §8 | 795-796 | +| T5 DLL placement | W10 | `feat/702-link` | the placement edge in `ninja_backend.cppm`; its internal command in `cli.cppm`; the closure walk shared with `mcpp pack` | 797-798 | +| T6 documents and release | W11 | `feat/runtime-library-dir` | specifications, docs and their zh pairs, CHANGELOG, the version, the xlings pin | | + +### 11.2 Dependencies + +- T1 to T4 are independent and proceed in parallel, each in a worktree from #702's head. +- T5 reads `LinkIntent::runtimeSearchDirs`, which `[runtime] runtime_search_dirs` already + feeds, so it proceeds in parallel against the manifest key; T3's `prepare` stamps join its + inputs at integration. +- The shared files are `prepare.cppm` (T1, T2, T3), `ninja_backend.cppm` (T1, T3, T4, T5), + `cli.cppm` (T3, T5) and `build_database.cppm` (T1, T2); each task changes its own functions. + Integration merges T3 first (it moves the protocol), then T4 and T5, then T1, then T2. +- T6 follows all five: each task reports its specification and documentation text. + +### 11.3 Cross-repository sequence + +1. mcpp #702 carries T1 to T6, the version and the xlings pin; every CI leg is green except the + xcode-27 legs of #669. +2. Before the release, the index and mcpp-plugins are searched for action roles outside the + five (P refuses them) and for link flags that escape `$` by hand (F1 changes their meaning). +3. The release: `release.yml`; each archive is uploaded to GitCode with the local gtc as it + appears; the xim-pkgindex bump is merged by a maintainer; the index's `latest` is read back. +4. The ecosystem: the published mcpp is verified in an xlings sandbox with the CN mirror + (`xlings subos use --sandbox --cmd ...`): installation, `new`/`build`/`run`, the compile + database, `emit` over a workspace with a failing member, a build program with + `runtime_search_dir` and a `prepare` action, and index packages including the rules plugins. +5. mcppls: S2 §3.4's sentence on `data` with error diagnostics, in its repository. +6. mcpp-plugins: SPEC-007 is relayed by the maintainer, and the plugins move after the release. +7. #699, #701 and #703 are closed with the release's readings. + +### 11.4 Review axes + +| Axis | What holds it | +|---|---| +| Architecture | one record for both databases (P1); one directory names a configuration; one mechanism per kind of work (SPEC-007 §1); engine changes name no tool (P9) | +| Stability | the fast path gains a copy and no plan; no compile command changes; links change only where F1 corrects a run path or a program declares a runtime search directory | +| Simplicity | one function publishes the root file on both paths; one stamp rule for both stamped roles; one role table decodes constants and strings | +| User experience | an editor keeps every member that plans; a deleted database returns; a first installation is usable on the first build; a Windows program started by hand finds its DLLs | +| Compatibility and upgrade | protocol 12 is additive; `kindVersion` stays 1; cache entries of older programs replay unchanged; a foreign database and a rerun input inside a `prepare` directory are warnings; the one new refusal, an unknown role, is searched for in the index before the release | +| Cross-platform | C3 in native spelling; the root file copied, never linked; R1' with a Windows leg; W on Windows CI; C4 and C5 measured on MSVC before their MSVC form is emitted | +| Consistency | SPEC-004, SPEC-005, SPEC-007, docs/01, docs/04, docs/30, docs/31, docs/50 and their zh pairs change in the same pull request as the code | +| Test coverage | each item has a criterion that fails on 2026.9.26.1 or states a new property; the full e2e suite runs on the integration branch before the pull request is pushed | + +## 12. Implementation record + +The version is 2026.9.26.2, the next date version on the day of the release; #702 was opened as +2026.9.27.1. Four worktrees implemented T1 to T5 in parallel from #702's head and merged into it. + +### 12.1 Where the implementation departs from or adds to the design + +| Item | Design | Implementation | Reason | +|---|---|---|---| +| C5 (W3) | the plan carries the standard-library description | `recover_invocation` moved from `build_database.cppm` to `plan.cppm`, re-exported under its old name | `compile_commands.cppm` must render the units for the build's own database and cannot import `build_database.cppm`, which imports it | +| C4 (W2) | the flag for every dialect, MSVC after a measurement | GCC and Clang dialects; MSVC omitted, stated in SPEC-005 R3.7 | the measurement of clang-cl-mode clangd remains open (§9) | +| §3.2 item 4 | the fast path restores the root file | one function publishes the root file on both paths, comparing bytes; one function words the warning | a record of size and time would be a second definition of "identical" | +| P, post-condition | the directory exists | the directory holds at least one file other than the action's stamps | ninja creates a declared output's parent before the command runs, so a stamp inside the directory made "exists" true for a command that wrote nothing (e2e 791 checks both placements) | +| P, ordering | the declaring package's compile and link edges | the declaring package's compile edges and every link edge of the plan except static archives | the link-global directives of every package merge into one `LinkIntent`, which every link line reads | +| P, refusal | unknown roles refused | also a `prepare` action without `output_dir` | the post-condition has nothing to check without it | +| W, inputs | the program and the `prepare` stamps; depfile | the same, and the stamp records the names placed, which the next run resolves outside the program's directory (`ClosureReadInput::notInFirstDir`) | without the stamps a DLL a `prepare` action replaced was placed one build late; without the record, the copy beside the program shadowed the newer DLL (both found by e2e 797) | +| W, deploy edges | nothing else changes | a deployed DLL is an order-only input of the link, not an implicit one | the linker reads the import library; the plan-time listing of runtime search directories, kept for one train, otherwise relinked the program on the second build | +| F1 | SPEC-004 §8 for link flags | also: dependency flags propagate word by word; the link directives spell paths as one word; the first plan names an element whose words changed | the renderer, the propagation and the directives are three writers of one list | +| Cache | none stated | build-program cache epoch 3 | a cached `ldflag` value and a cached unknown role both change meaning | +| S2 | one sentence in §3.4 | S2 0.3.0: `data` with `error` diagnostics, optional `path`, S2-3.4-12 and S2-3.4-13 | Sunrisepeak/mcpp-language-server#25 | + +### 12.2 Tests + +| Item | e2e | Fails on 2026.9.26.1 | CI | +|---|---|---|---| +| C1 | 781 | yes | shards | +| C2 | 782, 783 | yes | shards; 783 in the llvm job | +| C3 | 784, 211 | yes (gcc row) | llvm job | +| C4 | 785 | yes | llvm job | +| C5, D5b | 786, 688 | yes | llvm job; shards | +| E1, E2, E3 | 787, 788, 789 | yes | shards | +| R1' | 779 (Linux), 794 (Windows), 796 (mingw-cross, wine) | new surface | shards; Windows shards; mingw-cross job | +| R2 | 780 | yes | shards | +| P | 790, 791, 792, 793 | new surface | shards | +| F1 | 795 | yes | shards | +| W | 797 | new surface | mingw-cross job | + +Unit tests: `test_compile_commands` (the merge key, `directory`, the flag position), +`test_build_directives` (the directive, the roles, `output_dir`, the cache round trip), +`test_ninja_backend` (the placement edge, its absence off PE and without directories, the +order-only deployed DLLs). + +## Appendix A. Measurement record + +All runs use `~/.xlings/data/xpkgs/xim-x-mcpp/2026.9.26.1/bin/mcpp`, clangd and clang-tidy from +`~/.xlings/data/xpkgs/xim-x-llvm-tools/22.1.8/bin`, and scratch directories. + +### A.1 The compile database (a copy of `hello`) + +``` +report 3.6: rm compile_commands.json; mcpp build + last line: Finished dev in 0.00s; compile_commands.json NOT regenerated; after --no-cache: present +report 3.1: xmake's sample moved to the copy, touch src/main.cpp, mcpp build + from the root: 6 entries (g++ bits/std.cc; clang++ greet.cppm, main.cpp, test.cppm; g++ src/greet.cppm, src/main.cpp) + from src/: 4 entries (the two relative ones pruned) +report 3.2: replay with -o redirected, interfaces first + gcc from the output directory: 3 of 3 rc=0, objects written + gcc from directory: greet.cppm rc=0; test.cppm rc=1; main.cpp rc=1; gcm.cache/ written into the project root + llvm from either: 3 of 3 rc=0 +two toolchains: mcpp test (llvm), then mcpp build --toolchain gcc@16.1.0 + g++ greet.cppm | g++ main.cpp | g++ test.cppm | clang++ tests/test_smoke.cpp +configurations: target/x86_64-linux-gnu/ holds two fingerprint directories after build, test and + --no-cache runs with one toolchain and a build with the other; after `mcpp build` and `mcpp test` + in one configuration, the three build entries and the test entry all name one directory (6413abdf...) +``` + +### A.2 clangd and clang-tidy + +``` +.ixx without -x c++-module: E [fe_expected_compiler_job] ... expected exactly one compiler job +.ixx with the flag: All checks completed, 0 errors +llvm@20.1.7 database: Failed to build module std; due to Don't get the module unit for module std + [ast_file_version_too_old] ... std.pcm; 1 error ++ mcpp:std units: Built module std; Built module hello.greet; 0 errors +gcc@16.1.0 database: module 'std' not found; 1 error (unchanged with bits/std.cc appended) +missing directory: clangd: VFS: failed to set CWD ...; 0 errors + clang-tidy: LLVM ERROR: Cannot chdir into "..."! (exit 134) +one unknown key in an entry: clangd: Failed to load compilation database; clang-tidy: cannot load it +GCC 16 on .ixx: g++ -std=c++23 -fmodules -c m.ixx: rc=0, object written +``` + +### A.3 #699 + +``` +emit --workspace (good, bad, user): rc=1, no data, PLAN_FAILED bad: build.mcpp exited with 1 (build aborted) +emit in user: rc=1, no data, PLAN_FAILED building host tool 'tool:t' failed: build failed +mcpp build in user (verbose): [1/5] mcpp __action-stamp .../lupdate.stamp -- /bin/false; FAILED +emit --workspace (good): rc=0, sets ['good/good'] +``` + +### A.4 `directory` in other producers + +``` +CMake 4.4.2, Ninja: directory ; file /lib/lib.cpp; replay: object written +CMake 4.4.2, Unix Makefiles: directory /lib; file /lib/lib.cpp; replay: object written +ninja -t compdb, mcpp graph: directory /target/x86_64-linux-gnu/ +``` + +### A.5 F1 + +``` +mcpp::link_flag("-Wl,-rpath,$ORIGIN/../lib") and [build] ldflags, gcc@16.1.0: + build.ninja: -Wl,-rpath,$$ORIGIN/../lib + program RPATH: [.../xim-x-glibc/2.44/lib64:.../xim-x-gcc/16.1.0/lib64:/../lib:.../subos/default/lib] +``` diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 2345781b7..483783b11 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -310 records. +311 records. ## By subject @@ -30,6 +30,7 @@ Records that declare one. Everything else is listed by date below. ### design +- [The compile database, `emit build-database`, and #701/#702: triage against the specifications, and one design](2026-09-26-compile-database-and-issue-699-design.md) — landed - [Issues #693 to #696: triage against mcpp's contracts, and one repair plan](2026-09-25-issues-693-696-triage-and-repair-plan.md) — landed - [Workspace inheritance, flag scoping and the published form: a unified repair plan (#690)](2026-09-25-issue-690-workspace-build-inheritance-consistency.md) — landed - [MSVC toolset 的选择、#685、#687 与工具链管理规范：总体设计](2026-09-24-toolchain-selection-and-payload-trust-design.md) — active @@ -101,6 +102,7 @@ Records that declare one. Everything else is listed by date below. ### 2026-09 +- [The compile database, `emit build-database`, and #701/#702: triage against the specifications, and one design](2026-09-26-compile-database-and-issue-699-design.md) — landed - [Issues #693 to #696: triage against mcpp's contracts, and one repair plan](2026-09-25-issues-693-696-triage-and-repair-plan.md) — landed - [Workspace inheritance, flag scoping and the published form: a unified repair plan (#690)](2026-09-25-issue-690-workspace-build-inheritance-consistency.md) — landed - [#690: self-review before release, engine and ecosystem](2026-09-25-issue-690-self-review.md) — landed diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index f7ad1f041..909f5f7e3 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -449,3 +449,49 @@ jobs: "ok: a program over a C++ shared library runs on one C++ runtime" \ "ok: a stated self-contained program over a coupled C++ shared library is refused" \ "PASS: 700 a program over a C++ shared library has one C++ runtime" + + # The compile database (#699 report, design 2026-09-26 §3): the scripts + # whose rows need llvm declare `# requires: llvm`, which no shard has, so + # they run here and each is held to the lines it prints only when the + # property it names was checked. + - name: "Compile database e2e that needs llvm (783, 784, 785, 786)" + run: | + set -o pipefail + # The scripts read the database with python3, which this container + # lacks. It is not a toolchain: the job's property (no host compiler, + # no host C runtime) was asserted before the checkout and still holds. + if ! command -v python3 >/dev/null 2>&1; then + apt-get install -y -qq python3 >/dev/null + if command -v gcc >/dev/null 2>&1; then + echo "::error::installing python3 brought a host compiler"; exit 1 + fi + fi + export PATH="$HOME/.xlings/subos/current/bin:$PATH" + export MCPP + run_and_assert() { # $1 = script; the remaining arguments are lines it must print + local script="$1"; shift + local log="$RUNNER_TEMP/$(basename "$script").log" + bash "$script" 2>&1 | tee "$log" + for line in "$@"; do + grep -qxF "$line" "$log" || { echo "::error::$script did not print: $line"; exit 1; } + done + } + run_and_assert tests/e2e/783_cdb_switches_whole_with_the_configuration.sh \ + "ok: mcpp test then mcpp build keep the test entry in one configuration" \ + "ok: switching toolchain switches the whole root file" \ + "ok: switching back restores that configuration's entries, test units included" \ + "PASS: 783 one compile database per configuration" + run_and_assert tests/e2e/784_cdb_replays_from_its_directory.sh \ + "ok: gcc row replays every entry from its directory, no gcm.cache/ in the project root" \ + "ok: llvm row replays every entry from its directory" \ + "PASS: 784 the compile database replays from its own directory" + run_and_assert tests/e2e/785_cdb_interface_flag_module_extensions.sh \ + "ok: the interface entry carries -x c++-module immediately before -c, and replays" \ + "ok: the non-module unit carries no language flag" \ + "PASS: 785 the compile database states a module interface's language explicitly" + run_and_assert tests/e2e/786_std_unit_in_the_database_and_build_id.sh \ + "ok: the std unit's directory is the shared std cache, not the project's output directory" \ + "ok: emit --spec compile-commands renders the same std entry as the build's database" \ + "ok: provides['std'] names the std-cache BMI" \ + "ok: build-id is present and stable across two runs" \ + "PASS: 786 the std unit in the database, emit/build agreement, provides and build-id" diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index 0fabc96e3..d5edf236e 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -370,6 +370,38 @@ jobs: export MCPP_VENDORED_XLINGS="$XLINGS_BIN" bash tests/e2e/248_pack_library_fat_pe_leg.sh + # SPEC-007 R4.3: the engine places a Windows program's runtime DLLs + # beside it after its link, so a program started by hand finds them. + # Named here for the reason the steps above are: the Linux shards skip + # `# requires: mingw-cross wine`. Each property is held to the line the + # script prints only when that property was checked. + # The runtime_search_dir directive on PE (#701): `mcpp run` finds a DLL + # through PATH and `mcpp pack` places it. The native leg is e2e 794 on + # the Windows shards; this is its Linux-hosted counterpart. + - name: "e2e: runtime_search_dir on PE under wine" + run: | + set -o pipefail + export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + log="$RUNNER_TEMP/796.log" + bash tests/e2e/796_runtime_search_dir_on_pe_under_wine.sh 2>&1 | tee "$log" + grep -qxF "PASS: 796_runtime_search_dir_on_pe_under_wine" "$log" \ + || { echo "::error::796 did not print its PASS line"; exit 1; } + + - name: "e2e: a Windows program finds its runtime DLLs beside it" + run: | + set -o pipefail + export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + log="$RUNNER_TEMP/797.log" + bash tests/e2e/797_a_windows_program_finds_its_dlls_beside_it.sh 2>&1 | tee "$log" + for line in \ + " ok: after the first build the program started by hand finds its DLL" \ + " ok: a build with nothing changed neither relinks nor places again" \ + " ok: a DLL replaced in its directory replaces the copy" \ + " ok: no system DLL is copied" \ + "PASS: a Windows program finds its runtime DLLs beside it"; do + grep -qxF "$line" "$log" || { echo "::error::797 did not print: $line"; exit 1; } + done + # ── windows → linux ─────────────────────────────────────────────────────── # The mirror of mingw-cross-wine. Two jobs because a Windows runner cannot # execute the ELF it produces; the artefact is handed to a Linux job and diff --git a/CHANGELOG.md b/CHANGELOG.md index a26656663..38d402cc6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,85 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 +## [2026.9.26.2] - 2026-09-26 + +### 编译数据库:一个配置一个数据库(#699 的报告;#397 C-1、#677 B1) + +`compile_commands.json` 此前是一份跨命令、跨工具链合并的文件:删除后的下一次构建不再写回它, +另一个工具写入的条目与上一个工具链的条目都被保留,`directory` 写的是工程根目录而编译器在输出目录 +中运行。现在: + +- 每个配置一份数据库,位于 `target///compile_commands.json`,只在本配置之内 + 合并,因此 `mcpp test` 之后的 `mcpp build` 保留测试单元的条目。工程根目录的文件是当前配置数据库 + 的副本,整体替换、从不合并,内容相同时不写;切换工具链或 profile 就切换整个文件,切回时恢复该 + 配置的条目。根目录的符号链接照旧按其指向写入。 +- 根文件被删除后,下一次构建从配置数据库恢复它,快路径也是如此,不需要规划。 +- 被替换的根文件含有 mcpp 没有写的条目时(条目的 `output` 不在本工程的 `target/` 或 mcpp 主目录 + 之下),构建以一条警告说明其数量。 +- `directory` 与 S1 的 `work-directory` 是编译器运行的输出目录,与 JSON Compilation Database 格式、 + S1-8-2 以及 CMake、ninja 的写法一致;从 `directory` 重放 GCC 的条目不再在源码树中写出 + `gcm.cache/`。 +- 提供模块的单元在 `-c` 之前带上接口语言 flag(clang 为 `-x c++-module`,GCC 为 `-x c++`), + clangd 因此能处理 `.ixx` 接口;MSVC 的写法待 Windows 实测后再加。 +- 导入 `std` 的构建在 `compile_commands.json` 与 `emit --spec compile-commands` 中列出标准库单元 + (S1-12-1),另一个版本的 clangd 因此能自己构建 `std`。S1 文档中标准库单元的 `provides` 指向 + 共享 std 缓存中的 BMI,工具链带 `build-id`,版本与构建身份完全一致的读者可以直接复用。 +- `mcpp new` 写出的 `.gitignore` 包含 `compile_commands.json`。 + +### `emit build-database` 按成员规划(#699) + +`emit build-database --workspace` 此前在第一个规划失败的成员处停止,已规划成员的集合全部丢失。 +现在每个选中的成员独立规划:规划失败的成员不贡献集合,贡献一条 `error` 诊断,`path` 为它的 +`mcpp.toml`;至少一个成员规划成功时文档带 `data`,`watch` 也列出失败成员的 `mcpp.toml` 与 +`build.mcpp`;有任何错误时退出码为 1。在 `emit` 下,构建失败的宿主工具是警告 +`MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT`,规划继续,构建程序得到该工具将被发布的路径;构建程序失败 +的包不带该程序的指令地被描述,并得到一条 `MCPP_BUILD_DATABASE_PROGRAM_FAILED` 错误,`path` 为它的 +`build.mcpp`。`mcpp build` 的行为不变。带错误诊断的 `data` 是 S2 0.3.0 的部分回答 +(Sunrisepeak/mcpp-language-server#25)。 + +### 构建程序:`runtime_search_dir`、`prepare` 角色与 stamp 规则(#701、#702) + +- `mcpp::runtime_search_dir(dir)`(`mcpp:runtime-search-dir=`,协议 12)是 `[runtime] + runtime_search_dirs` 的构建程序形态,并入同一个字段:ELF 与 Mach-O 的运行路径、`mcpp run` 的加载 + 路径、`mcpp pack` 的闭包搜索与运行时校验都照常读取它,依赖的声明到达消费方的可执行文件。目录在 + 构建程序运行时不必存在。保留字段 `[runtime] library_dirs` 不获得指令。 +- 新角色 `prepare` 用于文件名在构建程序运行时未知的施工(安装一个前缀、解开一个 SDK):命令填充 + 用 `a.output_dir(dir)` 声明的目录,引擎写 stamp。命令成功而目录不存在或除 stamp 外不含文件时, + 这条边失败并点名目录。声明包的每条编译边与计划中的每条链接边等待它,构建输出以 `PREPARE` + 标注。构建程序的重新运行依据落在某个 `prepare` 目录内时,构建给出警告。 +- 角色以常量书写:`mcpp::roles::{source, check, object, artifact, prepare}`。引擎拒绝这五个之外的 + 角色字符串并列出它们;此前未知的字符串被静默读作 `source`。 +- `check` 与 `prepare` 的命令成功后,引擎创建缺失的 stamp,并把已存在的 stamp 更新到当前时间; + 此前已存在的 stamp 不被更新,一个输入改变一次之后,该 action 在此后每次构建中都会重新运行。 + +### Windows 程序的 DLL 在链接后放到程序旁 + +PE 映像没有运行路径,运行时搜索目录中的 DLL 此前只服务于 `mcpp run` 与 `mcpp pack`,从构建目录直接 +启动的程序找不到它。现在计划中带运行时搜索目录的 PE 程序在链接后多一条边 `mcpp place-dlls`:按 +`mcpp pack` 的闭包求解与系统规则,把程序直接或间接导入、且在这些目录中解析到的 DLL 放到程序旁, +字节不同时才写;DLL 在其目录中被替换(包括被 `prepare` action 替换)后,同一次构建会再次放置它。 +部署到程序旁的 DLL 从链接边的隐式输入改为 order-only 输入,DLL 变化不再使程序重新链接。 + +### 链接 flag 按词读取(#703) + +`[build] ldflags`、`mcpp::link_flag` 与依赖传播的链接 flag 此前只为 ninja 转义,Linux 与 macOS 的 +`sh` 展开其中的 `$ORIGIN`,程序的运行路径因此含有 `/../lib`,即宿主的 `/lib`。SPEC-004 §8 的词读法 +现在同样适用于链接 flag:每个词原样到达链接器;依赖的链接 flag 按词传播;`link_search`、`link_lib`、 +`link_script` 由路径构成的值是一个词,含空格的目录不再被拆开。为 shell 或 ninja 手工转义的写法 +(`\$ORIGIN`、`'$$ORIGIN'`)现在按写法读取,首次规划以 `build/flag-words` 指出这样的元素; +索引中没有这样的写法。 + +### 构建插件规范 SPEC-007 + +新增 `docs/specs/build-plugins.md`(草案 0.2):插件的配置、施工与校验各用一种机制,环境不完整时 +警告而不失败,配置不依赖施工结果,运行时库以 `runtime_search_dir` 声明,构建期的网络访问在离线 +构建中不发生。mcpp 只提供通用机制,某一个工具的知识只属于它的插件。 + +### 兼容性 + +- 构建程序缓存的 epoch 升到 3,升级后每个构建程序重新运行一次。 +- 构建程序协议升到 12;使用新指令、新角色或角色常量的构建程序在旧引擎上编译失败并指出名字。 + ## [2026.9.26.1] - 2026-09-26 ### 路径与文本统一为 UTF-8(#693) diff --git a/docs/01-getting-started.md b/docs/01-getting-started.md index 1c9b0475e..c2b7196b0 100644 --- a/docs/01-getting-started.md +++ b/docs/01-getting-started.md @@ -152,6 +152,31 @@ Run it only in a trusted workspace. The process exit code and the resulting `compile_commands.json` are the stable integration contract; stdout remains human-readable. +`compile_commands.json` at the project root is a copy of the current +configuration's own database, written at +`target///compile_commands.json` — the directory the +toolchain, target, profile and features together name, and the directory the +compiler actually runs in. Every command that plans in that configuration +(`build`, `test`, `run`, `--configure-only`) merges its fresh plan into that +one file, so a unit `mcpp test` added stays listed after a plain `mcpp +build`. The root copy is never merged: it is replaced whole with the +configuration's file, and left untouched when the two already agree, so +switching `--toolchain` or `--profile` switches the whole root file and +switching back restores that configuration's entries, test units included. A +symlink at the project root is followed, and the copy lands at its target. +When the replaced root file held entries mcpp did not write, one warning +states how many; the file then holds this project's own configuration. +Deleting the root file is not a way to reset it: the next build (even one +that finds nothing to recompile) restores it from the configuration's +database. + +A project that imports `std` also finds an entry for the toolchain's +standard-library units, in both `compile_commands.json` and the S1 document +`emit build-database` prints — see below. Because a BMI is readable only by +the compiler that built it, listing the unit lets a reader whose toolchain +differs from the one mcpp built with compile its own copy of `std` instead of +failing on a version mismatch. + An editor that must not write into the project asks for the same plan on stdout *(mcpp 2026.9.15.1+)*: diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index df94f6896..91ad3c597 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -422,10 +422,12 @@ bmi_schedule = "auto" # Module-edge scheduling: auto (= off) | on | #### Compile-flag syntax *(mcpp 2026.9.17.1+)* An element of `cflags`, `cxxflags` or `asmflags` stands for one or more compiler -arguments ("words"). The syntax is the same on every host, wherever the list is -written: `[build]`, `[targets.]`, a `flags` glob entry, a feature, a -`[target..build]` section, an xpkg descriptor, and the `mcpp:cflag=` / -`mcpp:cxxflag=` directives of a build program. +arguments ("words"), and an element of `ldflags` for one or more linker +arguments (mcpp 2026.9.26.2+). The syntax is the same on every host, wherever +the list is written: `[build]`, `[targets.]`, a `flags` glob entry, a +feature, a `[target..build]` section, an xpkg descriptor, and the +`mcpp:cflag=` / `mcpp:cxxflag=` / `mcpp:link-flag=` directives of a build +program. | Written | Words the compiler receives | |---|---| @@ -437,6 +439,7 @@ written: `[build]`, `[targets.]`, a `flags` glob entry, a feature, a | `"-I/opt/my\\ dir/include"` | `-I/opt/my dir/include` | | `"-IC:\\sdk\\include"` | `-IC:\sdk\include` | | `"-DNAME=a$b"` | `-DNAME=a$b` | +| `"-Wl,-rpath,$ORIGIN/../lib"` (in `ldflags`) | `-Wl,-rpath,$ORIGIN/../lib` | The rules, stated on the element's text (after TOML or Lua has removed its own escapes): @@ -452,8 +455,16 @@ escapes): taken verbatim, as in every earlier release. A `defines` entry is one value and is not read by this syntax: `defines = -["NAME=\"text\""]` passes the single word `-DNAME="text"`. `ldflags`, -`dialect_cxxflags` and `std-module-flags` are not covered by this section. +["NAME=\"text\""]` passes the single word `-DNAME="text"`. `dialect_cxxflags` +and `std-module-flags` are not covered by this section. + +`ldflags` follows the syntax from mcpp 2026.9.26.2 (#703). Before, a link-flag +element was escaped for ninja and not quoted for the shell, so on Linux and +macOS a `$ORIGIN` reached the program's run path as `/../lib`. A `-L` or +`-Wl,-rpath,` word with a package-relative path resolves against the package +root, and a dependency's `ldflags` reach its consumer word by word. An element +escaped for ninja or the shell by hand (`\$ORIGIN`, `'$$ORIGIN'`) now reads as +written; the first plan names such an element under `build/flag-words`. `compile_commands.json` and `mcpp emit build-database` list the same words in `arguments`, ready to execute without a shell. @@ -1447,7 +1458,7 @@ Link intent keeps discovery stages separate: |---|---|---|---| | `link_library_dirs` | `-L` | `-L` | `-L` or `/LIBPATH:` | | `transitive_needed_dirs` | `-Wl,-rpath-link` | no flag | no flag | -| `runtime_search_dirs` | RUNPATH/rpath only, never `-L` | rpath only | no flag | +| `runtime_search_dirs` | RUNPATH/rpath only, never `-L` | rpath only | no flag; after the link, the DLLs the program imports from these directories are placed beside it *(2026.9.26.2+)* | | `frameworks` | no flag | `-framework` | no flag | | `deploy_files` | copy edge | copy edge | copy beside the output; never a linker flag | | `deploy` *(2026.9.12.2+)* | copy edge into `bin//` | copy edge into `bin//` | copy edge into `bin//`; never a linker flag | @@ -1473,6 +1484,27 @@ For one compatibility train, `library_dirs` maps only to runtime search, `capabilities` maps to required run-phase capability requirements. None of these legacy fields creates a provider. +`runtime_search_dirs` has a build-program form for a directory only a +build.mcpp can locate (a vcpkg prefix's `bin/`, a Qt SDK's `bin/`, a directory +a `prepare` action populates): `mcpp::runtime_search_dir(dir)` (protocol 12; +[30 — Build Programs](30-build-mcpp.md)), which joins this same field +directly — not the legacy `library_dirs` above, which gains no directive of +its own. The directory need not exist when the program runs; a `prepare` +action may populate it later, at build time. + +A PE image has no run path, so on Windows a runtime search directory serves +`mcpp run`, which puts it on `PATH`, and `mcpp pack`, which stages the closure. +From 2026.9.26.2 the link of a PE program whose plan has runtime search +directories is followed by one more edge, `mcpp place-dlls`, which reads the +program's import closure as `mcpp pack` does and places beside the program +every DLL it imports, directly or through another DLL, that resolves in one of +those directories. System DLLs and API sets are never copied, a copy is written +only when its bytes differ, and a DLL replaced in its directory is placed again +on the next build. A program started by hand from the build directory therefore +finds them, as it does after vcpkg's applocal step or CMake's +`$`. When two directories offer one name, the first in +search order is placed and a note names both. + `target///resolution.json` schema 2 stores the RuntimeBinding, canonical requirements/providers/artifacts, LinkIntent, platform search mechanism, and post-link verdict. `mcpp why runtime` is a pure interpreter of diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index a0b365e5c..c3236847a 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -46,6 +46,44 @@ int main() { mcpp build # compiles + runs build.mcpp, then builds the project ``` +## Three kinds of work + +Everything a `build.mcpp` or a rule package it imports does is exactly one of +three kinds (SPEC-007 §1, `docs/specs/build-plugins.md`), and each has one +mechanism: + +| Kind | Decides | Happens | Mechanism | +|---|---|---|---| +| Configuration | the shape of the build: which sources, which flags, what to link, where to find a library at launch | while `build.mcpp` runs — **planning** | a directive (this chapter's tables) | +| Construction | files the build reads: generating a source, compiling device code, installing a dependency prefix, packaging | **build** time | `mcpp::action` | +| Verification | whether the environment or the product satisfies a requirement, producing no file the build reads | **build** time | an action with `role = check` | + +**Configuration depends only on declared inputs.** A `build.mcpp` reads its +package's manifest, its declared payloads (`xpkg_dir`), and whatever it named +with `rerun_if_changed`/`rerun_if_changed_glob`/`rerun_if_env_changed` — never +a file an action produces, because planning (`mcpp emit build-database`) and +the first build both happen *before* any action runs, and a program that read +one would configure differently the first time and the second. See +[the `prepare` role](#construction-whose-file-names-are-not-known-yet-the-prepare-role-protocol-12) +below for the warning this earns. + +**Construction is never done while `build.mcpp` runs.** Writing a file +directly in `main()` happens once per plan, serially, for the whole set, and a +failure is reported as "build.mcpp exited 1" three edges away from its cause. +Declare it instead (`mcpp::action`, below) and it becomes an edge: incremental, +parallel, and attributable. + +**An incomplete environment is a warning, never a failed configure.** A +program that cannot find an SDK module or a library says so with +`mcpp::warning` (below) and emits whatever it could still configure; +compilation or linking fails, if it must, at the place that is actually +missing the thing. Exiting non-zero here would fail builds that do not need +what is missing. + +**One behaviour under planning and building.** `mcpp emit build-database` +computes the same plan `mcpp build --configure-only` does, so a build program +that follows the rule above needs no signal telling it which one is asking. + ## Directives Print these to stdout (one per line). Any line that does not start with `mcpp:` @@ -67,6 +105,7 @@ is ignored, so diagnostics may be logged freely. | `mcpp:windows-subsystem=:` *(2026.9.12.2+)* | set the PE subsystem (`console` or `windows`) of the executable `` of **this** package, the same field as `[targets.] windows_subsystem` (docs/04). Reaches that target's link and no other, never a consumer, and renders nothing on a target that is not PE. A target the package does not declare with `kind = "bin"`, a value outside the set, and a value that contradicts mcpp.toml are each refused before any directive is applied | | `mcpp:windows-entry=:` *(2026.9.12.2+)* | set the entry function (`main`, `wmain`, `WinMain` or `wWinMain`) of the executable ``, the same field as `windows_entry`; the scope and the refusals are those of `windows-subsystem` | | `mcpp:deploy=\t` *(2026.9.12.3+, protocol 11)* | place a file this program produced or selected beside the artifact, at ``, relative to the executable's directory — the build-program form of `[runtime] deploy` (docs/04 §2.11). `` may be absolute (an action's own declared output) or resolved against the package root; TAB-separated, because an absolute Windows `` contains a colon. **Reaches the consumer**, joining the same `LinkIntent` `link-lib`/`link-search`/`link-flag` feed — see below | +| `mcpp:runtime-search-dir=` *(protocol 12)* | add `` to the launch-time search path — the build-program form of `runtime_search_dirs` (docs/04 §2.11). Relative resolves against the package root; the directory need not exist yet (a `prepare` action may populate it later). **Reaches the consumer**, joining `LinkIntent::runtimeSearchDirs` directly: RUNPATH/rpath on ELF and Mach-O, never `-L`, and `mcpp pack`'s closure search — see below | | `mcpp:link-script=` *(2026.8.19+)* | link with this **linker script** (`-T`; relative resolves against the package root, and the emitted path is absolute because the link runs in the build directory). Reaches the **consumer**, unlike `include-dir` — a board's memory layout is the one thing a consumer cannot write for itself | | `mcpp:warning=` *(2026.8.21.2+)* | say something to the user and **keep going**. The one directive that changes no compile line, no link line and no source set. Survives the build cache — see below | | `mcpp:fact==` *(2026.9.5.2+)* | state something the program **established about the machine** (`cuda.driver=12.4`). Compared against floors before anything is compiled; see below | @@ -134,6 +173,7 @@ int main() { | `mcpp::link_flag(s)` *(2026.9.6.5+)* | `mcpp:link-flag=` | | `mcpp::windows_subsystem(target, value)` / `mcpp::windows_entry(target, value)` *(2026.9.12.2+)* | `mcpp:windows-subsystem=` / `mcpp:windows-entry=` | | `mcpp::deploy(from, to)` *(2026.9.12.3+, protocol 11)* | `mcpp:deploy=\t` — see below | +| `mcpp::runtime_search_dir(dir)` *(protocol 12)* | `mcpp:runtime-search-dir=` — see below | | `mcpp::link_script(p)` *(2026.8.19+)* | `mcpp:link-script=` | | `mcpp::runner(tok)` *(2026.8.19.2+)* | `mcpp:runner=` — see below | | `mcpp::xpkg_dir(ns, name)` / `mcpp::xpkg_dir(name)` *(2026.8.19+)* | the payload directory of a package declared in `[xlings.workspace]` — by this manifest, or by a dependency compiled into this build program *(2026.9.6.6+)*; `""` when it was not declared or is not installed (see below) | @@ -426,6 +466,21 @@ including `[tools.overrides]` and `reexport = true` (which is how a library provides the whole toolchain, so a project declares **one** dependency instead of four). +**Under `mcpp emit build-database`, a host tool that fails to build is a +warning, not a lost plan.** Planning continues for every member, and +`dep_bin` answers with the path the tool *would* be published at — the same +path a successful build would occupy, fixed before the build ever runs. +Naming the tool this way configures exactly as it would after a successful +build. **Calling it** — running it while configuring, the pattern above stops +short of — fails, because the path names nothing that exists; that failure +costs this package's own directives (its build program's part of the +description), never the whole command. The tool is still built during an +actual `mcpp build`: a program may run it there, and its blocking checks still +run, because building it unverified into the store would be worse than not +building it. Prefer running a host tool from inside an `action`'s command +(construction), not from `main()` (configuration), so that whether the tool +happens to build does not decide whether planning sees this package at all. + ### Globbing inputs: `rerun_if_changed_glob` (2026.8.6.2+) The re-run key is built from *declared* inputs. Declare files and it works; @@ -479,7 +534,7 @@ int main() { const std::string out = std::string(mcpp::out_dir()) + "/foo.pb.cc"; mcpp::action a; a.id = "protoc:foo"; - a.role = "source"; // "source" | "check" | "object" | "artifact" + a.role = mcpp::roles::source; // source | check | object | artifact | prepare a.arg(mcpp::dep_bin("protobuf", "protoc")) .arg("--cpp_out=...").arg("proto/foo.proto") .input("proto/foo.proto") @@ -488,7 +543,7 @@ int main() { } ``` -Four roles, one primitive — `role` only decides where the edge's outputs +Five roles, one primitive — `role` only decides where the edge's outputs attach: | `role` | Outputs | Ordering | Typical | @@ -497,13 +552,24 @@ attach: | `check` | a stamp file, written by mcpp | runs **alongside** compilation; `blocking = true` makes the package's compile edges wait for it | clang-tidy, a format or ABI check | | `object` | join the **link** set | the link edge consumes them | a resource compiler, `objcopy` embedding a blob, a generated `.def`, a pre-built `.o` | | `artifact` | a new file | its *inputs* are link outputs, so it runs after the link | codesign, packaging, size budgets | +| `prepare` *(protocol 12)* | a stamp file, written by mcpp, plus one declared directory (`output_dir`) the command populates | **every compile edge and every link edge in the plan wait for it** — see below | installing a vcpkg manifest or a CMake subproject into a prefix, unpacking an SDK | `artifact` is also the only role that may name `${mcpp.stage_dir}` — see [Producing a distributable](#producing-a-distributable-pack_format--stage_dir-20269111) -below. The other three run before or alongside the link, so there is nothing +below. The other roles run before or alongside the link, so there is nothing staged for them to read, and mcpp refuses the placeholder rather than expanding it to a path that happens to exist. +**Spell the role with the `mcpp::roles` constants** (protocol 12), not a bare +string. A build.mcpp that writes `a.role = mcpp::roles::prepare;` fails to +*compile* on an older engine, naming the missing constant — the same +compile-time signal `mcpp::deploy` and `mcpp::runtime_search_dir` give. An +engine from protocol 12 on refuses a role string that names none of the five, +listing them; before this, an unrecognised string was silently read as +`source`, which changed an action's meaning without a word. The frozen +surface (a hand-written `printf("mcpp:action=...")` program) may still spell +the string directly. + No phase machinery is involved. `object` and `artifact` are sequenced by ninja's own file dependencies — which is also why an `artifact` action cannot double-apply itself the way a naive "post-build hook" would. `source` and a @@ -543,11 +609,22 @@ matters for already emits one: `glslangValidator --depfile`, `glslc -MD -MF`, > would be permanently dirty. **A check's command does not have to write its stamp** (mcpp 2026.8.29.1+). -The verdict is the exit code; the stamp is bookkeeping the graph needs, and -mcpp creates it when the command succeeds. Before this, every check needed a -wrapper script to touch the file — and a command is an argv with no shell -assumed, so that wrapper could not be written portably at all. A command that -already writes its own stamp is unaffected: an existing file is left alone. +The verdict is the exit code; the stamp is bookkeeping the graph needs. +Before this, every check needed a wrapper script to touch the file — and a +command is an argv with no shell assumed, so that wrapper could not be +written portably at all. + +**After a `check` or `prepare` command succeeds, every declared stamp is +newer than every input, whether or not the command wrote it** (protocol 12). +mcpp creates each stamp that is missing and sets the modification time of +each *existing* one to the present — unconditionally, not only when an input +changed — so a check whose command writes nothing (clang-tidy, an installer +run through a rule package) never re-runs merely because its own stamp +stayed older than an input it had already reacted to. A command that writes +its own stamp is unaffected: a file the command itself created or rewrote +during this run is exactly as new as the command left it, and touching it +again changes nothing a build reads — a stamp feeds no compile or link edge, +only the order-only edge described below. On failure nothing is written. > A missing stamp does **not** fail the build. ninja leaves the declared output > absent and re-runs that edge on every build afterwards, which looks like a @@ -656,6 +733,116 @@ int main() { | `.apk` | `assets/myapp.resources/` | | web | the same relative path in the static directory, served beside `.js`. A project that wants the files inside the `.data` preload instead links with `--preload-file @/`, an ordinary link flag | +### A launch-time search directory: `runtime_search_dir` (protocol 12) + +`runtime_search_dirs` (docs/04 §2.11) names a directory to search when the +artifact runs. It is a fixed TOML array, so it cannot name a directory a +build.mcpp only discovers — a vcpkg prefix's `bin/` (Windows) or `lib/` +(everywhere else), a Qt SDK's `bin/`, or a directory a `prepare` action +populates (below). `mcpp::runtime_search_dir` is that same declaration, +reached from a build program: + +```cpp +import mcpp; +#include + +int main() { + const std::string qtPrefix = locate_qt_prefix(); // however this + // package finds it + // Import libraries live in `lib/`; the DLLs `mcpp run` and `mcpp pack` + // need to find at launch time live in `bin/` on Windows. + mcpp::link_search((qtPrefix + "/lib").c_str()); + mcpp::runtime_search_dir((qtPrefix + "/bin").c_str()); +} +``` + +- **Joins `LinkIntent::runtimeSearchDirs` directly**, the same field + `runtime_search_dirs` populates from TOML — not the retiring `[runtime] + library_dirs`, which gains no directive of its own (docs/04 §2.11). A + directive-declared directory reaches every reader of that field exactly as + one written in `mcpp.toml` would: `mcpp run`'s loader path, `mcpp pack`'s + closure search, and RUNPATH/rpath on ELF and Mach-O (never `-L` — a + launch-time search directory is not a link-library search path). +- **`dir` may be absolute or package-relative.** A relative value resolves + against the package root, like every other `AbsPath` directive + (`include-dir`, `deploy`'s `from`). +- **The directory need not exist when this program runs.** A `prepare` + action may populate it later, at build time — see below. +- **Reaches a consumer.** A dependency's declared directory joins its + consumer's executable through the same merge the manifest key uses + (`resolve_runtime_contract`, `plan.cppm`). +- **Replayed on a cache hit.** A `runtime-search-dir` directive is persisted + in the build cache like `deploy` and `warning`; a cached run restores it + exactly as a fresh run would. + +### Construction whose file names are not known yet: the `prepare` role (protocol 12) + +`source`, `check`, `object` and `artifact` all name their outputs at +submission, because mcpp fixes the source set, the fingerprint and the +module graph during prepare (`action`, above). Installing a vcpkg manifest +or a CMake subproject into a prefix, or unpacking an SDK, does not fit that +shape: the files construction writes are not known until the command runs — +often not even by name, only by *directory*. `prepare` is the role for +exactly this: + +```cpp +import mcpp; +#include + +int main() { + const std::string prefix = std::string(mcpp::out_dir()) + "/vcpkg-install"; + mcpp::action a; + a.id = "vcpkg:install"; + a.role = mcpp::roles::prepare; + a.arg("vcpkg").arg("install").arg("--x-install-root").arg(prefix.c_str()) + .input("vcpkg.json") + .output((prefix + ".stamp").c_str()) + .output_dir(prefix.c_str()) + .submit(); + + mcpp::link_search((prefix + "/lib").c_str()); + mcpp::runtime_search_dir((prefix + "/lib").c_str()); +} +``` + +- **Outputs.** One or more stamps, written by the engine exactly as for + `check` (see the stamp rule above), and one declared directory, + `output_dir(dir)`, which the *command* populates. **If the command + succeeds and that directory does not exist or holds no file other than the + action's stamps, the engine writes no stamp and fails the edge, naming the + directory** — whether or not the command created it: a Qt `lupdate` + action's directory, for instance, is an existing source tree it rewrites, + not a fresh one. A stamp may lie inside the directory or beside it. +- **Ordering.** Every compile edge of the declaring package waits for it, + through the same order-only edge a blocking `check` uses; so does every + *link* edge in the plan, because a `runtime_search_dir` (or any other + link-global directive) any package declares is merged into the one + `LinkIntent` the whole plan's link lines share — a directory `prepare` + populates has to exist before any link that might read it, not only the + declaring package's own. +- **References to its products are names fixed at configuration time.** Once + the directory exists, the build reads it *by directory*: + `include_dir`/`link_search`/`runtime_search_dir` for the directory itself, + `link_flag` for one file's full path inside it. Nothing about `prepare` + changes what those directives already do. +- **Why a role, and not `check` with `blocking = true`.** A role is read by + engine decisions the way `blocking` alone is not: labelling construction as + verification would make it indistinguishable from a check the moment a + future policy treats checks specially (SPEC-007 §3). +- **Progress lines** for a `prepare` action are labelled `PREPARE`, next to + `CHECK`, `OBJECT`, `ARTIFACT` and `GENERATE`. + +> **Do not read a `prepare` directory's contents while configuring.** A +> `rerun_if_changed`/`rerun_if_changed_glob` naming a path inside another +> package's (or this package's own) `prepare` directory reads a construction +> result while this program *configures*: correct on the second build, once a +> previous build has populated the directory, and wrong on the first. mcpp +> warns, naming both paths, rather than refusing — the program still +> configures correctly today. This is the pattern of watching an install +> prefix to place a first installation's libraries on the *next* plan; +> declare `runtime_search_dir` for the prefix instead, and let a `prepare` +> action do the installing. + ### Producing a distributable: `pack_format` / `stage_dir` (2026.9.11.1+) An `.msi`, a `.deb`, an AppImage and a signed `.app` are none of the four roles' diff --git a/docs/31-authoring-a-rule-package.md b/docs/31-authoring-a-rule-package.md index a518282d1..d4f925d55 100644 --- a/docs/31-authoring-a-rule-package.md +++ b/docs/31-authoring-a-rule-package.md @@ -186,9 +186,13 @@ a.submit(); The strings must outlive the action. `a.id = ("toy:" + stem).c_str()` hands it a pointer into a temporary that is gone by `submit()`. -### The four roles +### The five roles -`role` decides where the edge's outputs go and when the edge runs. +`role` decides where the edge's outputs go and when the edge runs. Spell it +with the `mcpp::roles` constants (`mcpp::roles::source`, `::check`, +`::object`, `::artifact`, `::prepare`), not a bare string: an engine that does +not know a constant refuses to compile the rule package, naming it, instead +of reading a misspelt string as `source` (SPEC-007 R3.6). | `role` | outputs | ordering | |---|---|---| @@ -196,6 +200,14 @@ pointer into a temporary that is gone by `submit()`. | `check` | a stamp file | runs alongside compilation; `blocking = true` makes compiles wait | | `object` | join the **link** set | the link edge consumes them | | `artifact` | a new file | its inputs are link outputs, so it runs after the link | +| `prepare` | a stamp file, plus one declared directory (`output_dir`) the command populates | every compile edge *and* every link edge in the plan wait for it | + +`prepare` is for construction whose file names are not known when the rule +runs — installing a package-manager prefix, unpacking an SDK — and is what +SPEC-007 (`docs/specs/build-plugins.md`) asks a dependency-adapter rule to use +instead of a blocking `check`; see docs/30's own section on the role for the +full contract (the post-condition on `output_dir`, and the R1.3 warning a +rerun input inside it earns). ### Declared inputs, and the compiler among them diff --git a/docs/50-machine-output.md b/docs/50-machine-output.md index 48a3e61a3..4c7f2040c 100644 --- a/docs/50-machine-output.md +++ b/docs/50-machine-output.md @@ -463,21 +463,47 @@ whatever it would print to `` instead. The content of the document, the no-write guarantee and the `watch` rules are [SPEC-005](specs/build-database.md). -A failure omits `data` and exits 1, with the diagnostic code -`MCPP_BUILD_DATABASE_NO_PROJECT` outside a project, -`MCPP_OFFLINE_DOWNLOAD_REQUIRED` when an offline plan (`--offline`, -`MCPP_OFFLINE`, `MCPP_NO_AUTO_INSTALL`) needs something that has to be -downloaded (a toolchain, a package, a git revision or the package index; the -message names the first one), or `MCPP_BUILD_DATABASE_PLAN_FAILED` when planning -fails for any other reason. The first of the three is not a defect of the -project: one run without `--offline` removes it. Warnings leave the -document in place: - -| code | | -|---|---| -| `MCPP_LOCK_WOULD_CHANGE` | the resolution differs from the project's `mcpp.lock`, which the command does not write | -| `MCPP_GENERATED_FILE_NOT_MATERIALIZED` | a root `[build] generated_files` entry is missing or stale on disk, and the command does not write it | -| `MCPP_BUILD_DATABASE_STD_UNIT_UNDESCRIBED` | no standard-library build command names its module source, so that unit is not listed | +`emit` plans every selected member on its own (#699 item 1): one member's +planning failure does not cost its siblings'. Outside a project, or when +every selected member fails to plan, the envelope omits `data` and exits 1, +with one diagnostic per failed member: `MCPP_BUILD_DATABASE_NO_PROJECT` +outside a project; `MCPP_OFFLINE_DOWNLOAD_REQUIRED` when an offline plan +(`--offline`, `MCPP_OFFLINE`, `MCPP_NO_AUTO_INSTALL`) needs something that has +to be downloaded (a toolchain, a package, a git revision or the package +index; the message names the first one); or `MCPP_BUILD_DATABASE_PLAN_FAILED` +when planning fails for any other reason. The offline code is not a defect +of the project: one run without `--offline` removes it. Each member's +diagnostic carries `path`, that member's `mcpp.toml` relative to the +workspace root. + +When at least one selected member planned, `data` is present and describes +every member that did: a member that failed contributes no set, one `error` +diagnostic as above, and its `mcpp.toml` and `build.mcpp` (when present) join +`watch`. The exit status is still 1 whenever any diagnostic is an error, so a +consumer reads three outcomes structurally — no `data`; `data` with `error` +diagnostics, describing everything except what they name; `data` with none — +without parsing a message. + +A package whose build program fails (#699 item 2) is described without that +program's directives: the manifest's own configuration, the toolchain, the +module graph and the standard-library units are described as usual, and one +`error` diagnostic, `MCPP_BUILD_DATABASE_PROGRAM_FAILED`, names it, with +`path` naming its `build.mcpp`. A later failure that follows from the missing +directives fails the whole member instead, under the rule above. A host tool +a package requested that fails to build is a warning instead, +`MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT`, naming the tool, its package and the +first line of the failure; planning continues, and a build program that only +names the tool configures as it would after a successful build. `mcpp build` +is unaffected by either: a build program or a host tool that fails there +still fails the build. + +| code | severity | | +|---|---|---| +| `MCPP_LOCK_WOULD_CHANGE` | warning | the resolution differs from the project's `mcpp.lock`, which the command does not write | +| `MCPP_GENERATED_FILE_NOT_MATERIALIZED` | warning | a root `[build] generated_files` entry is missing or stale on disk, and the command does not write it | +| `MCPP_BUILD_DATABASE_STD_UNIT_UNDESCRIBED` | warning | no standard-library build command names its module source, so that unit is not listed | +| `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT` | warning | a requested host tool failed to build; the tool is still built and its `check` actions still run | +| `MCPP_BUILD_DATABASE_PROGRAM_FAILED` | error | a build program failed; its package is described without its directives | `--protocol-version` declares `init-mcpp-home`, `read-project`, `network`, `write-global-cache` and `exec-build-script` for the command, and never diff --git a/docs/README.md b/docs/README.md index fb7766aec..b1b533e06 100644 --- a/docs/README.md +++ b/docs/README.md @@ -162,3 +162,4 @@ downstream tooling. - [SPEC-004 — `mcpp.toml` planes, conditioning shape, resolution axes and naming](specs/manifest-semantics.md) - [SPEC-005 — The build database `mcpp emit build-database` prints](specs/build-database.md) - [SPEC-006 — Toolchain management: identity, origin, selection and the payload contract](specs/toolchain-management.md) + - [SPEC-007 — Build plugins: configuration, construction and verification, and the runtime and planning obligations](specs/build-plugins.md) diff --git a/docs/specs/README.md b/docs/specs/README.md index 6df637697..6bfb1f504 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -33,9 +33,10 @@ | [SPEC-001](package-identity.md) | 包身份(`package.namespace` / `package.name`)、`[dependencies]` 选择器与匹配机制 | 评审中 v1.1 | 2026-08-03 | mcpp >= 0.0.106 | | [SPEC-002](target-side.md) | 目标侧模型与能力声明(`mcpp:` 保留命名空间、五层、三条规则) | 评审中 v1.0 | 2026-08-24 | mcpp >= 2026.8.24.2 | | [SPEC-003](exit-codes.md) | 退出码契约(分类、语义、稳定性承诺) | 评审中 v1.0 | 2026-09-01 | mcpp >= 2026.9.1.1 | -| [SPEC-004](manifest-semantics.md) | `mcpp.toml` 的平面划分、条件化形状、解析轴与命名规约 | 草案 v1.4 | 2026-09-15 | 条件化形状 mcpp >= 2026.8.29.1;目标轴 mcpp >= 2026.9.6.4;`linkage` 默认值 mcpp >= 2026.9.15.2 | -| [SPEC-005](build-database.md) | 构建数据库:`mcpp emit build-database` 的内容、取值规则与不写工程目录的保证 | 评审中 v1.0 | 2026-09-15 | mcpp >= 2026.9.15.1 | +| [SPEC-004](manifest-semantics.md) | `mcpp.toml` 的平面划分、条件化形状、解析轴与命名规约 | 草案 v1.7 | 2026-09-26 | 条件化形状 mcpp >= 2026.8.29.1;目标轴 mcpp >= 2026.9.6.4;`linkage` 默认值 mcpp >= 2026.9.15.2;链接 flag 的词读法 mcpp >= 2026.9.26.2 | +| [SPEC-005](build-database.md) | 构建数据库:`mcpp emit build-database` 的内容、取值规则与不写工程目录的保证 | 评审中 v1.3 | 2026-09-26 | mcpp >= 2026.9.15.1;v1.3 条款 mcpp >= 2026.9.26.2 | | [SPEC-006](toolchain-management.md) | 工具链管理:身份、来源、选择与载荷契约 | 草案 v0.2 | 2026-09-24 | 逐条标注;已实现条款 mcpp >= 2026.9.24.1 | +| [SPEC-007](build-plugins.md) | 构建插件:配置、施工与校验的分工,运行时与规划期的义务 | 草案 v0.2 | 2026-09-26 | 逐条标注;mcpp >= 2026.9.26.2 | ## 文档约定 diff --git a/docs/specs/build-database.md b/docs/specs/build-database.md index 60b42f3ed..82db5b509 100644 --- a/docs/specs/build-database.md +++ b/docs/specs/build-database.md @@ -4,13 +4,13 @@ |---|---| | 规范编号 | SPEC-005 | | 标题 | mcpp 输出的构建数据库:内容、取值规则与不写工程目录的保证 | -| 状态 | 评审中 v1.2 | -| 版本 | 1.2 | -| 最后修改 | 2026-09-17 | -| 对应实现 | mcpp >= 2026.9.15.1 | -| 相关设计文档 | `.agents/docs/2026-09-14-636-build-database-and-the-latest-xlings.md` | -| 相关 issue | #636, #648, #655 | -| 依据的外部规范 | S1「C++ Build Database: IDE Profile」profile 0.2.0 与 S2 0.2.0 §3.4,取自 https://github.com/Sunrisepeak/lsp-mcpp-private 提交 `b82859d`(schema 自提交 `28ecd6e` 起未变);JSON Compilation Database | +| 状态 | 评审中 v1.3 | +| 版本 | 1.3 | +| 最后修改 | 2026-09-26 | +| 对应实现 | mcpp >= 2026.9.15.1;v1.3 修改的 R2.5、R3.7、R3.8、R4.1、R5.2 为 mcpp >= 2026.9.26.2 | +| 相关设计文档 | `.agents/docs/2026-09-14-636-build-database-and-the-latest-xlings.md`
`.agents/docs/2026-09-26-compile-database-and-issue-699-design.md` | +| 相关 issue | #636, #648, #655, #699, #702 | +| 依据的外部规范 | S1「C++ Build Database: IDE Profile」profile 0.2.0 与 S2 0.2.0 §3.4,取自 https://github.com/Sunrisepeak/lsp-mcpp-private 提交 `b82859d`(schema 自提交 `28ecd6e` 起未变);S2 0.3.0 §3.4 的部分回答(S2-3.4-12、S2-3.4-13,Sunrisepeak/mcpp-language-server#25);JSON Compilation Database | ## 0. 适用范围 @@ -53,7 +53,10 @@ Database 定义,本规范不重复它们的字段定义,只规定 mcpp 作为生 每个输出一条警告 `MCPP_GENERATED_FILE_NOT_MATERIALIZED`。**已实现** - **R2.5** 构建程序照常运行,工作目录为包根,与 `mcpp build` 相同;构建程序在 `MCPP_OUT_DIR` 之外写入的内容不在本保证之内。依赖提供的宿主工具照常构建到全局 - 工具库。**已实现** + 工具库,它声明的 `check` 动作照常运行。构建失败的宿主工具在本命令下降级为 + 警告 `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT`,消息点名工具、其所属包与失败信息 + 的第一行;规划继续,请求该工具的构建程序收到的是该工具本应发布到的路径。 + `mcpp build` 不受影响,宿主工具构建失败在其中仍使目标失败。**已实现** - **R2.6** `mcpp --protocol-version` 为这条命令声明 `init-mcpp-home`、`read-project`、 `network`、`write-global-cache` 与 `exec-build-script`,不声明 `write-project`。 **已实现** @@ -98,12 +101,21 @@ mcpp 输出的 S1 文档满足 S1 等级 2,不输出 `ide.options`。等级 3 - **R3.7** 除 NASM 单元外,构建计划中的每个编译单元是一个翻译单元。`source`、 `work-directory`、`arguments`、`object` 与 `compile_commands.json` 中对应条目的 `file`、`directory`、`arguments`、`output` 取自同一条记录,因而逐字相同。 - `arguments` 中的每一项是编译器收到的一个参数,不带任何宿主的引号或转义,不经 shell - 即可执行:单元自己的 flag 列表按 SPEC-004 §8 读成的词列出,引擎为宿主渲染的文本 - 按该宿主的读取规则(POSIX `sh` 或 MSVCRT)还原。**已实现** -- **R3.8** `provides` 把单元提供的模块名映射到空字符串,命令不执行构建(S1-8-6); - `requires` 为单元导入的模块名,分区写全名 `M:P`。`private` 为 `false`,理由同 R3.4。 - **已实现** + `work-directory` 是编译器实际运行的目录——即输出目录 + `target//`——对每个工程单元与每种工具链皆然;标准库单元 + 保留它们本来所在的共享 std 缓存目录(§3.4)。`arguments` 中的每一项是编译器收到 + 的一个参数,不带任何宿主的引号或转义,不经 shell 即可执行:单元自己的 flag 列表 + 按 SPEC-004 §8 读成的词列出,引擎为宿主渲染的文本按该宿主的读取规则(POSIX `sh` + 或 MSVCRT)还原。提供某个模块的单元,`arguments` 在 `-c ` 之前带有该 + 单元的模块接口语言标记(GCC、Clang 方言);MSVC 方言在 Windows 上量出 clang-cl + 模式的 clangd 是否接受 `/interface` 之前留空。**已实现** +- **R3.8** 工程单元的 `provides` 把单元提供的模块名映射到空字符串,因为这条命令 + 不执行构建(S1-8-6);`requires` 为单元导入的模块名,分区写全名 `M:P`。`private` + 为 `false`,理由同 R3.4。标准库单元的 `provides` 例外:把 `std`、`std.compat` + 映射到构建会写出的 BMI 在共享 std 缓存中的路径——这条路径由缓存键决定,不需要 + 真的编译就能得到(§3.4)。`ide.toolchains..build-id` 给出编译器的构建标识, + 取自 mcpp 已经算出的驱动身份(工具链指纹的同一个字段),同一工具链的两次运行 + 之间保持稳定。**已实现** - **R3.9** `ide.role` 取自扫描器读到的模块声明形式: | 声明 | `ide.role` | @@ -137,19 +149,28 @@ mcpp 输出的 S1 文档满足 S1 等级 2,不输出 `ide.options`。等级 3 ## 4. `--spec compile-commands` - **R4.1** 文档为 `mcpp build --configure-only` 在同一组选择器下写入 - `compile_commands.json` 的条目,差别只在输出路径位于 §2 的工作目录之下。标准库模块 - 的单元不在其中。**已实现** + `compile_commands.json` 的条目,差别只在输出路径位于 §2 的工作目录之下。标准库 + 模块的单元也在其中,遵循 S1-12-1 的导出规则:S1 文档里 `mcpp:std` 集合的每个 + 单元同样导出为一条 `compile_commands.json` 条目。**已实现** ## 5. 信封 - **R5.1** `kind` 为 `mcpp.build-database`,`kindVersion` 为 1。`data` 含 `spec` (`{"name": "s1", "version": "0.2.0"}` 或 `{"name": "compile-commands"}`)、 `database`、`watch` 与 `inputs-fingerprint`。**已实现** -- **R5.2** 失败时信封不含 `data`,`diagnostics` 至少含一条 `error`,退出码为 1:不在 - 工程中为 `MCPP_BUILD_DATABASE_NO_PROJECT`;离线运行而规划需要下载时为 +- **R5.2** 命令独立规划每一个被选中的成员:一个成员规划失败只影响它自己,不影响 + 其余成员的集合(#699 第 1 项)。规划失败的成员不贡献任何集合,只贡献一条 `error` + 诊断,`path` 为该成员的 `mcpp.toml`,相对工作区根目录;诊断码为:不在工程中时 + `MCPP_BUILD_DATABASE_NO_PROJECT`;该成员的规划因离线而需要下载时 `MCPP_OFFLINE_DOWNLOAD_REQUIRED`,消息指出需要下载的第一项;其他规划失败为 - `MCPP_BUILD_DATABASE_PLAN_FAILED`。工作区中消息指出成员;任一成员规划失败,整次命令 - 失败。**已实现**(离线诊断码:mcpp >= 2026.9.16.1) + `MCPP_BUILD_DATABASE_PLAN_FAILED`。`data` 在至少一个被选中的成员规划成功时出现, + 并描述每一个规划成功的成员;被选中的成员全部规划失败时,信封不含 `data`。规划成功 + 的成员中,构建程序失败的包被描述为不含该程序产生的指令(清单自身的配置、工具链、 + 模块图与标准库单元仍照常描述),`diagnostics` 另有一条 `error`, + `MCPP_BUILD_DATABASE_PROGRAM_FAILED`,`path` 为该包的 `build.mcpp`;后续失败若是 + 由缺失的指令引起,则按前一条规则使整个成员失败。只要 `diagnostics` 中有一条 + `error`,退出码就是 1,无论 `data` 是否出现。**已实现**(离线诊断码: + mcpp >= 2026.9.16.1;成员独立规划、`path` 与构建程序失败的描述:mcpp >= 2026.9.26.2) - **R5.3** 信封的 `effects` 为 `read-project` 与 `write-global-cache`,运行了构建程序时 另有 `exec-build-script`,本次运行启动过网络子进程(索引刷新、安装、git 远程操作, 失败或超时的也算)时另有 `network`。**已实现**(`network`:mcpp >= 2026.9.16.1) @@ -178,3 +199,4 @@ mcpp 输出的 S1 文档满足 S1 等级 2,不输出 `ide.options`。等级 3 | 1.0 | 2026-09-14 | 首版(#636)。 | | 1.1 | 2026-09-16 | R5.2 增加离线诊断码 `MCPP_OFFLINE_DOWNLOAD_REQUIRED`;R5.3 的 `network` 按观测列出;新增 R5.4(子进程不继承调用方描述符,xlings 子进程有期限并随 mcpp 结束)(#648)。 | | 1.2 | 2026-09-17 | R3.7 陈述 `arguments` 的每一项是编译器收到的参数,单元 flag 按 SPEC-004 §8 的词列出(#655)。 | +| 1.3 | 2026-09-26 | R2.5:`emit` 下构建失败的宿主工具是警告。R3.7:`work-directory` 是输出目录,模块接口单元的 `arguments` 带语言 flag。R3.8:标准库单元的 `provides` 指向 std 缓存中的 BMI,工具链带 `build-id`。R4.1:compile-commands 文档包含标准库单元(S1-12-1)。R5.2:成员各自规划,构建程序失败的包不带其指令地被描述(#699,#702)。 | diff --git a/docs/specs/build-plugins.md b/docs/specs/build-plugins.md new file mode 100644 index 000000000..8bd9ed7f2 --- /dev/null +++ b/docs/specs/build-plugins.md @@ -0,0 +1,185 @@ +# SPEC-007:构建插件:配置、施工与校验的分工,运行时与规划期的义务 + +| 项 | 值 | +|---|---| +| 规范编号 | SPEC-007 | +| 标题 | 构建插件:配置、施工与校验的分工,运行时与规划期的义务 | +| 状态 | 草案 v0.2 | +| 版本 | 0.2 | +| 最后修改 | 2026-09-26 | +| 对应实现 | 逐条标注。未注明版本的「已实现」条款对应 mcpp >= 2026.9.26.1;注明 mcpp#702 的条款对应 mcpp >= 2026.9.26.2 | +| 相关设计文档 | `.agents/docs/2026-09-26-compile-database-and-issue-699-design.md`(§5) | +| 相关 issue | mcpp#699、mcpp#701、mcpp#702、mcpp#703 | +| 使用文档 | [docs/30 - build.mcpp](../30-build-mcpp.md)、[docs/31 - 编写规则包](../31-authoring-a-rule-package.md) | + +本规范规定构建插件对引擎和对消费方承担的义务,以及引擎为此提供的机制。docs/31 说明怎样编写 +插件,本规范规定插件必须满足什么。 + +规范用语与实现状态标记见 [规范索引](README.md)。只约束插件作者、引擎不做检查的条款标注 +「作者义务」。 + +## 0. 适用范围 + +构建插件是为其他包的构建贡献工作的包,以及它导入消费方构建程序的模块(`rule_module`、 +`host-module`)。 + +| 类别 | 例 | 贡献 | +|---|---|---| +| 规则包 | `rules-qt`、`rules-spirv`、`rules-cuda` | 代码生成,设备语言的编译 | +| 依赖适配包 | `deps-vcpkg`、`deps-cmake` | 把外部包管理器或外部构建系统的产物接入构建 | +| 分发成员 | `dist-wix`、`dist-apk` | 由链接产物生成可安装的分发物 | + +项目自己的 `build.mcpp` 中的同类代码同样适用本规范。 + +**分工。** mcpp 只提供通用机制:指令、action 的角色、stamp、运行时库的搜索与放置。某一个 +工具(vcpkg、CMake、Qt)的知识只属于它的插件。插件遇到的缺口若是通用的,由 mcpp 以通用机制 +补上(第 3.3 节的 `prepare`、第 4.1 节的 `runtime_search_dir`、第 4.3 节的 DLL 放置),而不 +由插件绕过。 + +## 1. 三类工作 + +插件所做的每一件事属于且只属于下表的一类。 + +| 类 | 定义 | 发生在 | 机制 | +|---|---|---|---| +| 配置 | 决定构建的形状:编译哪些源、用哪些选项、链接什么、运行时在哪里找库 | 构建程序运行时,即规划期 | 指令(第 2 节) | +| 施工 | 产生构建读取的文件:生成源码、编译设备代码、安装依赖前缀、打包 | 构建期 | `mcpp::action`(第 3 节) | +| 校验 | 判断环境或产物是否满足要求,不产生构建读取的文件 | 构建期 | `role = "check"` 的 action | + +- **R1.1** 施工**必须**声明为 action,**禁止**在构建程序运行期间进行。构建程序可以运行工具 + 来**查询**配置所需的答案(版本、选项、路径),**禁止**借此产生构建读取的文件。(作者义务; + action 自 2026.8.5.1 **已实现**) +- **R1.2** 校验**必须**是 `check` action。构建程序发现环境不完整(缺少一个 SDK 模块、缺少 + 一个库)时,**必须**用 `mcpp::warning` 报告,并输出它能确定的全部配置;**禁止**因环境不完整 + 以非零状态退出。编译或链接会在缺失处失败,位于该警告之后。(作者义务) +- **R1.3** 配置**必须**只取决于构建程序声明的输入:包的清单、声明的载荷,以及 + `rerun_if_changed`、`rerun_if_changed_glob`、`rerun_if_env_changed` 列出的文件、glob 与 + 环境变量。配置**禁止**依赖施工的结果,例如枚举一个安装前缀中由 action 产生的文件:规划 + (`mcpp emit build-database`)与第一次构建都发生在施工之前,依赖施工结果的配置在第一次与 + 第二次构建之间不同。(作者义务;构建程序的重新运行依据落在一个 `prepare` action 声明的目录内 + 时,引擎给出一条警告并点名两者,**已实现**,mcpp#702) +- **R1.4** 构建程序在规划与构建中**必须**行为相同。引擎不提供「正在规划」的信号,因为规划 + 所描述的计划与 `mcpp build --configure-only` 计算的计划相同(SPEC-005 R1.2)。(**已实现**) + +## 2. 配置:指令 + +- **R2.1** 下表中的配置**必须**用对应的指令表达;**禁止**用 `link_flag`、`cxxflag` 拼写表中 + 已有指令所表达的内容(例如以 `-Wl,-rpath,` 代替运行时搜索目录,以 `-I` 代替头文件目录)。 + 指令由引擎按平台渲染、去重,并进入缓存与构建数据库。(作者义务) + + | 配置 | C++ 接口 | 线格式 | 自 | + |---|---|---|---| + | 头文件目录 | `include_dir`、`include_dir_after` | `include-dir`、`include-dir-after` | 协议 1 | + | 编译选项 | `cxxflag`、`cflag` | `cxxflag`、`cflag` | 协议 1 | + | 宏 | `define` | `cfg` | 协议 1 | + | 链接库与库目录 | `link_lib`、`link_search` | `link-lib`、`link-search` | 协议 1 | + | 其他链接选项(含库的完整路径) | `link_flag` | `link-flag` | 协议 8 | + | 运行时搜索目录 | `runtime_search_dir` | `runtime-search-dir` | 协议 12(mcpp#702) | + | 放到程序旁的文件 | `deploy` | `deploy` | 协议 11 | + | 生成的源 | `generated`、`source` 与 `role = "source"` 的 action | `generated`、`source`、`action` | 协议 1 | + | 重新运行构建程序的依据 | `rerun_if_changed`、`rerun_if_changed_glob`、`rerun_if_env_changed` | 同名 | 协议 1、2 | + | 报告与探测 | `warning`;`fact`、`floor` | 同名 | 协议 5、7 | + +- **R2.2** 相对路径按声明它的包的根目录解析。插件**禁止**把宿主系统目录(`/usr/include`、 + `/usr/lib`、`/lib`、`C:\Windows\System32` 及同类)声明为头文件、链接或运行时搜索目录;SDK + 与工具的路径取自声明的载荷(`mcpp::xpkg_dir`)或依赖边(`mcpp::dep_dir`、`mcpp::dep_bin`)。 + (作者义务;对图目标的链接,引擎检查 `-L`,mcpp#696 **已实现**) +- **R2.3** 插件**必须**使用现行字段的指令,**禁止**依赖只为兼容而保留的字段 + (`[runtime] library_dirs`,docs/04 §2.11)。(作者义务) +- **R2.4** 链接选项的一个元素按 SPEC-004 §8 读成词,每个词原样到达链接器。插件**禁止**依赖 + 引擎内部的转义拼写(例如 `'$$ORIGIN'`)。(**已实现**,mcpp#703;此前 `ldflags` 与 + `link_flag` 中的 `$` 会被宿主 shell 展开) + +## 3. 施工:action + +- **R3.1** action 的命令是 argv,**禁止**假定 shell。命令调用的工具**必须**列为输入;命令 + 自己发现的输入用 depfile 报告。(argv 与 depfile **已实现**;工具作为输入是作者义务) +- **R3.2** action **必须**在提交时命名它的输出文件,因为引擎在规划期确定源集合、指纹与模块 + 图。输出文件名在施工前无法得知的工作(安装一个前缀、解开一个 SDK)使用 `prepare` 角色。 + (**已实现**;`prepare` 随 mcpp#702) +- **R3.3** 角色: + + | `role` | 输出 | 顺序 | + |---|---|---| + | `source` | 可编译的输出加入声明包的编译集 | 声明包的每条编译边等待它 | + | `object` | 加入链接集 | 链接边消费它 | + | `artifact` | 新文件,输入是链接产物 | 链接之后 | + | `check` | 引擎写的 stamp | 与编译并行;`blocking = true` 时声明包的编译边等待它 | + | `prepare` | 引擎写的 stamp;命令填充它用 `output_dir` 声明的目录,构建按目录引用其内容 | 声明包的每条编译边与计划中的每条链接边(静态库归档除外)等待它 | + + `prepare`(**已实现**,mcpp#702)是施工,不是校验,任何只针对校验的策略都不作用于它;构建 + 输出以 `PREPARE` 标注它。它的产物由配置以目录为单位引用(`include_dir`、`link_search`、 + `runtime_search_dir`),或以 `link_flag` 中的完整路径引用;这些名字在配置时确定,内容在施工 + 时到达。一个 `prepare` action **必须**用 `output_dir` 声明它填充的目录(缺少时引擎拒绝该 + action);命令成功而该目录不存在,或除该 action 的 stamp 之外不含任何文件时,引擎不写 + stamp,并以指出该目录的消息使这条边失败。链接边等待所有 `prepare`,因为每个包的链接全局 + 指令并入整个计划共用的一份链接意图。 +- **R3.4** `check` 只用于校验,**禁止**用来表示施工。(作者义务) +- **R3.5** `check` 与 `prepare` 的命令成功后,引擎创建或更新它们的 stamp,使 stamp 新于该 + action 的每个输入;命令失败时不写 stamp。命令无需自己写 stamp。(创建自 2026.8.29.1 + **已实现**;更新 **已实现**,mcpp#702;此前一个已存在的 stamp 不被更新,输入改变一次后该 + action 在此后每次构建中都会重新运行) +- **R3.6** 角色**应当**以常量书写(`mcpp::roles::source`、`check`、`object`、`artifact`、 + `prepare`),使不认识该角色的旧引擎在编译构建程序时拒绝它;引擎拒绝未知的角色字符串,并列出 + 五个角色。(**已实现**,mcpp#702;此前引擎把未知的角色字符串当作 `source`) +- **R3.7** 构建期**不应**访问网络:下载属于安装期(载荷的安装、依赖的解析)。一个必须在构建期 + 下载的 action(例如由包管理器取得源码)**必须**在其说明中写明,并在离线构建中 + (`--offline` 或 `MCPP_OFFLINE=1`;前者在进程环境中设置后者,action 继承之)不访问网络: + 从缓存完成,或以指出缺失内容的消息失败。(作者义务;环境传递 **已实现**) + +## 4. 运行时:程序依赖的共享库的查找 + +- **R4.1** 一个目录中的共享库由施工产生(`prepare`)或文件名不定时,插件**必须**用 + `runtime_search_dir` 声明该目录。引擎把它用于 ELF 与 Mach-O 的运行路径(RUNPATH/rpath, + 从不作为 `-L`)、`mcpp run` 的加载路径、`mcpp pack` 的闭包搜索与运行时校验,并把依赖包的 + 声明传到消费方的可执行文件。(**已实现**,mcpp#702) +- **R4.2** 一个在配置时已知的文件需要位于程序旁的某个相对位置时(Qt 的平台插件、Vulkan 的 + ICD 清单),插件用 `deploy`。(**已实现**,协议 11) +- **R4.3** Windows 的可执行文件没有运行路径。`mcpp run` 通过 `PATH` 使用运行时搜索目录, + `mcpp pack` 把闭包需要的 DLL 放到程序旁(**已实现**)。链接之后,引擎把程序直接或间接导入的、 + 位于其运行时搜索目录中的非系统 DLL 放到程序旁,使从构建目录直接启动的程序同样能找到它们; + 闭包的求解与 `mcpp pack` 相同,DLL 在其目录中被替换后下一次构建再次放置。(**已实现**, + mcpp#702) +- **R4.4** 插件**禁止**在 `link_flag` 中写运行路径(`-Wl,-rpath,...`),**必须**使用 R4.1。 + (作者义务) + +## 5. 规划期的义务(`mcpp emit build-database`) + +- **R5.1** 规划运行构建程序,不运行任何 action(SPEC-005 R2.2)。插件遵守 R1.3 时,规划得出 + 的配置与构建相同。(**已实现**) +- **R5.2** 一个包的构建程序在规划中失败时,该包只按其清单描述,并得到一条错误诊断;成员的 + 其余部分照常描述。插件遵守 R1.2 时,环境不完整不会使构建程序失败。(**已实现**,mcpp#702) +- **R5.3** 插件所需的宿主工具在规划中构建失败时,规划继续,构建程序收到该工具将被发布的路径, + 并产生一条警告。插件**应当**在 action 中运行宿主工具,而不是在构建程序中运行,使规划不依赖 + 工具能否构建。(引擎部分 **已实现**,mcpp#702;「应当」为作者义务) + +## 6. 环境与载荷 + +- **R6.1** 插件驱动的工具与 SDK **必须**声明为载荷(`[xlings]` 或 `[feature-xlings]`),需要 + 时以目标轴选择器门控,并在构建程序中用 `mcpp::xpkg_dir` 取得路径。声明**必须**放在查询发生 + 的包上:`xpkg_dir` 为正在构建的包回答;`host-module` 的声明对编入它的每个构建程序可见 + (docs/31)。(**已实现**) +- **R6.2** 插件**禁止**探测宿主路径来寻找工具或 SDK;未声明的依赖不可复现。(作者义务) + +## 7. 版本与兼容 + +- **R7.1** 使用协议 N 的指令或角色的插件,**必须**在其文档中写明第一个支持协议 N 的 mcpp + 版本。索引测量该插件时,CI 所用的 mcpp 版本移到该版本;索引的 `min_mcpp` 不因此改变。旧引擎 + 编译该构建程序时因缺少函数或常量而失败,并指出其名称。(编译期失败 **已实现**) +- **R7.2** 插件**禁止**依赖引擎内部的拼写与未写入文档的行为(R2.3、R2.4)。(作者义务) + +## 8. 判据 + +- **R8.1** 插件**必须**在它声明支持的每个平台上有一个在该平台运行的判据。`# requires: gcc` + 只在 Linux 成立,不能作为 Windows 或 macOS 的判据。(作者义务) +- **R8.2** 每个判据**必须**在它所验证的改动之前失败。(作者义务) +- **R8.3** 插件**应当**有一个规划判据:它的一个消费方工程,在没有安装其载荷的机器上运行 + `mcpp emit build-database`,得到一份文档,其中该插件只贡献警告,或只使声明它的包缺少 + 构建程序的指令(R5.2),而不是整次失败。(作者义务) + +## 9. 变更记录 + +| 版本 | 日期 | 变更 | +|---|---|---| +| 0.1 | 2026-09-26 | 首版草案(mcpp#699、#701、#702、#703)。 | +| 0.2 | 2026-09-26 | 随 mcpp 2026.9.26.2 落地:R1.3 的警告、R2.1 的 `runtime_search_dir`、R2.4、R3.3 的 `prepare`(目录须含文件;链接边等待所有 `prepare`)、R3.5、R3.6、R4.1、R4.3、R5.2、R5.3 标为已实现。 | diff --git a/docs/specs/manifest-semantics.md b/docs/specs/manifest-semantics.md index 79f409fa0..ae6d04e2a 100644 --- a/docs/specs/manifest-semantics.md +++ b/docs/specs/manifest-semantics.md @@ -5,8 +5,8 @@ | **规范编号** | SPEC-004 | | **标题** | `mcpp.toml` 的平面划分、条件化形状、解析轴与命名规约 | | **状态** | **草案(Draft)** | -| **版本** | 1.6 | -| **最后修改** | 2026-09-25 | +| **版本** | 1.7 | +| **最后修改** | 2026-09-26 | | **最低实现版本** | 条件化形状:mcpp **2026.8.29.1**(`[target..build-dependencies]` 起齐备);目标轴:mcpp **2026.9.6.4** | | **作者/维护** | mcpp-community | | **相关设计文档** | `.agents/docs/2026-09-07-mcpp-toml-unified-semantics-design.md`
`.agents/docs/2026-06-04-manifest-schema-ownership.md`
`.agents/docs/2026-09-03-xlings-workspace-as-the-one-table.md`
`.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md` | @@ -353,14 +353,18 @@ feature-deps feature-xlings ← 限定词是门 客户端上都能构建;两次发布的归档逐字节相同;缺少 `version` 的兄弟 `path` 边被拒绝且报错给出 应写的一行;无需修改的包的归档与此前逐字节相同 (`tests/e2e/772_a_published_member_is_self_contained.sh`)。 +15. §8 对链接 flag 的判据:`[build] ldflags` 与构建程序的 `mcpp::link_flag` 中写出的 + `-Wl,-rpath,$ORIGIN/../lib` 原样到达程序的运行路径,不出现 `/../lib`;依赖传播的同一 + 元素同样原样到达;含空格的 `link_search` 目录是一个参数 + (`tests/e2e/795_a_link_flag_reaches_the_linker_as_written.sh`)。 -## 8. 编译 flag 列表的元素 +## 8. flag 列表的元素 -`cflags`、`cxxflags` 与 `asmflags` 的一个元素是一段文本,代表零个或多个词;编译器收到的 -参数就是这些词,按列表顺序排列。本节在元素的文本上陈述(TOML 或 Lua 先去掉自己的转义)。 -该读法对这三个键的每一个出现位置相同:`[build]`、`[targets.]`、`flags` 的 glob 条目、 -feature、`[profile.]`、`[target..build]`、xpkg 描述符,以及构建程序的 -`mcpp:cflag=` 与 `mcpp:cxxflag=` 指令。 +`cflags`、`cxxflags`、`asmflags` 与 `ldflags` 的一个元素是一段文本,代表零个或多个词; +编译器或链接器收到的参数就是这些词,按列表顺序排列。本节在元素的文本上陈述(TOML 或 Lua +先去掉自己的转义)。该读法对这四个键的每一个出现位置相同:`[build]`、`[targets.]`、 +`flags` 的 glob 条目、feature、`[profile.]`、`[target..build]`、xpkg 描述符, +以及构建程序的 `mcpp:cflag=`、`mcpp:cxxflag=` 与 `mcpp:link-flag=` 指令。 1. 未加引号的空格与制表符分隔词,连续的分隔符等同于一个。 2. `'` 开启一段单引号区域,区域内的字符按字面取到下一个 `'` 为止。 @@ -382,14 +386,20 @@ feature、`[profile.]`、`[target..build]`、xpkg 描述符,以及 `[build]`、各个命中的 `[target..build]`。实现**必须**满足:同一宏名的后一个 条目在原位替换前一个;条目 `!NAME` 移除宏名 `NAME`;一个包的每个编译单元对每个宏名 至多收到一个 `-D` 词;`defines` 条目取代同一个包的 `cflags`、`cxxflags` 中读作单个词 -且宏名相同的 `-D` 词。实现向这三个列表插入一个词 `w` 时,**必须**使用一个按上述规则读回恰为 `w` 的拼写。 +且宏名相同的 `-D` 词。实现向这四个列表插入一个词 `w` 时,**必须**使用一个按上述规则读回恰为 `w` 的拼写; +构建程序的 `mcpp:link-lib=`、`mcpp:link-search=` 与 `mcpp:link-script=` 指令由名字或路径构成的 +值属于这种插入。 -实现**必须**把每个词原样交给编译器,与宿主的命令行读取规则(POSIX `sh`、MSVCRT)无关; -`compile_commands.json` 与构建数据库(SPEC-005 R3.7)列出的参数**必须**是这些词。 +实现**必须**把每个词原样交给编译器或链接器,与宿主的命令行读取规则(POSIX `sh`、MSVCRT) +无关;`compile_commands.json` 与构建数据库(SPEC-005 R3.7)列出的参数**必须**是这些词。 +依赖的 `ldflags` 传播给消费者时按词传播,包内相对的搜索路径按词解析为绝对路径。 +规则 7 对链接 flag 的一个推论:`$ORIGIN` 等加载器记号原样到达链接器。为 shell 或 ninja +手工转义的写法(`\$ORIGIN`、`'$$ORIGIN'`)按上述规则读取,不再是转义。 -`ldflags`、`dialect_cxxflags` 与 `std-module-flags` 不在本节范围内。 +`dialect_cxxflags` 与 `std-module-flags` 不在本节范围内。 -**状态:已实现(mcpp 2026.9.17.1;`defines` 的集合语义 mcpp 2026.9.25.1)。** +**状态:已实现(mcpp 2026.9.17.1;`defines` 的集合语义 mcpp 2026.9.25.1;`ldflags` 与链接指令 +mcpp 2026.9.26.2,#703)。** ## 9. 工作空间继承与构建需求的作用域 @@ -425,3 +435,4 @@ feature、`[profile.]`、`[target..build]`、xpkg 描述符,以及 | 1.4 | 2026-09-15 | 库目标的默认链接形态 `linkage`(mcpp 2026.9.15.2):§3.1.1 补默认值的语义、优先顺序与拒绝条件;§7 补第 9 条判据。 | | 1.5 | 2026-09-17 | 编译 flag 列表元素的读法(mcpp 2026.9.17.1,#655):新增 §8 与 §7 第 10 条判据。 | | 1.6 | 2026-09-25 | 工作空间继承与构建需求的作用域(mcpp 2026.9.25.1,#690):§8 补 `defines` 的集合语义;新增 §9 与 §7 第 11 至 14 条判据。 | +| 1.7 | 2026-09-26 | §8 的读法扩展到 `ldflags` 与构建程序的链接指令(mcpp 2026.9.26.2,#703):`$ORIGIN` 原样到达链接器;§7 补第 15 条判据。 | diff --git a/docs/zh/01-getting-started.md b/docs/zh/01-getting-started.md index d081df364..45ee84cae 100644 --- a/docs/zh/01-getting-started.md +++ b/docs/zh/01-getting-started.md @@ -147,6 +147,24 @@ toolchain、lock/resolution 元数据以及构建目录元数据都可能被更 可信的 workspace 中运行。进程退出码与生成的 `compile_commands.json` 是 稳定的集成契约；标准输出仍是面向人的文本。 +工程根目录下的 `compile_commands.json` 是当前配置自己那份数据库的副本， +后者写在 `target///compile_commands.json`——这个目录 +由 toolchain、target、profile 与 feature 共同命名，也正是编译器实际运行 +所在的目录。每条在这个配置下做规划的命令(`build`、`test`、`run`、 +`--configure-only`)都把新鲜的计划合并进这一份文件，因此 `mcpp test` 加入 +的条目在之后一次普通 `mcpp build` 后仍然留在里面。根目录的这份副本从不 +合并：它整体替换为配置数据库的内容，两者已经一致时保持不变；因此切换 +`--toolchain` 或 `--profile` 会整体切换根文件，切回去也会恢复那个配置的 +条目，包括它的测试单元。根目录若是一个符号链接，写入会跟随到链接指向的 +位置。被替换掉的根文件如果持有 mcpp 未写入的条目，会输出一条警告说明 +条目数；之后该文件只持有本工程自己的配置。删除根文件不是重置它的办法： +下一次构建(即使没有任何东西需要重新编译)也会把它从配置数据库中恢复。 + +工程只要 `import std`，`compile_commands.json`与 `emit build-database` +打印的 S1 文档中都会各有一条编译器标准库单元的记录(见下文)。因为 BMI +只能被写出它的那个编译器读取，列出这个单元让工具链与 mcpp 不同的读取方 +可以自己编译一份 `std`，而不是在版本不匹配时直接失败。 + 不允许写入工程目录的编辑器，在标准输出上取得同一份计划 *(mcpp 2026.9.15.1+)*： diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 73057d18e..e45e29733 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -424,10 +424,11 @@ bmi_schedule = "auto" # Module-edge scheduling: auto (= off) | on | #### 编译 flag 的写法 *(mcpp 2026.9.17.1+)* `cflags`、`cxxflags` 或 `asmflags` 里的一个元素代表一个或多个编译器 -参数（「词」）。这套语法在每个宿主上都相同，无论写在哪张表里： +参数（「词」），`ldflags` 里的一个元素代表一个或多个链接器参数 +（mcpp 2026.9.26.2+）。这套语法在每个宿主上都相同，无论写在哪张表里： `[build]`、`[targets.]`、`flags` glob 条目、feature、 `[target..build]` 小节、xpkg 描述符，以及构建程序的 -`mcpp:cflag=` / `mcpp:cxxflag=` 指令。 +`mcpp:cflag=` / `mcpp:cxxflag=` / `mcpp:link-flag=` 指令。 | 写法 | 编译器收到的词 | |---|---| @@ -439,6 +440,7 @@ bmi_schedule = "auto" # Module-edge scheduling: auto (= off) | on | | `"-I/opt/my\\ dir/include"` | `-I/opt/my dir/include` | | `"-IC:\\sdk\\include"` | `-IC:\sdk\include` | | `"-DNAME=a$b"` | `-DNAME=a$b` | +| `"-Wl,-rpath,$ORIGIN/../lib"`（在 `ldflags` 中） | `-Wl,-rpath,$ORIGIN/../lib` | 以下规则施加于元素的文本上（TOML 或 Lua 已经去掉了它自己的转义之后）： @@ -453,8 +455,14 @@ bmi_schedule = "auto" # Module-edge scheduling: auto (= off) | on | 与以往每一个版本相同。 一条 `defines` 条目是一个值，不受这套语法解析：`defines = -["NAME=\"text\""]` 传出单独一个词 `-DNAME="text"`。`ldflags`、 -`dialect_cxxflags` 与 `std-module-flags` 不受本节约束。 +["NAME=\"text\""]` 传出单独一个词 `-DNAME="text"`。`dialect_cxxflags` 与 +`std-module-flags` 不受本节约束。 + +`ldflags` 自 mcpp 2026.9.26.2 起遵循这套语法（#703）。此前链接 flag 的元素只为 +ninja 转义、没有为 shell 加引号，所以在 Linux 与 macOS 上 `$ORIGIN` 以 `/../lib` +进入程序的运行路径。带包内相对路径的 `-L` 或 `-Wl,-rpath,` 词相对包根解析，依赖的 +`ldflags` 按词传给消费者。为 ninja 或 shell 手工转义的元素（`\$ORIGIN`、 +`'$$ORIGIN'`）现在按写法读取；首次 plan 会在 `build/flag-words` 下点名这样的元素。 `compile_commands.json` 与 `mcpp emit build-database` 在 `arguments` 里列出同样的词，可以不经 shell 直接执行。 @@ -1381,7 +1389,7 @@ Link intent 把各个发现阶段分开处理： |---|---|---|---| | `link_library_dirs` | `-L` | `-L` | `-L` 或 `/LIBPATH:` | | `transitive_needed_dirs` | `-Wl,-rpath-link` | 无旗标 | 无旗标 | -| `runtime_search_dirs` | 仅 RUNPATH/rpath，从不是 `-L` | 仅 rpath | 无旗标 | +| `runtime_search_dirs` | 仅 RUNPATH/rpath，从不是 `-L` | 仅 rpath | 无旗标；链接之后，程序从这些目录导入的 DLL 被放到程序旁 *(2026.9.26.2+)* | | `frameworks` | 无旗标 | `-framework` | 无旗标 | | `deploy_files` | 拷贝边 | 拷贝边 | 拷贝到输出旁边；从不是链接器旗标 | | `deploy` *（2026.9.12.2+）* | 拷贝边，进 `bin//` | 拷贝边，进 `bin//` | 拷贝边，进 `bin//`；从不是链接器旗标 | @@ -1405,6 +1413,22 @@ Link intent 把各个发现阶段分开处理： 映射到必需的运行期 soname 要求，`capabilities` 映射到必需的运行期 能力要求。这些遗留字段都不创建提供者。 +`runtime_search_dirs` 有一个构建程序形态，用于一个只有 build.mcpp 才能定位的 +目录（一个 vcpkg 前缀的 `bin/`、一个 Qt SDK 的 `bin/`、一个 `prepare` action +填充的目录）：`mcpp::runtime_search_dir(dir)`（protocol 12； +[30 —— 构建程序](30-build-mcpp.md)），直接并入的就是这一个字段——不是上面 +那个不再获得新指令的遗留字段 `library_dirs`。这个目录在该程序运行时不必 +存在；一个 `prepare` action 可能在构建期之后才把它填充起来。 + +PE 映像没有运行路径，所以在 Windows 上运行时搜索目录服务于 `mcpp run`（把它放进 +`PATH`）与 `mcpp pack`（暂存闭包）。自 2026.9.26.2 起，计划中带运行时搜索目录的 +PE 程序在链接之后多一条边 `mcpp place-dlls`：它按 `mcpp pack` 的方式读取程序的 +导入闭包，把程序直接或经另一个 DLL 间接导入、且在这些目录之一中解析到的每个 DLL +放到程序旁。系统 DLL 与 API set 从不复制，副本只在字节不同时写入，某个 DLL 在 +其目录中被替换后，下一次构建会再次放置它。于是从构建目录直接启动的程序能找到 +它们，与 vcpkg 的 applocal 步骤或 CMake 的 `$` 之后相同。两个 +目录提供同一个名字时，放置按搜索顺序的第一个，并以一条说明指出两者。 + `target///resolution.json` schema 2 存储 RuntimeBinding、 规范化后的要求/提供者/产物、LinkIntent、平台发现机制与链接后判定。 `mcpp why runtime` 是对最新存储文件的一个纯粹解读器：它既不会重新 diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index 16fab4d63..08bf2a872 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -43,6 +43,39 @@ int main() { mcpp build # compiles + runs build.mcpp, then builds the project ``` +## 三类工作 + +一个 `build.mcpp`，或者它导入的规则包，所做的每一件事都且只属于下面三类之一 +（SPEC-007 第 1 节，`docs/specs/build-plugins.md`），每一类都只有一种机制： + +| 类 | 决定的内容 | 发生在 | 机制 | +|---|---|---|---| +| 配置 | 构建的形状：编译哪些源、用哪些选项、链接什么、运行时在哪里找库 | `build.mcpp` 运行期间——**规划** | 一条指令（本章下面的表） | +| 施工 | 构建要读取的文件：生成源码、编译设备代码、安装一个依赖前缀、打包 | **构建**期 | `mcpp::action` | +| 校验 | 环境或产物是否满足某个要求，不产生构建要读取的文件 | **构建**期 | `role = check` 的一个 action | + +**配置只取决于声明过的输入。** 一个 `build.mcpp` 读它自己包的清单、它声明过的 +载荷（`xpkg_dir`），以及它用 `rerun_if_changed`/`rerun_if_changed_glob`/ +`rerun_if_env_changed` 点过名的东西——绝不读一个 action 产生的文件，因为规划 +（`mcpp emit build-database`）与第一次构建都发生在任何 action 运行**之前**， +读了施工结果的程序会在第一次构建与第二次构建给出不同的配置。这一点在下文 +[`prepare` role](#施工时文件名未知的构造prepare-roleprotocol-12) 那节会得到 +一条警告。 + +**施工绝不在 `build.mcpp` 运行期间完成。** 直接在 `main()` 里写文件，是每次 +规划都跑一遍、全量、串行的，失败还只报「build.mcpp exited 1」，离真正的原因 +隔着三条边。**声明**它（见下文 `mcpp::action`），它就成为一条边：增量、并行、 +可归因。 + +**一个不完整的环境是警告，绝不是配置失败。** 一个找不到某个 SDK 模块或某个库 +的程序，用 `mcpp::warning`（见下）说出来，并把它还能配出来的部分照常发出； +编译或链接会在真正缺东西的地方失败，如果非失败不可的话。在这里以非零状态 +退出，会让不需要那个缺失物的构建也失败。 + +**规划与构建下行为相同。** `mcpp emit build-database` 算出的计划，与 +`mcpp build --configure-only` 算出的是同一个，所以一个遵守上面这条规则的 +构建程序不需要任何信号告诉它这次是被谁问的。 + ## 指令 把这些打印到 stdout（每行一条）。任何不以 `mcpp:` 开头的行都会被忽略，因此可以 @@ -64,6 +97,7 @@ mcpp build # compiles + runs build.mcpp, then builds the project | `mcpp:windows-subsystem=:` *(2026.9.12.2+)* | 设置**本包**可执行目标 `` 的 PE 子系统（`console` 或 `windows`），与 `[targets.] windows_subsystem`（docs/04）是同一字段。只到达该目标的链接，不到达其他目标或消费者，在非 PE 目标上不产生任何标志。本包未以 `kind = "bin"` 声明该目标、取值不在集合内、取值与 mcpp.toml 的声明矛盾，这三种情形都在应用任何指令之前被拒绝 | | `mcpp:windows-entry=:` *(2026.9.12.2+)* | 设置可执行目标 `` 的入口函数（`main`、`wmain`、`WinMain` 或 `wWinMain`），与 `windows_entry` 是同一字段；作用域与拒绝条件同 `windows-subsystem` | | `mcpp:deploy=\t` *(2026.9.12.3+,protocol 11)* | 把本程序生成或选中的一个文件放到产物旁边的 ``（相对可执行文件所在目录）——`[runtime] deploy`（docs/04 §2.11）的构建程序形态。`` 可以是绝对路径（某个 action 自己声明的输出），也可以按包根解析；用 TAB 分隔，因为一个绝对的 Windows `` 本身含冒号。**到达消费者**，并入同一个被 `link-lib`/`link-search`/`link-flag` 喂入的 `LinkIntent`——见下 | +| `mcpp:runtime-search-dir=` *(protocol 12)* | 把 `` 加入启动期搜索路径——`runtime_search_dirs`（docs/04 §2.11）的构建程序形态。相对路径按包根解析；该目录在此程序运行时不必存在(一个 `prepare` action 可能之后才把它填充起来)。**到达消费者**，直接并入 `LinkIntent::runtimeSearchDirs`：在 ELF 与 Mach-O 上是 RUNPATH/rpath，绝不是 `-L`，并进入 `mcpp pack` 的闭包搜索——见下 | | `mcpp:link-script=` *(2026.8.19+)* | 用这个**链接脚本**链接（`-T`；相对路径按包根解析，发出的是绝对路径，因为链接是在构建目录里跑的）。与 `include-dir` 不同，它**到达消费者** —— 板子的内存布局恰恰是消费者写不出来的那一项 | | `mcpp:warning=` *(2026.8.21.2+)* | 对用户说一句话并**继续**。唯一一条不改变编译行、链接行与源码集的指令。它**穿过构建缓存** —— 见下 | | `mcpp:fact==` *(2026.9.5.2+)* | 陈述程序**测得的机器事实**(`cuda.driver=12.4`)。在编译任何东西之前与 floor 比较；见下 | @@ -123,6 +157,7 @@ int main() { | `mcpp::link_flag(s)` *(2026.9.6.5+)* | `mcpp:link-flag=` | | `mcpp::windows_subsystem(target, value)` / `mcpp::windows_entry(target, value)` *(2026.9.12.2+)* | `mcpp:windows-subsystem=` / `mcpp:windows-entry=` | | `mcpp::deploy(from, to)` *(2026.9.12.3+,protocol 11)* | `mcpp:deploy=\t` —— 见下 | +| `mcpp::runtime_search_dir(dir)` *(protocol 12)* | `mcpp:runtime-search-dir=` —— 见下 | | `mcpp::link_script(p)` *(2026.8.19+)* | `mcpp:link-script=` | | `mcpp::runner(tok)` *(2026.8.19.2+)* | `mcpp:runner=` —— 见下 | | `mcpp::xpkg_dir(ns, name)` / `mcpp::xpkg_dir(name)` *(2026.8.19+)* | `[xlings.workspace]` 里声明的包的载荷目录 —— 本 manifest 声明的，或编进本构建程序的某个依赖声明的（2026.9.6.6+）；没声明或没安装时返回 `""`（见下） | @@ -366,6 +401,18 @@ mcpp 会**为构建机器**构建那个 `kind = "bin"` target（即使在 `--tar 完整契约（含 `[tools.overrides]` 与 `reexport = true` —— 库据此把整条工具链交给 调用方，因此只需写**一条**依赖而不是四条）见本章*依赖产出的 host 工具*。 +**在 `mcpp emit build-database` 下，一个构建失败的 host 工具是一条警告，不是 +丢掉的计划。** 规划对每个成员继续进行，`dep_bin` 答的是那个工具**将会**被发布 +到的路径——与一次成功构建会占据的同一个路径，在构建真正跑之前就已经定死。 +仅仅**指名**这个工具，配置结果与成功构建之后完全一样。而**调用**它——在配置期 +就把它跑起来，上一段的写法止步的地方——会失败，因为那个路径什么都不指向； +这次失败只算在这个包自己的指令头上（它构建程序那部分的描述），绝不会连累 +整条命令。这个工具在一次真正的 `mcpp build` 里仍然会被构建：程序在那里可能 +会运行它，它的 blocking check 也仍然会跑，因为把一个未经校验的工具放进 +store、用的还是校验通过才用的那把键，比不构建它更糟。优先从一个 `action` 的 +命令里（施工）运行 host 工具，而不是在 `main()` 里（配置），这样这个工具当下 +能不能构建，就不会决定规划是否还能看见这个包。 + ### 用通配符声明输入：`rerun_if_changed_glob`（2026.8.6.2+） 重跑键由**声明过的**输入构成。声明具体文件是可行的；而 glob 一个目录不可行 —— @@ -411,7 +458,7 @@ int main() { const std::string out = std::string(mcpp::out_dir()) + "/foo.pb.cc"; mcpp::action a; a.id = "protoc:foo"; - a.role = "source"; // "source" | "check" | "object" | "artifact" + a.role = mcpp::roles::source; // source | check | object | artifact | prepare a.arg(mcpp::dep_bin("protobuf", "protoc")) .arg("--cpp_out=...").arg("proto/foo.proto") .input("proto/foo.proto") @@ -420,7 +467,7 @@ int main() { } ``` -四种 role，一个原语 —— `role` 只决定这条边的输出接到哪： +五种 role，一个原语 —— `role` 只决定这条边的输出接到哪： | `role` | 输出 | 顺序 | 典型 | |---|---|---|---| @@ -428,12 +475,20 @@ int main() { | `check` | 一个 stamp 文件，由 mcpp 写入 | 与编译并行；`blocking = true` 让该包的编译边等它 | clang-tidy、格式/ABI 检查 | | `object` | 进**链接**集 | 链接边消费它们 | 资源编译器、`objcopy` 嵌 blob、生成的 `.def`、预编译 `.o` | | `artifact` | 一个新文件 | 它的**输入**是链接产物，所以在链接之后跑 | 签名、打包、size budget | +| `prepare` *(protocol 12)* | 一个由 mcpp 写入的 stamp，加上一个声明的目录(`output_dir`)，由命令去填充 | **该包每一条编译边、以及整个构建计划里每一条链接边都等它** —— 见下 | 把一份 vcpkg manifest 或一个 CMake 子工程安装进某个前缀、解开一个 SDK | `artifact` 也是唯一允许写 `${mcpp.stage_dir}` 的 role —— 见下文 -[产出可分发物](#产出可分发物pack_format-与-stage_dir20269111)。另外三个跑在链接之前 -或与链接并行，没有任何已暂存的东西可读，所以 mcpp 会拒绝这个占位符，而不是把它展开成 +[产出可分发物](#产出可分发物pack_format-与-stage_dir20269111)。其余几个 role 跑在链接 +之前或与链接并行，没有任何已暂存的东西可读，所以 mcpp 会拒绝这个占位符，而不是把它展开成 一个恰好存在的路径。 +**用 `mcpp::roles` 常量拼写 role**(protocol 12)，而不是裸字符串。一个写下 +`a.role = mcpp::roles::prepare;` 的 build.mcpp 在旧引擎上会直接**编译失败**，并指出 +缺失的那个常量——与 `mcpp::deploy`、`mcpp::runtime_search_dir` 给出的是同一种编译期 +信号。自 protocol 12 起，一个哪个都不是的 role 字符串会被引擎拒绝并列出这五个；在此之前， +一个未识别的字符串会被静默读成 `source`，这就在无声之间改变了一个 action 的含义。 +手写的冻结表面(一个手写 `printf("mcpp:action=...")` 的程序)仍然可以直接写字符串。 + 全程不涉及任何 phase 机制。`object` 与 `artifact` 由 ninja 自己的文件依赖定序 —— 这也是为什么 `artifact` 不会像朴素的「post 构建钩子」那样把自己重复施加一遍。 `source` 与 blocking 的 `check` 则由一条 order-only 边定序：从声明它的那个包的 @@ -464,9 +519,16 @@ mcpp 为那条边写出 `depfile =` 与 `deps = gcc`，ninja 读取该文件并 > 承诺了该输出的边会永远是脏的。 **check 的命令不必自己写 stamp**(mcpp 2026.8.29.1+)。判定是退出码，stamp 是**构建图** -需要的记账；命令成功时由 mcpp 创建它。在此之前每个 check 都需要一个包装脚本去 touch -那个文件 —— 而 command 是 argv、不假设有 shell，所以那个包装器**根本没法可移植地写出来**。 -已经自己写 stamp 的命令不受影响：已存在的文件不会被动。 +需要的记账。在此之前每个 check 都需要一个包装脚本去 touch 那个文件 —— 而 command 是 +argv、不假设有 shell，所以那个包装器**根本没法可移植地写出来**。 + +**一个 `check` 或 `prepare` 命令成功之后，每个声明的 stamp 都比每个输入新，无论命令自己 +有没有写它**(protocol 12)。mcpp 会创建每一个缺失的 stamp，并把每一个**已存在**的 stamp +的修改时间设为当前时刻——无条件地，不只是在某个输入变化之后——这样一个什么都不写的 +check(clang-tidy、经由某个规则包运行的一次安装)就不会仅仅因为自己的 stamp 比它已经 +反应过的某个输入更旧而重新运行。自己写 stamp 的命令不受影响：这次运行中命令自己创建或 +改写过的文件本来就和命令留下的一样新，再 touch 一遍不改变任何构建会读到的东西——一个 +stamp 不会喂给任何编译或链接边，只喂给下文说的那条 order-only 边。命令失败时什么都不写。 > stamp 缺失**不会**让构建失败。ninja 只是留着那个声明的输出不存在，并在之后**每次构建 > 都重跑**那条边 —— 看起来像一个通过了的检查，实际上它从未被满足。 @@ -559,6 +621,100 @@ int main() { | `.apk` | `assets/myapp.resources/` | | web | 静态目录里同一个相对路径，与 `.js` 放在一起。想把文件放进 `.data` 预加载的项目改用链接标志 `--preload-file @/`，那是一条普通的链接标志 | +### 一个启动期搜索目录：`runtime_search_dir`（protocol 12） + +`runtime_search_dirs`（docs/04 §2.11）声明产物运行时要搜索的一个目录。 +它是一个固定的 TOML 数组，因此点不了一个只有 build.mcpp 才能发现的目录—— +一个 vcpkg 前缀的 `bin/`(Windows)或 `lib/`(其它平台)、一个 Qt SDK 的 +`bin/`，或一个由 `prepare` action(见下)填充的目录。`mcpp::runtime_search_dir` +就是同一次声明，从构建程序里发出： + +```cpp +import mcpp; +#include + +int main() { + const std::string qtPrefix = locate_qt_prefix(); // 本包自己怎么找到它 + // 导入库在 lib/ 下；mcpp run 与 mcpp pack 在启动时要找的那些 DLL + // 在 Windows 上是在 bin/ 下。 + mcpp::link_search((qtPrefix + "/lib").c_str()); + mcpp::runtime_search_dir((qtPrefix + "/bin").c_str()); +} +``` + +- **直接并入 `LinkIntent::runtimeSearchDirs`**，与 `runtime_search_dirs` 从 + TOML 填的是同一个字段——不是那个不再获得新指令的遗留字段 + `[runtime] library_dirs`（docs/04 §2.11）。一个由指令声明的目录，到达该字段 + 的每一个读者，与写在 `mcpp.toml` 里的完全一样：`mcpp run` 的加载器路径、 + `mcpp pack` 的闭包搜索，以及 ELF 与 Mach-O 上的 RUNPATH/rpath（绝不是 + `-L`——一个启动期搜索目录不是一个链接库搜索路径）。 +- **`dir` 可以是绝对路径或按包根解析的相对路径。** 相对路径按包根解析， + 与其它每一个 `AbsPath` 指令（`include-dir`、`deploy` 的 `from`）一样。 +- **该目录在此程序运行时不必存在。** 一个 `prepare` action 可能在构建期之后 + 才把它填充起来——见下。 +- **到达消费者。** 一个依赖声明的目录，经由清单键所走的同一个合并 + （`resolve_runtime_contract`，`plan.cppm`），并入它的消费者的可执行文件。 +- **在缓存命中时被重放。** `runtime-search-dir` 指令与 `deploy`、`warning` + 一样进入构建缓存；一次缓存命中的重跑会像真正跑过一样把它恢复回来。 + +### 施工时文件名未知的构造：`prepare` role（protocol 12） + +`source`、`check`、`object`、`artifact` 都在提交时就命名了自己的输出， +因为 mcpp 在 prepare 阶段就定死了源集合、指纹与模块图。把一份 vcpkg +manifest 或一个 CMake 子工程安装进某个前缀、或者解开一个 SDK，都不是这个 +形状：施工写出的文件在命令运行之前是不知道的——往往连名字都不知道，只知道 +**目录**。`prepare` 就是为此而设的 role： + +```cpp +import mcpp; +#include + +int main() { + const std::string prefix = std::string(mcpp::out_dir()) + "/vcpkg-install"; + mcpp::action a; + a.id = "vcpkg:install"; + a.role = mcpp::roles::prepare; + a.arg("vcpkg").arg("install").arg("--x-install-root").arg(prefix.c_str()) + .input("vcpkg.json") + .output((prefix + ".stamp").c_str()) + .output_dir(prefix.c_str()) + .submit(); + + mcpp::link_search((prefix + "/lib").c_str()); + mcpp::runtime_search_dir((prefix + "/lib").c_str()); +} +``` + +- **输出。** 一个或多个 stamp，由引擎按 `check` 的方式写入（见上文的 stamp + 规则），以及一个声明的目录，`output_dir(dir)`，由**命令**去填充。**如果 + 命令成功而那个目录不存在，或者除该 action 的 stamp 之外不含任何文件，引擎 + 不写 stamp，并以指出该目录的消息使这条边失败**——无论命令有没有创建过它： + 例如一个 Qt `lupdate` action 的目录，是它要改写的一棵既有源码树，从来不是 + 一个全新的目录。stamp 可以位于该目录之内，也可以在它旁边。 +- **顺序。** 声明它的那个包的每条编译边都等它，走的是 blocking `check` 用的 + 同一条 order-only 边；整个构建计划里的每一条**链接**边也都等它，因为任意 + 一个包声明的 `runtime_search_dir`（或任何其它 link-global 指令）都被并入 + 整个计划共用的**同一个** `LinkIntent`——一个 `prepare` 填充的目录，必须在 + 任何可能读它的链接之前就存在，不只是声明它的那个包自己的链接。 +- **对它产物的引用是配置时就定死的名字。** 一旦目录存在，构建就按**目录** + 去读它：目录本身用 `include_dir`/`link_search`/`runtime_search_dir`， + 目录里某个文件的完整路径用 `link_flag`。`prepare` 不改变这些指令原有的 + 行为。 +- **为什么是一个 role，而不是 `check` 加 `blocking = true`。** role 会被 + 引擎的决策读取，这是单靠 `blocking` 做不到的：把施工标成校验，会让它在 + 未来某条只针对 check 的策略出现的那一刻，与真正的校验混同。 +- **进度行**里，一个 `prepare` action 标为 `PREPARE`，与 `CHECK`、`OBJECT`、 + `ARTIFACT`、`GENERATE` 并列。 + +> **不要在配置期读一个 `prepare` 目录的内容。** 一个 `rerun_if_changed` / +> `rerun_if_changed_glob` 如果命中另一个包（或本包自己）声明的 `prepare` +> 目录内部的某个路径，就是在这个程序**配置**的同时读取一个**施工结果**： +> 第二次构建是对的（上一次构建已经把目录填好了），第一次是错的。mcpp 会 +> 警告，点名两条路径，而不是拒绝——这个程序今天仍然能正确地完成配置。这正是 +> 「监视一个安装前缀，把第一次安装的库放到下一次规划里」这种做法的形状； +> 应改为给这个前缀声明 `runtime_search_dir`，让一个 `prepare` action 去做 +> 安装本身。 + ### 产出可分发物：`pack_format` 与 `stage_dir`（2026.9.11.1+） 一个 `.msi`、一个 `.deb`、一个 AppImage、一个签过名的 `.app`，都不是那四个 role 的 diff --git a/docs/zh/31-authoring-a-rule-package.md b/docs/zh/31-authoring-a-rule-package.md index e1c81c33f..0efc25328 100644 --- a/docs/zh/31-authoring-a-rule-package.md +++ b/docs/zh/31-authoring-a-rule-package.md @@ -174,9 +174,12 @@ a.submit(); 这些字符串必须比 action 活得久。`a.id = ("toy:" + stem).c_str()` 交给它 的是一个指向临时对象的指针，而该临时对象在 `submit()` 之前就已经消失。 -### 四种 role +### 五种 role -`role` 决定这条边的输出去哪里，以及这条边什么时候跑。 +`role` 决定这条边的输出去哪里，以及这条边什么时候跑。用 `mcpp::roles` +常量拼写它(`mcpp::roles::source`、`::check`、`::object`、`::artifact`、 +`::prepare`)，不要用裸字符串：一个不认识某个常量的引擎会拒绝编译这个规则包 +并指出它，而不是把一个拼错的字符串读成 `source`(SPEC-007 R3.6)。 | `role` | 输出 | 次序 | |---|---|---| @@ -184,6 +187,12 @@ a.submit(); | `check` | 一个 stamp 文件 | 与编译并行；`blocking = true` 让编译等它 | | `object` | 进入**链接**集合 | 链接边消费它们 | | `artifact` | 一个新文件 | 它的输入是链接产物，所以它在链接之后跑 | +| `prepare` | 一个 stamp 文件，加上一个声明的目录(`output_dir`)由命令去填充 | 整个构建计划里的每条编译边**与**每条链接边都等它 | + +`prepare` 用于命令运行时文件名还不知道的施工——把一个包管理器的前缀装起来、 +解开一个 SDK——也是 SPEC-007(`docs/specs/build-plugins.md`)要求依赖适配类 +规则包使用的形态，而不是一个 blocking 的 `check`；完整的约定(`output_dir` +上的后置条件、命中它内部的重跑依据会得到的 R1.3 警告)见 docs/30 自己那一节。 ### 声明的输入，以及其中的编译器 diff --git a/docs/zh/50-machine-output.md b/docs/zh/50-machine-output.md index bc41c37a2..206316593 100644 --- a/docs/zh/50-machine-output.md +++ b/docs/zh/50-machine-output.md @@ -432,18 +432,39 @@ mcpp emit build-database [--spec s1|compile-commands] --format json 写入 ``。文档的内容、不写入项目目录这条保证，以及 `watch` 的规则，见 [SPEC-005](../specs/build-database.md)。 -失败时省略 `data` 并以 1 退出，诊断码为：不在项目中时是 +`emit` 独立规划每一个被选中的成员（#699 第 1 项）：一个成员的规划失败不会 +连累它的兄弟成员。不在项目中，或者被选中的成员全部规划失败时，信封省略 +`data` 并以 1 退出，每个失败的成员各带一条诊断：不在项目中是 `MCPP_BUILD_DATABASE_NO_PROJECT`；离线规划（`--offline`、`MCPP_OFFLINE`、 `MCPP_NO_AUTO_INSTALL`）需要下载某样东西（工具链、包、git 修订，或包索引， 消息会指出第一个）时是 `MCPP_OFFLINE_DOWNLOAD_REQUIRED`；因其他原因规划失败 -时是 `MCPP_BUILD_DATABASE_PLAN_FAILED`。三者中的第一种不是项目的缺陷：不带 -`--offline` 再运行一次即可消除它。警告不影响文档本身： - -| 诊断码 | | -|---|---| -| `MCPP_LOCK_WOULD_CHANGE` | 解析结果与项目的 `mcpp.lock` 不一致，命令不写这个文件 | -| `MCPP_GENERATED_FILE_NOT_MATERIALIZED` | 根包 `[build] generated_files` 中的某个文件缺失或内容已过期，命令不写这个文件 | -| `MCPP_BUILD_DATABASE_STD_UNIT_UNDESCRIBED` | 没有任何标准库构建命令点名它的模块源文件，该单元因此不被列出 | +时是 `MCPP_BUILD_DATABASE_PLAN_FAILED`。离线这一种不是项目的缺陷：不带 +`--offline` 再运行一次即可消除它。每个成员的诊断都带 `path`，即该成员的 +`mcpp.toml`，相对工作区根目录。 + +只要有一个被选中的成员规划成功，`data` 就会出现，并描述每一个规划成功的 +成员：规划失败的成员不贡献任何集合，只贡献上面那样一条 `error` 诊断，它的 +`mcpp.toml` 与存在时的 `build.mcpp` 一并加入 `watch`。只要诊断里有一条是 +`error`，退出码依然是 1——因此消费方靠结构就能读出三种结果：没有 `data`； +`data` 伴随若干 `error` 诊断，描述了诊断所指之外的一切；`data` 且没有 +`error`，不需要解析消息文本。 + +构建程序失败的包（#699 第 2 项）会被描述为不含该程序产生的指令：清单自身 +的那部分配置、工具链、模块图与标准库单元仍照常描述，另附一条 `error` 诊断 +`MCPP_BUILD_DATABASE_PROGRAM_FAILED` 点名它，`path` 为它的 `build.mcpp`。若 +后续失败是由缺失的指令引起的，则按上面的规则使整个成员失败。包请求的宿主 +工具构建失败则降级为警告 `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT`，点名工具、 +所属包与失败信息的第一行；规划继续进行，只点名该工具而不运行它的构建程序 +会像该工具构建成功时一样完成配置。这两者都不影响 `mcpp build`：构建程序或 +宿主工具在其中失败仍然会使构建失败。 + +| 诊断码 | 严重级别 | | +|---|---|---| +| `MCPP_LOCK_WOULD_CHANGE` | 警告 | 解析结果与项目的 `mcpp.lock` 不一致，命令不写这个文件 | +| `MCPP_GENERATED_FILE_NOT_MATERIALIZED` | 警告 | 根包 `[build] generated_files` 中的某个文件缺失或内容已过期，命令不写这个文件 | +| `MCPP_BUILD_DATABASE_STD_UNIT_UNDESCRIBED` | 警告 | 没有任何标准库构建命令点名它的模块源文件，该单元因此不被列出 | +| `MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT` | 警告 | 被请求的宿主工具构建失败；该工具仍会被构建，它的 `check` 动作仍会运行 | +| `MCPP_BUILD_DATABASE_PROGRAM_FAILED` | 错误 | 构建程序失败；它所属的包被描述为不含它产生的指令 | `--protocol-version` 为这条命令声明 `init-mcpp-home`、`read-project`、 `network`、`write-global-cache` 与 `exec-build-script`，从不声明 diff --git a/docs/zh/README.md b/docs/zh/README.md index 738861e11..d1207b333 100644 --- a/docs/zh/README.md +++ b/docs/zh/README.md @@ -157,3 +157,4 @@ - [SPEC-004 —— `mcpp.toml` 的平面划分、条件化形状、解析轴与命名规约](../specs/manifest-semantics.md) - [SPEC-005 —— `mcpp emit build-database` 输出的构建数据库](../specs/build-database.md) - [SPEC-006 —— 工具链管理：身份、来源、选择与载荷契约](../specs/toolchain-management.md) + - [SPEC-007 —— 构建插件：配置、施工与校验的分工，运行时与规划期的义务](../specs/build-plugins.md) diff --git a/mcpp.toml b/mcpp.toml index 96c19fc01..afdd732c2 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.9.26.1" +version = "2026.9.26.2" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/buildmcpp/src/directives.cppm b/modules/buildmcpp/src/directives.cppm index 2328774bb..a1fd4b13e 100644 --- a/modules/buildmcpp/src/directives.cppm +++ b/modules/buildmcpp/src/directives.cppm @@ -164,6 +164,23 @@ enum class Slot : std::size_t { // splitting it into two slots would let a build with N deploy directives // pair them up wrong the moment N > 1. Deploy, + // A LAUNCH-TIME SEARCH DIRECTORY, THE BUILD-PROGRAM FORM OF + // `runtime_search_dirs` (docs/04 §2.11). A dependency that brings a + // prebuilt shared library (a vcpkg prefix's `bin/`, a Qt SDK's `bin/`, a + // directory a `prepare` action populates) knows where it lives only at + // build-program time, and the manifest key is a fixed TOML array. + // `Transform::AbsPath`: relative resolves against the package root, like + // `include-dir`. Feeds `LinkIntent::runtimeSearchDirs` DIRECTLY -- + // `runtime_search_dirs`' own field, not the retiring `[runtime] + // library_dirs` / `RuntimeConfig::libraryDirs` (docs/04 §2.11 keeps that + // one "for one compatibility train" and gains no directive of its own) -- + // so every reader of `runtimeSearchDirs` (the plan merge, RUNPATH/rpath on + // ELF and Mach-O, `mcpp run`'s loader path, `mcpp pack`'s closure search) + // sees a directive-declared directory through the one merge that already + // exists for the manifest key (`resolve_runtime_contract`, plan.cppm), no + // second path to keep in sync. The directory need not exist when the + // program runs: a `prepare` action may populate it later, at build time. + RuntimeSearchDir, Count }; inline constexpr std::size_t kSlotCount = static_cast(Slot::Count); @@ -262,7 +279,7 @@ struct Def { int sinceProtocol; }; -inline constexpr std::array kTable{{ +inline constexpr std::array kTable{{ // wire tag slot scope transform must missingPrefix missingSuffix since {"cxxflag", "cxxflag", Slot::CxxFlags, Scope::PackagePrivate, Transform::Verbatim, false, "", "", 1}, {"cflag", "cflag", Slot::CFlags, Scope::PackagePrivate, Transform::Verbatim, false, "", "", 1}, @@ -413,6 +430,32 @@ inline constexpr std::array kTable{{ // before `apply` on both the run path and the cache-hit path, so a cached // replay refuses exactly what a fresh run would. {"deploy", "deploy", Slot::Deploy, Scope::LinkGlobal, Transform::Deploy, false, "", "", 11}, + // v12 (mcpp#701/#702, design §5.4 R1'). The build-program form of + // `runtime_search_dirs` (docs/04 §2.11): a launch-time search directory, + // for a dependency (a vcpkg prefix's `bin/`, a Qt SDK's `bin/`, a + // directory a `prepare` action populates) whose location a build.mcpp + // learns rather than one an author can write into TOML. Scope::LinkGlobal, + // exactly as `deploy` above, because `apply` folds it DIRECTLY into + // `LinkIntent::runtimeSearchDirs` -- NOT `RuntimeConfig::libraryDirs`, + // which docs/04 §2.11 keeps readable "for one compatibility train" and + // which gains no directive -- so `resolve_runtime_contract`'s per-package + // merge (plan.cppm) carries it into a consumer's `linkIntent + // .runtimeSearchDirs` through the one path that already exists for the + // manifest key, rather than a second one this table would have to keep in + // sync. `mustExistAfterRun` is FALSE: a directory is not the + // declared-output contract's shape (`generated=`/`source=` name a file), + // and the directory need not even exist when this program runs -- a + // `prepare` action (§3) may populate it later, at build time, which is + // the reason this directive exists at all rather than a plain manifest + // array. + // + // kCacheEpoch is NOT bumped, the same reasoning `warning`/`pack-format` + // state explicitly: an entry written before this row carries no + // `d runtime-search-dir` line, and the program that wrote it could not + // emit one -- so replaying it yields exactly what that program said, and + // the entry is still correct. An older engine reading a newer entry + // already discards the whole record through the unknown-tag path. + {"runtime-search-dir", "runtime-search-dir", Slot::RuntimeSearchDir, Scope::LinkGlobal, Transform::AbsPath, false, "", "", 12}, }}; // ── Collected output of one run ──────────────────────────────────────────── @@ -682,9 +725,15 @@ std::string transformed(const Def& def, std::string_view raw, const fs::path& root) { switch (def.transform) { case Transform::Verbatim: return std::string(raw); - case Transform::LibFlag: return mcpp::toolchain::lib_flag_for(dial, raw); - case Transform::LibSearchPath: return std::string(dial.libSearchPrefix) - + abs_against(root, raw); + // A link-flag element is read into words (SPEC-004 §8, #703), so a + // value the engine builds from a name or a path is spelled to read + // back as one word: a library or a directory with a space in its path + // stays one argument. + case Transform::LibFlag: + return mcpp::manifest::flag_element(mcpp::toolchain::lib_flag_for(dial, raw)); + case Transform::LibSearchPath: + return mcpp::manifest::flag_element(std::string(dial.libSearchPrefix) + + abs_against(root, raw)); // A define is one word of a compile-flag list, whatever it contains. case Transform::DefinePrefix: return mcpp::manifest::flag_element(std::string(dial.definePrefix) + std::string(raw)); @@ -692,7 +741,9 @@ std::string transformed(const Def& def, std::string_view raw, // Absolute on purpose: the link runs in the build directory, so a // relative script path resolves against the wrong root and lld // answers "cannot find linker script link.ld" — measured. - case Transform::LinkerScript: return "-T " + abs_against(root, raw); + // Two words, `-T` and the path, the second spelled as one word. + case Transform::LinkerScript: + return "-T " + mcpp::manifest::flag_element(abs_against(root, raw)); // Resolve `from` NOW, while `root` (this build.mcpp's package root) is // still in hand -- `apply` is never given it. `to` is left untouched: // it is a destination relative to an executable this package has not @@ -945,6 +996,15 @@ void apply(mcpp::manifest::Manifest& m, const Directives& d) { for (auto const& p : d.at(Slot::IncludeDirsAfter)) bc.includeDirsAfter.emplace_back(p); + // `runtime-search-dir`: already absolute from the AbsPath transform, like + // IncludeDirs above. Joins `LinkIntent::runtimeSearchDirs` DIRECTLY, the + // exact field `[runtime] runtime_search_dirs` populates from TOML -- not + // the retiring `RuntimeConfig::libraryDirs` -- so a consumer sees it + // exactly as it sees the manifest key, folded in by the SAME merge + // (plan.cppm's `resolve_runtime_contract`), never a second one. + for (auto const& p : d.at(Slot::RuntimeSearchDir)) + m.runtimeConfig.linkIntent.runtimeSearchDirs.emplace_back(p); + // Claims join the runtime declarations the manifest could have carried // itself, so the version-floor check in prepare reads ONE list and never // learns which spelling a claim arrived in. @@ -1006,15 +1066,55 @@ void apply(mcpp::manifest::Manifest& m, const Directives& d) { } } +// The five roles `mcpp::action::role` may name, in the order +// `mcpp::manifest::BuildAction::Role` declares them (mcpp#702). A build.mcpp +// SHOULD spell one of `mcpp::roles::{source, check, object, artifact, +// prepare}` (SPEC-007 R3.6): those constants exist only from protocol 12, so +// a program that misspells the role fails to COMPILE on an older engine and +// names the missing constant, instead of reaching this decoder as an +// unrecognised string. A hand-written `printf("mcpp:action=...")` program +// (the frozen surface, protocol 0) has no such guard, which is the case this +// list is for. +constexpr std::array kActionRoles{ + "source", "check", "object", "artifact", "prepare"}; + +bool is_known_action_role(std::string_view role) { + return std::ranges::find(kActionRoles, role) != kActionRoles.end(); +} + +// The bare `role` an `mcpp:action=` payload named, or "source" when the key +// is absent (the same default `decode_action` uses) -- independent of +// whether the REST of the payload is well-formed, so `action_error` can name +// an unrecognised role even when the action is otherwise malformed, instead +// of folding both failures into one generic message. +std::optional action_role_of(std::string_view payload) { + try { + auto j = nlohmann::json::parse(payload); + return j.value("role", std::string{"source"}); + } catch (...) { + return std::nullopt; + } +} + std::optional decode_action(std::string_view payload) { try { auto j = nlohmann::json::parse(payload); mcpp::manifest::BuildAction a; a.id = j.value("id", std::string{}); auto role = j.value("role", std::string{"source"}); + // An unrecognised role is refused here, not read as Source: before + // mcpp#702 this fell through to Source silently, changing an action's + // meaning without a word (SPEC-007 R3.6). `action_error` names the + // offending value before anything is applied; this path is also the + // one a cache-hit replay runs through `apply` (which calls no + // validator), so it has to refuse on its own rather than assume + // `action_error` already ran -- `epoch` 3 (program_protocol.cppm) + // makes sure no such entry survives from before this row existed. + if (!is_known_action_role(role)) return std::nullopt; a.role = role == "check" ? mcpp::manifest::BuildAction::Role::Check : role == "artifact" ? mcpp::manifest::BuildAction::Role::Artifact : role == "object" ? mcpp::manifest::BuildAction::Role::Object + : role == "prepare" ? mcpp::manifest::BuildAction::Role::Prepare : mcpp::manifest::BuildAction::Role::Source; auto arr = [&](const char* k, std::vector& dst) { if (auto it = j.find(k); it != j.end() && it->is_array()) @@ -1030,7 +1130,18 @@ std::optional decode_action(std::string_view payloa a.blocking = j.value("blocking", false); a.depfile = j.value("depfile", std::string{}); a.description = j.value("description", std::string{}); + // Omitted by every role but Prepare, exactly as `depfile` is omitted + // by an action that declares none -- the decoder's default (empty) + // is identical either way, so a cache entry written before this field + // existed replays as one that never set it. + a.outputDir = j.value("output_dir", std::string{}); if (a.command.empty() || a.outputs.empty()) return std::nullopt; + // Prepare only: `output_dir` is the whole point of the role (R3.2, + // R3.3) -- a `prepare` action with none declared would have a stamp + // and nothing else, indistinguishable from a `check` that merely + // forgot `blocking = true`. + if (a.role == mcpp::manifest::BuildAction::Role::Prepare && a.outputDir.empty()) + return std::nullopt; if (a.id.empty()) a.id = a.outputs.front(); return a; } catch (...) { @@ -1144,6 +1255,18 @@ std::string action_error(const Directives& d) { "and cannot be used.\n" " payload: {}", payload); } + // Named BEFORE the general decode below, so the message identifies + // the unrecognised VALUE and lists the five roles the engine knows, + // rather than folding it into "malformed action" -- which is what an + // unknown role used to become (silently read as Source) until + // mcpp#702 (SPEC-007 R3.6). + if (auto role = action_role_of(payload); role && !is_known_action_role(*role)) { + return std::format( + "build.mcpp declared an action with role \"{}\", which is not " + "one of the five mcpp knows: source, check, object, artifact, " + "prepare.\n" + " payload: {}", *role, payload); + } if (decode_action(payload)) continue; // A malformed action is a hard error, never a skip: an action that // silently does not exist produces a build missing generated sources, @@ -1152,7 +1275,10 @@ std::string action_error(const Directives& d) { "build.mcpp declared a malformed action.\n" " Every action needs a non-empty `command` and at least one\n" " declared `output` — mcpp fixes the source set during prepare,\n" - " so an output whose NAME is unknown cannot be built.\n" + " so an output whose NAME is unknown cannot be built. A\n" + " `prepare` action also needs a non-empty `output_dir`\n" + " (mcpp::action::output_dir), the directory its command\n" + " populates.\n" " payload: {}", payload); } return {}; @@ -1192,6 +1318,14 @@ void prepare_actions(std::vector& actions, // simply never match the file the command actually wrote. if (!a.depfile.empty() && a.depfile.find("${mcpp.") == std::string::npos) a.depfile = abs_against(pkgRoot, a.depfile); + // Prepare only: the same anchoring, for the same reason -- the + // directory the command populates and the post-condition the engine + // checks after that command succeeds (`__action-stamp`, src/cli.cppm) + // both need it absolute, or a package-relative spelling would be + // interpreted relative to the ninja build directory instead of the + // package root it was written against. + if (!a.outputDir.empty() && a.outputDir.find("${mcpp.") == std::string::npos) + a.outputDir = abs_against(pkgRoot, a.outputDir); if (a.role != mcpp::manifest::BuildAction::Role::Source) continue; for (auto const& o : a.outputs) { if (o.find("${mcpp.") != std::string::npos) continue; diff --git a/modules/buildmcpp/src/program_protocol.cppm b/modules/buildmcpp/src/program_protocol.cppm index 8b1dbf826..c712789d3 100644 --- a/modules/buildmcpp/src/program_protocol.cppm +++ b/modules/buildmcpp/src/program_protocol.cppm @@ -81,7 +81,22 @@ export namespace mcpp::build::program_protocol { // at a path relative to the executable. Same cost as v5's: a package calling // `mcpp::deploy()` fails on an older engine at the build.mcpp COMPILE, because // that engine's bundled module has no such function. -inline constexpr int kProtocolVersion = 11; +// v12 (mcpp#701/#702): adds `runtime-search-dir` -- the build-program form of +// `runtime_search_dirs`: a launch-time search directory for a dependency (a +// vcpkg prefix's `bin/`, a Qt SDK's `bin/`, a directory a `prepare` action +// populates) whose location a build.mcpp learns rather than one an author can +// write into TOML. Same cost as v5's: a package calling +// `mcpp::runtime_search_dir()` fails on an older engine at the build.mcpp +// COMPILE, because that engine's bundled module has no such function. +// +// v12 also adds the `prepare` role: construction whose file names are not +// known when the build program runs. Its constants +// (`mcpp::roles::{source, check, object, artifact, prepare}`) and its +// `mcpp::action::output_dir()` method live in the bundled module for the +// same reason `deploy` and `runtime_search_dir` do -- a program that uses +// them fails to COMPILE on an older engine, naming the missing symbol, +// rather than reaching that engine as a string it misreads. +inline constexpr int kProtocolVersion = 12; // ── Cache-format epoch ───────────────────────────────────────────────────── // @@ -94,7 +109,20 @@ inline constexpr int kProtocolVersion = 11; // Epoch 2 (#359): entries gained `glob` records. An engine that does not know // them would replay a strict subset of the declared inputs and call a stale // build fresh, which is exactly the silent-wrong-answer this guard exists for. -inline constexpr int kCacheEpoch = 2; +// Epoch 3 (#703): an `ldflag` value is read into words (SPEC-004 §8), and the +// link directives spell a path as one word. A value an earlier engine cached, +// `-L/opt/my sdk/lib` for instance, would now read as two words. +// Epoch 3 also covers mcpp#702: `decode_action` now refuses an `mcpp:action=` payload +// whose `role` names none of the five the engine knows, where it used to read +// an unrecognised string as `Source` silently. That is an INTERPRETATION +// change on an existing tag (`action`, since v1) rather than a new one: an +// entry cached under epoch 2 could hold such a role and decoded successfully +// then; replaying it through `apply` (which runs no validator) would now +// either refuse or, worse, silently drop the action, neither of which is what +// a fresh run of the SAME build.mcpp would do. Bumping the epoch forces one +// fresh run, which goes through `action_error` and reports exactly what +// changed, instead of a stale entry reinterpreted two ways by two code paths. +inline constexpr int kCacheEpoch = 3; // ── Run bound ────────────────────────────────────────────────────────────── // diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index e04aa713b..88f35a432 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -485,7 +485,17 @@ inline bool is_empty(const BuildInputs& b) { // prepare, and all of them need to know which files exist. Content may arrive // later; names may not. struct BuildAction { - enum class Role { Source, Check, Object, Artifact }; + // `Prepare` (mcpp#702): construction whose file names are not known when + // the build program runs -- installing a vcpkg manifest or a CMake + // subproject into a prefix, unpacking an SDK. Its outputs are stamps (the + // engine writes them exactly as for `Check`) and one declared directory, + // `outputDir` below, which the command populates and which the build + // reads BY DIRECTORY through `include_dir`/`link_search`/ + // `runtime_search_dir` rather than by naming files. A role of its own, + // not `Check` with `blocking = true`: a role is read by engine decisions + // (the ordering below, and any future one) that must not also catch every + // verification action just because it blocks compilation. + enum class Role { Source, Check, Object, Artifact, Prepare }; std::string id; // diagnostics + edge naming // Which package's `build.mcpp` declared this. Filled by the engine when @@ -546,6 +556,14 @@ struct BuildAction { std::vector imports; // Check only: make compilation wait for this to pass. Off by default. bool blocking = false; + // Prepare only: the directory the command populates (mcpp#702). Absolute + // once `prepare_actions` has run, exactly as `outputs`/`depfile` are. + // Empty for every other role. The post-condition it carries -- the + // directory must exist after the command succeeds, whether or not the + // command created it -- is checked by the engine's stamp wrapper + // (`__action-stamp`, src/cli.cppm), the same one `Check`'s stamp goes + // through, not by anything in this struct. + std::string outputDir; // A Make-style dependency file the action's COMMAND writes as a side // effect — ninja reads it once the command exits and folds it into its // own dependency log, the same `deps = gcc` mechanism a `cxx_object` edge diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index 8723bc1f1..60c3bcdd0 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.9.26.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.9.26.2"; } // namespace mcpp diff --git a/src/build/build_database.cppm b/src/build/build_database.cppm index 429595aa6..efbc618cb 100644 --- a/src/build/build_database.cppm +++ b/src/build/build_database.cppm @@ -66,15 +66,23 @@ struct Rendered { nlohmann::json compileCommands = nlohmann::json::array(); std::vector watch; std::string inputsFingerprint; - // Conditions found while rendering; reported as warnings by the command. + // Conditions found while rendering, at the severity the command reports + // them: most are warnings the document is still complete despite, and a + // failed build program (#699 item 2, E3) is an error whose `path` names + // its `build.mcpp`, rewritten here to the workspace-relative form. std::vector notes; }; // The S1 document for `members`, and the inputs whose change changes it. +// `failedMemberRoots` is every selected workspace member whose planning +// failed (#699 item 1, E1): it contributes no set, but its `mcpp.toml` and +// `build.mcpp` (when present) still join `watch`, exactly as a planned +// member's do — fixing either file is what should make a consumer ask again. // `workspaceRoot` anchors the relative `watch` patterns; `selector` is the // command's own selection (target, toolchain, profile, members), which enters // the fingerprint because the same files answer differently under another one. Rendered render(std::span members, + std::span failedMemberRoots, const std::filesystem::path& workspaceRoot, std::string_view selector); @@ -92,26 +100,15 @@ std::string_view stdlib_name(std::string_view stdlibId); std::string toolchain_id(const mcpp::toolchain::Toolchain& tc, std::string_view compilerTriple); -// Splits a command string mcpp rendered for the host shell into words, undoing -// its quoting: POSIX `sh` rules, or the Microsoft C runtime's rules on Windows. -// The reader is mcpp::manifest::host_command_words; this name is kept for the -// standard library units' recovery below. -std::vector split_command_words(std::string_view command, bool windows); - -// The working directory and argument vector of the command in `commands` whose -// words name `source`, recovered from the rendering: a leading `cd`, an `env` -// word and environment assignments, and redirections are removed. A driver -// path that the rendering left unquoted despite a space is rejoined. Empty when -// no command names the source. -struct Invocation { - std::filesystem::path workDirectory; - std::vector arguments; -}; -std::optional recover_invocation(const std::vector& commands, - const std::filesystem::path& source, - const std::filesystem::path& driver, - const std::filesystem::path& defaultDirectory, - bool windows); +// The standard-library units' recovery (mcpp.build.plan::recover_invocation) +// now runs once, in prepare.cppm, onto BuildPlan::stdModuleUnits (design +// 2026-09-26 §3.5, D5a/D5b: the plan carries the standard-library +// description, so this document and compile_commands.json render the exact +// same record and cannot disagree, P1). Re-exported here under the names +// this file's tests use. +using Invocation = mcpp::build::RecoveredInvocation; +using mcpp::build::recover_invocation; +using mcpp::build::split_command_words; } // namespace mcpp::build::database @@ -131,29 +128,6 @@ std::string qualified_name(const mcpp::manifest::Manifest& m) { : m.package.namespace_ + "." + m.package.name; } -bool is_assignment(std::string_view w) { - auto eq = w.find('='); - if (eq == std::string_view::npos || eq == 0) return false; - if (!(std::isalpha(static_cast(w[0])) || w[0] == '_')) return false; - for (std::size_t i = 1; i < eq; ++i) { - const unsigned char c = static_cast(w[i]); - if (!(std::isalnum(c) || c == '_')) return false; - } - return true; -} - -// `2>&1`, `>file`, `nul`, ``, `2>`, `<`, -// `>>`: the target is the next word. -enum class Redirect { None, Attached, Detached }; -Redirect redirect_kind(std::string_view w) { - std::size_t i = 0; - while (i < w.size() && std::isdigit(static_cast(w[i]))) ++i; - if (i >= w.size() || (w[i] != '>' && w[i] != '<')) return Redirect::None; - std::size_t j = i + 1; - if (j < w.size() && w[j] == w[i]) ++j; // `>>` - return j == w.size() ? Redirect::Detached : Redirect::Attached; -} - bool names_path(std::string_view word, const std::filesystem::path& path, const std::filesystem::path& cwd) { std::filesystem::path w{std::string(word)}; @@ -234,6 +208,23 @@ void split_baseline(nlohmann::json& set) { set["baseline-arguments"] = std::move(baseline); } +// D5b, S1 §6: "the compiler's build revision ... equal versions do not imply +// compatible BMIs" (S1-11.2-3 gates the authoritative reuse of a build BMI on +// this value matching exactly). mcpp already computes it, as one field of the +// toolchain fingerprint (mcpp.toolchain.fingerprint::compute_fingerprint, +// field 3): the compiler's normalized `--version` banner when the toolchain +// probe recorded one (`Toolchain::driverIdent`), which changes with a build +// commit even when the reported version string does not, and a hash of the +// driver binary otherwise. Deriving build-id from the SAME computation, +// rather than a second one, is what keeps the two stable together and each +// stable across two runs of one toolchain. +std::string build_id(const mcpp::toolchain::Toolchain& tc) { + return !tc.driverIdent.empty() + ? mcpp::toolchain::hash_string(tc.driverIdent) + : (tc.binaryPath.empty() ? std::string{} + : mcpp::toolchain::hash_file(tc.binaryPath)); +} + nlohmann::json toolchain_json(const mcpp::toolchain::Toolchain& tc, std::string_view compilerTriple, const std::vector& invocations) { @@ -250,6 +241,7 @@ nlohmann::json toolchain_json(const mcpp::toolchain::Toolchain& tc, if (!tc.stdlibVersion.empty()) stdlib["version"] = tc.stdlibVersion; j["stdlib"] = std::move(stdlib); } + if (auto id = build_id(tc); !id.empty()) j["build-id"] = id; return j; } @@ -308,74 +300,11 @@ std::string toolchain_id(const mcpp::toolchain::Toolchain& tc, return std::format("{}-{}-{}", tc.compiler_family(), tc.version, compilerTriple); } -std::vector split_command_words(std::string_view s, bool windows) { - return mcpp::manifest::host_command_words(s, windows); -} - -std::optional recover_invocation(const std::vector& commands, - const std::filesystem::path& source, - const std::filesystem::path& driver, - const std::filesystem::path& defaultDirectory, - bool windows) { - const std::string driverText = driver.string(); - for (auto const& command : commands) { - auto words = split_command_words(command, windows); - std::vector> segments(1); - for (auto& w : words) { - if (w == "&&") segments.emplace_back(); - else segments.back().push_back(std::move(w)); - } - std::filesystem::path cwd; - for (auto& seg : segments) { - if (seg.empty()) continue; - if (seg.front() == "cd") { - std::size_t k = 1; - if (k < seg.size() && (seg[k] == "/d" || seg[k] == "/D")) ++k; - if (k < seg.size()) cwd = std::filesystem::path{seg[k]}; - continue; - } - std::size_t b = 0; - if (b < seg.size() && seg[b] == "env") ++b; - while (b < seg.size() && is_assignment(seg[b])) ++b; - std::vector argv; - for (std::size_t k = b; k < seg.size(); ++k) { - switch (redirect_kind(seg[k])) { - case Redirect::Attached: continue; - case Redirect::Detached: ++k; continue; - case Redirect::None: argv.push_back(seg[k]); - } - } - if (argv.empty()) continue; - // A driver path rendered without quotes splits at its spaces. - if (argv.front() != driverText - && driverText.find(' ') != std::string::npos) { - std::string joined = argv.front(); - std::size_t k = 1; - while (k < argv.size() && joined.size() < driverText.size()) { - joined += ' '; - joined += argv[k]; - ++k; - } - if (joined == driverText) { - argv.erase(argv.begin() + 1, argv.begin() + static_cast(k)); - argv.front() = driverText; - } - } - const bool named = std::ranges::any_of(argv, [&](const std::string& w) { - return names_path(w, source, cwd.empty() ? defaultDirectory : cwd); - }); - if (!named) continue; - return Invocation{cwd.empty() ? defaultDirectory : cwd, std::move(argv)}; - } - } - return std::nullopt; -} - Rendered render(std::span members, + std::span failedMemberRoots, const std::filesystem::path& workspaceRoot, std::string_view selector) { Rendered r; - const bool windows = mcpp::platform::is_windows; nlohmann::json toolchains = nlohmann::json::object(); nlohmann::json sets = nlohmann::json::array(); @@ -471,40 +400,48 @@ Rendered render(std::span members, }); } - if (ctx.stdModule) { - const auto& sm = *ctx.stdModule; - auto add_std = [&](const std::filesystem::path& source, - const std::vector& commands, - const std::filesystem::path& object, - std::string_view module, - std::vector requires_) { - if (source.empty() || commands.empty()) return; - auto inv = recover_invocation(commands, source, ctx.tc.binaryPath, - sm.cacheDir, windows); - if (!inv) { - r.notes.push_back({"MCPP_BUILD_DATABASE_STD_UNIT_UNDESCRIBED", - std::format("no command that builds the {} module names its " - "source '{}'; the unit is not listed", - module, source.string())}); - return; - } - auto& set = set_for(member.setPrefix + std::string(kStdSetName), - std::string(kStdSetName), "library"); + // D5a/D5b (design 2026-09-26 §3.5): the standard-library units, + // already recovered onto the plan (prepare.cppm, from the same + // derivation `ensure_built` and `describe_std_module` both read), so + // this rendering cannot list a command or a BMI path that + // compile_commands.json (mcpp.build.compile_commands) disagrees with + // (P1). S1-12-1: they are translation units of the build like any + // other, so `--spec compile-commands` lists them too. + if (!ctx.plan.stdModuleUnits.empty()) { + auto& set = set_for(member.setPrefix + std::string(kStdSetName), + std::string(kStdSetName), "library"); + for (auto const& unit : ctx.plan.stdModuleUnits) { + const auto sourceStr = native_string(unit.source); + const auto objectStr = native_string(unit.object); + const auto workDirStr = native_string(unit.workDirectory); + r.compileCommands.push_back(nlohmann::json{ + {"directory", workDirStr}, + {"file", sourceStr}, + {"arguments", unit.arguments}, + {"output", objectStr}, + }); + nlohmann::json requires_ = nlohmann::json::array(); + for (auto const& name : unit.requiresModules) requires_.push_back(name); set.units.push_back(nlohmann::json{ - {"source", native_string(source)}, - {"work-directory", native_string(inv->workDirectory)}, - {"arguments", std::move(inv->arguments)}, - {"object", native_string(object)}, + {"source", sourceStr}, + {"work-directory", workDirStr}, + {"arguments", unit.arguments}, + {"object", objectStr}, {"private", false}, - {"provides", {{std::string(module), ""}}}, + // D5b, S1-8-6: the path of the BMI the build writes, in + // the shared std cache that `emit` and the build share. + // The planning pass never compiles (SPEC-005 R2.2), but + // this path is the cache key's, not a compiler output it + // would have to run to learn. + {"provides", {{unit.module, native_string(unit.bmi)}}}, {"requires", std::move(requires_)}, {"ide", {{"role", "module-interface"}}}, }); - }; - add_std(ctx.tc.stdModuleSource, sm.stdCommands, sm.objectPath, "std", {}); - add_std(ctx.tc.stdCompatSource, sm.compatCommands, sm.compatObjectPath, - "std.compat", {"std"}); + } } + // A recovery failure is now reported where the recovery runs + // (prepare.cppm, onto BuildContext::planNotes) and reaches `r.notes` + // through the unconditional copy below, with every other plan note. for (auto const& name : order) { auto& set = groups.at(name); @@ -544,7 +481,31 @@ Rendered render(std::span members, for (auto const& f : declared.files) watch_file(f); for (auto const& g : declared.globs) watch_glob(declared.root, g); } - for (auto const& note : ctx.planNotes) r.notes.push_back(note); + for (auto note : ctx.planNotes) { + // `note.path`, when set, is the absolute path a `plan_only` + // branch recorded (#699 item 2, E3); every other `path` in this + // document is relative to the workspace root, and a note's is + // rewritten to match before it reaches the command. + if (!note.path.empty()) { + const std::filesystem::path abs{note.path}; + if (auto rel = relative_to(abs, workspaceRoot); rel && !rel->empty()) + note.path = *rel; + else + note.path = native_string(abs.lexically_normal()); + } + r.notes.push_back(std::move(note)); + } + } + + // Every member whose planning failed still has its likely cause watched + // (#699 item 1, E1): editing its `mcpp.toml`, or the `build.mcpp` that may + // have failed, is what should make a consumer ask again for the members + // that could not be described this time. + for (auto const& failedRoot : failedMemberRoots) { + watch_file(failedRoot / "mcpp.toml"); + std::error_code ec; + if (std::filesystem::exists(failedRoot / "build.mcpp", ec)) + watch_file(failedRoot / "build.mcpp"); } r.database = nlohmann::json{ diff --git a/src/build/compile_commands.cppm b/src/build/compile_commands.cppm index 93afe61ca..24214e2d5 100644 --- a/src/build/compile_commands.cppm +++ b/src/build/compile_commands.cppm @@ -7,10 +7,33 @@ // Uses the `arguments` array format (preferred over `command` string // per clangd docs). // -// Output location: /compile_commands.json so clangd finds -// it via its default upward directory walk — zero configuration needed. +// TWO FILES, ONE RULE EACH (design 2026-09-26, .agents/docs/2026-09-26- +// compile-database-and-issue-699-design.md §3.2). // -// See .agents/docs/2026-05-12-compile-commands-design.md. +// - The CONFIGURATION's database, `/compile_commands.json`, is +// the merged record of every command that has planned in this exact +// output directory (`build`, `test`, `run`, `--configure-only`): the +// fresh plan's entries, plus the entries it already holds whose `file`, +// resolved against `directory`, the fresh plan lacks and which still +// exist. Every entry in it was written by mcpp in this one configuration, +// so no ownership test is needed and none is made. +// - The ROOT file, `/compile_commands.json` (the path clangd's +// own upward search finds, or `plan.compileDbPath` where a symlink there +// is written through), is a COPY of the current configuration's database: +// replaced whole, never merged, and left untouched when byte-identical so +// clangd is not triggered for nothing. Switching toolchain or profile +// switches the whole file; switching back restores that configuration's +// entries, its test units included. +// +// `write_compile_commands` performs both steps for a freshly rendered plan +// (the full build path, ninja_backend.cppm). `publish_root_compile_commands` +// is exported on its own because the FAST path (execute.cppm) restores the +// root file from a configuration database already on disk, with no plan at +// all (C1: a deleted root file used to stay deleted forever, because nothing +// on the fast path ever reached a writer). +// +// See .agents/docs/2026-05-12-compile-commands-design.md and the 2026-09-26 +// design record above. export module mcpp.build.compile_commands; @@ -18,17 +41,19 @@ import std; import mcpp.source_kind; import mcpp.build.plan; import mcpp.build.flags; +import mcpp.home; import mcpp.libs.json; import mcpp.platform.fs; import mcpp.platform; import mcpp.manifest.flag_words; +import mcpp.toolchain.model; export namespace mcpp::build { // The words the compiler receives from one flag string the engine rendered // for a ninja `command =` line: ninja's `$` escapes are undone, then the -// host's command-line reader splits the text (POSIX `sh`, or the MSVCRT rules -// on Windows; mcpp::manifest::host_command_words). +// host's command-line reader splits the text (POSIX `sh`, or the MSVCRT +// rules on Windows; mcpp::manifest::host_command_words). // // It serves the rendered strings only, the global `$cflags`/`$cxxflags`/ // `$asmflags`. A unit's own flag lists are never rendered and re-read: the @@ -67,16 +92,24 @@ std::vector unit_invocations(const BuildPlan& plan, std::string emit_compile_commands(const BuildPlan& plan, const CompileFlags& flags); // Merge freshly-emitted CDB text (`fresh`, from the current build plan) with a -// prior CDB on disk (`existing`). A prior entry is preserved ONLY when its -// `file` is absent from `fresh` AND still exists on disk (per `fileExists`); +// prior CDB of the SAME CONFIGURATION on disk (`existing`). A prior entry is +// preserved ONLY when its `file`, resolved against its `directory` when not +// already absolute, is absent from `fresh` (under the same resolution) AND +// still exists on disk (per `fileExists`, probed against the resolved path); // everything else comes from `fresh`. Result is sorted by `file` for stable // output. A malformed `existing` is ignored (falls back to `fresh`). // -// Rationale: `mcpp build` regenerates the CDB from a plan that lacks test files -// / dev-deps, while `mcpp test` writes them in. Without merging, whichever ran -// last wins and clangd loses coverage for tests/ (no completion). Merging makes -// the CDB the union of every command's real plan — offline-safe, no extra -// dependency resolution. See .agents/docs/2026-06-25-cdb-test-coverage-design.md. +// Rationale: `mcpp build` regenerates the database from a plan that lacks +// test files / dev-deps, while `mcpp test` writes them in; both plan into the +// same output directory, so without merging, whichever ran last wins and +// clangd loses coverage for tests/ (no completion). Merging makes the +// configuration's database the union of every command's real plan — +// offline-safe, no extra dependency resolution. `directory` enters the +// identity because a unit's own `directory` need not be the process's +// working directory (the standard-library units always name the shared std +// cache; every project unit now names the output directory, §3.3). See +// .agents/docs/2026-06-25-cdb-test-coverage-design.md and the 2026-09-26 +// design record. std::string merge_compile_commands( std::string_view fresh, std::string_view existing, @@ -85,6 +118,11 @@ std::string merge_compile_commands( struct CompileCommandsWriteResult { bool changed = false; std::size_t commandCount = 0; + // Set only by publish_root_compile_commands (and by write_compile_commands, + // which calls it): the number of entries the REPLACED root file held that + // were not mcpp's own. Zero when the root was not replaced, held nothing + // foreign, or is being written for the first time. + std::size_t foreignEntries = 0; }; struct CompileCommandsWriteError { @@ -95,6 +133,11 @@ using ReplaceFile = std::function; +// Writes the CONFIGURATION's database at `path` (normally +// `plan.outputDir / "compile_commands.json"`): merges `fresh` with whatever +// `path` already holds (a symlink there is followed, as at the root), +// leaves the file untouched when the result is byte-identical, and replaces +// it atomically otherwise. std::expected publish_compile_commands( const std::filesystem::path& path, @@ -102,17 +145,44 @@ publish_compile_commands( const std::function& fileExists, ReplaceFile replaceFile = mcpp::platform::fs::replace_file); +// Copies the configuration database's CURRENT CONTENT on disk at +// `configDbPath` over the root file at `rootPath` (following a symlink +// there, as today): replaced WHOLE, never merged, and left untouched when +// byte-identical. This is the one function both the full build (right after +// it writes the configuration database) and the fast path (which has no +// plan, and reads the configuration database instead of holding it in +// memory) call to publish the root file — design §3.2 items 2 to 4. +// +// `targetRoot` and `mcppHome` decide which of the root file's PRIOR entries +// (by `output`, resolved against `directory`) are mcpp's own: under +// `targetRoot` (this project's whole `target/` tree — a toolchain or +// profile switch replaces the root file too, and that is not a foreign +// write) or under `mcppHome` (the shared std cache the standard-library +// units of §3.5 write into). Every other prior entry is counted in the +// result's `foreignEntries`, so the caller can warn once. +std::expected +publish_root_compile_commands( + const std::filesystem::path& configDbPath, + const std::filesystem::path& rootPath, + const std::filesystem::path& targetRoot, + const std::filesystem::path& mcppHome, + ReplaceFile replaceFile = mcpp::platform::fs::replace_file); + +// Writes the configuration's database for `plan`, then publishes the root +// copy from it. `commandCount` is the configuration database's entry count; +// `foreignEntries` is the root publish's (see publish_root_compile_commands). std::expected write_compile_commands(const BuildPlan& plan, const CompileFlags& flags); +// The one sentence both publishers of the root file (the full build and the +// fast path) print when the file they replaced held `n` entries mcpp did not +// write. One function, so the two paths cannot word it differently. +std::string foreign_entries_warning(std::size_t n); + } // namespace mcpp::build namespace mcpp::build { -namespace { - -} // namespace - // The order is the one ninja and the host apply: ninja replaces `$ `, `$:` and // `$$` while it builds the command line, and only then does the host read the // line into words. flags.cppm escapes for ninja before it quotes for the host, @@ -194,6 +264,160 @@ CompileCommandsWriteError write_error(std::string message) { return CompileCommandsWriteError{std::move(message)}; } +// `file` (or, for the ownership test below, `output`) resolved against +// `directory` when it is not already absolute: the identity the within- +// configuration merge and the root's ownership test both use (design §3.2 +// items 1 and 3). Every path mcpp itself writes into either field is already +// absolute (a unit's source and its output are both absolute; see +// plan.cppm), so this only matters for an entry another tool wrote, or for a +// future producer that follows the JSON format's licence to write a relative +// one. +std::filesystem::path resolve_against_directory(const nlohmann::json& entry, + std::string_view field) { + const std::string key(field); + if (!entry.is_object() || !entry.contains(key) || !entry.at(key).is_string()) + return {}; + std::filesystem::path p(entry.at(key).get()); + if (p.is_absolute()) return p.lexically_normal(); + if (!entry.contains("directory") || !entry.at("directory").is_string()) return p; + return (std::filesystem::path(entry.at("directory").get()) / p).lexically_normal(); +} + +// A key for the merge's dedup set: the resolved path, in one spelling. Not +// just `.string()` — a prior CDB written before the mixed-separator fix +// (#390) carries `root\generated/modules\x.cppm` entries that are the SAME +// file as a fresh `root\generated\modules\x.cppm` one, and a literal string +// comparison would keep both. Normalizing makes the merge self-healing. +std::string dedup_key(const std::filesystem::path& resolved) { + auto p = resolved.lexically_normal(); + p.make_preferred(); + return p.string(); +} + +// §3.2 item 3: an entry is mcpp's own when its `output`, resolved against its +// `directory`, lies under this project's `target/` (every configuration, not +// only the current one — switching toolchain or profile replaces the root +// file too, and that is not a foreign write) or under the mcpp home (the +// shared std cache the standard-library units of §3.5 write their objects +// into, outside `target/`). An entry with no `output` at all — the shape a +// hand-written or another tool's database uses — is never mcpp's: mcpp +// always writes one. +bool is_mcpps_entry(const nlohmann::json& entry, + const std::filesystem::path& targetRoot, + const std::filesystem::path& mcppHome) { + auto output = resolve_against_directory(entry, "output"); + if (output.empty()) return false; + auto under = [&](const std::filesystem::path& base) { + if (base.empty()) return false; + auto rel = output.lexically_relative(base.lexically_normal()); + if (rel.empty() || rel == std::filesystem::path(".")) return false; + return *rel.begin() != std::filesystem::path(".."); + }; + return under(targetRoot) || under(mcppHome); +} + +// Reads `path` through one symlink hop (as the root file's redirect always +// has) and returns the real file to publish to, plus its current content +// when it exists. A first build has nothing to resolve: symlink_status +// reports the missing path with type()==not_found on every standard library +// (the error code category differs — generic ENOENT vs system +// ERROR_FILE_NOT_FOUND — so that case is not an error). +struct ExistingDocument { + std::filesystem::path publishPath; + std::optional content; +}; + +std::expected +read_existing_document(const std::filesystem::path& path) { + std::filesystem::path publishPath = path; + std::error_code statusEc; + const auto linkStatus = std::filesystem::symlink_status(path, statusEc); + if (statusEc && linkStatus.type() != std::filesystem::file_type::not_found) { + return std::unexpected(write_error(std::format( + "cannot inspect compile database '{}': {}", path.string(), + statusEc.message()))); + } + if (linkStatus.type() == std::filesystem::file_type::symlink) { + auto target = std::filesystem::read_symlink(path, statusEc); + if (statusEc) { + return std::unexpected(write_error(std::format( + "cannot resolve compile database link '{}': {}", path.string(), + statusEc.message()))); + } + publishPath = target.is_absolute() ? target : path.parent_path() / target; + } + + std::optional existing; + std::ifstream input(publishPath, std::ios::binary); + if (input) { + std::stringstream ss; + ss << input.rdbuf(); + if (input.bad()) { + return std::unexpected(write_error(std::format( + "cannot read existing compile database '{}'", publishPath.string()))); + } + existing = ss.str(); + } else { + std::error_code existsEc; + auto exists = std::filesystem::exists(publishPath, existsEc); + if (existsEc) { + return std::unexpected(write_error(std::format( + "cannot inspect existing compile database '{}': {}", + publishPath.string(), existsEc.message()))); + } + if (exists) { + return std::unexpected(write_error(std::format( + "cannot read existing compile database '{}'", publishPath.string()))); + } + } + return ExistingDocument{std::move(publishPath), std::move(existing)}; +} + +std::expected +atomic_replace_document(const std::filesystem::path& publishPath, + const std::string& content, const ReplaceFile& replaceFile) { + static std::atomic sequence{0}; + const auto nonce = std::random_device{}(); + // 临时文件和链接目标同目录，避免 rename 跨文件系统；随机量降低跨进程碰撞概率。 + auto temp = publishPath.parent_path() + / std::format(".{}.tmp.{}.{}.{}", publishPath.filename().string(), + std::chrono::steady_clock::now().time_since_epoch().count(), + static_cast(nonce), + sequence.fetch_add(1, std::memory_order_relaxed)); + auto cleanup_temp = [&] { + std::error_code cleanupEc; + std::filesystem::remove(temp, cleanupEc); + }; + + std::ofstream output(temp, std::ios::binary | std::ios::trunc); + if (!output) { + return std::unexpected(write_error(std::format( + "cannot open temporary compile database '{}'", temp.string()))); + } + output << content; + output.flush(); + if (!output) { + output.close(); + cleanup_temp(); + return std::unexpected(write_error(std::format( + "cannot write temporary compile database '{}'", temp.string()))); + } + output.close(); + if (!output) { + cleanup_temp(); + return std::unexpected(write_error(std::format( + "cannot close temporary compile database '{}'", temp.string()))); + } + + std::error_code ec; + if (!replaceFile(temp, publishPath, ec)) { + cleanup_temp(); + return std::unexpected(write_error(std::format( + "cannot replace '{}': {}", publishPath.string(), ec.message()))); + } + return {}; +} + } // namespace std::vector unit_asm_flags(const CompileUnit& cu) { @@ -211,6 +435,14 @@ std::vector unit_invocations(const BuildPlan& plan, std::vector out; out.reserve(plan.compileUnits.size()); + // C4: the driver's own suffix→language table is private and version- + // dependent (BmiTraits::moduleInterfaceLangFlag's note on why the build + // states it unconditionally instead of trusting a driver to infer it); + // this record states the same flag at the same position, so a reader + // that replays `arguments` verbatim reads a module-extension source the + // way the build did rather than guessing from its name. + const auto traits = mcpp::toolchain::bmi_traits(plan.toolchain); + for (auto& cu : plan.compileUnits) { // NASM units carry a command line no CDB consumer (clangd, …) can // interpret — a bogus entry actively harms LSP diagnostics, so they @@ -228,7 +460,12 @@ std::vector unit_invocations(const BuildPlan& plan, UnitInvocation inv; inv.unit = &cu; inv.output = native_string(plan.outputDir / cu.object); - inv.directory = native_string(plan.projectRoot); + // C3: every project unit runs in the output directory — the directory + // ninja actually invokes the compiler in — never the project root. + // The standard-library units keep their own `directory` (the shared + // std cache), which unit_invocations never touches: they are + // synthesised separately (mcpp.build.build_database::render). + inv.directory = native_string(plan.outputDir); inv.file = native_string(cu.source); // Build arguments array. @@ -240,6 +477,18 @@ std::vector unit_invocations(const BuildPlan& plan, for (auto& f : isGasSource ? mcpp::manifest::flag_words(unit_asm_flags(cu)) : package_flag_args(cu, isCSource)) inv.arguments.push_back(std::move(f)); + // C4: immediately before `-c`, since the GNU dialects read the flag + // positionally (BmiTraits::moduleInterfaceLangFlag). Only a unit that + // actually provides a module needs it — an implementation unit (a + // plain source, or `module M;` inside a module-extension file) is + // not the "which language is this extension" ambiguity the flag + // removes. MSVC's form (`/interface /TP`) waits for a Windows + // measurement of clang-cl-mode clangd (design §3.4, "Open"). + if (!cu.providesModule.empty() + && plan.toolchain.compiler != mcpp::toolchain::CompilerId::MSVC) { + for (auto& w : split_flags(traits.moduleInterfaceLangFlag)) + inv.arguments.push_back(std::move(w)); + } inv.arguments.push_back("-c"); inv.arguments.push_back(inv.file); inv.arguments.push_back("-o"); @@ -266,6 +515,24 @@ std::string emit_compile_commands(const BuildPlan& plan, const CompileFlags& fla entries.push_back(std::move(entry)); } + // D5a, S1-12-1: the standard-library units are translation units of this + // build like any other, recovered once onto the plan (prepare.cppm, + // BuildPlan::stdModuleUnits) from the SAME command + // mcpp.build.build_database's S1 rendering uses, so the two documents + // cannot list different arguments for the same unit (P1). + for (auto const& unit : plan.stdModuleUnits) { + nlohmann::json args = nlohmann::json::array(); + for (auto const& a : unit.arguments) args.push_back(a); + + nlohmann::json entry; + entry["directory"] = native_string(unit.workDirectory); + entry["file"] = native_string(unit.source); + entry["arguments"] = std::move(args); + entry["output"] = native_string(unit.object); + + entries.push_back(std::move(entry)); + } + return entries.dump(2) + "\n"; } @@ -277,38 +544,29 @@ std::string merge_compile_commands( if (freshJ.is_discarded() || !freshJ.is_array()) return std::string(fresh); - // Dedup key = the file's PATH, spelled the way a fresh plan spells it - // (native separators). A prior CDB written before the mixed-separator - // fix (#390) carries `root\generated/modules\x.cppm` entries that are - // the SAME file as the fresh `root\generated\modules\x.cppm` — a literal - // string comparison would keep both and the user's upgrade would not - // visibly fix anything. Normalizing makes the merge self-healing: the - // stale mixed entry is skipped on the first `mcpp build` after upgrade. - // fileExists still probes the raw spelling — Windows accepts both. - auto norm_key = [](std::string_view f) { - auto p = std::filesystem::path(std::string(f)).lexically_normal(); - p.make_preferred(); - return p.string(); - }; - // Files the current plan already covers — those entries are authoritative. + // The key resolves `file` against `directory` (resolve_against_directory): + // every path mcpp writes is already absolute, so this is a no-op for a + // fresh entry, but it is what makes an EXISTING entry from before this + // resolution rule (or, in principle, one with a relative `file`) compare + // correctly against it. std::set freshFiles; for (auto const& e : freshJ) { - if (e.contains("file") && e["file"].is_string()) - freshFiles.insert(norm_key(e["file"].get())); + auto resolved = resolve_against_directory(e, "file"); + if (!resolved.empty()) freshFiles.insert(dedup_key(resolved)); } // Keep fresh order, then append still-valid prior entries the plan doesn't // cover (e.g. tests/ from a previous `mcpp test`). Drop entries for files - // that no longer exist so the CDB never accrues dead references. + // that no longer exist so the database never accrues dead references. nlohmann::json merged = freshJ; auto existingJ = nlohmann::json::parse(existing, nullptr, /*allow_exceptions=*/false); if (!existingJ.is_discarded() && existingJ.is_array()) { for (auto const& e : existingJ) { - if (!e.contains("file") || !e["file"].is_string()) continue; - auto f = e["file"].get(); - if (freshFiles.contains(norm_key(f))) continue; // fresh wins - if (!fileExists(std::filesystem::path(f))) continue; // pruned + auto resolved = resolve_against_directory(e, "file"); + if (resolved.empty()) continue; + if (freshFiles.contains(dedup_key(resolved))) continue; // fresh wins + if (!fileExists(resolved)) continue; // pruned merged.push_back(e); } } @@ -330,64 +588,17 @@ publish_compile_commands( "fresh compile database for '{}' is not a JSON array", path.string()))); } - std::filesystem::path publishPath = path; - std::error_code statusEc; - const auto linkStatus = std::filesystem::symlink_status(path, statusEc); - // A first build has no prior CDB to inspect. symlink_status reports the - // missing path with type()==not_found on libstdc++/libc++/MSVC (the error - // code category differs: generic ENOENT vs system ERROR_FILE_NOT_FOUND), - // so that is the normal fresh-workspace case, not an error. - if (statusEc && linkStatus.type() != std::filesystem::file_type::not_found) { - return std::unexpected(write_error(std::format( - "cannot inspect compile database '{}': {}", path.string(), - statusEc.message()))); - } - const bool isLink = linkStatus.type() == std::filesystem::file_type::symlink; - if (isLink) { - auto target = std::filesystem::read_symlink(path, statusEc); - if (statusEc) { - return std::unexpected(write_error(std::format( - "cannot resolve compile database link '{}': {}", path.string(), - statusEc.message()))); - } - publishPath = target.is_absolute() ? target : path.parent_path() / target; - } - - std::optional existing; - { - std::ifstream input(publishPath, std::ios::binary); - if (input) { - std::stringstream ss; - ss << input.rdbuf(); - if (input.bad()) { - return std::unexpected(write_error(std::format( - "cannot read existing compile database '{}'", publishPath.string()))); - } - existing = ss.str(); - } else { - std::error_code existsEc; - auto exists = std::filesystem::exists(publishPath, existsEc); - if (existsEc) { - return std::unexpected(write_error(std::format( - "cannot inspect existing compile database '{}': {}", - publishPath.string(), existsEc.message()))); - } - if (exists) { - return std::unexpected(write_error(std::format( - "cannot read existing compile database '{}'", publishPath.string()))); - } - } - } // input closed before the atomic replace: Windows cannot replace an open file. + auto doc = read_existing_document(path); + if (!doc) return std::unexpected(doc.error()); // 完全相同的有效输入不重写文件，避免 clangd 因 mtime 变化重复索引。 - if (existing && *existing == fresh) { - return CompileCommandsWriteResult{false, freshJson.size()}; - } + if (doc->content && *doc->content == fresh) + return CompileCommandsWriteResult{false, freshJson.size(), 0}; std::string content(fresh); - if (existing) { + if (doc->content) { // 保留仍存在但当前 plan 未覆盖的条目，主要是之前 test 生成的 TU。 - content = merge_compile_commands(content, *existing, fileExists); + content = merge_compile_commands(content, *doc->content, fileExists); } auto finalJson = nlohmann::json::parse(content, nullptr, /*allow_exceptions=*/false); @@ -398,58 +609,75 @@ publish_compile_commands( sort_entries_by_file(finalJson); content = finalJson.dump(2) + "\n"; - if (existing && *existing == content) { - return CompileCommandsWriteResult{false, finalJson.size()}; - } + if (doc->content && *doc->content == content) + return CompileCommandsWriteResult{false, finalJson.size(), 0}; - static std::atomic sequence{0}; - const auto nonce = std::random_device{}(); - // 临时文件和链接目标同目录，避免 rename 跨文件系统；随机量降低跨进程碰撞概率。 - auto temp = publishPath.parent_path() - / std::format(".{}.tmp.{}.{}.{}", publishPath.filename().string(), - std::chrono::steady_clock::now().time_since_epoch().count(), - static_cast(nonce), - sequence.fetch_add(1, std::memory_order_relaxed)); - auto cleanup_temp = [&] { - std::error_code cleanupEc; - std::filesystem::remove(temp, cleanupEc); - }; + if (auto r = atomic_replace_document(doc->publishPath, content, replaceFile); !r) + return std::unexpected(r.error()); - std::ofstream output(temp, std::ios::binary | std::ios::trunc); - if (!output) { - return std::unexpected(write_error(std::format( - "cannot open temporary compile database '{}'", temp.string()))); - } - output << content; - output.flush(); - if (!output) { - output.close(); - cleanup_temp(); - return std::unexpected(write_error(std::format( - "cannot write temporary compile database '{}'", temp.string()))); + return CompileCommandsWriteResult{true, finalJson.size(), 0}; +} + +std::expected +publish_root_compile_commands( + const std::filesystem::path& configDbPath, + const std::filesystem::path& rootPath, + const std::filesystem::path& targetRoot, + const std::filesystem::path& mcppHome, + ReplaceFile replaceFile) { + std::string content; + { + std::ifstream input(configDbPath, std::ios::binary); + if (!input) { + return std::unexpected(write_error(std::format( + "cannot read configuration compile database '{}'", configDbPath.string()))); + } + std::stringstream ss; + ss << input.rdbuf(); + if (input.bad()) { + return std::unexpected(write_error(std::format( + "cannot read configuration compile database '{}'", configDbPath.string()))); + } + content = ss.str(); } - output.close(); - if (!output) { - cleanup_temp(); + + auto configJson = nlohmann::json::parse(content, nullptr, /*allow_exceptions=*/false); + if (configJson.is_discarded() || !configJson.is_array()) { return std::unexpected(write_error(std::format( - "cannot close temporary compile database '{}'", temp.string()))); + "configuration compile database '{}' is not a JSON array", + configDbPath.string()))); } - std::error_code ec; - if (!replaceFile(temp, publishPath, ec)) { - cleanup_temp(); - return std::unexpected(write_error(std::format( - "cannot replace '{}': {}", publishPath.string(), ec.message()))); + auto doc = read_existing_document(rootPath); + if (!doc) return std::unexpected(doc.error()); + + // §3.2 item 2: identical → untouched, so clangd is not triggered for + // nothing (and the fast path costs one read, not one write, on a hit). + if (doc->content && *doc->content == content) + return CompileCommandsWriteResult{false, configJson.size(), 0}; + + // §3.2 item 3: count the replaced file's foreign entries BEFORE replacing + // it, so the caller can warn with a number. A prior file this parser + // cannot read as a JSON array at all (another tool's own format, or one + // corrupted) is replaced the same way, silently — there is no entry + // count to report for a document that was never a list of entries. + std::size_t foreign = 0; + if (doc->content) { + auto priorJson = nlohmann::json::parse(*doc->content, nullptr, false); + if (!priorJson.is_discarded() && priorJson.is_array()) { + for (auto const& e : priorJson) + if (!is_mcpps_entry(e, targetRoot, mcppHome)) ++foreign; + } } - return CompileCommandsWriteResult{true, finalJson.size()}; + if (auto r = atomic_replace_document(doc->publishPath, content, replaceFile); !r) + return std::unexpected(r.error()); + + return CompileCommandsWriteResult{true, configJson.size(), foreign}; } std::expected write_compile_commands(const BuildPlan& plan, const CompileFlags& flags) { - auto path = plan.compileDbPath.empty() - ? plan.projectRoot / "compile_commands.json" - : plan.compileDbPath; // A JSON document holds UTF-8 text only, and the serialiser throws on any // other byte (`type_error.316`). The entry points refuse or skip a path // with no UTF-8 spelling (#693), so a string reaching this point in another @@ -464,11 +692,37 @@ write_compile_commands(const BuildPlan& plan, const CompileFlags& flags) { "it cannot be written as JSON, which holds UTF-8 text only ({})", e.what()))); } - return publish_compile_commands( - path, fresh, - [](const std::filesystem::path& candidate) { - return std::filesystem::exists(candidate); - }); + + auto fileExists = [](const std::filesystem::path& candidate) { + return std::filesystem::exists(candidate); + }; + + // §3.2 item 1: the configuration's own database, merged within itself. + auto configPath = plan.outputDir / "compile_commands.json"; + auto configResult = publish_compile_commands(configPath, fresh, fileExists); + if (!configResult) return configResult; + + // §3.2 items 2-3: the root is a copy of it, replaced whole. `emit` + // (`mcpp emit build-database`) never reaches this function (SPEC-005 + // R2.1: it writes nothing into the project), so in practice + // `plan.compileDbPath` is always the project root's file here; the + // fallback exists because the field itself is more general (plan.cppm). + auto rootPath = plan.compileDbPath.empty() + ? plan.projectRoot / "compile_commands.json" + : plan.compileDbPath; + auto targetRoot = plan.outputDir.parent_path().parent_path(); + auto rootResult = publish_root_compile_commands( + configPath, rootPath, targetRoot, mcpp::home::root()); + if (!rootResult) return rootResult; + + return CompileCommandsWriteResult{ + rootResult->changed, configResult->commandCount, rootResult->foreignEntries}; +} + +std::string foreign_entries_warning(std::size_t n) { + return std::format( + "compile_commands.json held {} entr{} mcpp did not write; " + "the file now holds mcpp's configuration", n, n == 1 ? "y" : "ies"); } } // namespace mcpp::build diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 6694eec60..476c25345 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -11,6 +11,7 @@ export module mcpp.build.execute; import std; import mcpp.build.build_program; // #359 glob inputs the mtime sweep cannot see +import mcpp.build.compile_commands; // C1: the fast path restores a deleted root CDB import mcpp.build.prepare; import mcpp.pack; // #622 A10: mcpp::pack::Options / Format import mcpp.pack.pipeline; // #622 A10: build_and_pack, for `run --format` @@ -1405,6 +1406,34 @@ export int list_runners(const std::string& package_filter, return 0; } +// C1: a deleted root compile_commands.json used to stay deleted forever on +// the fast path, because nothing on it ever reached a writer (the fast path +// is defined as "skip preparation" — see the hooksActive check below, and +// design .agents/docs/2026-09-26-compile-database-and-issue-699-design.md +// §3.2 item 4). This restores it from the configuration's own database, +// already on disk at `outputDir` — no plan is built, so P3 (the fast path +// replays a build) holds: the root file is a copy, never a fresh plan. +// Errors are reported as warnings and never fail the fast build itself: a +// permission problem here is exactly what a normal build would already warn +// about (`write_compile_commands`), not a reason to fall back to the full +// path. +void restore_root_compile_commands(const std::filesystem::path& projectRoot, + const std::filesystem::path& outputDir) { + auto configPath = outputDir / "compile_commands.json"; + std::error_code ec; + if (!std::filesystem::exists(configPath, ec) || ec) return; + auto rootPath = projectRoot / "compile_commands.json"; + auto targetRoot = outputDir.parent_path().parent_path(); + auto result = mcpp::build::publish_root_compile_commands( + configPath, rootPath, targetRoot, mcpp::home::root()); + if (!result) { + mcpp::ui::warning(std::format( + "compile_commands.json was not updated: {}", result.error().message)); + } else if (result->foreignEntries > 0) { + mcpp::ui::warning(mcpp::build::foreign_entries_warning(result->foreignEntries)); + } +} + export std::optional try_fast_build(const std::filesystem::path& projectRoot, bool verbose, bool no_cache, std::string_view currentTarget = "") { @@ -1519,6 +1548,11 @@ export std::optional try_fast_build(const std::filesystem::path& projectRoo if (!validatedBefore) return std::nullopt; // All inputs are older than build.ninja → fast-path: just run ninja. + // C1: this configuration is confirmed current, so the root database is + // restored here rather than left to whatever a NEXT full build happens to + // do — a project that never edits a source again would otherwise never + // see it back. + restore_root_compile_commands(projectRoot, outputDir); std::chrono::milliseconds elapsed{}; auto rc = run_ninja_fast(ninjaProgram, outputDir, ninjaPath, verbose, runtimeEnvKey, runtimeEnvValue, &elapsed); @@ -1667,7 +1701,8 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, if (!validatedBefore) return std::nullopt; // Fresh → run ninja (picks up any incremental object/link work) then - // exec the cached exe path directly. + // exec the cached exe path directly. C1, same reason as try_fast_build's. + restore_root_compile_commands(projectRoot, outputDir); auto rc = run_ninja_fast(ninjaProgram, outputDir, ninjaPath, /*verbose=*/false, match->runtimeEnvKey, match->runtimeEnvValue); if (!rc) return std::nullopt; diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 90673ed4d..9b21fdccf 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -364,24 +364,30 @@ std::string escape_path(const std::filesystem::path& p) { return escape_ninja_chars(p.string()); } -std::string normalize_ldflag(const std::filesystem::path& root, const std::string& flag) { +// One WORD of the link-flag list, with a package-relative search path made +// absolute against the package root. The result is the word the linker +// receives, not ninja text: the caller quotes it for the host and escapes it +// for ninja (`ninja_command_word`). Until #703 this function escaped the +// element for ninja only, so the `sh` that runs a POSIX link expanded a +// `$ORIGIN` the author wrote, and the program's run path held `/../lib`. +std::string normalize_ldflag(const std::filesystem::path& root, const std::string& word) { auto absolute_path = [&](std::string_view raw) { std::filesystem::path p{std::string(raw)}; if (p.is_absolute() || is_loader_relative_search_path(raw)) return p; return root / p; }; - if (flag.starts_with("-L") && flag.size() > 2) { - return "-L" + escape_path(absolute_path(std::string_view(flag).substr(2))); + if (word.starts_with("-L") && word.size() > 2) { + return "-L" + absolute_path(std::string_view(word).substr(2)).string(); } constexpr std::string_view rpathPrefix = "-Wl,-rpath,"; - if (flag.starts_with(rpathPrefix) && flag.size() > rpathPrefix.size()) { + if (word.starts_with(rpathPrefix) && word.size() > rpathPrefix.size()) { return std::string(rpathPrefix) - + escape_path(absolute_path(std::string_view(flag).substr(rpathPrefix.size()))); + + absolute_path(std::string_view(word).substr(rpathPrefix.size())).string(); } - return flag; + return word; } } // namespace @@ -1046,11 +1052,17 @@ CompileFlags compute_flags(const BuildPlan& plan) { plan.manifest.buildConfig.cxxRuntime))); } - // User link flags + // User link flags: `[build] ldflags`, the `link_flag` and `link_lib` + // directives, and what the dependencies propagate. SPEC-004 §8's reading + // applies to them as to the compile flags (#703): each element is read + // into words, and each word reaches the linker verbatim, quoted for the + // host and then escaped for ninja. A `$ORIGIN` therefore reaches the + // linker as written, and an element that packs several tokens is several + // words on every host. std::string user_ldflags; - for (auto const& flag : plan.manifest.buildConfig.ldflags) { + for (auto const& word : mcpp::manifest::flag_words(plan.manifest.buildConfig.ldflags)) { user_ldflags += ' '; - user_ldflags += normalize_ldflag(plan.projectRoot, flag); + user_ldflags += ninja_command_word(normalize_ldflag(plan.projectRoot, word)); } // C standard. The file-level `$cflags` carries the engine default, a diff --git a/src/build/hostprogram.cppm b/src/build/hostprogram.cppm index aecbf8398..5d47b8d1f 100644 --- a/src/build/hostprogram.cppm +++ b/src/build/hostprogram.cppm @@ -183,6 +183,41 @@ inline void windows_entry(const char* target, const char* value) { inline void deploy(const char* from, const char* to) { std::printf("mcpp:deploy=%s\t%s\n", from, to); } +// The build-program form of `runtime_search_dirs` (docs/04 §2.11): a +// directory to search at LAUNCH time, for a dependency (a vcpkg prefix's +// `bin/`, a Qt SDK's `bin/`, a directory a `prepare` action populates) whose +// location this program learns rather than one an author can write into +// `mcpp.toml`. Reaches the consumer, joining the SAME `LinkIntent` field the +// manifest key populates directly -- not the retiring `[runtime] +// library_dirs` -- and gets the same treatment: RUNPATH/rpath on ELF and +// Mach-O, never `-L`, and `mcpp pack`'s closure search. Relative paths +// resolve against this package's root, like every other AbsPath directive. +// The directory need not exist when this program runs: a `prepare` action +// (below) may populate it later, at build time. +inline void runtime_search_dir(const char* dir) { + std::printf("mcpp:runtime-search-dir=%s\n", dir); +} +// The five action roles (mcpp#702), as CONSTANTS rather than bare string +// literals: `a.role = mcpp::roles::prepare` fails to COMPILE on an engine +// whose bundled module has no such name, instead of reaching that engine's +// decoder as the plain string "prepare", which an engine older than protocol +// 12 reads as Source (see `decode_action`, modules/buildmcpp/src/ +// directives.cppm). Spelled inside THIS module, not the engine's own headers, +// because these names have to be visible from a build.mcpp translation unit, +// which imports this module and nothing else of the engine's. SPEC-007 R3.6 +// makes writing the constant the author's obligation; the engine still +// accepts the bare string from a hand-written frozen-surface program (protocol +// 0), and refuses one that names none of the five (`action_error`). +namespace roles { + inline constexpr const char* source = "source"; + inline constexpr const char* check = "check"; + inline constexpr const char* object = "object"; + inline constexpr const char* artifact = "artifact"; + // Construction whose file names are not known when this program runs: + // installing a vcpkg manifest or a CMake subproject into a prefix, + // unpacking an SDK. See `action::output_dir` below for its contract. + inline constexpr const char* prepare = "prepare"; +} // ── Build-graph nodes (mcpp 2026.8.5.1+) ──────────────────────────────── // Declare WORK instead of doing it. A build program is a good place to decide // what the build looks like and a bad place to perform it: work done here is @@ -194,7 +229,7 @@ inline void deploy(const char* from, const char* to) { // cannot be built. Content may arrive later; names may not. struct action { const char* id = ""; - const char* role = "source"; // "source" | "check" | "object" | "artifact" + const char* role = roles::source; // one of `roles::{source, check, object, artifact, prepare}` const char* description = ""; bool blocking = false; // check only: gate compilation on it // A Make-style dependency file the COMMAND writes as a side effect (gcc/ @@ -219,6 +254,15 @@ struct action { // its target out of ${mcpp.target_file:NAME}; an Object runs before the link // and has no such handle, so it has to say the name. action& target(const char* n) { add(targets_, n); return *this; } + // `prepare` only: the directory the COMMAND populates, which the build + // reads BY DIRECTORY (`include_dir`, `link_search`, `runtime_search_dir`) + // rather than by naming files -- the whole reason this role exists (O1, + // mcpp#702). One directory, not a list like `output()`: the post-condition + // the engine checks after the command succeeds (does this directory now + // exist, whether or not the command created it) is a single answer, and a + // `prepare` action with no `output_dir` is refused (`action_error`) as a + // `check` that forgot to declare what it built. + action& output_dir(const char* p) { outputDir_ = p; return *this; } void submit() const { std::printf("mcpp:action={\"id\":"); esc(id); std::printf(",\"role\":"); esc(role); @@ -232,6 +276,10 @@ struct action { // has nothing to do with depfiles. The decoder's default (empty // string) is identical either way, so omission costs nothing on read. if (depfile[0]) { std::printf(",\"depfile\":"); esc(depfile); } + // Same omission rule as `depfile`, for the same reason: an action + // that never calls `output_dir()` serialises to the same bytes it did + // before the method existed. + if (outputDir_[0]) { std::printf(",\"output_dir\":"); esc(outputDir_); } // Set only when the process could not allocate memory for a list. // A declaration cut short would otherwise be INVALID rather than // obviously wrong -- the engine turns this marker into a diagnostic @@ -295,6 +343,9 @@ private: } }; list inputs_, outputs_, command_, provides_, imports_, targets_; + // `prepare` only: see `output_dir()` above. A plain `const char*`, not a + // `list`: it is one directory, never a JSON array. + const char* outputDir_ = ""; mutable bool overflow_ = false; static void esc(const char* s) { std::putchar('"'); diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 2d260ee75..9eeb6c1c4 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -673,11 +673,17 @@ runtime_env_for_dirs(const std::vector& dirs) { // `Object` outputs are link inputs and `Artifact` inputs are link outputs, so // ninja's file dependencies already order both. `Source` may produce a header, -// which is never an edge input, and a `Check` gates compilation exactly when it -// says it does — the `blocking` flag, which nothing read before mcpp#534. +// which is never an edge input, a `Check` gates compilation exactly when it +// says it does — the `blocking` flag, which nothing read before mcpp#534 — and +// a `Prepare` (mcpp#702) gates it UNCONDITIONALLY: the build reads its +// declared directory by directory (`include_dir`, `link_search`, +// `runtime_search_dir`), which a compile edge cannot express as a file +// dependency any more than a `Source` header can, so it needs the same +// order-only edge every time, with no opt-in flag of its own. bool action_precedes_compilation(const mcpp::manifest::BuildAction& a) { return a.role == mcpp::manifest::BuildAction::Role::Source - || (a.role == mcpp::manifest::BuildAction::Role::Check && a.blocking); + || (a.role == mcpp::manifest::BuildAction::Role::Check && a.blocking) + || a.role == mcpp::manifest::BuildAction::Role::Prepare; } // A phony per package. Not a path — ninja resolves it in the build dir, and no @@ -690,6 +696,27 @@ std::string action_phony_name(std::string_view pkg) { return s; } +// Packages whose `prepare` action gates every LINK edge in the plan, not only +// their own compile edges (mcpp#702, design §5.4 P). `plan.linkIntent` — and +// therefore the `$ldflags` every link rule interpolates, `runtime_search_dir` +// included — is ONE value merged from every package's LinkGlobal directives +// (`resolve_runtime_contract`, plan.cppm), read by every link unit in this +// plan alike: the declaring package's own link, and the link of every package +// that depends on it, which in mcpp's flat per-plan merge is indistinguishable +// from "every link unit this plan emits". A directory a `prepare` action +// populates therefore has to exist before ANY such link runs, not only the +// declaring package's. File-scope for the same reason +// `action_precedes_compilation`/`action_phony_name` are: the emitter and +// `check_action_ordering` ask the identical question. +std::set packages_with_prepare_actions(const BuildPlan& plan) { + std::set pkgs; + for (auto const& a : plan.actions) + if (a.role == mcpp::manifest::BuildAction::Role::Prepare + && !a.packageName.empty() && !a.outputs.empty()) + pkgs.insert(a.packageName); + return pkgs; +} + } // namespace // WHAT `windows_subsystem` AND `windows_entry` RENDER TO (#618). @@ -1887,6 +1914,33 @@ std::string emit_ninja_string(const BuildPlan& plan) { append(" command = $mcpp coff-def --output $out --name $def_name $in\n"); append(" description = DEF $out\n\n"); + // A WINDOWS PROGRAM'S RUNTIME DLLS, PLACED AFTER ITS LINK (SPEC-007 R4.3). + // A PE image has no run path, so a DLL in a runtime search directory + // serves `mcpp run` (through `PATH`) and `mcpp pack`, and not a program + // started by hand from the build directory. `mcpp place-dlls` reads the + // program's import closure as `mcpp pack` does and publishes each DLL it + // resolves in those directories beside the program. The names are not + // known when this graph is written, because an action may populate the + // directory, so the edge's output is a stamp and its depfile names the + // DLLs it placed: a DLL replaced in its directory is placed again on the + // next build. ELF and Mach-O keep their run paths, and a plan without + // runtime search directories has no such edge. + const bool placeRuntimeDlls = [&] { + if (plan.linkIntent.runtimeSearchDirs.empty()) return false; + const auto t = mcpp::toolchain::triple::parse(plan.toolchain.targetTriple); + return t ? t->is_pe() : bool(mcpp::platform::is_windows); + }(); + if (placeRuntimeDlls) { + std::string dirs; + for (auto const& d : plan.linkIntent.runtimeSearchDirs) + dirs += " " + ninja_command_word(d.string()); + append("rule place_dlls\n"); + append(" command = $mcpp place-dlls --output $out --depfile $out.d $in" + dirs + "\n"); + append(" depfile = $out.d\n"); + append(" deps = gcc\n"); + append(" description = DLLS $in\n\n"); + } + append("rule runtime_alias\n"); if constexpr (mcpp::platform::is_windows) { // PE has no soname symlink, so the alias is a copy — and a copy of a @@ -2267,6 +2321,15 @@ std::string emit_ninja_string(const BuildPlan& plan) { : stagedOrderOnly + " " + phony; }; + // Order-only suffix for every LINK edge (mcpp#702, design §5.4 P): the + // phony of every package whose `prepare` action gates the whole plan's + // link, per `packages_with_prepare_actions` above. `ar` never reads + // `$ldflags`, so a static archive is not one of the edges this reaches — + // applied at the link-unit loop below, keyed on `lu.kind`. + std::string linkOrderOnly; + for (auto const& pkg : packages_with_prepare_actions(plan)) + linkOrderOnly += " " + action_phony_name(pkg); + if (dyndep) { // ── Phase 1: scan edges (one .ddi per TU). ────────────────────── // .ddi is placed beside the object so multi-version mangling can @@ -2681,13 +2744,19 @@ std::string emit_ninja_string(const BuildPlan& plan) { for (auto& input : lu.implicitInputs) { implicit += " " + escape_ninja_path(input); } - // Windows runtime-DLL deployment: an executable takes an implicit + // Windows runtime-DLL deployment: an executable takes an ORDER-ONLY // dependency on each staged dep DLL (bin/), so ninja copies them - // beside the .exe before the build is considered done. Empty on RPATH - // platforms (no *.dll deps), so other targets are unaffected. + // beside the .exe whenever the .exe is built. Order-only rather than + // implicit, because the linker never reads a deployed DLL: it links + // the import library. As an implicit input, a DLL that changed, or a + // deploy entry that a later plan added (a runtime search directory an + // action populated after the first plan), relinked a program whose + // link inputs had not changed. Empty on RPATH platforms (no *.dll + // deps), so other targets are unaffected. + std::string orderOnly; if (lu.kind == LinkUnit::Binary || lu.kind == LinkUnit::TestBinary) { for (auto const& d : deployFiles) - implicit += " " + escape_ninja_path(d.dest); + orderOnly += " " + escape_ninja_path(d.dest); } // The import library is a SECOND output of this edge, declared as an @@ -2740,9 +2809,15 @@ std::string emit_ninja_string(const BuildPlan& plan) { if (!lu.defFile.empty()) implicit += " " + escape_ninja_path(lu.defFile); - std::string out_line = std::format("build {}{} : {}{}{}\n", + // `ar` (StaticLibrary) never reads `$ldflags`, so it never reads a + // `runtime_search_dir` either — see `linkOrderOnly`, above. The + // deployed DLLs (`orderOnly`) and the `prepare` phonies share the one + // order-only list of the edge. + if (lu.kind != LinkUnit::StaticLibrary) orderOnly += linkOrderOnly; + std::string out_line = std::format("build {}{} : {}{}{}{}\n", escape_ninja_path(lu.output), implicitOut, rule, ins, - implicit.empty() ? std::string{} : " |" + implicit); + implicit.empty() ? std::string{} : " |" + implicit, + orderOnly.empty() ? std::string{} : " ||" + orderOnly); if (auto flag = shared_soname_flag(lu, plan); !flag.empty()) out_line += " soname_flag = " + flag + "\n"; if (auto flag = shared_soname_default(lu, plan); !flag.empty()) @@ -2863,6 +2938,29 @@ std::string emit_ninja_string(const BuildPlan& plan) { append("default " + elf + ".bin\n\n"); } + // The placement edge (see `place_dlls` above), after the link it reads. + // + // The stamps of the graph's `prepare` actions are implicit inputs. + // A `prepare` command rewrites files ninja does not know as outputs, + // so a DLL it replaces in a runtime search directory is newer than + // the depfile's record only after ninja has already decided what is + // dirty; without the stamps, the new DLL would be placed one build + // late. With them, the edge runs again in the build that ran the + // action, after it. + if (placeRuntimeDlls + && (lu.kind == LinkUnit::Binary || lu.kind == LinkUnit::TestBinary)) { + const auto exe = escape_ninja_path(lu.output); + std::string prepareStamps; + for (auto const& a : plan.actions) + if (a.role == mcpp::manifest::BuildAction::Role::Prepare) + for (auto const& o : a.outputs) + prepareStamps += " " + escape_ninja_path(o); + append("build " + exe + ".dlls: place_dlls " + exe + + (prepareStamps.empty() ? std::string{} : " |" + prepareStamps) + + "\n"); + append("default " + exe + ".dlls\n\n"); + } + for (auto const& alias : lu.runtimeAliases) { append(std::format("build {} : runtime_alias {}\n", escape_ninja_path(alias), @@ -2941,7 +3039,10 @@ std::string emit_ninja_string(const BuildPlan& plan) { // nothing on success — and the obvious workaround, a wrapper script, // cannot be written portably because an action's command is an argv // with no shell assumed. So the engine wraps it: mcpp is already on - // disk on every platform it runs on. + // disk on every platform it runs on. A PREPARE (mcpp#702) shares the + // wrapper: its command populates a directory, not the stamp, and the + // wrapper's post-condition is what checks that directory exists once + // the command has succeeded (R3.2/R3.3) — `--require-dir`, below. // // Prepended rather than appended, and the original command is passed // through unchanged after `--`, so a command that DOES write its own @@ -2951,7 +3052,8 @@ std::string emit_ninja_string(const BuildPlan& plan) { // action with none at parse time — and is kept because this loop reads // as if it could see one, and a wrapper with no stamp to write would // be a command that swallows its own exit code. - if (a.role == mcpp::manifest::BuildAction::Role::Check + if ((a.role == mcpp::manifest::BuildAction::Role::Check + || a.role == mcpp::manifest::BuildAction::Role::Prepare) && !a.outputs.empty()) { // `mcpp_exe_path()`, not `self_exe_path()` directly: this file // already has one spelling of "where am I" and a second would be @@ -2965,6 +3067,14 @@ std::string emit_ninja_string(const BuildPlan& plan) { std::string wrapped = shell_quote_arg(escape_ninja_chars(mcpp_exe_path().string())) + " __action-stamp"; + // Before the stamp list, so the wrapper can tell the flag from a + // stamp path without an allowlist of extensions. `directives.cppm` + // refuses a `prepare` action with no `output_dir`, so this is + // reached with a non-empty directory whenever the role is Prepare. + if (a.role == mcpp::manifest::BuildAction::Role::Prepare) { + wrapped += " --require-dir " + + shell_quote_arg(escape_ninja_chars(a.outputDir)); + } for (auto const& o : a.outputs) wrapped += " " + shell_quote_arg(escape_ninja_chars(o)); wrapped += " -- " + cmd; @@ -2974,6 +3084,7 @@ std::string emit_ninja_string(const BuildPlan& plan) { append(std::format(" command = {}\n", cmd)); append(std::format(" description = {} {}\n", a.role == mcpp::manifest::BuildAction::Role::Check ? "CHECK" + : a.role == mcpp::manifest::BuildAction::Role::Prepare ? "PREPARE" : a.role == mcpp::manifest::BuildAction::Role::Artifact ? "ARTIFACT" : a.role == mcpp::manifest::BuildAction::Role::Object ? "OBJECT" : "GENERATE", @@ -3182,6 +3293,58 @@ std::optional check_action_ordering(const std::string& manifest, " This is a build-emitter defect; please report it.", pkg); } + + // mcpp#702, design §5.4 P: every LINK edge must wait for every package's + // `prepare` action — see `packages_with_prepare_actions`'s comment for + // why the set is plan-wide rather than scoped to one package the way the + // compile-edge check above is. `ar` (a StaticLibrary) never reads + // `$ldflags` and is excluded for the same reason the emitter excludes it. + auto preparePkgs = packages_with_prepare_actions(plan); + if (!preparePkgs.empty()) { + std::set linkOutputs; + for (auto const& lu : plan.linkUnits) { + if (lu.kind == LinkUnit::StaticLibrary) continue; + linkOutputs.insert(escape_ninja_path(lu.output)); + } + std::map seenLinks; // link output -> edges checked + for (auto line : manifest | std::views::split('\n')) { + std::string_view l{line.begin(), line.end()}; + if (!l.starts_with("build ")) continue; + auto colon = l.find(" : "); + if (colon == std::string_view::npos) continue; + auto outs = l.substr(6, colon - 6); + for (auto const& obj : linkOutputs) { + if (outs != obj && outs.find(obj) == std::string_view::npos) continue; + ++seenLinks[obj]; + for (auto const& pkg : preparePkgs) { + auto phony = action_phony_name(pkg); + if (l.find(phony) == std::string_view::npos) + return std::format( + "internal: link edge '{}' does not wait for '{}', " + "the phony of package '{}', which declares a " + "`prepare` action.\n" + " A directory that action populates may be " + "named by a `runtime_search_dir` this link reads " + "through the plan's one merged LinkIntent " + "(mcpp#702).\n" + " This is a build-emitter defect; please " + "report it.", + outs, phony, pkg); + } + break; + } + } + for (auto const& obj : linkOutputs) { + if (seenLinks[obj] > 0) continue; + return std::format( + "internal: a linked image '{}' was expected in the manifest " + "while the plan declares a `prepare` action, but no link edge " + "for it was found there — the ordering guard cannot see what " + "it is supposed to check.\n" + " This is a build-emitter defect; please report it.", + obj); + } + } return std::nullopt; } @@ -3484,6 +3647,11 @@ std::expected NinjaBackend::build(const BuildPlan& plan } mcpp::ui::warning(std::format( "compile_commands.json was not updated: {}", cdb.error().message)); + } else if (cdb->foreignEntries > 0) { + // §3.2 item 3: the root file is replaced whole, never merged; when it + // held another writer's entries, say so once rather than silently + // discard them. + mcpp::ui::warning(mcpp::build::foreign_entries_warning(cdb->foreignEntries)); } // A SHARED LIBRARY ON A TARGET WHOSE LINK IS DRIVEN BY THE LINKER, SAID diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 85d82d252..f9f0f68bc 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -224,6 +224,49 @@ StaticPlacement place_static_packages( const std::map& sharedImages, const std::set& boundaries); +// The standard-library units this configuration's build compiles, when it +// imports `std` (design 2026-09-26 §3.5, D5a/D5b). Populated once, in +// prepare.cppm, by recovering the invocation from the command mcpp actually +// ran to build the module into the shared std cache +// (mcpp.toolchain.stdmod::StdModuleDescription) — recovered here, on the +// plan, rather than separately by each renderer, so compile_commands.json, +// `emit --spec compile-commands` and the S1 document cannot list different +// arguments for the same unit (P1, mcpp.build.compile_commands's header). +// Zero, one (`std`) or two (`std`, `std.compat`) entries. +struct StdModuleUnit { + std::filesystem::path source; + std::filesystem::path workDirectory; // the shared std cache, not outputDir + std::vector arguments; // driver first, as recovered + std::filesystem::path object; + std::filesystem::path bmi; // the BMI the build writes (S1 `provides`) + std::string module; // "std" | "std.compat" + std::vector requiresModules; // {} for std, {"std"} for std.compat +}; + +// Splits a command string mcpp rendered for the host shell into words, +// undoing its quoting: POSIX `sh` rules, or the Microsoft C runtime's rules +// on Windows. The reader is mcpp::manifest::host_command_words; this name is +// kept for the standard-library units' recovery below. +std::vector split_command_words(std::string_view command, bool windows); + +// The working directory and argument vector of the command in `commands` +// whose words name `source`, recovered from the rendering: a leading `cd`, an +// `env` word and environment assignments, and redirections are removed. A +// driver path that the rendering left unquoted despite a space is rejoined. +// Empty when no command names the source. Used to populate +// StdModuleUnit::workDirectory/arguments (prepare.cppm) from +// mcpp.toolchain.stdmod::StdModuleDescription's recorded commands. +struct RecoveredInvocation { + std::filesystem::path workDirectory; + std::vector arguments; +}; +std::optional recover_invocation( + const std::vector& commands, + const std::filesystem::path& source, + const std::filesystem::path& driver, + const std::filesystem::path& defaultDirectory, + bool windows); + struct BuildPlan { mcpp::manifest::Manifest manifest; // Packages whose declared `[build] c_standard` the compiler does not apply, @@ -307,6 +350,8 @@ struct BuildPlan { // (possibly read-only) registry directory, and deriving the path would put // an IDE database there. Empty → projectRoot, the historical default. std::filesystem::path compileDbPath; + // See StdModuleUnit above. Empty when the build does not import `std`. + std::vector stdModuleUnits; // GCC only: a specs file that replaces the pristine `*link:`, so the // payload's own (patched by every home that ever installed against it) // cannot inject rpath entries into this build's artifacts. Empty for @@ -2410,4 +2455,108 @@ package_link_objects(const BuildPlan& plan, std::string_view packageName) { return objects; } +namespace { + +// `V=x`: an environment assignment word, the shape mcpp's own `env` prefix +// renders before a driver's argv. Recognized so it can be skipped rather +// than mistaken for the driver or an input path. +bool std_command_is_assignment(std::string_view w) { + auto eq = w.find('='); + if (eq == std::string_view::npos || eq == 0) return false; + if (!(std::isalpha(static_cast(w[0])) || w[0] == '_')) return false; + for (std::size_t i = 1; i < eq; ++i) { + const unsigned char c = static_cast(w[i]); + if (!(std::isalnum(c) || c == '_')) return false; + } + return true; +} + +// `2>&1`, `>file`, `nul`, ``, `2>`, `<`, +// `>>`: the target is the next word. +enum class StdCommandRedirect { None, Attached, Detached }; +StdCommandRedirect std_command_redirect_kind(std::string_view w) { + std::size_t i = 0; + while (i < w.size() && std::isdigit(static_cast(w[i]))) ++i; + if (i >= w.size() || (w[i] != '>' && w[i] != '<')) return StdCommandRedirect::None; + std::size_t j = i + 1; + if (j < w.size() && w[j] == w[i]) ++j; // `>>` + return j == w.size() ? StdCommandRedirect::Detached : StdCommandRedirect::Attached; +} + +bool std_command_names_path(std::string_view word, const std::filesystem::path& path, + const std::filesystem::path& cwd) { + std::filesystem::path w{std::string(word)}; + const auto want = path.lexically_normal(); + if (w.lexically_normal() == want) return true; + if (!w.is_absolute() && !cwd.empty() && (cwd / w).lexically_normal() == want) + return true; + return false; +} + +} // namespace + +std::vector split_command_words(std::string_view s, bool windows) { + return mcpp::manifest::host_command_words(s, windows); +} + +std::optional recover_invocation( + const std::vector& commands, + const std::filesystem::path& source, + const std::filesystem::path& driver, + const std::filesystem::path& defaultDirectory, + bool windows) { + const std::string driverText = driver.string(); + for (auto const& command : commands) { + auto words = split_command_words(command, windows); + std::vector> segments(1); + for (auto& w : words) { + if (w == "&&") segments.emplace_back(); + else segments.back().push_back(std::move(w)); + } + std::filesystem::path cwd; + for (auto& seg : segments) { + if (seg.empty()) continue; + if (seg.front() == "cd") { + std::size_t k = 1; + if (k < seg.size() && (seg[k] == "/d" || seg[k] == "/D")) ++k; + if (k < seg.size()) cwd = std::filesystem::path{seg[k]}; + continue; + } + std::size_t b = 0; + if (b < seg.size() && seg[b] == "env") ++b; + while (b < seg.size() && std_command_is_assignment(seg[b])) ++b; + std::vector argv; + for (std::size_t k = b; k < seg.size(); ++k) { + switch (std_command_redirect_kind(seg[k])) { + case StdCommandRedirect::Attached: continue; + case StdCommandRedirect::Detached: ++k; continue; + case StdCommandRedirect::None: argv.push_back(seg[k]); + } + } + if (argv.empty()) continue; + // A driver path rendered without quotes splits at its spaces. + if (argv.front() != driverText + && driverText.find(' ') != std::string::npos) { + std::string joined = argv.front(); + std::size_t k = 1; + while (k < argv.size() && joined.size() < driverText.size()) { + joined += ' '; + joined += argv[k]; + ++k; + } + if (joined == driverText) { + argv.erase(argv.begin() + 1, argv.begin() + static_cast(k)); + argv.front() = driverText; + } + } + const bool named = std::ranges::any_of(argv, [&](const std::string& w) { + return std_command_names_path(w, source, cwd.empty() ? defaultDirectory : cwd); + }); + if (!named) continue; + return RecoveredInvocation{cwd.empty() ? defaultDirectory : cwd, std::move(argv)}; + } + } + return std::nullopt; +} + } // namespace mcpp::build diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index e40927bfe..aa79fd470 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -100,6 +100,7 @@ import mcpp.pm.lock_io; import mcpp.version_req; import mcpp.ui; import mcpp.log; +import mcpp.wire; // Severity, for PlanNote (#699 item 2, E3) import mcpp.fallback.install_integrity; import mcpp.bmi_cache; import mcpp.project; @@ -587,6 +588,18 @@ std::vector previous_release_words(std::string element, bool define } void report_flag_words_changes(const mcpp::manifest::Manifest& m) { + auto show = [](const std::vector& words) { + std::string out = "["; + for (auto const& w : words) + out += std::format("{}'{}'", out.size() > 1 ? ", " : "", w); + return out + "]"; + }; + auto note_change = [&](std::string what, std::string hint) { + auto note = std::pair{std::move(what), std::move(hint)}; + auto& notes = pending_flag_words_notes(); + if (std::ranges::find(notes, note) == notes.end()) notes.push_back(std::move(note)); + }; + auto const who = m.package.name.empty() ? std::string("(root)") : m.package.name; auto check = [&](std::string_view where, const std::vector& list, bool define) { for (auto const& e : list) { @@ -594,27 +607,45 @@ void report_flag_words_changes(const mcpp::manifest::Manifest& m) { : mcpp::manifest::flag_words(e); auto before = previous_release_words(e, define); if (now == before) continue; - auto show = [](const std::vector& words) { - std::string out = "["; - for (auto const& w : words) - out += std::format("{}'{}'", out.size() > 1 ? ", " : "", w); - return out + "]"; - }; - auto note = std::pair{std::format( + note_change(std::format( "{}: {} element '{}' reaches the compiler as {}; mcpp before " "2026.9.17.1 passed {} on this host", - m.package.name.empty() ? std::string("(root)") : m.package.name, - where, e, show(now), show(before)), + who, where, e, show(now), show(before)), std::string( "a compile-flag element is read by one syntax on every host, and a " "`defines` entry is one value (docs/04-mcpp-toml.md, " "\"Compile-flag syntax\"); spell the element so that it reads as the " - "words meant")}; - auto& notes = pending_flag_words_notes(); - if (std::ranges::find(notes, note) == notes.end()) notes.push_back(std::move(note)); + "words meant")); + } + }; + // THE SAME QUESTION FOR THE LINK FLAGS, which take the reading from + // 2026.9.26.2 (#703). Before, a `-L` or `-Wl,-rpath,` element was escaped + // for ninja, so its text reached the host's reader as written, and any + // other element was pasted into the ninja rule, so ninja replaced its `$` + // sequences first. `$ORIGIN` written plainly reads the same under both + // models, because neither reproduces the shell's expansion that lost it; + // an element escaped for ninja or for the shell by hand is what differs. + auto check_link = [&](std::string_view where, const std::vector& list) { + for (auto const& e : list) { + auto now = mcpp::manifest::flag_words(e); + auto before = e.starts_with("-L") || e.starts_with("-Wl,-rpath,") + ? mcpp::manifest::host_command_words(e, mcpp::platform::is_windows) + : previous_release_words(e, false); + if (now == before) continue; + note_change(std::format( + "{}: {} element '{}' reaches the linker as {}; mcpp before " + "2026.9.26.2 passed {} on this host", + who, where, e, show(now), show(before)), + std::string( + "a link-flag element is read by the compile-flag syntax, so `$ORIGIN` " + "reaches the linker as written and an element escaped for ninja or the " + "shell by hand is no longer unescaped (docs/04-mcpp-toml.md, " + "\"Compile-flag syntax\"); spell the element so that it reads as the " + "words meant")); } }; auto const& bc = m.buildConfig; + check_link("[build] ldflags", bc.ldflags); check("[build] cflags", bc.cflags, false); check("[build] cxxflags", bc.cxxflags, false); check("[build] defines", bc.defines, true); @@ -1089,11 +1120,21 @@ export std::string_view cache_mode_name(CacheMode m) { } // A condition a planning pass reports instead of acting on (plan_only): the -// code is stable and the message is for people. Emitted as warning diagnostics -// by the command that asked for the plan. +// code is stable and the message is for people. Emitted as diagnostics by the +// command that asked for the plan, at the severity carried here — most notes +// are warnings the document is still complete despite (a lock that would +// change, a generated file left unmaterialized); a build program whose run +// failed under `plan_only` (#699 item 2, E3) is an error, because the sets it +// would have shaped are described without its directives. export struct PlanNote { std::string code; std::string message; + mcpp::wire::Severity severity = mcpp::wire::Severity::Warning; + // The absolute, native path of the file the condition is about (a + // package's `build.mcpp`), empty when the note names no file. + // `mcpp.build.build_database::render` rewrites it to the workspace- + // relative form every other `path` in the document uses. + std::string path; }; export struct BuildContext { @@ -7103,9 +7144,14 @@ prepare_build(bool print_fingerprint, const mcpp::manifest::Manifest& depManifest) -> std::vector { + // Word by word (SPEC-004 §8, #703): a search path is made absolute + // per word, and each word is written back as an element that reads as + // exactly that word, so the consumer's renderer reads the dependency's + // flags with the same reading its own flags receive, and an element + // that packs several tokens is several words on both sides. std::vector added; - for (auto const& flag : depManifest.buildConfig.ldflags) { - auto normalized = normalizeDepLdflag(depRoot, flag); + for (auto const& word : mcpp::manifest::flag_words(depManifest.buildConfig.ldflags)) { + auto normalized = mcpp::manifest::flag_element(normalizeDepLdflag(depRoot, word)); m->buildConfig.ldflags.push_back(normalized); added.push_back(std::move(normalized)); } @@ -10351,6 +10397,26 @@ prepare_build(bool print_fingerprint, continue; } + // #699 item 2 (E2): under `emit build-database` + // (`plan_only`), a host tool that fails to build is a + // warning, not a refusal that costs the whole plan — the + // requesting member is still worth describing, and its + // build program receives the path the tool would have + // been published at (`binOut`, fixed above before any of + // this runs). `mcpp build` is unchanged below: it still + // returns `std::unexpected` and the target fails. + auto host_tool_unbuilt = [&](std::string_view failure) { + // The first line only: a nested build's message can + // run to several, and the warning names the tool and + // its package, not the whole log. + const auto first = failure.substr(0, failure.find('\n')); + planNotes.push_back({"MCPP_BUILD_DATABASE_HOST_TOOL_UNBUILT", + std::format("host tool '{}' of package '{}' did not " + "build: {}", toolName, depName, first), + mcpp::wire::Severity::Warning}); + record(binOut); + }; + mcpp::ui::status("Building", std::format( "host tool {}:{} from {} v{} (once per package source and " "host toolchain)", depName, toolName, depName, @@ -10440,6 +10506,10 @@ prepare_build(bool print_fingerprint, /*includeDevDeps=*/false, /*extraTargets=*/{}, sub); if (!subCtx) { + if (overrides.plan_only) { + host_tool_unbuilt(subCtx.error()); + continue; + } return std::unexpected(std::format( "building host tool '{}:{}' failed: {}{}", depName, toolName, subCtx.error(), subContext())); @@ -10453,6 +10523,12 @@ prepare_build(bool print_fingerprint, if (lu.targetName == toolName) { goal = lu.output; break; } } if (goal.empty()) { + if (overrides.plan_only) { + host_tool_unbuilt("produced no link unit — its " + "required_features may not be satisfiable on " + "this platform"); + continue; + } return std::unexpected(std::format( "host tool '{}:{}' produced no link unit — its " "required_features may not be satisfiable on this " @@ -10471,6 +10547,10 @@ prepare_build(bool print_fingerprint, bopt.verbose = true; auto br = be->build(subCtx->plan, bopt); if (!br) { + if (overrides.plan_only) { + host_tool_unbuilt(br.error().message); + continue; + } auto diag = br.error().diagnosticOutput; if (diag.empty()) diag = "(the inner build produced no diagnostic " @@ -10481,6 +10561,11 @@ prepare_build(bool print_fingerprint, subContext(), diag)); } if (br->exitCode != 0) { + if (overrides.plan_only) { + host_tool_unbuilt(std::format( + "build exited with {}", br->exitCode)); + continue; + } return std::unexpected(std::format( "building host tool '{}:{}' failed (exit {}){}", depName, toolName, br->exitCode, subContext())); @@ -10492,6 +10577,11 @@ prepare_build(bool print_fingerprint, std::error_code cpEc; auto produced = subCtx->plan.outputDir / goal; if (!std::filesystem::exists(produced, cpEc)) { + if (overrides.plan_only) { + host_tool_unbuilt(std::format( + "built but '{}' is missing", produced.string())); + continue; + } return std::unexpected(std::format( "host tool '{}:{}' built but '{}' is missing", depName, toolName, produced.string())); @@ -10503,6 +10593,11 @@ prepare_build(bool print_fingerprint, std::filesystem::copy_file(produced, tmp, std::filesystem::copy_options::overwrite_existing, cpEc); if (cpEc) { + if (overrides.plan_only) { + host_tool_unbuilt(std::format( + "staging failed: {}", cpEc.message())); + continue; + } return std::unexpected(std::format( "staging host tool '{}:{}' failed: {}", depName, toolName, cpEc.message())); @@ -10514,6 +10609,11 @@ prepare_build(bool print_fingerprint, std::filesystem::perm_options::add, cpEc); std::filesystem::rename(tmp, binOut, cpEc); if (cpEc) { + if (overrides.plan_only) { + host_tool_unbuilt(std::format( + "publishing failed: {}", cpEc.message())); + continue; + } return std::unexpected(std::format( "publishing host tool '{}:{}' failed: {}", depName, toolName, cpEc.message())); @@ -10610,6 +10710,24 @@ prepare_build(bool print_fingerprint, pkg.manifest, pkg.root, host->first, host->second, pkg.manifest.cppStandard, bpEnv); !r) { + // #699 item 2 (E3): under `emit build-database` (`plan_only`), + // a failing build program describes its package without that + // program's directives, instead of costing the whole plan — + // the manifest's own configuration, the toolchain and the + // module graph are still worth describing. Nothing is applied + // either way: `run_build_program` returns before + // `Directives::apply` on every failure path. A later failure + // that follows from the missing directives (a source the + // program would have added, say) fails the member under the + // ordinary rule (E1). + if (overrides.plan_only) { + planNotes.push_back({"MCPP_BUILD_DATABASE_PROGRAM_FAILED", + std::format("dependency '{}': {}", + pkg.manifest.package.name, r.error()), + mcpp::wire::Severity::Error, + (pkg.root / "build.mcpp").string()}); + continue; + } return std::unexpected(std::format( "dependency '{}': {}", pkg.manifest.package.name, r.error())); } @@ -12111,7 +12229,8 @@ prepare_build(bool print_fingerprint, facts.hasSources = !mcpp::modgraph::package_source_files( packages[i].root, pkg).empty(); facts.carriesForeignLinkInputs = - lf::carries_foreign_link_inputs(pkg.buildConfig.ldflags); + lf::carries_foreign_link_inputs( + mcpp::manifest::flag_words(pkg.buildConfig.ldflags)); facts.isDistribution = mcpp::pack::is_distribution_package(pkg); for (auto const& artifact : pkg.runtimeConfig.artifacts) { if (artifact.role == "static-library") facts.shipsStatic = true; @@ -12389,120 +12508,147 @@ prepare_build(bool print_fingerprint, // snapshot. Anything past this index is a `mcpp::deploy()` residue // that needs the same mirror the flag/source tails get below. const auto rdeployN = m->runtimeConfig.linkIntent.deploy.size(); + // Same reason, one field wide: `mcpp::runtime_search_dir()` residue + // needs the same mirror `deploy` does, or `resolve_runtime_contract` + // (which reads `packages[0]`'s snapshot, not `*m`) never sees it. + const auto rsearchDirN = m->runtimeConfig.linkIntent.runtimeSearchDirs.size(); // What the dependencies supplied as runners, before the root's program // speaks. The root's emissions are appended to the same slots, so a // name both supply becomes one argv joining the two (#634, §9 item 8, // measured: `run-A.sh run-B.sh `). const auto runnerBeforeRoot = bcRoot.runner; const auto namedBeforeRoot = bcRoot.namedRunners; - if (auto bp = mcpp::build::run_build_program( - *m, *root, host->first, host->second, - m->cppStandard, bpEnv); - !bp) { + auto bp = mcpp::build::run_build_program( + *m, *root, host->first, host->second, + m->cppStandard, bpEnv); + if (!bp && !overrides.plan_only) { return std::unexpected(bp.error()); } - // THE SAME RULE THE DEPENDENCIES ARE HELD TO, WITH THE ROOT AS A PARTY. - // Two suppliers of one runner are refused naming both, and the - // manifest is the way to choose: a `[target.]` runner the - // project writes outranks every supplied one where the runner is - // looked up, so a name the manifest declares is not refused here. - { - const auto rowKey = [&]() -> std::string { - if (!tc) return {}; - auto t = mcpp::toolchain::triple::parse(tc->targetTriple); - return t ? t->str() : tc->targetTriple; - }(); - const auto row = m->targetOverrides.find(rowKey); - const auto manifestNames = [&](std::string_view name) { - if (row == m->targetOverrides.end()) return false; - if (name.empty()) return !row->second.runner.empty(); - return row->second.namedRunners.contains(std::string(name)); - }; - if (!runnerProvider.empty() && !runnerBeforeRoot.empty() - && bcRoot.runner.size() > runnerBeforeRoot.size() - && !manifestNames({})) { - return std::unexpected(std::format( - "the dependency '{}' and this project's build program both " - "supply the runner for this target, and the two would be " - "joined into one argv.\n" - " Drop one of them, or state the runner in " - "[target.{}].runner.", - runnerProvider, rowKey)); - } - for (auto const& [name, nr] : bcRoot.namedRunners) { - auto before = namedBeforeRoot.find(name); - auto who = namedRunnerProvider.find(name); - if (before == namedBeforeRoot.end() || before->second.argv.empty() - || who == namedRunnerProvider.end() || who->second.empty()) - continue; - if (nr.argv.size() <= before->second.argv.size()) continue; - if (manifestNames(name)) continue; - return std::unexpected(std::format( - "the dependency '{}' and this project's build program both " - "supply a runner named '{}' for this target, and the two " - "would be joined into one argv.\n" - " Drop one of them, or state it in " - "[target.{}.runners].{}.", - who->second, name, rowKey, name)); - } - } - auto& pkg0 = packages[0]; - // Compile-visible tail → privateBuild: the shared fold (same owner - // as the dep loop; the root's TUs read privateBuild). - foldDirectiveTailIntoPrivateBuild(pkg0, *m, mark); - // Before the source residues are mirrored below: adopting an action's - // outputs APPENDS to bcRoot.sources, and those appends must be inside - // the tail that gets copied into the packages[0] snapshot the scan reads. - adoptActionOutputs(*m, *root, ractN); - // The root's build program has spoken; a floor it stated is checked - // now, with the facts every package (it included) established. - if (auto err = checkVersionFloors(); err) return std::unexpected(*err); - // Root residues — apply() mutated *m, but packages[0].manifest is a - // value-copy snapshot taken at makePackageRoot, so everything the - // scan/fingerprint read from the snapshot needs the tail mirrored: - // sources → the scan walks packages[0].manifest, not *m. - pkg0.manifest.buildConfig.sources.insert( - pkg0.manifest.buildConfig.sources.end(), - bcRoot.sources.begin() + rsrcN, bcRoot.sources.end()); - pkg0.manifest.modules.sources.insert( - pkg0.manifest.modules.sources.end(), - m->modules.sources.begin() + rmodN, m->modules.sources.end()); - // Fingerprint metadata (canonical_package_build_metadata folds - // packages[].manifest.buildConfig) — mirror the flag/include tails, - // as the old pre-snapshot ordering implicitly did. - pkg0.manifest.buildConfig.cflags.insert( - pkg0.manifest.buildConfig.cflags.end(), - bcRoot.cflags.begin() + static_cast(mark.cflags), - bcRoot.cflags.end()); - pkg0.manifest.buildConfig.cxxflags.insert( - pkg0.manifest.buildConfig.cxxflags.end(), - bcRoot.cxxflags.begin() + static_cast(mark.cxxflags), - bcRoot.cxxflags.end()); - pkg0.manifest.buildConfig.includeDirs.insert( - pkg0.manifest.buildConfig.includeDirs.end(), - bcRoot.includeDirs.begin() + static_cast(mark.includeDirs), - bcRoot.includeDirs.end()); - pkg0.manifest.buildConfig.includeDirsAfter.insert( - pkg0.manifest.buildConfig.includeDirsAfter.end(), - bcRoot.includeDirsAfter.begin() - + static_cast(mark.includeDirsAfter), - bcRoot.includeDirsAfter.end()); - // Link flags → the final link reads *m (already applied); keep the - // linkUsage snapshot and fingerprint metadata equivalent too. - pkg0.linkUsage.ldflags.insert(pkg0.linkUsage.ldflags.end(), - bcRoot.ldflags.begin() + rldN, bcRoot.ldflags.end()); - pkg0.manifest.buildConfig.ldflags.insert( - pkg0.manifest.buildConfig.ldflags.end(), - bcRoot.ldflags.begin() + rldN, bcRoot.ldflags.end()); - // #622 A4: `mcpp::deploy()` residue → `packages[0].manifest`, the - // object `resolve_runtime_contract` (plan.cppm) actually reads. - // Without this mirror a directive-sourced deploy entry lands in `*m` - // and nowhere the planner looks — the same gap this block already - // closes for sources/flags, one more field wide. - pkg0.manifest.runtimeConfig.linkIntent.deploy.insert( - pkg0.manifest.runtimeConfig.linkIntent.deploy.end(), - m->runtimeConfig.linkIntent.deploy.begin() + static_cast(rdeployN), - m->runtimeConfig.linkIntent.deploy.end()); + // #699 item 2 (E3): under `emit build-database` (`plan_only`), a + // failing root build program describes the package without its + // directives rather than costing the whole plan. Every mirror below + // reads what the program would have added to `*m`, so skipping + // straight past it (nothing runs on this path) is what "without its + // directives" means; a later failure that follows from the gap + // fails the member under the ordinary rule (E1). + if (!bp) { + planNotes.push_back({"MCPP_BUILD_DATABASE_PROGRAM_FAILED", + bp.error(), mcpp::wire::Severity::Error, + (*root / "build.mcpp").string()}); + } + if (bp) { + // THE SAME RULE THE DEPENDENCIES ARE HELD TO, WITH THE ROOT AS A PARTY. + // Two suppliers of one runner are refused naming both, and the + // manifest is the way to choose: a `[target.]` runner the + // project writes outranks every supplied one where the runner is + // looked up, so a name the manifest declares is not refused here. + { + const auto rowKey = [&]() -> std::string { + if (!tc) return {}; + auto t = mcpp::toolchain::triple::parse(tc->targetTriple); + return t ? t->str() : tc->targetTriple; + }(); + const auto row = m->targetOverrides.find(rowKey); + const auto manifestNames = [&](std::string_view name) { + if (row == m->targetOverrides.end()) return false; + if (name.empty()) return !row->second.runner.empty(); + return row->second.namedRunners.contains(std::string(name)); + }; + if (!runnerProvider.empty() && !runnerBeforeRoot.empty() + && bcRoot.runner.size() > runnerBeforeRoot.size() + && !manifestNames({})) { + return std::unexpected(std::format( + "the dependency '{}' and this project's build program both " + "supply the runner for this target, and the two would be " + "joined into one argv.\n" + " Drop one of them, or state the runner in " + "[target.{}].runner.", + runnerProvider, rowKey)); + } + for (auto const& [name, nr] : bcRoot.namedRunners) { + auto before = namedBeforeRoot.find(name); + auto who = namedRunnerProvider.find(name); + if (before == namedBeforeRoot.end() || before->second.argv.empty() + || who == namedRunnerProvider.end() || who->second.empty()) + continue; + if (nr.argv.size() <= before->second.argv.size()) continue; + if (manifestNames(name)) continue; + return std::unexpected(std::format( + "the dependency '{}' and this project's build program both " + "supply a runner named '{}' for this target, and the two " + "would be joined into one argv.\n" + " Drop one of them, or state it in " + "[target.{}.runners].{}.", + who->second, name, rowKey, name)); + } + } + auto& pkg0 = packages[0]; + // Compile-visible tail → privateBuild: the shared fold (same owner + // as the dep loop; the root's TUs read privateBuild). + foldDirectiveTailIntoPrivateBuild(pkg0, *m, mark); + // Before the source residues are mirrored below: adopting an action's + // outputs APPENDS to bcRoot.sources, and those appends must be inside + // the tail that gets copied into the packages[0] snapshot the scan reads. + adoptActionOutputs(*m, *root, ractN); + // The root's build program has spoken; a floor it stated is checked + // now, with the facts every package (it included) established. + if (auto err = checkVersionFloors(); err) return std::unexpected(*err); + // Root residues — apply() mutated *m, but packages[0].manifest is a + // value-copy snapshot taken at makePackageRoot, so everything the + // scan/fingerprint read from the snapshot needs the tail mirrored: + // sources → the scan walks packages[0].manifest, not *m. + pkg0.manifest.buildConfig.sources.insert( + pkg0.manifest.buildConfig.sources.end(), + bcRoot.sources.begin() + rsrcN, bcRoot.sources.end()); + pkg0.manifest.modules.sources.insert( + pkg0.manifest.modules.sources.end(), + m->modules.sources.begin() + rmodN, m->modules.sources.end()); + // Fingerprint metadata (canonical_package_build_metadata folds + // packages[].manifest.buildConfig) — mirror the flag/include tails, + // as the old pre-snapshot ordering implicitly did. + pkg0.manifest.buildConfig.cflags.insert( + pkg0.manifest.buildConfig.cflags.end(), + bcRoot.cflags.begin() + static_cast(mark.cflags), + bcRoot.cflags.end()); + pkg0.manifest.buildConfig.cxxflags.insert( + pkg0.manifest.buildConfig.cxxflags.end(), + bcRoot.cxxflags.begin() + static_cast(mark.cxxflags), + bcRoot.cxxflags.end()); + pkg0.manifest.buildConfig.includeDirs.insert( + pkg0.manifest.buildConfig.includeDirs.end(), + bcRoot.includeDirs.begin() + static_cast(mark.includeDirs), + bcRoot.includeDirs.end()); + pkg0.manifest.buildConfig.includeDirsAfter.insert( + pkg0.manifest.buildConfig.includeDirsAfter.end(), + bcRoot.includeDirsAfter.begin() + + static_cast(mark.includeDirsAfter), + bcRoot.includeDirsAfter.end()); + // Link flags → the final link reads *m (already applied); keep the + // linkUsage snapshot and fingerprint metadata equivalent too. + pkg0.linkUsage.ldflags.insert(pkg0.linkUsage.ldflags.end(), + bcRoot.ldflags.begin() + rldN, bcRoot.ldflags.end()); + pkg0.manifest.buildConfig.ldflags.insert( + pkg0.manifest.buildConfig.ldflags.end(), + bcRoot.ldflags.begin() + rldN, bcRoot.ldflags.end()); + // #622 A4: `mcpp::deploy()` residue → `packages[0].manifest`, the + // object `resolve_runtime_contract` (plan.cppm) actually reads. + // Without this mirror a directive-sourced deploy entry lands in `*m` + // and nowhere the planner looks — the same gap this block already + // closes for sources/flags, one more field wide. + pkg0.manifest.runtimeConfig.linkIntent.deploy.insert( + pkg0.manifest.runtimeConfig.linkIntent.deploy.end(), + m->runtimeConfig.linkIntent.deploy.begin() + static_cast(rdeployN), + m->runtimeConfig.linkIntent.deploy.end()); + // `mcpp::runtime_search_dir()` residue → `packages[0].manifest`, the + // same object and the same reason as the `deploy` mirror above: without + // it a directive-sourced entry lands in `*m` and `resolve_runtime_contract` + // never looks there. + pkg0.manifest.runtimeConfig.linkIntent.runtimeSearchDirs.insert( + pkg0.manifest.runtimeConfig.linkIntent.runtimeSearchDirs.end(), + m->runtimeConfig.linkIntent.runtimeSearchDirs.begin() + + static_cast(rsearchDirN), + m->runtimeConfig.linkIntent.runtimeSearchDirs.end()); + } } // ── Every device source must reach some action ───────────────────────── @@ -12571,6 +12717,87 @@ prepare_build(bool print_fingerprint, " drop them from `[build] sources`.")); } + // ── R1.3: a re-run input inside a `prepare` directory (SPEC-007 §3) ───── + // + // A build program's re-run set is declared BEFORE anything is built + // (`rerun_if_changed`/`rerun_if_changed_glob`), and a `prepare` action's + // directory is filled AFTER a build program has already run once for + // this build — it is a ninja edge, scheduled after `mcpp build`'s + // configure step ends. A program that also names a file or a glob inside + // such a directory as its own re-run input reads a CONSTRUCTION RESULT + // while it configures: correct on the SECOND build, once a previous + // build's `prepare` action has populated the directory, and wrong on the + // first — the exact pattern of a plugin placing a first installation's + // libraries on the NEXT plan (design §5.2's route on 2026.9.26.1, which + // this directive and role exist to remove). + // + // WARNED, NOT REFUSED: the program still configures correctly today (its + // FIRST run sees what the tree already held), and R1.2 already asks a + // program to say what it could not find with `mcpp::warning`. This is the + // engine naming an author obligation SPEC-007 states (R1.3), not a build + // it can complete no differently. + // + // `declared_program_inputs` reads back what every package's build.mcpp + // just declared (or, on a cache hit, declared on its last run) from the + // caches under `/target/.build-mcpp`, so no extra plumbing is + // needed to carry the re-run set out of `run_build_program`. + { + std::map ownerName; + std::vector> prepareDirs; + for (std::size_t i = 0; i < packages.size(); ++i) { + auto const& mm = (i == 0) ? *m : packages[i].manifest; + ownerName.emplace(packages[i].root.lexically_normal(), mm.package.name); + for (auto const& a : mm.buildConfig.actions) { + if (a.role != mcpp::manifest::BuildAction::Role::Prepare) continue; + if (a.outputDir.empty()) continue; + prepareDirs.emplace_back(mm.package.name, + std::filesystem::path(a.outputDir).lexically_normal()); + } + } + if (!prepareDirs.empty()) { + // `p` reaches strictly inside `dir`: equal paths and a sibling + // that merely shares a prefix (`lexically_relative` starting with + // `..`) both do not count. + auto isUnder = [](const std::filesystem::path& p, + const std::filesystem::path& dir) { + auto rel = p.lexically_relative(dir); + if (rel.empty()) return false; + auto s = rel.generic_string(); + return s != "." && s.compare(0, 2, "..") != 0; + }; + for (auto const& decl : mcpp::build::declared_program_inputs(workRoot)) { + std::vector watched(decl.files); + for (auto const& pattern : decl.globs) { + // The glob's fixed prefix — everything before its first + // wildcard character — is enough to answer whether the + // PATTERN reaches into a `prepare` directory; resolving it + // into the file set it matches is not needed for that. + auto wildcard = pattern.find_first_of("*?["); + auto fixed = wildcard == std::string::npos + ? pattern : pattern.substr(0, wildcard); + watched.push_back((decl.root / fixed).lexically_normal()); + } + auto ownerIt = ownerName.find(decl.root.lexically_normal()); + const std::string declName = + ownerIt != ownerName.end() ? ownerIt->second : decl.root.string(); + for (auto const& w : watched) { + for (auto const& [pkgName, dir] : prepareDirs) { + if (!isUnder(w, dir)) continue; + mcpp::ui::warning(std::format( + "{}'s build.mcpp re-runs on '{}', which is inside " + "'{}', the directory package '{}' declared with a " + "`prepare` action's output_dir. That directory is " + "populated at BUILD time, after build.mcpp has " + "already configured, so this program sees the " + "PREVIOUS build's contents, never the current " + "one's (SPEC-007 R1.3).", + declName, w.string(), dir.string(), pkgName)); + } + } + } + } + } + // [targets.*] required_features gate: a target is emitted only when ALL its // required features are active in this build; otherwise it is silently // skipped. A pure build-selection knob — it runs before the modgraph/plan @@ -13319,6 +13546,22 @@ prepare_build(bool print_fingerprint, stdObjectPath = sm->objectPath; stdCompatBmiPath = sm->compatBmiPath; stdCompatObjectPath = sm->compatObjectPath; + // C5 / D5a (design 2026-09-26 §3.5): compile_commands.json and the + // S1 document list the standard-library units too, so the plan + // needs the commands mcpp ran to build them (§13396 below), not + // only their output paths. `describe_std_module` is the pure + // derivation `ensure_built` itself reads before running anything + // (mcpp.toolchain.stdmod's header); calling it again here starts + // no process and cannot name a different command or directory. + // A failure here is not this build's failure -- `ensure_built` + // above already succeeded with the same inputs -- so it only + // means the description is unavailable for the plan, silently. + auto described = mcpp::toolchain::describe_std_module( + *tc, m->package.standard, stdFlagAndDialect, + mcpp::platform::macos::deployment_target( + stdTargetIsMacos, m->buildConfig.macosDeploymentTarget), + mcpp::toolchain::default_cache_root(), stdCrt); + if (described) describedStdModule = std::move(*described); } } @@ -13355,7 +13598,12 @@ prepare_build(bool print_fingerprint, } ctx.stdBmi = stdBmiPath; ctx.stdObject = stdObjectPath; - ctx.stdModule = std::move(describedStdModule); + // Copied, not moved: `describedStdModule` is read again once `ctx.plan` + // exists (below), to recover the standard-library units' commands onto + // it (StdModuleUnit, C5 / D5a-b). A `std::optional` move leaves the + // source engaged with a moved-from value, so a plain move here would + // hand build_database.cppm's render() a value and the plan an empty one. + ctx.stdModule = describedStdModule; // Every directory a package payload may legitimately have been INSTALLED // into. There is more than one: the global registry, plus the two // project-local data roots a custom git index installs into @@ -13611,6 +13859,48 @@ prepare_build(bool print_fingerprint, // plan, and every reader of `compute_flags` runs after this line. ctx.plan.targetSide = resolvedTargetSide; + // C5 / D5a-b (design 2026-09-26 §3.5): the standard-library units this + // configuration's build compiles, when it imports `std`. Recovered here, + // once, from the SAME command derivation `ensure_built` and + // `describe_std_module` both read (mcpp.toolchain.stdmod), and carried on + // the plan (BuildPlan::stdModuleUnits) so compile_commands.json, + // `emit --spec compile-commands` and the S1 document render the exact + // same record and cannot disagree (P1, mcpp.build.compile_commands). + if (describedStdModule) { + const auto& sm = *describedStdModule; + auto add_std_unit = [&](const std::filesystem::path& source, + const std::vector& commands, + const std::filesystem::path& object, + const std::filesystem::path& bmi, + std::string_view module, + std::vector requiresModules) { + if (source.empty() || commands.empty()) return; + auto inv = mcpp::build::recover_invocation( + commands, source, tc->binaryPath, sm.cacheDir, + mcpp::platform::is_windows); + if (!inv) { + planNotes.push_back({"MCPP_BUILD_DATABASE_STD_UNIT_UNDESCRIBED", + std::format("no command that builds the {} module names its " + "source '{}'; the unit is not listed", + module, source.string())}); + return; + } + ctx.plan.stdModuleUnits.push_back(mcpp::build::StdModuleUnit{ + .source = source, + .workDirectory = std::move(inv->workDirectory), + .arguments = std::move(inv->arguments), + .object = object, + .bmi = bmi, + .module = std::string(module), + .requiresModules = std::move(requiresModules), + }); + }; + add_std_unit(tc->stdModuleSource, sm.stdCommands, sm.objectPath, + sm.bmiPath, "std", {}); + add_std_unit(tc->stdCompatSource, sm.compatCommands, sm.compatObjectPath, + sm.compatBmiPath, "std.compat", {"std"}); + } + // A DEPENDENCY'S C++ SHARED LIBRARY OVER A C++ RUNTIME THAT IS A PACKAGE // (#641, item 5). // diff --git a/src/build/runtime_validation.cppm b/src/build/runtime_validation.cppm index d96d20153..3110f2b45 100644 --- a/src/build/runtime_validation.cppm +++ b/src/build/runtime_validation.cppm @@ -916,7 +916,10 @@ check_symbol_provision(const mcpp::build::BuildPlan& plan, // The flags this build hands the linker, as ONE vector, because the // question is whether ANY of them took the export decision away from // mcpp. Per-unit flags join below; these are the whole-build ones. - std::vector globalFlags = plan.manifest.buildConfig.ldflags; + // Read as words (SPEC-004 §8, #703), which is what the linker receives: + // an element that packs `-Wl,-E` with another token is still a request. + std::vector globalFlags = + mcpp::manifest::flag_words(plan.manifest.buildConfig.ldflags); auto searchDirs = runtime_search_dirs(plan); diff --git a/src/cli.cppm b/src/cli.cppm index 16d0ed29c..8bf13376b 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -929,6 +929,11 @@ int run(int argc, char** argv) { .option(cl::Option("verify").takes_value().value_name("MODE") .help("How an existing destination is judged up to date: content (default) | size")) .action(wrap_rc(cmd_stage))) + .subcommand(cl::App("place-dlls") + .description("(internal: invoked by ninja) Place beside a Windows program the DLLs it imports from its runtime search directories") + .option(cl::Option("output").takes_value().value_name("PATH").help("the stamp to write")) + .option(cl::Option("depfile").takes_value().value_name("PATH").help("the depfile naming every DLL placed")) + .action(wrap_rc(cmd_place_dlls))) .subcommand(cl::App("coff-def") .description("(internal: invoked by ninja) Write a .def of every exportable symbol in the given COFF objects") .option(cl::Option("output").takes_value().value_name("PATH").help("the .def to write")) @@ -974,8 +979,9 @@ int run(int argc, char** argv) { return 0; } - // `mcpp __action-stamp ... -- ...` — internal, and absent - // from the usage text on purpose: nobody types it, ninja does. + // `mcpp __action-stamp [--require-dir ] ... -- ...` — + // internal, and absent from the usage text on purpose: nobody types it, + // ninja does. // // WHY IT EXISTS. A `role = "check"` action's output is a stamp file, and // until now the COMMAND had to create it. Analysers do not: clang-tidy's @@ -984,17 +990,45 @@ int run(int argc, char** argv) { // no shell assumed, which is right for Windows and is exactly what made // the wrapper unwritable there. The role with no ecosystem consumer also // had no portable way to acquire one, and that was not a coincidence. + // `role = "prepare"` (mcpp#702) shares the same stamp -- its command + // populates a directory rather than writing the stamp itself either. // // The engine is the portable wrapper. It is already on disk on every // platform mcpp runs on, so this needs no shell, no `touch`, and no // per-platform spelling. // - // A command that already creates its stamp is unaffected: existing files - // are left alone, so the pre-2026.8.29.1 wrapper scripts keep working - // byte-for-byte. + // A command that writes its own stamp is unaffected: a stamp the command + // created or rewrote is left as the command left it, so the + // pre-2026.8.29.1 wrapper scripts keep working byte-for-byte. + // + // EVERY STAMP IS NEWER THAN EVERY INPUT AFTER SUCCESS (R2, design §5.4), + // in its simple form: the engine creates each stamp that is missing and + // sets the modification time of each EXISTING one to the present, + // whether or not the command wrote it. No record of a stamp's time + // before the run is needed — a stamp feeds no compile or link edge (a + // blocking `check` or a `prepare` action orders edges through an + // order-only edge instead), so unconditionally touching a stamp the + // command already wrote changes no build. Until 2026.9.26.2 an existing + // stamp was left alone whatever the command did, and every check whose + // command writes nothing -- clang-tidy, an installer run through + // `mcpp-deps` -- re-ran on every build after its first input change, + // because its output stayed older than that input forever. if (std::string_view(argv[1]) == "__action-stamp") { - std::vector stamps; int i = 2; + // `prepare` only: the directory its command populates + // (`mcpp::action::output_dir`). Parsed before the stamp list, which + // is otherwise everything up to `--`, so this flag cannot be mistaken + // for a stamp path. + std::string requireDir; + if (i < argc && std::string_view(argv[i]) == "--require-dir") { + if (i + 1 >= argc) { + std::println(stderr, "error: --require-dir requires a directory"); + return 2; + } + requireDir = argv[i + 1]; + i += 2; + } + std::vector stamps; for (; i < argc && std::string_view(argv[i]) != "--"; ++i) stamps.emplace_back(argv[i]); if (i >= argc || stamps.empty()) { @@ -1014,8 +1048,53 @@ int run(int argc, char** argv) { const int r = mcpp::platform::process::run_exec(cmd); // The stamps are written ONLY on success. Writing them anyway would // make ninja consider the edge satisfied, so the next build would skip - // a check that had never passed. + // a check (or a `prepare`) that had never passed. if (r != 0) return r; + // `prepare`'s post-condition (design §5.4 P, SPEC-007 R3.2/R3.3): the + // declared directory must exist once the command has succeeded, + // whether or not the command itself created it — a Qt `lupdate` + // action's directory, for instance, is an existing source tree it + // rewrites, never a fresh one. Checked BEFORE any stamp is written, + // so a missing directory fails the edge exactly as a failing command + // does: no stamp, and the next build tries again. + // + // POPULATED, NOT MERELY PRESENT. ninja creates the parent directory of + // every declared output before it runs the command, so a stamp declared + // inside the directory makes the directory exist whatever the command + // did. The directory therefore has to hold at least one file that is + // not one of this action's stamps; directories alone do not count. + // The walk stops at the first such file, so an installed prefix of + // thousands of files costs one lookup. + if (!requireDir.empty()) { + std::error_code ec; + const auto dir = + mcpp::platform::fs::extended_length(std::filesystem::path{requireDir}); + std::vector stampPaths; + for (auto const& s : stamps) + stampPaths.push_back(std::filesystem::absolute( + mcpp::platform::fs::extended_length(std::filesystem::path{s}), ec) + .lexically_normal()); + bool populated = false; + if (std::filesystem::is_directory(dir, ec)) { + for (auto it = std::filesystem::recursive_directory_iterator( + dir, std::filesystem::directory_options::skip_permission_denied, ec); + !ec && it != std::filesystem::recursive_directory_iterator(); + it.increment(ec)) { + std::error_code tec; + if (it->is_directory(tec)) continue; + const auto entry = std::filesystem::absolute(it->path(), tec).lexically_normal(); + if (std::ranges::find(stampPaths, entry) != stampPaths.end()) continue; + populated = true; + break; + } + } + if (!populated) { + std::println(stderr, + "error: prepare action succeeded, but its declared " + "directory '{}' does not exist or holds no file", requireDir); + return 1; + } + } for (auto const& s : stamps) { std::error_code ec; // Relative to the build directory, which can be deep enough to @@ -1023,7 +1102,15 @@ int run(int argc, char** argv) { const auto p = mcpp::platform::fs::extended_length(std::filesystem::path{s}); if (!p.parent_path().empty()) std::filesystem::create_directories(p.parent_path(), ec); - if (std::filesystem::exists(p, ec)) continue; + if (std::filesystem::exists(p, ec)) { + std::filesystem::last_write_time( + p, std::filesystem::file_time_type::clock::now(), ec); + if (ec) { + std::println(stderr, "error: cannot update check stamp '{}': {}", s, ec.message()); + return 1; + } + continue; + } std::ofstream out(p, std::ios::trunc); if (!out) { std::println(stderr, "error: cannot write check stamp '{}'", s); @@ -1122,6 +1209,7 @@ int run(int argc, char** argv) { "update", "search", "publish", "pack", "emit", "xpkg", "toolchain", "cache", "index", "self", "explain", "version", "dyndep", "why", "resolve", "stage", "bmi-equal", "coff-def", + "place-dlls", "bmi-compile", "bmi-supervise", "bmi-await", }); bool ok = false; diff --git a/src/cli/cmd_build.cppm b/src/cli/cmd_build.cppm index c9589c05e..e26940cbc 100644 --- a/src/cli/cmd_build.cppm +++ b/src/cli/cmd_build.cppm @@ -303,10 +303,13 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) } return 0; }; - // A failure is one envelope with diagnostics and no `data` (S2 0.2.0 §3.4: - // a command without data has failed), and exit 1. - auto failed = [&](std::string code, std::string message) -> int { - diagnostics.push_back({std::move(code), Severity::Error, std::move(message)}); + // A failure with nothing to describe is one envelope with diagnostics and + // no `data` (S2 §3.4: a command without data has failed), and exit 1. `fail_no_data` finishes with whatever `diagnostics` already holds — + // used once a single diagnostic is pushed onto it (`failed`, below, for a + // usage error decided before any member is tried) and once every + // selected member's own planning has failed in turn (#699 item 1, E1: a + // workspace where nothing planned still names each member's own reason). + auto fail_no_data = [&]() -> int { if (!envelope) { for (auto const& d : diagnostics) std::println(stderr, "{}: {}", @@ -322,6 +325,10 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) (void)publish(text); return 1; }; + auto failed = [&](std::string code, std::string message) -> int { + diagnostics.push_back({std::move(code), Severity::Error, std::move(message)}); + return fail_no_data(); + }; auto root = mcpp::project::find_manifest_root(std::filesystem::current_path()); if (!root) @@ -340,21 +347,46 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) requests.emplace_back(std::string{}, ov); } + // An offline plan that needs a download is not a defect of the project, and + // a client that plans offline by default (an editor) has to tell the two + // apart without reading the message (#648 A1). The code is taken right + // where a member's planning ends: `take()` clears the sink, so a refusal + // recorded by a member that failed cannot relabel a later member's own + // reason (the per-member analogue of the rule this used to apply once). + auto plan_failure_code = [&] { + const bool offline = mcpp::platform::env::offline_mode() + || mcpp::platform::env::no_auto_install(); + const bool wasOfflineDownload = mcpp::build::refusal::take() + == mcpp::build::refusal::Code::OfflineDownloadRequired; + return std::string(wasOfflineDownload && offline + ? "MCPP_OFFLINE_DOWNLOAD_REQUIRED" : "MCPP_BUILD_DATABASE_PLAN_FAILED"); + }; + std::vector contexts; std::vector workDirs; std::vector prefixes; std::vector>> testDiscovery; - std::optional planError; + // The root of every member whose planning failed: `mcpp.toml` and + // `build.mcpp` (when it exists) join `watch` exactly as a planned + // member's do (render(), below), so an edit that might fix the failure is + // what wakes a consumer to ask again. + std::vector failedMemberRoots; { // Planning narrates on stdout and may start programs that inherit it; // the document is printed after this scope, alone. mcpp::platform::terminal::StdoutToStderr narration; for (auto& [member, mo] : requests) { + const auto memberRoot = member.empty() ? *root : *root / member; + const auto memberPath = member.empty() ? std::string("mcpp.toml") + : member + "/mcpp.toml"; auto discovered = mcpp::build::discover_test_targets(*root, mo.package_filter); if (!discovered) { - planError = member.empty() ? discovered.error() - : std::format("{}: {}", member, discovered.error()); - break; + diagnostics.push_back({plan_failure_code(), Severity::Error, + member.empty() ? discovered.error() + : std::format("{}: {}", member, discovered.error()), + memberPath}); + failedMemberRoots.push_back(memberRoot); + continue; } // As `--configure-only`: tests and dev-dependencies are part of the // surface an editor needs. @@ -368,9 +400,12 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) includeDevDeps, std::move(discovered->targets), mo); if (!ctx) { - planError = member.empty() ? ctx.error() - : std::format("{}: {}", member, ctx.error()); - break; + diagnostics.push_back({plan_failure_code(), Severity::Error, + member.empty() ? ctx.error() + : std::format("{}: {}", member, ctx.error()), + memberPath}); + failedMemberRoots.push_back(memberRoot); + continue; } contexts.push_back(std::move(*ctx)); workDirs.push_back(mo.work_dir); @@ -378,20 +413,13 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) testDiscovery.push_back(std::move(discovery)); } } - // An offline plan that needs a download is not a defect of the project, and - // a client that plans offline by default (an editor) has to tell the two - // apart without reading the message (#648 A1). The code is taken only while - // the run is offline, so a refusal recorded on a path that recovered cannot - // relabel an unrelated failure. - if (planError) { - const bool offline = mcpp::platform::env::offline_mode() - || mcpp::platform::env::no_auto_install(); - if (mcpp::build::refusal::take() - == mcpp::build::refusal::Code::OfflineDownloadRequired - && offline) - return failed("MCPP_OFFLINE_DOWNLOAD_REQUIRED", *planError); - return failed("MCPP_BUILD_DATABASE_PLAN_FAILED", *planError); - } + // Every selected member was planned independently (#699 item 1, E1): one + // that failed contributed its own diagnostic above and nothing else. + // Only when none of them planned is there nothing left to describe. A + // document with `data` and error diagnostics is S2 0.3.0's partial answer + // (S2-3.4-12, S2-3.4-13): it describes everything the errors do not name. + if (contexts.empty()) + return fail_no_data(); // The lock this planning produced, against the project's. The project's is // never written; a difference is reported. @@ -430,35 +458,47 @@ export int cmd_emit_build_database(const mcpplibs::cmdline::ParsedArgs& parsed) spec, ov.target_triple, mcpp::platform::env::get("MCPP_TOOLCHAIN").value_or(""), ov.profile, ov.features, ov.capabilities, ov.accel, ov.force_static, ov.package_filter, parsed.is_flag_set("workspace")); - auto rendered = mcpp::build::database::render(members, *root, selector); + auto rendered = mcpp::build::database::render(members, failedMemberRoots, + *root, selector); + // A note's severity is its own (E3's program-failure note is an error; + // every other note today is a warning) and its `path`, when set, already + // names the file relative to the workspace root — render() rewrote it. for (auto& note : rendered.notes) - diagnostics.push_back({std::move(note.code), Severity::Warning, - std::move(note.message)}); + diagnostics.push_back({std::move(note.code), note.severity, + std::move(note.message), std::move(note.path)}); auto document = spec == "s1" ? std::move(rendered.database) : std::move(rendered.compileCommands); + // The exit status is 1 whenever an error is present (#699 item 1, E1; + // item 2, E3) even though `data` is: a workspace member's own failure, or + // a build program's, is still a failure this command reports through its + // exit code, only not by withholding the sets that DID plan. + const bool hasError = std::ranges::any_of(diagnostics, + [](const Diagnostic& d) { return d.severity == Severity::Error; }); if (!envelope) { for (auto const& d : diagnostics) std::println(stderr, "{}: {}", mcpp::wire::severity_name(d.severity), d.message); - return publish(document.dump(2) + "\n"); + if (const auto rc = publish(document.dump(2) + "\n"); rc != 0) return rc; + return hasError ? 1 : 0; } std::vector effects{Effect::ReadProject, Effect::WriteGlobalCache}; if (ranBuildPrograms) effects.push_back(Effect::ExecBuildScript); nlohmann::json specJson{{"name", spec}}; if (spec == "s1") specJson["version"] = std::string(mcpp::build::database::kProfileVersion); - return publish(mcpp::wire::to_json(mcpp::wire::Envelope{ - .kind = "mcpp.build-database", - .effects = std::move(effects), - .data = nlohmann::json{ - {"spec", std::move(specJson)}, - {"database", std::move(document)}, - {"watch", std::move(rendered.watch)}, - {"inputs-fingerprint", std::move(rendered.inputsFingerprint)}, - }, - .diagnostics = diagnostics, - }).dump(2) + "\n"); + if (const auto rc = publish(mcpp::wire::to_json(mcpp::wire::Envelope{ + .kind = "mcpp.build-database", + .effects = std::move(effects), + .data = nlohmann::json{ + {"spec", std::move(specJson)}, + {"database", std::move(document)}, + {"watch", std::move(rendered.watch)}, + {"inputs-fingerprint", std::move(rendered.inputsFingerprint)}, + }, + .diagnostics = diagnostics, + }).dump(2) + "\n"); rc != 0) return rc; + return hasError ? 1 : 0; } export int cmd_run(const mcpplibs::cmdline::ParsedArgs& parsed, diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index e304d57d4..c5a16be19 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -36,6 +36,77 @@ export int cmd_publish(const mcpplibs::cmdline::ParsedArgs& parsed) { parsed.is_flag_set("dry-run"), parsed.is_flag_set("allow-dirty")); } +// `mcpp place-dlls --output --depfile ...` -- the +// edge that follows a Windows program's link when its plan has runtime search +// directories (mcpp.pack's `place_runtime_dlls`, SPEC-007 R4.3). Internal: +// only a generated build.ninja names it, and it runs on whatever host builds, +// because it reads the program's import table rather than asking a loader. +// +// The depfile names every DLL placed, so ninja runs the edge again when one of +// them changes in its directory; the stamp is the edge's only declared output, +// because the DLL names are not known when the graph is written, and it holds +// the names placed, one per line, for the next run. +export int cmd_place_dlls(const mcpplibs::cmdline::ParsedArgs& parsed) { + const std::filesystem::path stamp{parsed.option_or_empty("output").value()}; + const std::filesystem::path depfile{parsed.option_or_empty("depfile").value()}; + if (stamp.empty() || depfile.empty() || parsed.positional_count() < 1) { + std::println(stderr, "error: place-dlls requires --output, --depfile and a program"); + return 2; + } + const std::filesystem::path program{parsed.positional(0)}; + std::vector dirs; + for (std::size_t i = 1; i < parsed.positional_count(); ++i) + dirs.emplace_back(parsed.positional(i)); + + // What the previous run placed, recorded in the stamp itself: those copies + // are this edge's, not the program's, and resolve again from their + // directories (see `place_runtime_dlls`). + std::vector placedBefore; + { + std::ifstream prev(stamp); + for (std::string line; std::getline(prev, line);) + if (!line.empty()) placedBefore.push_back(line); + } + auto placed = mcpp::pack::place_runtime_dlls(program, dirs, placedBefore); + if (!placed) { + std::println(stderr, "error: {}", placed.error().message); + return 1; + } + for (auto const& n : placed->notes) std::println("note: {}", n); + + // The depfile syntax ninja reads (`deps = gcc`): a space and `#` are + // escaped with a backslash, and `$` is doubled. + auto dep_word = [](const std::filesystem::path& p) { + std::string out; + for (char c : p.generic_string()) { + if (c == ' ' || c == '#') out.push_back('\\'); + if (c == '$') out.push_back('$'); + out.push_back(c); + } + return out; + }; + std::error_code ec; + if (depfile.has_parent_path()) + std::filesystem::create_directories(depfile.parent_path(), ec); + { + std::ofstream d(depfile, std::ios::trunc); + d << dep_word(stamp) << ':'; + for (auto const& src : placed->sources) d << " \\\n " << dep_word(src); + d << '\n'; + if (!d) { + std::println(stderr, "error: cannot write '{}'", depfile.string()); + return 1; + } + } + std::ofstream st(stamp, std::ios::trunc); + for (auto const& n : placed->names) st << n << '\n'; + if (!st) { + std::println(stderr, "error: cannot write '{}'", stamp.string()); + return 1; + } + return 0; +} + namespace { int cmd_pack_body(const mcpplibs::cmdline::ParsedArgs& parsed, diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index c9c2c83bb..1a3a35d8f 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -38,6 +38,7 @@ export module mcpp.pack; import std; import mcpp.build.loader_contract; +import mcpp.build.stage; // place_runtime_dlls publishes through the staging primitive import mcpp.config; import mcpp.pack.binfmt; import mcpp.pack.host_requirements; @@ -332,6 +333,11 @@ struct ClosureReadInput { std::vector platformDirs; // Android std::vector forceBundle; // Pe: overrides the system list std::string arch; // MachO: the slice of a fat file + // Pe: names never resolved in `searchDirs.front()`. `place_runtime_dlls` + // passes the copies an earlier placement put beside the program, so that + // each resolves again where it came from instead of shadowing a newer + // version there. `mcpp pack` leaves it empty. + std::vector notInFirstDir; }; // A name that resolved to a file, and where that file is staged: `dest` is @@ -358,6 +364,40 @@ struct ClosureRead { // exists. ClosureRead read_closure(const ClosureReadInput& in); +// WHAT A WINDOWS PROGRAM NEEDS BESIDE IT AFTER ITS LINK (SPEC-007 R4.3). +// +// A PE image has no run path: the loader searches the program's directory, +// the system directories and `PATH`, and nothing the program carries. A DLL in +// a runtime search directory therefore serves `mcpp run`, which puts those +// directories on `PATH`, and `mcpp pack`, which stages the closure; a program +// started by hand from the build directory does not find it. The platform's +// own answer is placement: vcpkg's integration copies a program's imported +// DLLs beside it after the link, and CMake names the same set +// `$`. +// +// This is that placement, as one engine mechanism that names no tool. The +// closure is `mcpp pack`'s (`read_closure` under `ClosureRule::Pe`, with its +// system rule), searched in the program's own directory first and then in the +// runtime search directories, in their order. Every resolved DLL outside the +// program's directory is published beside the program through the staging +// primitive, which writes only when the bytes differ. `sources` lists what was +// resolved, for the edge's depfile, so that a DLL replaced in its directory is +// placed again on the next build; `names` lists the DLLs placed, which the +// caller records and passes back as `placedBefore` on the next run, so that a +// copy this function put beside the program is resolved again from its +// directory rather than taken for a file of the program's own; `notes` names +// each DLL that more than one directory offers, with the one the search order +// chose. +struct RuntimeDllPlacement { + std::vector sources; + std::vector names; + std::vector notes; +}; +std::expected +place_runtime_dlls(const std::filesystem::path& program, + const std::vector& searchDirs, + const std::vector& placedBefore = {}); + // Build a Plan from already-resolved inputs. Caller is expected to have // already run `mcpp build` (or equivalent) and pass the resulting // binary path in. @@ -1320,6 +1360,66 @@ make_tarball(const std::filesystem::path& stagingRoot, } // namespace detail +std::expected +place_runtime_dlls(const std::filesystem::path& program, + const std::vector& searchDirs, + const std::vector& placedBefore) +{ + const auto programDir = program.parent_path(); + auto same_dir = [](const std::filesystem::path& a, const std::filesystem::path& b) { + std::error_code ec; + if (std::filesystem::equivalent(a, b, ec)) return true; + return a.lexically_normal() == b.lexically_normal(); + }; + + ClosureReadInput in; + in.object = program; + in.rule = ClosureRule::Pe; + in.searchDirs.push_back(programDir.empty() ? std::filesystem::path(".") : programDir); + for (auto const& d : searchDirs) + if (!same_dir(d, in.searchDirs.front())) in.searchDirs.push_back(d); + in.notInFirstDir = placedBefore; + const auto read = read_closure(in); + + // The program itself is the one object the caller chose, so a program that + // cannot be read is an error; `read_closure` reports it as unresolved + // under the program's own name. + for (auto const& u : read.unresolved) + if (u.name == program.filename().string()) + return std::unexpected(Error{std::format( + "cannot read the imports of '{}': {}", program.string(), u.why)}); + + RuntimeDllPlacement out; + for (auto const& m : read.members) { + if (same_dir(m.source.parent_path(), in.searchDirs.front())) continue; + auto staged = mcpp::build::stage::stage_file(m.source, programDir / m.name); + if (!staged) + return std::unexpected(Error{std::format( + "cannot place '{}' beside '{}': {}", m.source.string(), + program.filename().string(), staged.error().message)}); + out.sources.push_back(m.source); + out.names.push_back(m.name); + // Search order decides between two directories that offer one name, + // and the decision is stated, because the other copy may be the one + // the author meant. + std::vector offering; + for (std::size_t i = 1; i < in.searchDirs.size(); ++i) { + std::error_code ec; + if (std::filesystem::is_regular_file(in.searchDirs[i] / m.name, ec)) + offering.push_back(in.searchDirs[i]); + } + if (offering.size() > 1) { + std::string others; + for (std::size_t i = 1; i < offering.size(); ++i) + others += (others.empty() ? "" : ", ") + offering[i].string(); + out.notes.push_back(std::format( + "'{}' is offered by {} and by {}; the program receives the first, in " + "runtime search order", m.name, offering.front().string(), others)); + } + } + return out; +} + ClosureRead read_closure(const ClosureReadInput& in) { namespace bf = mcpp::pack::binfmt; @@ -1381,8 +1481,12 @@ ClosureRead read_closure(const ClosureReadInput& in) continue; } std::optional hit; - for (auto const& dir : in.searchDirs) - if (is_file(dir / name)) { hit = dir / name; break; } + const bool skipFirst = std::ranges::any_of(in.notInFirstDir, + [&](const std::string& n) { return lower(n) == lower(name); }); + for (std::size_t d = 0; d < in.searchDirs.size(); ++d) { + if (d == 0 && skipFirst) continue; + if (is_file(in.searchDirs[d] / name)) { hit = in.searchDirs[d] / name; break; } + } if (hit) { out.members.push_back({name, *hit, std::filesystem::path(name)}); // Transitive: a staged library brings its own needs, and a diff --git a/src/scaffold/create.cppm b/src/scaffold/create.cppm index f83151ea3..9d7d4f7d1 100644 --- a/src/scaffold/create.cppm +++ b/src/scaffold/create.cppm @@ -403,8 +403,14 @@ int main() { { // `.mcpp/` is the per-project xlings sandbox (and, when no MCPP_HOME // can be resolved, the local BMI cache) — build state, never sources. + // `compile_commands.json` at the root names absolute paths of one + // machine's build (design 2026-09-26 §3.2): it is a copy of the + // current configuration's own database under `target/`, replaced + // whole on the next build, so a checkout gains nothing from shipping + // it and loses a warning the first time another machine's copy + // disagrees with this one. if (auto written = mcpp::scaffold::write_text_file( - root / ".gitignore", "target/\n.mcpp/\n"); !written) { + root / ".gitignore", "target/\n.mcpp/\ncompile_commands.json\n"); !written) { mcpp::ui::error(written.error()); return 1; } diff --git a/tests/e2e/211_configure_only_cdb.sh b/tests/e2e/211_configure_only_cdb.sh index 17f884cb9..a7afb0e2a 100755 --- a/tests/e2e/211_configure_only_cdb.sh +++ b/tests/e2e/211_configure_only_cdb.sh @@ -66,6 +66,11 @@ grep -q 'tests[\/][\/]*test_smoke.cpp' compile_commands.json || { exit 1 } if command -v python3 >/dev/null 2>&1; then + # The devkit path is derived from the entry's own `file`, in the spelling + # mcpp wrote: the shell's `$TMP` is not it on Windows (an MSYS root) or on + # macOS (`/var` against the resolved `/private/var`). `directory` no longer + # serves, because since C3 (design 2026-09-26 §3.3) it is the output + # directory the compiler runs in, not the project root. python3 - compile_commands.json <<'PY' import json, sys entries = json.load(open(sys.argv[1], encoding="utf-8")) @@ -73,16 +78,19 @@ normal = lambda p: p.replace("\\", "/").rstrip("/") test = next(e for e in entries if normal(e["file"]).endswith("/tests/test_smoke.cpp")) main = next(e for e in entries if normal(e["file"]).endswith("/src/main.cpp")) args = test["arguments"] -# Windows 原生进程与 MSYS 可能用不同根路径表示同一临时目录， -# 因此从 CDB 的 directory 字段推导相邻 devkit 路径。 -fixture = normal(test["directory"]).rsplit("/", 1)[0] -expected_include = f"{fixture}/devkit/include".casefold() -include_args = { +# Compared by the fixture directory's own name: Windows may spell the same +# temporary directory with its 8.3 short form in one field and its long form in +# another (`RUNNER~1` against `runneradmin`), and the name below it is the same +# in both. +project = normal(test["file"])[: -len("/tests/test_smoke.cpp")] +fixture_name = project.rsplit("/", 2)[-2] +expected_tail = f"/{fixture_name}/devkit/include".casefold() +include_args = [ normal(a[2:]).casefold() for a in args if a[:2].casefold() == "-i" -} -assert expected_include in include_args, args +] +assert any(i.endswith(expected_tail) for i in include_args), (expected_tail, args) assert any("MCPP_CONFIGURE_ONLY_TEST_FLAG=1" in a for a in args), args assert not any("MCPP_CONFIGURE_ONLY_TEST_FLAG=1" in a for a in main["arguments"]), main PY diff --git a/tests/e2e/47_cdb_prebuilt_module_path_abs.sh b/tests/e2e/47_cdb_prebuilt_module_path_abs.sh index 4b0cf2fb1..8a9d58438 100755 --- a/tests/e2e/47_cdb_prebuilt_module_path_abs.sh +++ b/tests/e2e/47_cdb_prebuilt_module_path_abs.sh @@ -3,12 +3,14 @@ # 47_cdb_prebuilt_module_path_abs.sh — `-fprebuilt-module-path` in # compile_commands.json must be an ABSOLUTE path, NOT a bare `pcm.cache`, # AND must not carry ninja-escape artefacts like `C$:` on Windows. -# Reason: CDB `directory` is the project root and clangd does `cd -# directory` before running the args, so a bare relative path points at -# `/pcm.cache` (missing) and a `C$:` prefix is treated as a -# literal string, not a Windows drive letter. Both modes silently break -# clangd's module resolution while `mcpp build` itself keeps working -# (ninja runs from outputDir AND unescapes its own escape sequences). +# Reason: since design 2026-09-26 §3.3 (C3), CDB `directory` IS the output +# directory the compiler runs in, so a bare relative `pcm.cache` would in +# fact resolve there too — but the flag is still rendered from the same +# absolute path the build's own command line uses (flags.cppm), and a +# `C$:` prefix would still be treated as a literal string, not a Windows +# drive letter, wherever `directory` points. Both modes would silently +# break clangd's module resolution while `mcpp build` itself keeps working +# (ninja unescapes its own escape sequences). set -e TMP=$(mktemp -d) @@ -89,9 +91,9 @@ while IFS= read -r v; do : else echo "FAIL: value is relative: '$v'" - echo " CDB 'directory' is the project root, but the BMI cache" - echo " lives under target/// — clangd resolves to" - echo " the wrong location and module imports fail." + echo " the flag must carry the same absolute BMI cache path" + echo " the build's own command line uses, whatever 'directory'" + echo " resolves to." fail=1 fi diff --git a/tests/e2e/688_emit_build_database.sh b/tests/e2e/688_emit_build_database.sh index 0ccffd860..70a9a59e7 100755 --- a/tests/e2e/688_emit_build_database.sh +++ b/tests/e2e/688_emit_build_database.sh @@ -200,17 +200,22 @@ EOF echo "ok: J, baseline and local arguments, private, config-files; K, the test's imports" # ── F (the S1 and compile-commands renderings agree) ────────────────────── +# D5a: the standard-library units are translation units of the build like any +# other (S1-12-1), so they are in BOTH renderings now, not just the S1 one -- +# no `mcpp:std` exclusion on the S1 side any more. "$MCPP" emit build-database --spec compile-commands > "$OUT/cc.json" 2> "$OUT/cc.err" \ || fail "F: --spec compile-commands exited non-zero" "$OUT/cc.err" "$PY" - "$OUT/env.json" "$OUT/cc.json" <<'EOF' || fail "F: the two renderings disagree" "$OUT/cc.json" import json, sys db = json.load(open(sys.argv[1]))["data"]["database"] cc = json.load(open(sys.argv[2])) -s1 = {u["source"]: u["arguments"] for s in db["sets"] if s["name"] != "mcpp:std" for u in s["translation-units"]} +s1 = {u["source"]: u["arguments"] for s in db["sets"] for u in s["translation-units"]} cdb = {e["file"]: e["arguments"] for e in cc} assert s1 == cdb, (sorted(s1), sorted(cdb)) +std_files = {u["source"] for s in db["sets"] if s["name"] == "mcpp:std" for u in s["translation-units"]} +assert std_files and std_files <= set(cdb), (std_files, sorted(cdb)) EOF -echo "ok: F, the S1 units and the compile-commands entries carry the same arguments" +echo "ok: F, the S1 units and the compile-commands entries carry the same arguments, std included" # ── G ────────────────────────────────────────────────────────────────────── set +e @@ -305,14 +310,23 @@ import json, sys emitted = json.load(open(sys.argv[1])) written = json.load(open(sys.argv[2])) # The one difference by construction is where the build writes: the planning -# pass writes under its work directory, configure-only under the project. +# pass writes under its work directory, configure-only under the project. The +# standard-library units' `output` names the shared std cache instead (D5a), +# outside either root, so the mapping is learned from a PROJECT entry -- +# `compile_commands.json` is sorted by `file` and a std source's absolute +# path may sort before a project one, so index 0 is not reliably a project +# entry any more. def slash(text): # One spelling for the comparison: a Windows argument may name a path with # either separator, and the mapping below is textual. return text.replace("\\", "/") -def write_root(entry): - return slash(entry["output"]).split("/target/")[0] -work, project = write_root(emitted[0]), write_root(written[0]) +def write_root(entries): + for e in entries: + s = slash(e["output"]) + if "/target/" in s: + return s.split("/target/")[0] + raise AssertionError("no project entry (an /target/ output) found") +work, project = write_root(emitted), write_root(written) def mapped(args): return [slash(a).replace(work, project) for a in args] e = {slash(x["file"]): mapped(x["arguments"]) for x in emitted} diff --git a/tests/e2e/779_a_build_program_declares_a_runtime_search_dir.sh b/tests/e2e/779_a_build_program_declares_a_runtime_search_dir.sh new file mode 100755 index 000000000..0de16ebaa --- /dev/null +++ b/tests/e2e/779_a_build_program_declares_a_runtime_search_dir.sh @@ -0,0 +1,217 @@ +#!/usr/bin/env bash +# requires: pack gcc +# 779_a_build_program_declares_a_runtime_search_dir.sh -- +# `mcpp::runtime_search_dir(dir)`: the build-program form of +# `runtime_search_dirs` (docs/04 §2.11). A dependency that brings a prebuilt +# shared library (a vcpkg prefix's `bin/`, a Qt SDK's `bin/`) knows where it +# lives only at build-program time, and the manifest key cannot be computed -- +# it is a fixed TOML array. This is the directive that closes that gap +# (design §5.4 R1', mcpp#701/#702). +# +# What this holds, each with the wrong answer it excludes: +# +# 1. THE DIRECTIVE GETS THE SAME RUNPATH TREATMENT THE MANIFEST KEY DOES: +# `-Wl,-rpath,` in build.ninja, never `-L` -- a launch-time search +# directory is not a link-library search path (docs/04 §2.11's own +# table), read from the emitted graph rather than inferred from a green +# build, the same way 62_runtime_library_dirs.sh checks the manifest key. +# 2. `mcpp run` FINDS THE LIBRARY ONLY THROUGH THAT RUNPATH: the executable +# is linked against it (DT_NEEDED), and the directory is nowhere else on +# any search path this build would otherwise consult. +# 3. THE CACHE TAG IS NON-EMPTY, SO A REPLAY CARRIES THE ENTRY: rebuilding +# on a build.mcpp CACHE HIT (not a re-run) must keep the RUNPATH and keep +# `mcpp run` working -- the same replay criterion `mcpp:deploy=` is held +# to in 651_a_build_program_deploys_what_it_generated.sh. +# 4. `mcpp pack --format dir` STAGES IT into the bundle's `lib/` -- the +# same closure search `runtime_search_dirs` already feeds. Checked +# LAST: `mcpp pack` builds under a different profile and would otherwise +# invalidate the `dev`-profile build.mcpp cache the replay check (3) +# depends on. +# 5. A DEPENDENCY'S DECLARATION REACHES THE CONSUMER'S EXECUTABLE: the +# directive is `Scope::LinkGlobal`, folded into `LinkIntent +# .runtimeSearchDirs` by the SAME per-package merge (`resolve_runtime_ +# contract`, plan.cppm) the manifest key goes through, so a path +# dependency that calls `runtime_search_dir` in its OWN build.mcpp makes +# its consumer's binary run without the consumer naming the directory at +# all. +# +# It lands DIRECTLY on `LinkIntent::runtimeSearchDirs`, not the retiring +# `RuntimeConfig::libraryDirs` (docs/04 §2.11) -- #702 as first written joined +# the wrong field (design §5.3 point 1); nothing here would tell them apart +# if it had not been fixed, since every reader below is fed FROM +# `runtimeSearchDirs` either way once the manifest key itself is also routed +# there. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +export MCPP_HOME=${MCPP_HOME:-$HOME/.mcpp} + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +cd "$TMP" +mkdir -p app/src app/rtlib + +# The prebuilt-plugin shape a real build.mcpp would locate (a vcpkg/Qt-style +# `bin/`): compiled directly with the toolchain here, standing in for that +# discovery the same way 62_runtime_library_dirs.sh's precompiled .so stands +# in for the manifest key's case. +cat > app/rtlib/plugin.c <<'EOF' +int runtime_plugin_answer(void) { return 42; } +EOF +gcc -shared -fPIC app/rtlib/plugin.c -o app/rtlib/libruntime_plugin.so + +cat > app/src/main.cpp <<'EOF' +extern "C" int runtime_plugin_answer(); +int main() { return runtime_plugin_answer() == 42 ? 0 : 1; } +EOF + +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[toolchain] +linux = "gcc@16.1.0" + +[targets.app] +kind = "bin" +main = "src/main.cpp" +EOF + +# The directive under test. `link_lib`/`link_search` also come from the +# program, on purpose: the point is a dependency the build.mcpp DISCOVERS, +# not one written into mcpp.toml by hand -- the case `runtime_search_dirs` +# cannot cover. +cat > app/build.mcpp <<'EOF' +import mcpp; +#include +int main() { + const std::string root = mcpp::manifest_dir(); + mcpp::link_lib("runtime_plugin"); + mcpp::link_search((root + "/rtlib").c_str()); + mcpp::runtime_search_dir((root + "/rtlib").c_str()); + return 0; +} +EOF + +MCPP="${MCPP:-mcpp}" +cd app +find_graph() { find target -name build.ninja | head -1; } + +# ── 1. a plain build gets the RUNPATH treatment, never -L ────────────────── +"$MCPP" build > b1.log 2>&1 || fail "build failed" b1.log +G=$(find_graph) +[ -n "$G" ] || fail "no build.ninja" b1.log +RTDIR=$(realpath rtlib) +grep -F -- "-Wl,-rpath,$RTDIR" "$G" >/dev/null \ + || fail "runtime-search-dir directive missing from RUNPATH intent" "$G" + +# ── 2. mcpp run finds the library only through that RUNPATH ──────────────── +"$MCPP" run > run1.log 2>&1 \ + || fail "run failed to find the plugin through the directive's RUNPATH" run1.log b1.log + +# ── 3. the replay criterion: a build.mcpp cache HIT keeps the directive ──── +# `mcpp pack` builds under a different profile and would invalidate this +# package's build.mcpp cache (a different ctx hash) before the replay is +# exercised, so the cache-hit rebuild happens BEFORE pack, on the same `dev` +# profile `mcpp build` and `mcpp run` already used. +touch src/main.cpp # past the whole-project no-op fast path, without + # touching build.mcpp itself +"$MCPP" build > b2.log 2>&1 || fail "second build failed" b2.log +grep -q "up to date (cached)" b2.log \ + || fail "the second build re-ran build.mcpp; the replay path was not exercised" b2.log +G2=$(find_graph) +[ -n "$G2" ] || fail "no build.ninja after the second build" b2.log +grep -F -- "-Wl,-rpath,$RTDIR" "$G2" >/dev/null \ + || fail "the RUNPATH directive did not survive a build.mcpp cache hit" "$G2" +"$MCPP" run > run2.log 2>&1 \ + || fail "run failed after the directive was replayed from a build.mcpp cache hit" run2.log b2.log + +# ── 4. mcpp pack --format dir stages it into the bundle's lib/ ───────────── +"$MCPP" pack --format dir > pack1.log 2>&1 || fail "pack --format dir failed" pack1.log +STAGED=$(find target/dist -name 'libruntime_plugin.so' | head -1) +[ -n "$STAGED" ] || fail "the runtime-search-dir closure was not staged by mcpp pack" pack1.log +case "$STAGED" in + */lib/libruntime_plugin.so) : ;; + *) fail "the staged library is not under the packed bundle's lib/" pack1.log ;; +esac + +cd "$TMP" + +# ── 5. a dependency's declaration reaches the consumer's executable ──────── +# `libdep` is a path dependency whose OWN build.mcpp declares the plugin's +# directory; `consumer` names none of this -- no `link_search`, no +# `runtime_search_dir`, not even `link_lib` in its own manifest or program -- +# and still runs, because the directive is LinkGlobal and the merge that +# folds a dependency's LinkIntent into its consumer's is the SAME one the +# manifest key goes through (plan.cppm ~:911-948). +mkdir -p libdep/src libdep/plugin + +cat > libdep/plugin/dep_plugin.c <<'EOF' +int dep_plugin_answer(void) { return 7; } +EOF +gcc -shared -fPIC libdep/plugin/dep_plugin.c -o libdep/plugin/libdep_plugin.so + +cat > libdep/src/lib.cpp <<'EOF' +extern "C" int dep_plugin_answer(); +int libdep_touch() { return dep_plugin_answer(); } +EOF + +cat > libdep/mcpp.toml <<'EOF' +[package] +name = "libdep" +version = "0.1.0" + +[modules] +sources = ["src/**/*.cpp"] + +[targets.libdep] +kind = "lib" +EOF + +cat > libdep/build.mcpp <<'EOF' +import mcpp; +#include +int main() { + const std::string root = mcpp::manifest_dir(); + mcpp::link_lib("dep_plugin"); + mcpp::link_search((root + "/plugin").c_str()); + mcpp::runtime_search_dir((root + "/plugin").c_str()); + return 0; +} +EOF + +mkdir -p consumer/src +cat > consumer/src/main.cpp <<'EOF' +extern "C" int dep_plugin_answer(); +int main() { return dep_plugin_answer() == 7 ? 0 : 1; } +EOF + +cat > consumer/mcpp.toml <<'EOF' +[package] +name = "consumer" +version = "0.1.0" + +[toolchain] +linux = "gcc@16.1.0" + +[dependencies] +libdep = { path = "../libdep" } + +[targets.consumer] +kind = "bin" +main = "src/main.cpp" +EOF + +cd consumer +DEPDIR=$(realpath ../libdep/plugin) +"$MCPP" build > b3.log 2>&1 || fail "consumer build failed" b3.log +G3=$(find_graph) +[ -n "$G3" ] || fail "no build.ninja for consumer" b3.log +grep -F -- "-Wl,-rpath,$DEPDIR" "$G3" >/dev/null \ + || fail "the dependency's runtime-search-dir did not reach the consumer's link" "$G3" +"$MCPP" run > run3.log 2>&1 \ + || fail "consumer run failed to find the dependency's plugin" run3.log b3.log + +echo "PASS: 779_a_build_program_declares_a_runtime_search_dir" diff --git a/tests/e2e/780_a_passing_check_moves_its_stamp_past_a_changed_input.sh b/tests/e2e/780_a_passing_check_moves_its_stamp_past_a_changed_input.sh new file mode 100755 index 000000000..7b3c8e2dc --- /dev/null +++ b/tests/e2e/780_a_passing_check_moves_its_stamp_past_a_changed_input.sh @@ -0,0 +1,77 @@ +#!/usr/bin/env bash +# 780_a_passing_check_moves_its_stamp_past_a_changed_input.sh — a `role = +# "check"` whose command writes no stamp, after one of its inputs changed. +# +# The engine writes such a check's stamp when the command succeeds (313). Until +# 2026.9.26.2 it only CREATED the stamp: an existing one was left alone. So once +# an input changed, the check ran and passed, and its stamp stayed older than +# that input -- and ninja ran it again on every build after, forever. Measured +# with mcpp-plugins' `deps-cmake`, whose check is a CMake build: every `mcpp +# build` after one edit re-ran configure, build and install. +# +# The criterion is the build AFTER the one that re-ran the check: it must not +# run the check again. Portable like 313: the command is the engine itself. +set -e + +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +mkdir -p src +cat > mcpp.toml <<'EOF' +[package] +name = "stampmove" +version = "0.1.0" +EOF +cat > src/main.cpp <<'EOF' +#include +int main() { std::printf("STAMP_MOVE_OK\n"); } +EOF +echo one > input.txt + +MCPP_HOST="$(host_path "$MCPP")" +INPUT_HOST="$(host_path "$TMP/input.txt")" +cat > build.mcpp < b1.log 2>&1 || { cat b1.log; echo "FAIL: build failed"; exit 1; } +ran b1.log || { cat b1.log; echo "FAIL: the first build did not run the check"; exit 1; } + +"$MCPP" build -v > b2.log 2>&1 || { cat b2.log; echo "FAIL: no-op rebuild failed"; exit 1; } +if ran b2.log; then cat b2.log; echo "FAIL: the check re-ran with nothing changed"; exit 1; fi + +# A changed input: the check runs once... +sleep 1 +echo two > input.txt +"$MCPP" build -v > b3.log 2>&1 || { cat b3.log; echo "FAIL: rebuild after the edit failed"; exit 1; } +ran b3.log || { cat b3.log; echo "FAIL: a changed input did not re-run the check"; exit 1; } + +# ...and not again: its stamp is now newer than the input. +"$MCPP" build -v > b4.log 2>&1 || { cat b4.log; echo "FAIL: rebuild after the check failed"; exit 1; } +if ran b4.log; then + cat b4.log + echo "FAIL: the check re-ran on the build after it passed -- its stamp stayed older than the input" + exit 1 +fi + +out="$("$MCPP" run 2>&1 | tail -1)" +[[ "$out" == *STAMP_MOVE_OK* ]] || { echo "FAIL: the program did not run: '$out'"; exit 1; } +echo "OK" diff --git a/tests/e2e/781_deleted_root_cdb_is_restored.sh b/tests/e2e/781_deleted_root_cdb_is_restored.sh new file mode 100755 index 000000000..a57cb6235 --- /dev/null +++ b/tests/e2e/781_deleted_root_cdb_is_restored.sh @@ -0,0 +1,65 @@ +#!/usr/bin/env bash +# requires: +# 781_deleted_root_cdb_is_restored.sh — C1 (design 2026-09-26 +# .agents/docs/2026-09-26-compile-database-and-issue-699-design.md §3.2 item +# 4): a deleted root compile_commands.json is restored by the very next +# `mcpp build`, through the FAST PATH -- no plan at all, since nothing on this +# tree changed. Before the fix (2026.9.26.1 and earlier), the fast path never +# reached a compile-database writer, so a deleted database stayed deleted +# forever (#397 C-1, #677 B1, report §3.6). +set -euo pipefail + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT + +cd "$TMP" +"$MCPP" new app > /dev/null +cd app + +"$MCPP" build > build1.log 2>&1 || { cat build1.log; echo "FAIL: the first build failed"; exit 1; } +[[ -s compile_commands.json ]] || { echo "FAIL: no compile_commands.json after the first build"; exit 1; } + +config_cdb=$(find target -name compile_commands.json | head -1) +[[ -n "$config_cdb" ]] || { echo "FAIL: no configuration database under target/"; exit 1; } +before=$(cat "$config_cdb") + +rm compile_commands.json + +"$MCPP" build > build2.log 2>&1 || { + cat build2.log + echo "FAIL: the build after deleting the root database failed" + exit 1 +} + +[[ -f compile_commands.json ]] || { + echo "FAIL: compile_commands.json was not restored" + cat build2.log + exit 1 +} + +# "Without a plan": nothing in the tree changed since the first build, so a +# real prepare pass has nothing to compile -- the restore must not trigger +# one. A plan prints a "Compiling " line; the fast path prints only +# "Finished ... in