From 3b3bf44819250b22a5980cf97a416e1310e18721 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Sat, 26 Sep 2026 02:42:10 +0800 Subject: [PATCH 1/3] 2026.9.26.1: paths in UTF-8, a C standard per package, a graph link without the host, and the level that was declared #693. mcpp held paths in the Windows ANSI code page, and the JSON it writes holds UTF-8 only, so every build under a non-ASCII project directory or home failed with an internal JSON exception, including names the code page can spell; a Latin-1 directory name fails the same way on Linux. mcpp.exe now declares the UTF-8 code page (res/mcpp.rc), build programs are linked with the same manifest, host tools default to it, and the new target key `windows_code_page` states it for a project's own executables. A path with no UTF-8 spelling is refused (project directory, MCPP_HOME), skipped and reported (a name inside a project), or refused by key (a build.mcpp directive), never an internal exception. The response files of the msvc dialect begin with a byte order mark, which cl.exe, link.exe and lib.exe need to read UTF-8 (measured), and Ninja's own encoding is checked when build.ninja is not ASCII. #695. `[build] c_standard` reached every C unit of the graph through the file-level $cflags, and a dependency's own value was not applied. Each package's C units now compile at that package's standard; an undeclared package at c11. #696. A link over a graph-supplied C library searched the host's library directories, so `-lm` linked glibc objects into a musl image. Such a link on ELF now carries --sysroot naming an empty directory, the hermetic check holds every -L to the store, the build directory and the graph's packages, and an unanswered -l fails with a note naming openkal-musl 0.19.2. #694. The musl -Og workaround is removed; the compile and the Finished line read one realised optimization level. Plan and measurements: .agents/docs/2026-09-25-issues-693-696-triage-and-repair-plan.md --- ...workspace-build-inheritance-consistency.md | 2 + ...5-issues-693-696-triage-and-repair-plan.md | 1282 +++++++++++++++++ .agents/docs/README.md | 4 +- .github/tools/check_unicode_paths.sh | 130 ++ .github/workflows/ci-windows.yml | 11 + .github/workflows/openkal-cross.yml | 9 +- CHANGELOG.md | 51 + docs/04-mcpp-toml.md | 111 +- docs/22-target-side.md | 23 + docs/30-build-mcpp.md | 15 + docs/91-toolchain-internals.md | 8 + docs/zh/04-mcpp-toml.md | 100 +- docs/zh/22-target-side.md | 17 + docs/zh/30-build-mcpp.md | 12 + docs/zh/91-toolchain-internals.md | 5 + mcpp.toml | 19 +- modules/buildmcpp/src/directives.cppm | 37 + modules/manifest/src/glob.cppm | 138 +- modules/manifest/src/toml.cppm | 21 +- modules/manifest/src/types.cppm | 39 +- modules/platform/src/windows/windows.cppm | 25 +- modules/versioning/src/version.cppm | 2 +- res/mcpp.exe.manifest | 8 + res/mcpp.rc | 12 + src/build/build_program.cppm | 30 + src/build/cache_key.cppm | 15 +- src/build/compile_commands.cppm | 16 +- src/build/execute.cppm | 12 +- src/build/flags.cppm | 88 +- src/build/hermetic.cppm | 62 +- src/build/ninja_backend.cppm | 184 ++- src/build/plan.cppm | 60 + src/build/prepare.cppm | 92 +- src/build/resources.cppm | 122 +- src/cli.cppm | 16 +- src/config.cppm | 16 + tests/e2e/190_link_rspfile_newlines.sh | 9 +- ...6_a_path_with_no_utf8_spelling_is_named.sh | 107 ++ ...applies_to_the_package_that_declares_it.sh | 98 ++ ...a_graph_link_searches_no_host_directory.sh | 96 ++ tests/unit/test_build_directives.cpp | 25 + tests/unit/test_build_resources.cpp | 37 +- tests/unit/test_c_standard_per_package.cpp | 243 ++++ tests/unit/test_hermetic_graph_link.cpp | 101 ++ tests/unit/test_manifest.cpp | 40 +- tests/unit/test_modgraph.cpp | 65 + tests/unit/test_ninja_backend.cpp | 282 +++- 47 files changed, 3775 insertions(+), 122 deletions(-) create mode 100644 .agents/docs/2026-09-25-issues-693-696-triage-and-repair-plan.md create mode 100755 .github/tools/check_unicode_paths.sh create mode 100644 res/mcpp.exe.manifest create mode 100644 res/mcpp.rc create mode 100755 tests/e2e/776_a_path_with_no_utf8_spelling_is_named.sh create mode 100755 tests/e2e/777_c_standard_applies_to_the_package_that_declares_it.sh create mode 100644 tests/e2e/778_a_graph_link_searches_no_host_directory.sh create mode 100644 tests/unit/test_c_standard_per_package.cpp create mode 100644 tests/unit/test_hermetic_graph_link.cpp diff --git a/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md b/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md index 9f5a13d7c..10a9968d0 100644 --- a/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md +++ b/.agents/docs/2026-09-25-issue-690-workspace-build-inheritance-consistency.md @@ -67,6 +67,8 @@ The decisions below are derived from these rules. Each rule names its source in | `dialect_cxxflags`, `c_standard`, `linkage`, `target`, `cxx_runtime`, `dependency_linkage`, `macos_deployment_target` | root manifest only | graph-wide | n/a | consistent | | `ios_deployment_target` | parsed and inherited | **refused** | n/a | **F3** | +**Correction (2026-09-26, mcpp#695).** The row above classifies `c_standard` as read from the root manifest only and calls that consistent. For C the classification is wrong. A C translation unit produces no BMI, so nothing requires one C standard across the graph, and the package's own cache key and fingerprint already recorded the value as the package's. The root's value reached every dependency's C units through the file-level `$cflags` line, and a dependency's own declaration was not applied. From mcpp 2026.9.26.1 each package's C units compile at that package's own standard (docs/04, "`c_standard` applies to the package that declares it"); the plan is `2026-09-25-issues-693-696-triage-and-repair-plan.md`. + ### 3.2 Inheritance placement: F1, F2, F8 **F1 (measured): `defines` are lost for a member reached as a `path` dependency.** On the dependency branch, `fold_build_defines_into_flags(dep_manifest->buildConfig)` (`src/build/prepare.cppm:8235`) folds and clears `defines`. `makePackageRoot` (`prepare.cppm:6709`) then calls `inherit_workspace_build`, which prepends the workspace `defines` to the folded manifest. It copies `cflags`/`cxxflags` into `privateBuild` without folding again. The failure is silent unless the source guards the macro. Workspace-wide defines are typically layout- or ABI-affecting (`_ITERATOR_DEBUG_LEVEL`, `_WIN32_WINNT`, `UNICODE`, `FMT_HEADER_ONLY`, `SPDLOG_ACTIVE_LEVEL`), so a member compiled without them is an ODR violation against members compiled with them. diff --git a/.agents/docs/2026-09-25-issues-693-696-triage-and-repair-plan.md b/.agents/docs/2026-09-25-issues-693-696-triage-and-repair-plan.md new file mode 100644 index 000000000..7cc3d17c4 --- /dev/null +++ b/.agents/docs/2026-09-25-issues-693-696-triage-and-repair-plan.md @@ -0,0 +1,1282 @@ +--- +subject: design +status: active +--- + +# Issues #693 to #696: triage against mcpp's contracts, and one repair plan + +- Issues: mcpp-community/mcpp#693 (Windows: a working directory outside the ANSI code + page ends `mcpp --version` with 0xC0000409), #694 (musl targets compile every + optimizing profile at `-Og`), #695 (`c_standard` on a dependency is accepted and + ignored), #696 (links over a graph-supplied C library still search the host's + `/usr/lib`). All four were filed on 2026-09-25. They are issues, not pull requests. +- Basis: `origin/main` 82867ad7 (mcpp 2026.9.25.1). Every citation of mcpp code is to that + commit. Other repositories: `openxlings/xlings` 84572b0, `mcpplibs/mcpp-index` 93781cf, + `mcpplibs/openkal-musl` f1f789f (0.19.1). +- Measurements: + - Linux x86_64, Ubuntu 24.04.1, glibc 2.39. The released mcpp 2026.9.25.1 was invoked by its + xlings store path. Toolchains: llvm@22.1.8, musl-gcc 15.1.0 and 16.1.0 (x86_64 native, + aarch64 cross), qemu-aarch64. + - Windows, through the temporary draft PR #697 on `windows-latest` (Windows Server 2025, + ACP 1252), with the released mcpp, the released xlings, and three toolchain rows (§6.2). + - The reporters measured on CachyOS (glibc 2.44) with 2026.9.21.3, and on Windows (ACP 936) + with 2026.9.25.1. +- Status: proposed. Nothing is implemented. + - Revision 1 (2026-09-25): the triage. + - Revision 2 (2026-09-26): records D1 to D4 as accepted; adds the Windows measurements for + #693, the UTF-8 model (§6.4) and the answer to D5; ends with a self-review (§11). + - Section 9 lists what remains for review. + +--- + +## 0. Summary + +| | Report | Verdict | Where the repair lives | The silent part | +|---|---|---|---|---| +| **#694** | musl targets compile every optimizing profile at `-Og` | **Engine defect.** A May 2026 workaround for a musl-gcc 15.1.0 ICE is keyed on the target triple, not on the compiler. Its trigger no longer reproduces on any input available today (§3.2). It also fixes the optimization level of both published Linux binaries of mcpp at `-Og`. | engine | the build prints `Finished release [optimized]` | +| **#695** | `c_standard` on a dependency is ignored; the root's value reaches every dependency | **Engine defect.** It violates P1 and P4 of the #690 record, contradicts docs/04 and docs/07, and disagrees with mcpp's own cache key and fingerprint. The #690 record classified the key incorrectly. | engine, after an ecosystem measurement | the key is parsed, hashed and never applied | +| **#696** | a link over a graph-supplied C library searches the host's library directories | **Engine defect (hermeticity), plus an ecosystem data gap.** The graph branch replaced the payload's `--sysroot` with nothing, and clang without a sysroot searches `/`. openkal-musl does not install the empty archives that musl's own `make install` provides. | openkal-musl (data) first, then engine | a glibc object inside a musl static image, and the hermetic check passes | +| **#693** | `mcpp --version` exits 0xC0000409, with no output, in a non-ACP directory | **The silent exit is an xlings defect**, measured: the xlings shim in front of mcpp throws during start-up, and `mcpp.exe` itself prints its version. **mcpp has a larger defect the report did not reach**, also measured: every build fails with an internal exception when the project or the mcpp home (the user profile) lies under a non-ASCII name. That includes names the ACP can represent, which on a Chinese system means any Chinese directory or user name. mcpp has no declared text encoding. | xlings (the silent exit); engine (a UTF-8 model, §6.4) | the xlings shim exits with no output | + +### 0.1 Classification of every finding + +| Issue | Finding | Class | +|---|---|---| +| #694 | `-Og` on musl targets for every compiler and every optimizing profile | engine defect | +| #694 | `Finished … [optimized]` derived from the declared level, not the level used | engine defect (reporting) | +| #694 | the reporter's `forced-o2` profile | usage-side workaround; valid, and it reaches only the root package | +| #694 | `mcpp.toml:52-56` says the aarch64 cross toolchain is gcc 15.1.0 | stale documentation | +| #695 | `c_standard` dropped on every non-root package; the root's value applied to all | engine defect | +| #695 | `cache_key.cppm:582` says the value "reaches its own C units" | engine defect (a false comment next to correct keying) | +| #695 | the #690 record lists `c_standard` as root-only and "consistent" | design-record error | +| #695 | compat.libaio, compat.libdrm, compat.libinput define `_GNU_SOURCE` and blame the value | usage-side workaround; valid, but the comments misattribute the cause | +| #695 | 13 descriptors and openkal-musl declare a standard that was never applied | ecosystem data; measured valid with clang on Linux (§4.6) | +| #696 | host library directories on graph links (ELF; PE through the MinGW driver) | engine defect | +| #696 | the hermetic check does not inspect `-L` and passes | engine defect (check coverage) | +| #696 | openkal-musl lacks the empty archives musl installs for `m`, `rt`, `pthread`, `crypt`, `util`, `xnet`, `resolv`, `dl` | ecosystem data gap | +| #696 | descriptors and users that link `-lm` on Linux | usage; correct | +| #693 | a silent exit with 0xC0000409 | **xlings defect** (F-693a, measured): an unhandled `std::system_error` in start-up, with no exception boundary | +| #693 | every build in a non-ASCII directory that the ACP can represent fails with an internal JSON exception, and so does every build under a non-ASCII user profile | **engine defect** (F-693b, F-693f, measured); not in the report, and wider than it | +| #693 | every build in a directory outside the ACP fails with an internal narrowing exception | engine defect (F-693c, measured) | +| #693 | the build-program contract has no encoding | engine defect (F-693d, measured) | +| #693 | on Linux, a directory name that is not UTF-8 fails the same way | engine defect (F-693e, measured) | +| #693 | process creation and environment through the -A APIs | engine design debt (W4e) | +| #693 | MinGW `as`, `ld`, `collect2` and `ar` without a UTF-8 manifest | external limitation; harmless while they receive relative paths (measured), to be named if that changes | +| #693 | a directory name outside the ACP | usage; correct | + +### 0.2 Three statements + +1. **No reported usage is wrong.** Every manifest, descriptor and command in the four reports + uses a documented feature correctly. The usage side has only data that the engine defects + hid. + - Thirteen descriptors and openkal-musl declare a `c_standard` that has never been applied to + them as dependencies. Measured with clang on Linux, all fourteen build at their declared + value (§4.6). + - Three descriptors define `_GNU_SOURCE` instead of declaring `gnu11`, and their comments + say the value has no effect. The cause is the dependency position, not the value. +2. **Every defect has a silent part, and the silent part does more damage.** Each engine path + overrides, drops or bypasses a declared value without a diagnostic. In #694 the build's own + label contradicts the flags it used. In #696 the check that exists for this class of error + reports success. In #693 the silent part is xlings's: mcpp's own failures print an error, + although the error is an internal exception rather than a diagnostic (§7). +3. **Two structural rules cover three of the four.** + - The file-level flag variables carry only graph-wide values. #695 is a case where they did + not, and #690 F7 was the case before it. + - What the build reports is what it used. #694's label, #695's cache key and fingerprint, and + #696's hermetic check each describe something the build did not do (§7). + +--- + +## 1. Method + +- Each finding is marked *measured* or *reasoned*. A measured finding has a command and its + output in Appendix A. A reasoned finding cites a code path. Measurements made by a reporter + are attributed to the reporter. Where this review repeated one, it says so. +- Statements about the implementation were read from `origin/main` with `git show`. The local + checkout was five commits behind and was not used. +- Probe builds of mcpp itself ran in a detached worktree of 82867ad7 under the session scratchpad. + One temporary profile was appended to `mcpp.toml` (§3.2) and removed afterwards. Nothing was + committed. +- The criterion for "a flag reached a unit" is that unit's entry in `compile_commands.json`. + Build success is never used as the criterion. +- Windows facts come from a temporary draft PR (#697). Its branch removes every other + workflow and runs one PowerShell script. Every observation is a `READING` line in the log; + nothing is uploaded, and a step fails only when the probe itself cannot run. Each + non-ASCII name is built from code points, so the script's source is ASCII. The PR is closed + once its readings are recorded here (Appendix A.10). + +--- + +## 2. Principles + +The #690 record's P1 to P8 apply unchanged. Two of them decide these issues. P1 is position +independence: a package's compile inputs do not depend on which consumer reached it. P4 is scope: +a package's private build requirements reach only its own units, and nothing flows from a +consumer into a dependency. Five further rules are used below. + +| | Rule | Source in mcpp | Elsewhere | +|---|---|---|---| +| **Q1** | A declared value is honoured, or refused with a diagnostic. It is never replaced or dropped silently. | `mcpp.diag`'s batch invariant: "a branch doing LESS because a precondition was not met owes the user an `impact` sentence" (`src/cli.cppm:115-117`). | Cargo warns on manifest keys it does not use. | +| **Q2** | Each question has one answerer. What the build reports is read from what the build used. | `src/build/execute.cppm:1034-1036` states this for the profile descriptor. | | +| **Q3** | The engine carries no toolchain-defect workarounds (decision D5, §9). A defect in a toolchain version is answered by the toolchain pin, which is data. A project that has to keep a defective version scopes its own mitigation with `[build] cxxflags` or `[build] flags = [{ glob, cxxflags }]`, which is usage. The engine's guarantee is that the level it realises is the level declared, and a property test enforces it. | The v0.0.1 comment named the compiler and a retirement condition that nothing executed; the code keyed the workaround on the target (§3.4). | A version pin is how every package manager excludes a defective release. | +| **Q4** | Host state reaches a build only through a named, minimal surface. When the graph supplies the C library, no host library directory, startup file or loader reaches the link. | The hermetic link model (`.agents/docs/2026-07-07-hermetic-toolchain-link-model-design.md`). | Bazel and Nix sandboxes; `--sysroot` as the standard way to scope a cross driver. | +| **Q5** | mcpp's text is UTF-8 on every platform. Everything it hands to another program is in the encoding that program reads. Text that enters from outside is validated where it enters (§6.4). | #516 and #518; the measurements of §6.2. | Ninja, CMake, LLVM and Cargo are UTF-8 inside and convert at the boundary (§6.4). | + +The routing rule for upstream reports applies to every item. The order of preference is usage, +project plugin, official plugin, ecosystem data, engine. An engine item is valid when the defect +is general. The rule filters features, not defects. + +--- + +## 3. #694: the musl `-Og` workaround + +### 3.1 The report + +On a `*-linux-musl` target, every profile with a non-zero `opt` compiles at `-Og`, with GCC and +with Clang. docs/04 §2.9 says `release` is `-O2` and `dist` is `-O3`. The build still prints +`[optimized]`. The condition dates from v0.0.1 (92f1335e), whose comment gave the reason: +musl-gcc 15.1.0 hit an ICE in `tree-ssa-ccp` on libstdc++'s `std::format` (`__write_padded`) at +`-O2`. The comment also said `TODO(musl-gcc-upstream): remove once musl-gcc@16+ ships`. The +reporter reasoned, without checking the artefacts, that both published Linux binaries of mcpp +are compiled at `-Og`. The proposed fix keeps the workaround for libstdc++ only. + +### 3.2 Verification + +- **Reasoned.** `src/build/flags.cppm:984-988`: + `opt_flag = isMuslTc && prof.optLevel != "0" ? " -Og" : …`. `isMuslTc` is + `is_musl_target(plan.toolchain)` (L858), which reads only the target triple. The flag is part of + the file-level `$cflags`/`$cxxflags` (L1072-1082), so it reaches every unit of every package. +- **Measured: the report's fixture on 2026.9.25.1.** `optlevel` with clang 22.1.8 and + `--release --target x86_64-linux-musl`: 1621 entries in `compile_commands.json` carry `-Og`, + and the 3 assembly entries carry no `-O`. The output ends with `Finished release [optimized]`. + This repeats the reporter's result on the current release. +- **Measured: the release configuration of mcpp itself.** In the 82867ad7 worktree, + `mcpp build --release --configure-only --target x86_64-linux-musl` resolves `gcc@16.1.0` + (the manifest's pin), and so does the same command for `aarch64-linux-musl` (the target table's + pin, `modules/toolchain-model/src/triple.cppm:450-451`). Every entry carries `-Og`, including + all 178 units of the binary: 127 in `src/`, 48 in `modules/` and 3 in `mcpplibs.cmdline`. + `release.yml` builds the published binaries with exactly these commands (L132, L348, L351), + and `mcpp.toml:14` sets `default-profile = "release"`. **Both published Linux binaries have + been compiled at `-Og` since v0.0.1.** This was established from the build configuration, not + from the artefacts, which are stripped and record no switches. +- **Measured: the premise no longer holds.** The ICE was not reproduced on any input available + on 2026-09-25: + + | Input | Compiler | Level | Result | + |---|---|---|---| + | The report's `std::format` line, header mode and `import std` module mode | musl-gcc 15.1.0 and 16.1.0, x86_64 and aarch64 | `-O2` (header mode also `-O3`) | compiles | + | mcpp at 82867ad7, root units raised to `-O2` by a probe profile | x86_64-linux-musl-gcc 16.1.0 | 127 root units at `-Og -O2` | builds; `--version` runs | + | the same | aarch64-linux-musl-gcc 16.1.0 (cross) | 127 root units at `-Og -O2` | builds; `--version` and `--help` run under qemu-aarch64 | + | mcpp at v0.0.1 (92f1335e), the code that hit the ICE | x86_64-linux-musl-gcc 15.1.0 | 23 root units at `-Og -O2` | every unit compiles, with no ICE; the link fails on an unrelated GCC 15 module defect (`undefined reference to std::optional::optional(optional&&)`) | + | mcpp at 82867ad7 | x86_64-linux-musl-gcc 15.1.0 | `-Og`, all 178 units | fails with `exposes TU-local entity`, a front-end check that does not depend on the optimization level | + + The last row means that current mcpp cannot be built with musl-gcc 15.1.0 at any level. For + mcpp's own build, the workaround protects nothing. The probe has one limitation. Profile flags + reach only the root package, so the 51 units outside `src/` stayed at `-Og`. The removal PR + compiles all 178 units at `-O2` in CI, and that run is the final criterion. + +### 3.3 Classification + +This is an engine defect. The profiles, the docs and the reporter's manifests are all correct. The +reporter's `forced-o2` profile is a usage-side workaround, and it reaches only the root package's +units. + +### 3.4 Root cause + +1. **A compiler defect was answered in the engine, and keyed on the wrong axis (Q3).** An ICE + is a property of one compiler version, so its place is the version pin, which is data. The + code instead reads the target, so when clang gained musl targets through openkal, it + inherited a GCC workaround. +2. **The override is silent, and the label reads the value that was overridden (Q1, Q2).** + `execute.cppm:1039-1040` derives `[optimized]` from `bc.optLevel`, while the compile uses + `opt_flag`. The comment above it says the descriptor "cannot disagree with the compiler flags", + and on musl it does. +3. **The retirement condition had nothing to execute it.** Two later changes touched the line: + 53f85a68 (#24) shortened the comment and dropped the reason, and 40215eb2 (#109) added the + `opt = 0` exception. Neither re-tested the premise, and the TODO's condition (musl-gcc 16 is + pinned) has been true for some time. + +### 3.5 Repair (W1) + +- **W1a.** Delete the musl branch of `opt_flag`. The optimization flag becomes a function of the + profile and the dialect only. +- **W1b.** Make the profile descriptor and the flag read one value (Q2), for example a single + function over the plan that both `compute_flags` and `execute.cppm` call. Once W1a lands they + agree anyway. The change is small, and it keeps them in agreement the next time something + adjusts the level. +- **W1c.** Correct `mcpp.toml:52-56`, which says the aarch64 cross toolchain is gcc 15.1.0 from + musl-cross-make. The target table pins `gcc@16.1.0`, and the probe resolved + `aarch64-linux-musl-gcc@16.1.0`. +- **Not recommended: the report's narrower patch**, which keeps `-Og` when the standard library + is libstdc++. It keeps a workaround whose trigger does not reproduce, on the one path where the + cost falls on mcpp's own release binaries. +- **No workaround layer (D5).** The engine needs no mechanism for toolchain workarounds, and + none is added. An inventory of `origin/main` for engine behaviour that responds to a compiler + defect found exactly one case that changes a declared value, the `-Og` above. Every other + match is one of two things. Some are a choice of mechanism that is the conventional contract, + such as publishing clang's reduced BMI rather than its full BMI (`model.cppm:477-501`, + measured). The rest arrange mcpp's own sources around a clang 22.1.8 miscompile, which + concerns mcpp's code and not the builds of its users. The general answer has three parts: + 1. **The toolchain is data, so a defective version is answered in data.** The target table + and `[toolchain]` pin a version without the defect, as `gcc@16.1.0` already does. + 2. **A project that must keep a defective version** mitigates it in its own manifest, scoped + to the unit that triggers it: `[build] flags = [{ glob = "src/x.cpp", cxxflags = [...] }]` + exists for this (`GlobFlags`, `modules/manifest/src/types.cppm:280`). The mitigation is + then visible, reviewable and limited to the one translation unit. + 3. **W1d, a property test instead of a rule.** For every row of the target table, both + compiler families and every profile level, the optimization token in `compute_flags` equals + the dialect's spelling of the declared level. A future change that overrides a declared + level fails this test, instead of depending on a reviewer remembering a rule. + +### 3.6 Criteria + +- W1d, a property test in `tests/unit/test_ninja_backend.cpp` next to the other + `compute_flags` cases. For every row of `kKnownTargets`, with GCC and with Clang, and for + `opt` in `0`, `1`, `2`, `3`, `s`, the realised optimization token is the dialect's spelling of + the declared level: `-O2` and no `-Og` for `opt = 2` on `x86_64-linux-musl`, `-Os` for + `opt = "s"`, and so on for every row. +- The report's fixture: `--release --target x86_64-linux-musl` produces zero `-Og` entries. +- The release workflow's two musl builds and the e2e suite pass, and the release job's + `compile_commands.json` has zero `-Og` entries. + +### 3.7 Risk + +The published Linux binaries change optimization level for the first time since v0.0.1. The +exposure is low. mcpp's code is already compiled at `-O2` on every other release platform (macOS +and Windows use clang). Linux CI builds mcpp with a plain `mcpp build` (`ci-linux.yml:143`, +`ci-linux-e2e.yml:71`), which is gcc 16.1.0 on x86_64-linux-gnu under the release profile, so the +e2e suite already runs against an `-O2` build from the same compiler family. The one musl leg in +CI (`ci-linux.yml:238`) is the one that runs at `-Og`. The release notes state the change. + +--- + +## 4. #695: `c_standard` on a dependency + +### 4.1 The report + +`[build] c_standard` takes effect only on the package being built. On a dependency it is parsed +and accepted but never used. This covers path dependencies, workspace members reached from +another member, and index packages, whether the key is in the package's `mcpp.toml` or in a +descriptor. The root's value (default `c11`) reaches every C unit of every dependency. The cache +key and the fingerprint record the value per package. + +### 4.2 Verification + +- **Measured: path dependency, 2026.9.25.1, clang 22.1.8, host target** (the report's `cdep` + and `app` fixture). `cdep.c` fails its `__STRICT_ANSI__` guard, and the file-level line in + `build.ninja` is `cflags = -std=c11`. The reporter's other positions (workspace member, + `compat.zlib` descriptor, openkal-musl's 1343 units) were not repeated here. They use the same + code path, which is unchanged on 82867ad7. +- **Reasoned.** The standard comes from the root manifest (`flags.cppm:1015-1016`) and is placed + in the file-level `$cflags` (L1078-1082). A unit's `$unit_cflags` carry its package's `cflags` + and no standard (`src/modgraph/scanner.cppm:1466-1477`). Meanwhile + `src/build/cache_key.cppm:581-584` adds `__c_standard=` under the + comment "A package may pin its own C standard; it reaches its own C units". That comment is + false. `src/build/prepare_inputs.cppm:735` fingerprints the package's value. `cache_key.cppm:529` + also keys on the root's value, so the cache is sound (P5), but it is keyed on an input the + package does not control. + +### 4.3 Classification + +This is an engine defect, and three sources say so. + +- **The contract.** docs/04 lists `c_standard` in the same `[build]` table as `cflags`, as + "Standard for C source files (default c11)". §3.4 of the same document shows it on a pure C + library. The merge table in docs/07 treats it as a member-level scalar. +- **The #690 principles.** Under P1, a dependency's C compile depends on its consumer. Under P4, + a consumer's value flows into its dependency. +- **The engine's own records.** The cache key and the fingerprint treat the value as belonging to + the package. + +The #690 record (§3.1) lists `c_standard` among the keys read "from the root manifest only" and +calls that "consistent". **That classification is wrong for C.** The C++ standard is graph-wide +for a reason: a BMI must be read at the level at which it was produced. A C translation unit +produces no BMI, and a program that links C objects compiled under different standards is +ordinary. The record needs a correction note in §3.1. + +On the usage side, nothing is wrong. `compat.libaio`, `compat.libdrm` and `compat.libinput` use +`-D_GNU_SOURCE` because `gnu11` did nothing. That is a correct response to the defect. Their +comments, however, place the fault in the value ("accepted and silently emits `-std=c11`"). The +value works on a root package; it is the dependency position that drops it. After W3 lands, the +comments should be corrected. Switching those descriptors to `gnu11` is optional. + +### 4.4 Root cause + +The C standard travels on the file-level `$cflags`, a graph-wide channel. The per-package value +exists in three places (manifest, fingerprint, cache key), but no code places it on the package's +units. This is the same family as #690 F7, the include directories broadcast through the same +channel, which #691 removed. + +### 4.5 Repair (W3) + +- **Semantics.** A package's effective C standard is its own declared value, after workspace + inheritance, or the engine default `c11`. It never comes from the consumer. The root is treated + like any other package, so a root's `c_standard` reaches only the root's own C units (decision + D2). +- **Mechanism.** The file-level `$cflags` carries the engine default, `-std=c11`, as a graph-wide + constant. A package whose effective standard differs gets its dialect's spelling appended to its + C units' per-unit flags, which come after `$cflags` in the `c_object` rule, so the later + `-std=` wins on GCC and Clang. This is the mechanism `implementationStandardFlag` already uses + for C++ (`src/build/plan.cppm:1725-1770`). Only the packages that declare a non-default value + see their command lines change. +- **W3b, the MSVC dialect: a separate step, measured first.** Today no C `/std:` is emitted for + `cl.exe` at all (`flags.cppm:1078`), so the key is silently dropped for every package, the + root included. Mapping a declared `c11` to `/std:c11`, or `c17` to `/std:c17`, is not a + neutral change. + - Microsoft documents that both options also switch on the conforming preprocessor + (`/Zc:preprocessor`), and that without them `cl` compiles C89 with Microsoft extensions. + - 90 descriptors declare `c11`, so all of them would change mode on that row at once. + - W3b therefore does two things. It maps only declared values, never the engine default. + It reports the values `cl` cannot express (`c99`, `gnu*`) once per package as `degraded`. + - It ships after mcpp-index CI has run the MSVC row against it. + - The llvm row on Windows is not affected, because it drives clang with GNU spellings and + takes W3 as is. +- **Cache.** Remove the root's `cStandard` from the dependency key (`cache_key.cppm:529`). After the + change it no longer reaches any dependency command. Keep the per-package `__c_standard` entry, + whose comment then becomes true. This also makes docs/04 §2.10 true for C dependencies: "A + dependency's artifacts do not depend on who consumes them". +- **Documents.** Update the docs/04 row: the standard applies to this package's C units, a + dependency keeps its own, and the default is `c11`. The docs/07 table stays as written and + becomes true. Append a correction note to the #690 record, §3.1. +- **Rejected: graph-wide with refusal on non-root packages.** It contradicts P1, P4 and the + documentation. It would refuse 13 descriptors and openkal-musl that are written per package. It + would also forbid something C permits without restriction. + +### 4.6 Blast radius (P8) + +Thirteen descriptors in mcpp-index declare a non-default standard: eleven declare `c99`, +`compat.ffmpeg` declares `c17` and `compat.freetype` declares `gnu11`. openkal-musl declares +`c99`. None of these has ever been compiled at its declared value as a dependency. The value is a +claim that has never been tested, and W3 is the first time it takes effect. + +**Measured, with no engine change.** Under the current engine, a consumer that declares a +dependency's value applies that value to the dependency, so the per-package semantics can be +emulated one package at a time. The side effect is that the dependency's transitive C +dependencies receive the value too, so failures are attributed by object path. Each row below is a +control build (root default `c11`) and a treatment build (root = the declared value), with +2026.9.25.1 and clang 22.1.8: + +| Package | Declared | C units at that standard in the treatment graph | Control | Treatment | +|---|---|---|---|---| +| compat.cjson 1.7.19 | `c99` | 1 | builds | builds | +| compat.ffmpeg 8.1.2 | `c17` | 2124 | builds | builds | +| compat.freetype 2.13.3 | `gnu11` | 60 | builds | builds | +| compat.glad 0.0.0-651a425 | `c99` | 1 | builds | builds | +| compat.hiredis 1.2.0 | `c99` | 7 | builds | builds | +| compat.libpng 1.6.43 | `c99` | 30 | builds | builds | +| compat.libuv 1.48.0 | `c99` | 35 | builds | builds | +| compat.lua 5.4.7 | `c99` | 32 | builds | builds | +| compat.md4c 0.5.3 | `c99` | 1 | builds | builds | +| compat.sdl2 2.32.10 | `c99` | 744 | builds | builds | +| compat.tray 0.0.0-8dd1358 | `c99` | 1 | builds | builds | +| compat.yyjson 0.12.0 | `c99` | 1 | builds | builds | +| compat.eui-neo 0.5.9.1 | `c99` | 641 | builds | builds | +| openkal-musl 0.19.1 (through openkal-llvm-runtime 0.15.1, `x86_64-linux-musl`) | `c99` | 1512 | builds | builds | + +**All fourteen build at their declared standard.** Before accepting the result, the probe was +checked for the failure shape "the treatment was served from a cache compiled at another +standard", in three ways. + +- No treatment build printed `Cached`. +- `.ninja_log` shows the treatment's object edges were executed as compiles, not restored. For + compat.libpng, the control restored the objects from the cache (3 ms per edge, and the build + printed `Cached compat.libpng v1.6.43 (15 units)`), while the treatment compiled them (29 to + 80 ms per edge). +- For the same object, the command hashes differ between control and treatment, and + `SDL_audio.c` carries `-std=c99` in the treatment. + +The measurement covers clang 22.1.8 on a Linux host only. + +Two more sources of evidence apply before release: mcpp-index CI run against the W3 branch +through `MCPP_SOURCE_REF`, and openkal's own measurement set. A descriptor that fails at its +declared value is a data error. It is corrected in mcpp-index before the engine release, not +worked around in the engine. + +### 4.7 Criteria + +- The report's fixture builds, and `cdep.c`'s entry carries `-std=gnu11`. +- A root that declares `c99` leaves a dependency's C units at the dependency's own value, or at + `c11` when the dependency declares none. +- A workspace member's declared value holds under `-p app` and under `-p lib`. +- A descriptor package (`compat.zlib`, `c11`) keeps `c11` while the root declares `c99`. +- On a musl target, openkal-musl's units carry `-std=c99`. +- Two consumers that differ only in `c_standard` share one cache entry for a C dependency: the + second build prints `Cached`. + +--- + +## 5. #696: host library directories on graph links + +### 5.1 The report + +When the C library comes from the dependency graph (openkal-musl), the link line that mcpp builds +still lets clang add the build machine's library directories. `-nostdlib` removes the startup +files and the default libraries, but not the search directories. A `-l` that the graph does not +answer is therefore looked up on the host. On aarch64-linux-musl, glibc's `libm.a` linker script +(`OUTPUT_FORMAT(elf64-x86-64)`) breaks the link. On x86_64-linux-musl, glibc objects are linked in +without any message. The hermetic check misses this too, because it inspects only startup objects +and the loader. + +### 5.2 Verification + +- **Measured: the driver on a second distribution** (Ubuntu 24.04, clang 22.1.8, the report's + mcpp-free commands). + + | Target | `-L` added by the driver | `-lm` resolves to | + |---|---|---| + | aarch64-unknown-linux-musl | `/lib/../lib64`, `/usr/lib64`, `/lib`, `/usr/lib` | `unable to find library -lm` | + | x86_64-unknown-linux-musl | eight directories, including `/usr/lib/gcc/x86_64-linux-gnu/13` and `/usr/lib/x86_64-linux-gnu` | `/lib/x86_64-linux-gnu/libm.a`, then `libm-2.39.a` and `libmvec.a` | + + The aarch64 symptom differs from CachyOS because Ubuntu keeps glibc's archives in the + multiarch directory. The cause is the same. The outcome depends on the host's layout, which is + the point of the report. +- **Measured: through mcpp 2026.9.25.1** (the report's `fmaximum` fixture, x86_64-linux-musl). + The build succeeds and the program prints `2`. `--why-extract` shows: + + ``` + obj/main.o /usr/lib/x86_64-linux-gnu/libm-2.39.a(s_fmaximum.o) fmaximum + ``` + + A glibc object sits inside a musl static image, and nothing reports it. +- **Measured: the repair's two halves.** `--sysroot=` removes every `-L` on + both targets, and `-lm` then fails with `unable to find library -lm`. Adding `-L` to a directory + that holds one empty `libm.a` (8 bytes, `!\n`) makes `-lm` resolve to it, and both + targets link. +- **Reasoned: the mechanism.** On a payload link, `lm.link_flags()` carries + `--sysroot=` (`flags.cppm:814`). The graph branch replaces the host's link model + (L785-810 and L1944-2042) and emits no sysroot in its place. A clang with no sysroot derives its + search directories from `/`, and on x86_64 from the host's GCC installation. **The replacement + removed the one token that had kept the host out.** `src/build/hermetic.cppm` dry-runs the + driver (L149) and checks CRT objects (L188) and the loader, but not `-L`. With `-nostdlib` there + is nothing for it to find, so it writes `.mcpp-hermetic-ok` (L139), including for the aarch64 + link that then fails. + +### 5.3 Classification + +- **Engine defect**, under Q4. It is the link-side twin of #664, which closed the compile side + with `-nostdlibinc`. The hermetic check does not cover the class it exists for. +- **Ecosystem data gap.** musl's own `make install` places empty archives for `m`, `rt`, + `pthread`, `crypt`, `util`, `xnet`, `resolv` and `dl` next to `libc.a`, because libc holds all of + their contents. openkal-musl does not provide them. +- **Usage: correct.** Linking `-lm` on Linux is right for glibc and for musl alike. In mcpp-index, + 30 of 233 descriptors link at least one of the eight names in code, with Lua comments removed. + The counts are `-lm` 13, `-lpthread` 21, `-ldl` 10, `-lrt` 6 and `-lresolv` 2. + +### 5.4 Repair (W2, three parts in a fixed order) + +- **W2a (openkal-musl, data).** Ship the eight empty archives in a package directory, and add a + package-relative `-L` for it to `[target.'cfg(os = "linux")'.build] ldflags`. mcpp already + resolves a dependency's relative `-L` against the dependency's root and forwards it to the + consumer's link line (`src/build/prepare.cppm:7064-7097`). No engine change is needed, which is + why the routing rule puts this half in data. Release chain: openkal-musl, then + openkal-llvm-runtime (pins are exact), then the index. +- **W2b (engine).** The graph branch passes `--sysroot=` to + clang, first on ELF targets. On PE targets that use the MinGW driver it follows one inventory + (measured in the table below: the same defect, the same repair). The inventory is the `-l` names + that the index's `x86_64-windows-musl` builds resolve from a host MinGW today, read from + `-Wl,--verbose` in index CI. Import libraries such as `-lws2_32` may be among them, and each must + be answered by the graph (openkal-windows) before the PE half lands. A `-l` the graph does not + answer then fails as `unable to find library`, which is the graph's true answer. Add a diagnostic for that failure shape on graph links. It names + the graph's C library and its version, following the precedent that explains `file not found` + under `-nostdlibinc` (`src/build/ninja_backend.cppm:140-156`). +- **W2c (engine).** Extend the hermetic check to graph links. Every `-L` in the driver's linker + invocation must lie under an allowed prefix: the xpkgs registry, the build directory, or the + root of a graph package. A later edit that brings a host directory back then fails in CI instead + of passing silently. +- **Other object formats and drivers.** + + | Link | Host search directories | Status | + |---|---|---| + | PE through clang's MinGW driver (`x86_64-w64-windows-gnu`, which is what mcpp hands clang for `x86_64-windows-musl`; `triple.cppm:485`) | This Linux host has Ubuntu's mingw-w64 installed. The driver adds `-libpath:/usr/lib/gcc/x86_64-w64-mingw32/13-win32`, `/usr/x86_64-w64-mingw32/lib` and `/usr/x86_64-w64-mingw32/mingw/lib`, and `-lm` reads `/usr/x86_64-w64-mingw32/lib/libm.a`. With `--sysroot=`, only directories under the empty sysroot remain, and `-lm` is `unable to find library`. | **measured**: same defect, same repair | + | PE through `lld-link` in MSVC mode | reads `%LIB%` unless `/lldignoreenv` is given | to be measured on a Windows host | + | GCC over the graph | its configured sysroot and library directories (#664 handled the compile side separately) | to be measured | + | Mach-O | ld64.lld searches `/usr/lib` and `/usr/local/lib`. The SDK is the declared platform anchor and is legitimate. | to be measured | + + The criterion is the same for each: the linker line from `-###` names no directory outside the + allowed prefixes. The PE row matters beyond Windows hosts. mcpp-index measures + `x86_64-windows-musl` on Linux runners, where the result depends on whether the runner image + has mingw-w64 installed. +- **Order and cliff.** W2a must be released, and pinned through the openkal chain, before W2b + ships. Otherwise every openkal consumer on a Linux target that links `-lm` breaks at once. + Consumers that pin an older openkal-llvm-runtime still break when W2b ships. D3 accepted that + cliff (§9.1): W2b ships without a warning-only release, and its diagnostic names the version + that fixes it. + +### 5.5 Criteria + +- The `fmaximum` fixture on x86_64-linux-musl stops at `undefined symbol: fmaximum`, and the + linker's `--verbose` output contains no host path. +- The report's aarch64 `-lm` fixture links, and the binary runs under qemu-aarch64. `-###` lists + only store, build-directory and package `-L`. +- A negative test: a graph link that receives `-L/usr/lib` through `ldflags` is refused by the + hermetic check, which names the directory. +- mcpp-index gains `x86_64-linux-musl` and `aarch64-linux-musl` rows in its openkal measurement. + Today it measures `x86_64-linux-gnu` and `x86_64-windows-musl` only (`tests/openkal/pins.toml`), + which is why this has not shown up there. + +--- + +## 6. #693: paths outside ASCII on Windows, and one text-encoding model + +### 6.1 The report + +On Windows with `GetACP() == 936`, mcpp 2026.9.25.1 exits with 0xC0000409 and prints nothing when +it runs `--version` or `build --configure-only` from a directory whose name contains U+1F9EA +(TEST TUBE), which code page 936 cannot represent. The same `mcpp.exe` works in an ASCII +directory. The Ninja that mcpp brings reports `Build file encoding: UTF-8`. + +The report proposes three steps: +1. a UTF-8 `activeCodePage` manifest in `mcpp.exe` and `build.mcpp.exe`; +2. an explicit boundary, with UTF-8 inside and UTF-16 wide APIs at Win32; +3. verification of each downstream tool. + +### 6.2 Measurement + +This host cannot run Windows, so the facts below come from a temporary draft PR (#697). The +branch keeps one measurement workflow and removes every other; it is closed once the readings are +recorded here. The runs are 36162457075, 36163057371, 36163850821 and 36164076852. The first +run's build readings are void because of a defect in the probe, recorded in Appendix A.10. + +**Setup.** +- Machine: `windows-latest`, Windows Server 2025, build 26100. The system ANSI code page is + 1252 and the OEM code page is 437. +- Binaries: + - the released mcpp 2026.9.25.1 from its release zip; + - a copy of the same `mcpp.exe` into which `mt.exe` embedded a UTF-8 `activeCodePage` + manifest (called "the UTF-8 copy" below); + - xlings 2026.9.20.1, installed the way a user installs it. +- Directories, written as code points: + + | Name used below | Path | In cp1252? | + |---|---|---| + | ascii | `C:\w\ascii` | yes | + | café | `C:\w\caf` + U+00E9 | yes, but its cp1252 bytes differ from its UTF-8 bytes | + | CJK | `C:\w\repro-` + U+6D4B U+8BD5 + `-` + U+1F9EA | no | + +- Toolchains: llvm@20.1.7 with the runner's MSVC 14.51 and Windows SDK 10.0.26100, `cl.exe` + through `msvc@system`, and MinGW-w64 `gcc@16.1.0`. The Ninja is 1.12.1, which reports + `Build file encoding: UTF-8`. + +**Q1. Which process ends with 0xC0000409.** Each entry point ran `--version` in each directory: + +| Entry point | ascii | café | CJK | +|---|---|---|---| +| `mcpp.exe` from the release zip | 0 | 0 | **0** | +| `mcpp.bat` from the release zip | 0 | 0 | **0** | +| the UTF-8 copy | 0 | 0 | 0 | +| the xlings shim `mcpp.exe`, inside the workspace that pins mcpp | 0, `mcpp 2026.9.25.1` | 0 | **0xC0000409, no output** | +| `xlings --version` | 0 | 0 | **0xC0000409, no output** | + +`cdb` was run on `xlings --version` in the CJK directory. It records a first-chance C++ +exception (`e06d7363`) whose `what()` is "No mapping for the Unicode character exists in the +target multi-byte code page." That is the MSVC STL's `std::system_error` from narrowing a path. +The exception is unhandled (second chance), which reaches `std::terminate` and the fast fail. +The stack is eleven frames inside `xlings.exe` (stripped), so the throw happens during start-up, +before any command runs. + +**The silent exit in #693 belongs to xlings, not to mcpp.** The reporter's `mcpp` resolved to +the xlings shim. + +**Q2 and Q3. Builds** (`mcpp build`, llvm row, three fixtures: `import std`, a narrow +`build.mcpp`, and a `build.mcpp` that uses the wide environment and prints UTF-8): + +| Directory | Released `mcpp.exe` | The UTF-8 copy | +|---|---|---| +| ascii | all three build and run | all three build and run | +| café | **every build fails**: `error: internal: unhandled exception: [json.exception.type_error.316] invalid UTF-8 byte at index 9: 0x2F` | the `import std` fixture builds and runs; `build.ninja` and `compile_commands.json` carry `63 61 66 C3 A9` (UTF-8) and no cp1252 form | +| CJK | every build fails: `error: internal: unhandled exception: No mapping for the Unicode character exists in the target multi-byte code page.` (exit 70) | the `import std` fixture builds and runs; `build.ninja` carries `E6 B5 8B E8 AF 95 2D F0 9F A7 AA` | + +Build programs, with the UTF-8 copy: + +| `build.mcpp` | café | CJK | +|---|---|---| +| narrow (`getenv`, `std::ofstream(std::string)`, bytes printed back) | The program writes its file and prints the path in cp1252 bytes. mcpp then fails with `internal: unhandled exception: No mapping for the Unicode character…`. | The program receives `C:\w\repro-??-??\…` and cannot open it. It exits 2, and mcpp reports that exit. | +| UTF-8 (wide environment and path, UTF-8 output) | builds and runs | builds and runs | + +**The other two Windows rows, with the UTF-8 copy.** Two fixtures ran (`#include` and +`import std`); every cell is "builds and runs", with UTF-8 bytes in `build.ninja`: + +| Row | ascii | café | CJK | +|---|---|---|---| +| MSVC (`cl.exe`, `msvc@system`) | builds and runs | builds and runs | builds and runs | +| MinGW-w64 (`gcc@16.1.0`, `x86_64-windows-gnu`) | builds and runs | builds and runs | builds and runs | + +In the MinGW payload, `gcc.exe`, `g++.exe`, `cc1.exe` and `cc1plus.exe` declare a UTF-8 +`activeCodePage`. `collect2.exe`, `as.exe`, `ld.exe` and `ar.exe` carry a manifest without one. +They still succeeded, because Ninja runs in the build directory and hands them relative, ASCII +paths. None of these projects produced a response file, so response-file encoding was not +exercised. + +**A non-ASCII `MCPP_HOME`, with an ASCII project.** This is the shape of a Windows account +whose user name is not ASCII, because the default home is `%USERPROFILE%\.mcpp`: + +| `MCPP_HOME` | Released `mcpp.exe` | The UTF-8 copy | +|---|---|---| +| `C:\mh-caf` + U+00E9 (in cp1252) | The toolchain downloads and installs. Every build then fails with the JSON exception of the café row above. | builds and runs | +| `C:\mh-` + U+6D4B U+8BD5 (not in cp1252) | fails at once: `error: cannot create 'C:\mh-??\bin'` | mcpp itself proceeds. The xlings it vendors cannot initialise its sandbox under that home: ``warning: `xlings self init` failed for sandbox at 'C:\mh-测试\registry'``, then `sandbox not initialized`. | + +**The Linux twin (measured on this host).** A project directory whose name is not valid UTF-8 +(the single byte 0xE9) fails the same way on Linux with mcpp 2026.9.25.1: +`internal: unhandled exception: [json.exception.type_error.316] invalid UTF-8 byte at index 123: +0x2F`. + +### 6.3 Findings + +| | Finding | Evidence | Home | +|---|---|---|---| +| **F-693a** | xlings throws an unhandled `std::system_error` during start-up when the working directory is outside the ACP. Its `main` has no exception boundary, so the process ends silently with 0xC0000409. | measured (Q1, cdb) | xlings | +| **F-693b** | In a directory the ACP can represent but that is not ASCII, every mcpp build fails with an internal JSON exception. mcpp holds paths as ACP bytes, and the JSON it writes requires UTF-8. On a Chinese system (ACP 936) this is every directory with a Chinese name, which is more common than the report's case: GBK byte sequences are, with rare accidental exceptions, not valid UTF-8. The first JSON document on that path is `compile_commands.json`, whose writer is `src/build/compile_commands.cppm`, and the file is absent after the failure. | measured on cp1252; the cp936 case and the writer are reasoned | mcpp | +| **F-693c** | Outside the ACP, every mcpp build fails with an internal narrowing exception. The failure is reported (exit 70), not silent. | measured | mcpp | +| **F-693d** | The build-program contract has no encoding. mcpp passes paths through the environment, which is converted through the program's own ACP, and reads the program's output as bytes. A narrow build program and mcpp disagree as soon as either one is not in the ACP. | measured | mcpp | +| **F-693e** | On POSIX, path bytes that are not UTF-8 reach the same JSON serializer. | measured (Linux) | mcpp | +| **F-693f** | F-693b applies to the home as well as the project. With 2026.9.25.1, a Windows user whose account name is not ASCII cannot build any project, even an ASCII one, when the name is in the ACP. That is the common case: a Chinese name on a Chinese system. Outside the ACP, the vendored xlings fails too, even under the UTF-8 copy. | measured on cp1252; the cp936 case is reasoned | mcpp; xlings for a home outside the ACP | + +- **Usage:** correct everywhere. A directory name with any Unicode character is valid on every + platform. +- **Root cause:** mcpp has no declared text encoding. On Windows its strings are in whatever the + process ACP is. The formats it writes and the tools it drives assume UTF-8: JSON by + definition, Ninja by its manifest, clang internally. On POSIX, bytes flow unvalidated into + formats that accept only UTF-8. + +### 6.4 One text-encoding model: UTF-8 everywhere + +The review asked whether mcpp can adopt UTF-8 as its single model, weighing compatibility, +cross-platform behaviour and established build conventions. **It can, and the measurements +support it.** Established practice: + +| Tool | Text inside | At the Windows boundary | Files it writes for other tools | +|---|---|---|---| +| Ninja 1.11 and later | bytes, read as UTF-8 | UTF-8 `activeCodePage` manifest (Windows 10 1903 and later); `ninja -t wincodepage` reports the build-file encoding it expects | — | +| CMake 3.2 and later | UTF-8 | wide APIs | build files in UTF-8; the Ninja generator had to stop writing ANSI once Ninja moved to UTF-8 | +| LLVM and clang | UTF-8 | wide APIs; the command line through `GetCommandLineW` | response files in the encoding each consumer reads: UTF-8 or UTF-16 for clang, UTF-16 for MSVC's `CL.exe` and `LINK.exe`, ANSI for GNU tools on MinGW (`clang::driver::ResponseFileSupport`) | +| MSVC tools | UTF-16 | wide | read response files and `.DEF` files as UTF-16 or UTF-8 **with a BOM**, and as ANSI otherwise | +| GNU tools on MinGW | narrow `argv` | the ANSI code page; a GCC patch (PR108865) embeds a UTF-8 manifest in the driver, and mcpp's `gcc@16.1.0` payload carries it in the driver and compilers but not in `as`, `ld` or `collect2` (measured) | read response files as ANSI only | +| Rust and Cargo | UTF-8 (WTF-8 for OS strings) | wide APIs only | — | +| Microsoft's guidance | — | `activeCodePage` UTF-8 lets code written against the -A APIs run in UTF-8 on Windows 10 1903 and later; convert with `CP_UTF8` explicitly, because `CP_ACP` equals `CP_UTF8` only under the manifest | — | + +**The model for mcpp, M1 to M7:** + +- **M1. One encoding inside.** Every string mcpp holds for a path, an argument, an environment + value or file content is UTF-8, on every platform. +- **M2. Windows process setting.** `mcpp.exe` declares `activeCodePage` UTF-8. Measured + sufficient: mcpp's own stages, and all three Windows rows end to end, in both kinds of + non-ASCII directory. Every path that writes to a console must render UTF-8. `std::print` + does, through `WriteConsoleW`. C stdio and iostream do not unless the console output code + page is 65001, and this has to be checked in the implementation, because CI has no console. +- **M3. The programs mcpp runs as part of a build share M2.** This covers `build.mcpp` and + rule-package programs (D4, measured necessary), and host tools run by actions (D6). It also + covers the `xlings.exe` that mcpp vendors and runs with its own paths: under a home outside + the ACP, mcpp in UTF-8 still fails until xlings is in UTF-8 too (measured, F-693f). +- **M4. Every file mcpp writes for another tool uses the encoding that tool reads.** + - `build.ninja` is UTF-8. Ninja 1.11 or later is already required through + `ninja_required_version = 1.11`. mcpp checks that the encoding Ninja declares + (`-t wincodepage`) equals its own: UTF-8 under M2, or ANSI in the legacy mode of M7. A + mismatch is refused by name. + - `compile_commands.json` is UTF-8. + - Response files are UTF-8 for LLVM tools and UTF-8 with a BOM for MSVC tools. For GNU tools + on MinGW, a path the ANSI code page cannot represent is refused with a diagnostic that names + the tool, instead of being written wrong. + - Resource scripts are already compiled as UTF-8: the rc tool receives `/C 65001` or + `--codepage=65001` (`src/build/prepare.cppm:14579-14582`). Nothing changes here. +- **M5. Validate at the point of entry.** Text that comes from outside mcpp is validated as + UTF-8 where it enters: POSIX file names, build-program output, and POSIX environment values + used as paths. Invalid input is refused with a diagnostic that names its source, never with + an internal exception. No JSON writer lets an exception reach `main`. +- **M6. Scope.** The programs mcpp builds for the user (`mcpp run`, the artefacts) keep their + own encoding model, and mcpp adds no manifest to them. A `[resources]` key that declares one is + a possible later feature, and `[resources] files` already allows it. POSIX behaviour is + unchanged apart from M5. +- **M7. Older hosts (D7).** Windows 10 1809 and earlier, and Windows Server 2019 (build 17763), + ignore the manifest. mcpp checks `GetACP()` at start-up and reports the legacy mode in one + `degraded` line when a path is not ASCII. W4e removes the dependency on the OS version. + +**Compatibility.** +- **ASCII paths.** UTF-8 and every ANSI code page agree on ASCII, so for ASCII paths + `build.ninja`, `compile_commands.json` and every command line are byte-identical. There are + no rebuilds and no cache changes. +- **Non-ASCII project and home paths.** Every build under one already fails on 2026.9.25.1 + (F-693b, F-693c, F-693f), so no working configuration of this kind can regress. State that + 2026.9.25.1 installed under a non-ASCII home remains usable: the UTF-8 copy built with the + toolchain the released binary had installed there (§6.2). +- **Names that #516 and #518 skipped.** A file outside the ACP inside an otherwise buildable + project used to be skipped, with a report naming its directory. Under M2 it is no longer + skipped, so a source glob that matched such a file now compiles it. Today's skip report + names every affected directory, so the projects concerned can be found before the release. +- **Build programs.** A narrow build program in a non-ASCII directory fails today, because + mcpp fails first. After M2 without M3 it would still fail (§6.2), which is why D4 and M3 are + one change. +- **Readers of mcpp's piped output.** Programs that decode mcpp's piped output as the ANSI code + page see UTF-8 for non-ASCII text. ASCII output is unchanged. + +### 6.5 Repair (W4) + +- **W4a (xlings, upstream).** Report F-693a and the xlings half of F-693f: + - the reproduction: `xlings --version` in a directory outside the ACP, and `self init` under a + home outside it; + - the exception text, and the missing boundary in `main`. + + The suggested fix is the M2 manifest for `xlings.exe`, and therefore for its shims, plus an + exception boundary. The report's symptom is xlings's, so this item closes #693 as reported. + mcpp vendors xlings (`registry/bin/xlings.exe`), so a home outside the ACP also waits on + this item. +- **W4b (mcpp).** + - The M2 manifest for `mcpp.exe`, through mcpp's own `[resources] files` (an `RT_MANIFEST` + at ordinal 1, docs/04 §2.15). + - The same manifest for build programs, through the engine's host-compile path (M3, D4), and + for host tools subject to D6. + - The start-up check of M7. +- **W4c (mcpp).** M5: validation at the entry points, with diagnostics instead of internal + exceptions. This covers build-program output, POSIX path names, and the JSON writers. +- **W4d (mcpp).** M4: the Ninja encoding check, and response files per tool. Measure + response-file encoding with a project large enough to need one, which this measurement did + not produce. +- **W4e (mcpp, later).** Wide APIs at the Win32 call sites, with explicit UTF-8 conversion for + `std::filesystem::path`, which means construction from `std::u8string`. This removes the + dependence on the manifest and on the OS version. The call sites are listed in Appendix A.11. +- **W4f (CI).** The measurement workflow becomes a regression job on `windows-latest`. It runs + the ascii, café and CJK directories against the llvm, MSVC and MinGW rows, plus the + build-program fixture. It asserts success, and it asserts UTF-8 bytes in `build.ninja` and + `compile_commands.json`. +- **Terminate handler, demoted to optional hardening.** mcpp's own `main` reported every mcpp + failure the measurement produced. A terminate handler would add coverage only for exceptions + that cross a `noexcept` boundary, and no measured case needs it. + +**Criteria.** +- W4f is green on all rows, including a non-ASCII `MCPP_HOME`. The row with a home outside the + ACP turns green once X lands and mcpp's xlings pin moves to that release. +- On Linux, a directory whose name is not UTF-8 is refused with a diagnostic that names it. +- A narrow build program that prints a path which is not UTF-8 is reported by name, and mcpp + does not stop with an internal exception. +- "Unicode paths are supported" is claimed only for the rows that W4f runs, as #518 required. + +--- + +## 7. What the four have in common + +**C1. Silence.** + +| | What went unreported | What should have reported it | +|---|---|---| +| #694 | a profile's level replaced by `-Og` | the `Finished` descriptor, which instead printed `[optimized]` | +| #695 | a declared key dropped | the manifest reader; three descriptors recorded it as "the value does nothing" | +| #696 | a glibc object linked into a musl image | the hermetic check, which wrote its OK marker | +| #693 | a process that died | xlings's `main`, which has no exception boundary; mcpp's own failures were reported, but as internal exceptions | + +The rule is Q1 together with P7. `mcpp.diag`'s `degraded` channel exists for exactly this case. + +**C2. The file-level flag variables work as a broadcast channel.** #695's `-std=` travels on +`$cflags`, which every C unit of every package reads. #690 F7 was the include-directory case on +the same channel. #694's `-Og` travels there too, and there it is legitimate: the optimization +level a profile chooses is graph-wide, as in Cargo. In #694 the defect is the override, not the +channel. + +The rule: file-level variables carry only graph-wide values, meaning the toolchain, the target, +the profile, and values that must be uniform (such as the C++ standard, for BMIs). Anything a +package declares for itself travels per unit. + +This rule is enforced by construction and by a test, not by a document (D5). The direct form is +a flag model in which each token carries its scope (graph, package or unit), and the backend +builds the file-level variables from graph-scoped tokens only. A misplaced package value is then +a type error rather than a review finding. The minimal form, which W3 adds, is a negative test: +build a plan whose root declares every package-private key (`c_standard`, `include_dirs`, +`private_include_dirs`, `defines`, `cflags`, `cxxflags`), and assert that none of those values +appears in the file-level `$cflags` or `$cxxflags`. Reasoned from §4.2, the tree at 82867ad7 +fails this test on `c_standard` alone, and passes it once W3 lands. + +**C3. Two answerers per question.** + +| Question | First answerer | Second answerer | +|---|---|---| +| which optimization level | the descriptor | the flag | +| which C standard | the cache key and fingerprint | the compile line | +| can the link reach the host | the hermetic check's model (CRT and loader) | the linker's actual inputs | +| which encoding | mcpp's strings (the ACP) | the JSON writer (UTF-8), Ninja (UTF-8), the build program (its own ACP) | + +Each pair agrees on the configurations CI exercises and disagrees on the others. The +configurations where they agree are: + +- glibc targets; +- graphs in which no dependency declares its own C standard; +- graph links in which the graph answers every `-l`; +- ASCII paths. + +A green build on a configuration where both answerers agree carries no information about the +difference between them. + +**C4. The coverage has the same shape.** Every defect sits on an axis that CI does not exercise: +musl with an optimizing profile, a C dependency with its own standard, a graph link on aarch64 +or with an unanswered `-l`, and Windows with a non-UTF-8 ACP and a non-ASCII path. The criteria +above sit on those axes. The CI additions are: + +1. a musl release build whose `compile_commands.json` is asserted; +2. e2e coverage of the per-package C standard in each position; +3. a graph link with `-lm` on aarch64-linux-musl under qemu, a PE graph link from a host that + has mingw-w64 installed, and the hermetic negative test; +4. the W4f job: on a runner whose ACP is verified not to be 65001, the ascii, café and CJK + directories against the llvm, MSVC and MinGW rows, a build program, and a non-ASCII home. + +**C5. A design record can be wrong in a table cell.** #690 §3.1 labelled `c_standard` +"consistent" without a criterion. A classification that nothing measured is not a finding. + +--- + +## 8. Plan + +The tracks below group the work by issue. §12.1 gives the pull requests, one per repository, +and the order in which they land. + +| Track | Content | Repository | Depends on | Size | +|---|---|---|---|---| +| **A** | W1a to W1d (#694): delete the workaround, one answerer for the level, the stale comment, the property test | mcpp | nothing | small | +| **B** | W3 (#695): semantics, mechanism, cache key, documents, and the C2 negative test | mcpp | mcpp-index CI through `MCPP_SOURCE_REF`, which adds GCC and the other hosts to the §4.6 sweep (clang on Linux: 14 of 14 build) | medium | +| **B2** | W3b: the MSVC dialect's `/std:` for declared C standards | mcpp | B, and mcpp-index CI on the MSVC row | small | +| **C1** | W2a: openkal-musl publishes the eight empty archives and the `-L` | openkal-musl | nothing | small | +| **C2** | openkal-llvm-runtime moves its openkal-musl pin; index registration | openkal-llvm-runtime, mcpp-index | C1 released and mirrored | small | +| **C3** | W2b for ELF links, and W2c (#696): graph-link `--sysroot`, the diagnostic, the hermetic `-L` check | mcpp | C2 in the index | medium | +| **C4** | mcpp-index gains linux-musl rows in its openkal measurement, and records which `-l` names its `x86_64-windows-musl` builds resolve from a host MinGW | mcpp-index | C3 released | small | +| **C5** | W2b for PE links through the MinGW driver | mcpp (and openkal-windows for any name C4 finds) | C4's inventory answered by the graph | small | +| **U0** | the measurement (§6.2, PR #697) | done | — | — | +| **U1** | W4b and W4f: the UTF-8 manifest for `mcpp.exe` and for build programs (and for host tools, per D6); the start-up check (M7); the Windows regression job over three directories and three rows | mcpp | nothing | medium | +| **U2** | W4c and W4d: validation at the entry points, response files per tool, `.rc` code page | mcpp | U1 | medium | +| **U3** | W4e: wide APIs at the Win32 call sites and explicit UTF-8 conversions | mcpp | U1 | medium to large | +| **X** | W4a: xlings start-up exception, the manifest for `xlings.exe` and its shims, an exception boundary in `main` | xlings (upstream issue) | nothing | small | + +Ordering notes, following the release discipline already recorded for this repository: + +- Registering a package and moving a pin are separate PRs. +- A consumer's pin moves only after the index entry has merged. +- Track B changes what 13 descriptors and openkal-musl compile, so it ships only after + mcpp-index CI is green against the branch. +- Track A changes the optimization level of the published Linux binaries, so the release note + states it. +- Tracks A, U1 and X depend on nothing and can go first. The U tracks are independent of + tracks B and C. +- #693 as reported closes with X. mcpp's own share, U1 and U2, fixes the larger defect that + the measurement found: every non-ASCII directory. +- U1 changes nothing for ASCII paths (§6.4, Compatibility), so it needs no ecosystem build + measurement before release. Two things are still checked: + - the index's Windows CI logs are searched for today's `path/codepage` skip reports, because + those files stop being skipped; + - the release note states that mcpp's piped output carries UTF-8 for non-ASCII text. + +--- + +## 9. Decisions + +### 9.1 Accepted in review (2026-09-26) + +D1 to D4 were accepted on the first review, and D5 to D7 on the second. + +| | Decision | +|---|---| +| **D1** (#694) | Remove the musl `-Og` workaround entirely (W1a). The report's narrower patch is not taken. | +| **D2** (#695) | A package's C standard is its own declared value, or the engine default `c11`; it never comes from the consumer (W3). | +| **D3** (#696) | openkal-musl ships musl's empty archives as data (W2a). W2b ships directly after the openkal chain, **with no warning-only release**. The cliff for consumers that pin an older openkal-llvm-runtime is handled by W2b's diagnostic, which names the version that fixes it. | +| **D4** (#693) | Build programs carry the same UTF-8 `activeCodePage` manifest as `mcpp.exe`. The measurement (§6.2) turned this from a recommendation into a requirement: with only `mcpp.exe` in UTF-8, a narrow build program fails in every non-ASCII directory. | +| **D5** | No workaround layer and no rules in a skill (§3.5, §7 C2). Defective toolchain versions are answered by the pin (data) and, in a project that keeps one, by per-file flags (usage). Two structural guarantees are enforced by tests: W1d (the level realised is the level declared) and the C2 negative test (no package-private value in a file-level variable). | +| **D6** | Host tools built for the graph carry the UTF-8 manifest by default. A target opts out with `windows_code_page = "legacy"` (§12.2), and the default yields to a manifest the package embeds itself (§12.4). | +| **D7** | On a Windows host that ignores the manifest, mcpp reports the legacy mode when a path is not ASCII, and keeps the skip-and-report behaviour of #516 and #518. Realised inside the diagnostics each such path reaches, which name the process code page (§12.4). | + +--- + +## 10. Incidental observations + +- `--toolchain gcc@15.1.0-musl` did not override `[target.x86_64-linux-musl] toolchain`. The + build resolved `gcc@16.1.0` and printed nothing about the flag it ignored. Whether the per-target + pin should outrank the flag was not investigated. The silence is the finding. +- mcpp at 82867ad7 cannot be built with musl-gcc 15.1.0 (`exposes TU-local entity`). v0.0.1 + compiles with it but does not link. On musl targets, mcpp's effective minimum GCC is 16. +- docs/04 §2.10 states that a dependency's artefacts do not depend on who consumes them. For C + dependencies this is false until W3 lands. +- The #690 record needs a correction note on `c_standard` in §3.1. +- `compile_commands.json` is an auxiliary artefact for editors, yet a failure to write it stops the + whole build (F-693b). After M5 it cannot fail on encoding. A failure to write an auxiliary + artefact should still be reported as `degraded` rather than abort the build that produces + it. + +--- + +## 11. Self-review + +Revision 2 was read end to end against four questions: whether each claim carries the evidence +it states, whether the sections agree, which risks the plan still hides, and what remains +unmeasured. + +**Corrections made in this revision.** +- Revision 1 predicted that non-ASCII paths the ACP can represent would fail at the first compile + edge, because Ninja reads the ACP bytes as UTF-8. The measurement found the failure earlier, in + mcpp's own JSON writer (F-693b), and wider: it covers the user profile as well (F-693f). The + conclusion held; the mechanism in revision 1 was wrong. +- Revision 1 left open which process dies in #693. It is xlings (F-693a), and the terminate + handler proposed for mcpp is demoted to optional hardening. +- The cp936 statements in F-693b and F-693f are marked as reasoned from the cp1252 measurement. + They were not measured on a cp936 machine. + +**Risks found during this review and folded into the plan.** +- **W3b.** Mapping a C standard to `/std:c11` on `cl.exe` also switches on the conforming + preprocessor, and 90 descriptors declare `c11`. The MSVC mapping is separated from W3 and + measured first (track B2). +- **W2b on PE.** Revision 1 would have applied `--sysroot` to PE links through the MinGW driver + at once. The index's `x86_64-windows-musl` builds may resolve import libraries from a host + MinGW today, so an inventory (C4) precedes the PE half (C5). +- **M4.** In the legacy mode of M7, Ninja declares ANSI. The check therefore compares the two + encodings; it does not require UTF-8. +- **Compatibility of M2.** Files that #516 and #518 skipped are no longer skipped. Today's skip + reports in the index's Windows CI are read before the release. +- **X is on mcpp's path.** A home outside the ACP needs the xlings fix and a move of mcpp's + xlings pin, not only the manifest in `mcpp.exe`. + +**Probe defects of this review, recorded so that the readings can be trusted.** +- In the first Windows run, the loop variable `$fx` overwrote the fixtures directory `$Fx`. + PowerShell names are case-insensitive, so every build "ran" in 0 s. Those readings are void, + and the script now records a start error as a probe defect. +- The §4.6 sweep was checked for the treatment being served from a cache compiled at another + standard. `.ninja_log` durations and command hashes rule it out. +- The #694 probe raised only the root package to `-O2`. The removal PR's CI covers the other 51 + units. + +**What remains unmeasured.** + +| Item | Where it gets measured | +|---|---| +| response-file encodings (M4, W4d); no project in the measurement produced one | W4d, with a project large enough to need one | +| console rendering under M2 through C stdio and iostream | the implementation, on a machine with a console, because CI has none | +| Windows hosts older than 1903 (M7, D7) | no such runner is available; reasoned from Microsoft's documentation | +| an actual cp936 machine | the reporter's machine can confirm; the mechanism is the one measured on cp1252 | +| host tools under the manifest (D6) | the implementation PR | +| GCC over graph links, Mach-O graph links, `lld-link` in MSVC mode (§5.4) | before W2b is extended to them | +| W3 with GCC and on non-Linux hosts | mcpp-index CI through `MCPP_SOURCE_REF` | + +**Scope, checked against the routing rule.** +- Every engine item repairs a defect; none adds a product feature. +- W2a is data, and X is upstream in xlings. +- The one item that would be a feature, a user-facing `[resources]` key for the code page, is + deferred, because `[resources] files` already allows it. + +--- + +## 12. Implementation + +### 12.1 Repositories, pull requests and order + +Each repository receives one pull request. The exception is mcpp-index, because registering a +version and moving a pin are always separate changes. The order follows the release chain: +a dependency is tagged, mirrored and registered before anything pins it. + +| # | Repository | Pull request | Depends on | Released as | +|---|---|---|---|---| +| 1 | openxlings/xlings | X: the UTF-8 `activeCodePage` manifest in `xlings.exe` (and therefore in its shims) through `[resources] files`; an exception boundary in `main`; a Windows CI step that runs xlings in a directory outside the ACP | nothing | xlings 2026.9.26.1, mirrored to GitCode | +| 2 | mcpplibs/openkal-musl | W2a: the eight empty archives and a package-relative `-L`; CI asserts that `-lm` resolves inside the package | nothing | 0.19.2, mirrored to GitCode | +| 3 | mcpplibs/mcpp-index | registers openkal-musl 0.19.2 | 2 | index artifact | +| 4 | mcpplibs/openkal-llvm-runtime | pins openkal-musl 0.19.2 | 3 | 0.15.2, mirrored to GitCode | +| 5 | mcpplibs/mcpp-index | registers openkal-llvm-runtime 0.15.2 | 4 | index artifact | +| 6 | mcpp-community/mcpp | W1a to W1d, W2b (ELF) and W2c, W3 with the W3b report, W4b to W4d, W4f, D6, D7, the documents, and `kXlingsVersion` moved to 2026.9.26.1 | 1 and 5 | mcpp 2026.9.26.x; GitCode by the local gtc; the xim-pkgindex bump | +| 7 | mcpplibs/mcpp-index | pins `tests/openkal/pins.toml` to runtime 0.15.2 and the new mcpp, adds the `x86_64-linux-musl` and `aarch64-linux-musl` rows, moves `latest_mcpp`, and corrects the three comments that blamed `gnu11` | 5 and 6 | index artifact | + +Three items are split out, each for a stated reason. +- **The W3b mapping for `cl.exe`.** No CI row builds the index with `cl.exe`: the Windows row of + `validate.yml` drives clang. Under P8, the mapping waits for such a row. Until then, this + round reports the unapplied value (Q1) and does not change what `cl.exe` compiles. +- **W4e, wide APIs at the Win32 boundary.** It is all or nothing. Converting only the process + calls would decode ACP strings as UTF-8 on the hosts that ignore the manifest. The manifest + covers Windows 10 1903 and later, and D7 covers the rest. +- **W2b for PE links (C5).** It waits for the C4 inventory of the import libraries that the + index's `x86_64-windows-musl` builds resolve from a host MinGW. + +### 12.2 mcpp, by change + +- **W1** (`src/build/flags.cppm`, `src/build/execute.cppm`, `mcpp.toml`, + `tests/unit/test_ninja_backend.cpp`): + - The musl branch of `opt_flag` goes. + - One function names the realised optimization level, and both the flags and the + `Finished` descriptor read it. + - The stale aarch64 comment is corrected. + - The W1d property test runs over every row of `kKnownTargets`. +- **W3** (`src/build/flags.cppm`, `src/build/plan.cppm`, `src/build/cache_key.cppm`, + `docs/04`, `docs/07`, tests): + - The file-level `$cflags` carries `-std=c11` as a graph-wide constant. + - A package whose effective standard differs gets its spelling in its C units' flags, + through the mechanism `implementationStandardFlag` uses for C++. + - The root's standard leaves the dependency cache key. + - The C2 negative test is added. + - On the MSVC dialect, one line per build reports the declared standards that `cl.exe` + does not apply (W3b). +- **W2** (`src/build/flags.cppm`, `src/build/hermetic.cppm`, `src/build/ninja_backend.cppm`, + tests): + - An ELF link over a graph-supplied C library passes `--sysroot` to an empty directory + that mcpp owns. + - A link that then fails with `unable to find library` names the graph's C library. + - The hermetic check refuses a `-L` outside the allowed prefixes on such a link. +- **W4, D6, D7**: + - **The manifest for user targets.** A new target key, `windows_code_page = "utf-8" | + "legacy"`, uses the vocabulary of the manifest element itself. It embeds the manifest in a + PE executable through the existing resource pipeline, where the synthesized script gains + one `RT_MANIFEST` entry. + - **Defaults.** Ordinary targets default to `legacy` (M6). Targets built as host tools + default to `utf-8` (D6). Build programs always carry the manifest (D4): `build_program.cppm` + compiles the resource with the host toolchain's rc tool and links it into + `build.mcpp.exe`. + - **mcpp's own executable** carries the manifest through `[resources] files`, because the + bootstrap engine that builds mcpp does not know the new key. + - **Validation (W4c).** Paths, build-program output and file names are validated as UTF-8 + where they enter (M5). A path mcpp cannot name is refused, or skipped and reported, and is + never an internal exception. Failing to write `compile_commands.json` is a `degraded` + report, not an abort. + - **The Ninja check (W4d).** Ninja's declared encoding is compared with mcpp's own. MSVC + response files carry a UTF-8 BOM when their content is not ASCII. + - **The start-up check (D7)** reads `GetACP()`. + - **The regression job (W4f)** turns the measurement of §6.2 into a CI job. + +### 12.3 Review by angle + +| Angle | How the plan answers it | +|---|---| +| Architecture | Each item lives where the routing rule puts it: stubs in data (openkal-musl), the silent exit upstream (xlings), and general defects in the engine. The engine gains no workaround layer (D5), and scope is carried by construction and enforced by tests (C2). | +| Stability | Every behaviour change was measured before it was planned: W1 on both musl targets, W3 on 14 packages, W2 on two distributions and on PE, W4 on three Windows rows. The changes that could not be measured are split out, not shipped blind. | +| Simplicity | One new key (`windows_code_page`). Everything else reuses existing mechanisms: the resource pipeline, the per-unit standard flag, the dependency `-L` normalisation, the `diag` channel. | +| User experience | Internal exceptions become diagnostics that name the path, the program or the tool. The `Finished` label states the level used. A cliff (D3) names the version that crosses it. | +| Compatibility | Manifests read by bootstrap engines gain no new key. For ASCII paths the output is byte-identical. Build programs and `mcpp.exe` change together (D4). | +| Cross-platform | The Windows rows (llvm, MSVC, MinGW) and the Linux twin are measured. macOS takes W3 like every host, and W1 and W2 do not reach it: no musl target, and Mach-O graph links are unchanged. | +| Consistency | C and C++ standards use one per-unit mechanism. The code-page vocabulary is Microsoft's. The key follows `windows_subsystem` and `windows_entry`. | +| Upgrade without surprise | Caches rebuild once only where the command line changes: musl builds (W1); C dependencies of a root that declared a non-default standard (W3); graph links (W2). A project with only ASCII paths sees no other difference. | +| Test coverage | W1d and the C2 negative test in the unit suite; e2e for the per-package standard, the graph link and the Linux non-UTF-8 refusal; the W4f job on Windows; the xlings and openkal-musl CI steps; the sandbox verification after release. | + +### 12.4 Implementation record (2026-09-26) + +**Status by repository.** Rows refer to §12.1. + +| # | Pull request | State | +|---|---|---| +| 1 | openxlings/xlings#613 | open, CI running; released as 2026.9.26.2, because 2026.9.26.1 had been released separately earlier the same day | +| 2 | mcpplibs/openkal-musl#43 | merged as `20b92683`; tag `0.19.2`; the GitHub archive (1157319 bytes, sha256 `e8043bcd...c82238`) and the GitCode copy compared byte for byte; the archive carries `port/lib/lib*.a`, eight files of eight bytes | +| 3 | mcpplibs/mcpp-index#467 | open, CI running | +| 4 to 7 | | follow in the order of §12.1 | + +**Where the implementation departs from §12.2, and why.** + +- **W4c distinguishes the entry points.** A project directory or `MCPP_HOME` with no UTF-8 + spelling is refused before anything is written; a name inside a project is skipped and + reported through the existing `path/codepage` channel of #516; a `build.mcpp` directive whose + text is not UTF-8 is refused by its key. Skipping rather than refusing keeps the #516 contract + for names that are test data. One function, `try_narrow`, decides all three, so the Windows + code page and the POSIX byte case share one predicate. A serialiser failure in + `compile_commands.json` becomes that document's write failure (a warning, or an error when the + database is required), never an exception that reaches `main`. +- **D7 is realised inside the diagnostics it affects.** On a host that ignores the manifest, + every non-ASCII path reaches one of three diagnostics (the refusal of a project or home, the + skip report, the Ninja check), and each of them names the process code page. A separate + start-up line would repeat them, and a legacy host with ASCII paths is unaffected and is told + nothing. +- **The Ninja check compares against UTF-8, not against mcpp's code page.** Every path in + `build.ninja` has passed the UTF-8 check, and every other string is UTF-8 by construction, so + the file is UTF-8 in every mode. A Ninja reading ANSI would misread it on a legacy host too. +- **The byte order mark is written on every msvc response file, not only on non-ASCII ones.** + Round 5 measured ASCII content with the mark as accepted by `cl.exe`, `link.exe` and `lib.exe`, + and a conditional mark would need a per-edge variable for no behavioural gain. +- **The host-tool default (D6) yields to a manifest the package embeds itself.** Both would sit + at ordinal 1; the package said nothing about code pages, and its own manifest is the one it + ships. A declared `windows_code_page = "utf-8"` beside such a manifest is refused instead. +- **mcpp's own manifest** is `res/mcpp.rc`, which names `mcpp.exe.manifest` beside the script. + Round 5b measured rc.exe and windres, and a local run measured llvm-rc, all finding the file + there when run from another directory; mcpp resolves it there as a build input. +- **The rc scanner** tracks a manifest named by the numeric type `24` and treats a statement as + a manifest only when a file name follows the type, so `FILEVERSION 24,1,0,0` is not one. + +**Test results on Linux** (the worktree at `fix/693-696`): + +| Suite | Result | +|---|---| +| unit: modgraph, build_directives, ninja_backend, c_standard_per_package, hermetic_graph_link, compile_commands, build_resources | 57, 55, 92, 6, 4, 23 and 15 tests pass | +| e2e 776, a path with no UTF-8 spelling is named (new) | passes; released 2026.9.25.1 fails at its first criterion with the JSON exception | +| e2e 777, `c_standard` applies to the package that declares it (new) | passes; 2026.9.25.1 compiles `cdep` and `plain` at the consumer's `c99` | +| e2e 778, a graph link searches no host directory (new), leg B | passes; 2026.9.25.1 links leg B from the host's `libm` | +| e2e 778, leg A | runs once openkal-musl 0.19.2 is in the index | +| e2e 190 | accepts the byte order mark before `$in_newline` | + +The Windows rows run in CI through `.github/tools/check_unicode_paths.sh` (W4f): llvm, MSVC and +MinGW in an ASCII directory, `caf` + U+00E9 and U+6D4B U+8BD5, plus a path through `build.mcpp`. + +--- + +## Appendix A. Measurement record + +The Linux commands ran on 2026-09-25 and 2026-09-26 under the session scratchpad; the Windows +runs are dated 2026-09-25 (UTC). `$M` is +`~/.xlings/data/xpkgs/xim-x-mcpp/2026.9.25.1/bin/mcpp` (`mcpp 2026.9.25.1`). + +**A.1 #694 fixture** (`optlevel`, the report's manifest and `main.cpp`): + +``` +$ $M build --offline --release --target x86_64-linux-musl + Finished release [optimized] in 4.14s +compile_commands.json optimization flags: {'-Og': 1621, '(none)': 3} +[ 42] [left ] [ mid ] [***3.142****] +``` + +**A.2 mcpp's release configuration** (82867ad7 worktree, unmodified manifest): + +``` +$ $M build --offline --release --target x86_64-linux-musl --configure-only # Resolved gcc@16.1.0 +$ $M build --offline --release --target aarch64-linux-musl --configure-only # Resolved gcc@16.1.0 +both: every compile_commands.json entry carries -Og (307 = 178 binary units + 127 tests + 2 gtest) +``` + +**A.3 The premise, minimal program** (`#include ` and `import std` variants of the report's +format line): + +``` +gcc 15.1.0 x86_64/aarch64, 16.1.0 x86_64/aarch64: header mode -O2 and -O3 rc=0; +module mode (std.cc and the TU both at -O2) rc=0. No internal compiler error. +``` + +**A.4 The premise, mcpp itself.** A probe profile was appended to the worktree's `mcpp.toml`: + +```toml +[profile.o2probe] +opt = 2 +debug = false +cxxflags = ["-O2"] +cflags = ["-O2"] +``` + +``` +x86_64-linux-musl (x86_64-linux-musl-gcc 16.1.0): Finished o2probe in 92.23s + 127 src/ units '-Og -O2'; 48 modules/ units '-Og'; 3 cmdline units '-Og'; binary: mcpp 2026.9.25.1 +aarch64-linux-musl (aarch64-linux-musl-gcc 16.1.0): Finished o2probe in 92.45s + same distribution; qemu-aarch64 runs --version and --help +v0.0.1 (92f1335e), gcc 15.1.0-musl: 23 src/ units '-Og -O2' compile, 0 ICE; + link: undefined reference to std::optional::optional(...&&) +82867ad7, gcc 15.1.0-musl, --release (178 units -Og): 'exposes TU-local entity' x2, 0 ICE +``` + +**A.5 #695 fixture** (`cdep`/`app`, host target): + +``` +failed: obj/mcpplibs_cdep/src/cdep.o +cdep.c:2:2: error: "strict ISO mode: this package's c_standard = \"gnu11\" did not reach this unit" +build.ninja: cflags = -std=c11 +``` + +**A.6 #696, driver only** (clang 22.1.8, `start.c` containing `void _start(void) {}`): + +``` +aarch64-unknown-linux-musl -###: "-L/lib/../lib64" "-L/usr/lib64" "-L/lib" "-L/usr/lib" + -lm: ld.lld: error: unable to find library -lm +x86_64-unknown-linux-musl -###: "-L/usr/lib/gcc/x86_64-linux-gnu/13" + "-L/usr/lib/gcc/x86_64-linux-gnu/13/../../../../lib64" + "-L/lib/x86_64-linux-gnu" "-L/lib/../lib64" + "-L/usr/lib/x86_64-linux-gnu" "-L/usr/lib64" "-L/lib" "-L/usr/lib" + -lm: /lib/x86_64-linux-gnu/libm.a, /usr/lib/x86_64-linux-gnu/libm-2.39.a, + /usr/lib/x86_64-linux-gnu/libmvec.a +--sysroot=: no -L on either target; -lm: unable to find library -lm +--sysroot= -L: -lm resolves to that file; both targets link + +x86_64-w64-windows-gnu and x86_64-pc-windows-gnu, identical -L lists +(MinGW driver; host has /usr/bin/x86_64-w64-mingw32-gcc): + lld-link ... -libpath:/usr/lib/gcc/x86_64-w64-mingw32/13-win32 + -libpath:/usr/x86_64-w64-mingw32/lib -libpath:/usr/x86_64-w64-mingw32/mingw/lib + pe.o /usr/x86_64-w64-mingw32/lib/libm.a + with --sysroot=: only /x86_64-w64-mingw32/lib, /x86_64-w64-mingw32/mingw/lib, + /lib; lld: error: unable to find library -lm +``` + +**A.7 #696 through mcpp** (`lmcross` with `fmaximum`, x86_64-linux-musl): + +``` + Finished dev [unoptimized + debuginfo] in 2.94s +2 +reference extracted symbol +obj/main.o /usr/lib/x86_64-linux-gnu/libm-2.39.a(s_fmaximum.o) fmaximum +``` + +**A.8 Ecosystem counts** (mcpp-index 93781cf, 233 descriptors; Lua comments removed before +matching, string literals kept). 30 descriptors link at least one of the eight names that musl +answers from libc: `-lm` 13, `-lpthread` 21, `-ldl` 10, `-lrt` 6, `-lresolv` 2, and `-lcrypt`, +`-lutil`, `-lxnet` 0. `c_standard` declarations in code: 90 `c11`, 11 `c99`, 1 `c17` +(compat.ffmpeg) and 1 `gnu11` (compat.freetype). The comments of compat.libaio, compat.libdrm and +compat.libinput record `gnu11` as having no effect. + +**A.9 The §4.6 sweep.** Each probe is a package with an empty `main`, `[toolchain] default = +"llvm@22.1.8"`, one dependency at its latest Linux version, and in the treatment +`[build] c_standard = ""`. The table in §4.6 is `results.tsv` from the sweep script, +and the `.ninja_log` checks are quoted there. + +**A.10 #693 on Windows** (PR #697, branch `measure/693-windows-acp`; the workflow and the script +are `.github/workflows/measure-693.yml` and `.github/measure-693/measure.ps1` on that branch). + +| Run | Content | Note | +|---|---|---| +| 36162457075 | Q1 and Q2 | The Q1 readings are valid. **The Q2 readings are void**: PowerShell variable names are case-insensitive, so the loop variable `$fx` overwrote the fixtures directory `$Fx`, every copy failed, and every build "ran" in 0 s with no process started. The script did not record the start error at the time. Both defects were corrected in the next run. | +| 36163057371 | Q1 with the shim inside its workspace; cdb; Q2 and Q3 on the llvm row | cdb was present at `Windows Kits\10\Debuggers\x64`. WER produced neither events nor dumps, even after it was enabled. | +| 36163850821 | the MSVC and MinGW rows; MinGW manifests | Replicates the Q1 and Q2 readings of the previous run. | +| 36164076852 | a non-ASCII `MCPP_HOME` | Replicates every earlier reading. | + +Selected reading lines, verbatim: + +``` +READING env.acp: system ACP=1252 OEMCP=437; GetACP() in pwsh=1252 +READING q1.nonacp.release: exit=0x00000000 out=mcpp 2026.9.25.1 +READING q1.nonacp.xlings-shim-in-workspace: exit=0xC0000409 out=(no output) +READING q1.nonacp.xlings: exit=0xC0000409 out=(no output) +cdb: (340.17cc): C++ EH exception - code e06d7363 (first chance) + "No mapping for the Unicode character exists in the target multi-byte code page." + (340.17cc): C++ EH exception - code e06d7363 (!!! second chance !!!) +READING q2.latin1.release.hello: exit=0x00000046 ... error: internal: unhandled exception: + [json.exception.type_error.316] invalid UTF-8 byte at index 9: 0x2F +READING q2.latin1.patched.hello.build-ninja: 4345 bytes; acp=0 utf8=4; + first 'caf' -> 63 61 66 C3 A9 5C 70 61 ... +READING q2.nonacp.release.hello: exit=0x00000046 ... error: internal: unhandled exception: + No mapping for the Unicode character exists in the target multi-byte code page. +READING q2.nonacp.patched.hello: exit=0x00000000 ... run: exit=0x00000000 hello from a project ... +READING q2.nonacp.patched.bmn: exit=0x00000002 ... build.mcpp: cannot open + C:\w\repro-??-??\patched-bmn\target\.build-mcpp\out/gen.cpp +READING q4.gnu-manifest.cc1plus.exe: UTF-8 +READING q4.gnu-manifest.ld.exe: manifest without activeCodePage +READING q5.home-latin1.release: config exit=0x00000000; build exit=0x00000046 +READING q5.home-nonacp.release: ... error: cannot create 'C:\mh-??\bin' +READING ninja: ... ninja.exe version=1.12.1 wincodepage=Build file encoding: UTF-8 +``` + +**A.11 The -A API call sites that W4e replaces** (82867ad7): +- `CreateProcessA`: `modules/platform/src/windows/bounded_process.cppm:333,476` and + `src/build/schedule/detach_codegen.cppm:350,402`. +- `GetEnvironmentStringsA`: `bounded_process.cppm:211`. +- `std::system`: `modules/platform/src/process.cppm:572`. +- `_putenv_s`: `modules/platform/src/env.cppm:145`. +- Every narrowing through `try_narrow`, which is `path::generic_string()`, in + `modules/manifest/src/glob.cppm:69-75`. + +**A.12 The Linux twin of F-693b**, on this host with 2026.9.25.1: + +``` +$ mkdir "caf"$'\xe9'; cd "caf"$'\xe9'/hello && mcpp build --offline +error: internal: unhandled exception: [json.exception.type_error.316] invalid UTF-8 byte at index 123: 0x2F +``` + +**A.13 Response-file encodings of the MSVC tools** (PR #697 round 5, windows-latest, code page +1252, MSVC 14.51.36231, Ninja 1.12.1 from Chocolatey). One response file per tool, directory +and encoding; `cl-include` names an include directory, `cl-source` a source file, `link-out` and +`lib-out` the output file: + +``` + ascii caf+U+00E9 U+6D4B U+8BD5 +UTF-8, no BOM pass fail (cl exit 2; link and lib LNK1104 'café') fail +UTF-8 with BOM pass pass pass +UTF-16LE with BOM pass pass pass +code page 1252 pass pass (not representable) +Ninja rspfile_content, no BOM: ascii pass, caf+U+00E9 fail, U+6D4B U+8BD5 fail +Ninja rspfile_content, BOM: pass in all three +READING R1n.wincodepage: Build file encoding: UTF-8 +``` + +**A.14 Where the resource compilers look for a file a statement names** (round 5b and this +host). The script is `res/.rc`, the tool runs from another directory with `/I` (or `-I`) +naming the project root: + +``` +READING R2.rc.beside: exit=0 (1 24 "m.manifest", the file beside the script) +READING R2.rc.rooted: exit=0 (1 24 "res/m.manifest", found through /I) +READING R2.windres.beside: exit=0 +READING R2.windres.rooted: exit=0 +llvm-rc 20.1.7 (Linux): beside exit=0; rooted through /I exit=0; rooted without /I exit=1 +``` diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 0d7f11573..6424b150f 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -309 records. +310 records. ## By subject @@ -30,6 +30,7 @@ Records that declare one. Everything else is listed by date below. ### design +- [Issues #693 to #696: triage against mcpp's contracts, and one repair plan](2026-09-25-issues-693-696-triage-and-repair-plan.md) — active - [Workspace inheritance, flag scoping and the published form: a unified repair plan (#690)](2026-09-25-issue-690-workspace-build-inheritance-consistency.md) — landed - [MSVC toolset 的选择、#685、#687 与工具链管理规范：总体设计](2026-09-24-toolchain-selection-and-payload-trust-design.md) — active - [openkal 生态：能力的时刻模型，以及 C 环境方案空间的划分](2026-09-20-openkal-c-environment-ecosystem-design.md) — active @@ -100,6 +101,7 @@ Records that declare one. Everything else is listed by date below. ### 2026-09 +- [Issues #693 to #696: triage against mcpp's contracts, and one repair plan](2026-09-25-issues-693-696-triage-and-repair-plan.md) — active - [Workspace inheritance, flag scoping and the published form: a unified repair plan (#690)](2026-09-25-issue-690-workspace-build-inheritance-consistency.md) — landed - [#690: self-review before release, engine and ecosystem](2026-09-25-issue-690-self-review.md) — landed - [#690: implementation plan](2026-09-25-issue-690-implementation-plan.md) — landed diff --git a/.github/tools/check_unicode_paths.sh b/.github/tools/check_unicode_paths.sh new file mode 100755 index 000000000..5dd3e1481 --- /dev/null +++ b/.github/tools/check_unicode_paths.sh @@ -0,0 +1,130 @@ +#!/usr/bin/env bash +# check_unicode_paths.sh: mcpp#693 on Windows: a project in a directory whose +# name is not ASCII builds and runs on every toolchain row, and the files mcpp +# writes for other tools hold that name in UTF-8. +# +# THE SHAPE OF THE FAILURE THIS GUARDS. Up to mcpp 2026.9.25.1 mcpp held paths in +# the process ANSI code page. A name that page could spell (U+00E9 on code page +# 1252, every Chinese name on code page 936) reached compile_commands.json as +# ANSI bytes and failed every build with +# error: internal: unhandled exception: [json.exception.type_error.316] +# and a name it could not spell failed with a narrowing exception. mcpp.exe now +# declares the UTF-8 code page, build programs are linked with the same +# declaration, and the response files of the MSVC tools begin with a byte order +# mark. Each of those is a separate way back to the failure, which is why every +# row runs in every directory. +# +# Rows: llvm (clang++, the Windows default), msvc (cl.exe) and mingw (gcc@16.1.0 +# for x86_64-windows-gnu), each in an ASCII directory, a directory named +# `caf` + U+00E9 (inside code page 1252) and one named U+6D4B U+8BD5 (outside +# it). One more project runs a build.mcpp that reads its own directory from the +# environment and prints it back as an include directory: the round trip of a +# path through a build program. Names are built from bytes, so this file stays +# ASCII. +# +# Usage: MCPP= bash .github/tools/check_unicode_paths.sh +set -uo pipefail + +: "${MCPP:?set MCPP to the mcpp.exe under test}" +ROOT=$(mktemp -d) +CAFE="caf$(printf '\xc3\xa9')" +CJK="$(printf '\xe6\xb5\x8b\xe8\xaf\x95')" +CAFE_ANSI="caf$(printf '\xe9')" +failed=0 + +ok() { echo " ok $*"; } +bad() { echo " FAIL $*"; failed=1; } + +# write_project DIR TOOLCHAIN_LINES +write_project() { + mkdir -p "$1/src" + printf '[package]\nname = "unicodeprobe"\nversion = "0.1.0"\n\n%b\n' "$2" > "$1/mcpp.toml" + cat > "$1/src/main.cpp" <<'EOF' +import std; +int main() { + std::println("unicode probe"); + return 0; +} +EOF +} + +# The generated file holds the directory name in UTF-8, and never in the ANSI +# code page. +check_bytes() { + local file=$1 name=$2 row=$3 + [ -f "$file" ] || { bad "$row: no $(basename "$file")"; return; } + if ! LC_ALL=C grep -qF "$name" "$file"; then + bad "$row: $(basename "$file") does not hold the directory name in UTF-8" + fi + if [ "$name" = "$CAFE" ] && LC_ALL=C grep -qF "$CAFE_ANSI" "$file"; then + bad "$row: $(basename "$file") holds the ANSI spelling of the directory name" + fi +} + +rows=( + 'llvm|[toolchain]\nwindows = "llvm@20.1.7"|' + 'msvc|[toolchain]\nwindows = "msvc@system"|' + 'mingw|[toolchain]\ndefault = "gcc@16.1.0"|--target x86_64-windows-gnu' +) + +for name in ascii "$CAFE" "$CJK"; do + for row in "${rows[@]}"; do + IFS='|' read -r rid tc args <<<"$row" + dir="$ROOT/$name/$rid" + label="$rid in '$name'" + write_project "$dir" "$tc" + cd "$dir" || { bad "$label: cannot enter the directory"; continue; } + # shellcheck disable=SC2086 + if ! "$MCPP" build $args > build.log 2>&1; then + bad "$label: the build failed"; tail -15 build.log | sed 's/^/ /'; continue + fi + # shellcheck disable=SC2086 + out=$("$MCPP" run $args 2>&1) + if ! grep -q 'unicode probe' <<<"$out"; then + bad "$label: the program did not run"; printf '%s\n' "$out" | tail -5 | sed 's/^/ /' + continue + fi + if [ "$name" != ascii ]; then + ninja=$(find target -name build.ninja -newer mcpp.toml | head -1) + check_bytes "$ninja" "$name" "$label" + check_bytes compile_commands.json "$name" "$label" + fi + ok "$label" + done +done + +# A path through a build program: MCPP_MANIFEST_DIR in, `mcpp:include-dir` out, +# and the header found at the directory the program printed. +dir="$ROOT/$CJK/buildprogram" +mkdir -p "$dir/src" "$dir/inc" +printf '[package]\nname = "unicodebp"\nversion = "0.1.0"\n\n[toolchain]\nwindows = "llvm@20.1.7"\n' > "$dir/mcpp.toml" +printf '#define UNICODE_BP 42\n' > "$dir/inc/unicode_bp.h" +cat > "$dir/build.mcpp" <<'EOF' +#include +#include +int main() { + const char* here = std::getenv("MCPP_MANIFEST_DIR"); + if (!here) return 1; + std::printf("mcpp:include-dir=%s/inc\n", here); + return 0; +} +EOF +cat > "$dir/src/main.cpp" <<'EOF' +#include "unicode_bp.h" +import std; +int main() { + std::println("build program {}", UNICODE_BP); + return 0; +} +EOF +cd "$dir" +if "$MCPP" build > build.log 2>&1 && "$MCPP" run 2>&1 | grep -q 'build program 42'; then + ok "a path through build.mcpp in '$CJK'" +else + bad "a path through build.mcpp in '$CJK'"; tail -15 build.log | sed 's/^/ /' +fi + +cd / && rm -rf "$ROOT" +[ "$failed" = 0 ] && echo "OK: every row builds in every directory" && exit 0 +echo "FAIL: at least one row did not build in a directory whose name is not ASCII" +exit 1 diff --git a/.github/workflows/ci-windows.yml b/.github/workflows/ci-windows.yml index f9d0d38d9..1a9159800 100644 --- a/.github/workflows/ci-windows.yml +++ b/.github/workflows/ci-windows.yml @@ -405,6 +405,17 @@ jobs: # restore the LLVM default for the remaining steps "$MCPP_SELF" toolchain default llvm@20.1.7 + # mcpp#693: a project in a directory whose name is not ASCII, inside the + # runner's code page 1252 and outside it, on the llvm, MSVC and MinGW rows, + # plus a path carried through build.mcpp. Each row reaches a different + # consumer of mcpp's text: Ninja's build file, the MSVC response files, + # and the MinGW driver. + - name: "Paths: non-ASCII project directories on every toolchain row (mcpp#693)" + shell: bash + run: | + export MCPP_VENDORED_XLINGS="$XLINGS_BIN" + MCPP="$MCPP_SELF" bash .github/tools/check_unicode_paths.sh + # GRAPHICS ON THIS HOST, BUILD ONLY, AND THAT IS THE WHOLE CLAIM. # diff --git a/.github/workflows/openkal-cross.yml b/.github/workflows/openkal-cross.yml index 4dec8d298..6e6bdaa2a 100644 --- a/.github/workflows/openkal-cross.yml +++ b/.github/workflows/openkal-cross.yml @@ -576,7 +576,7 @@ jobs: for t in tests/e2e/285_*.sh tests/e2e/286_*.sh tests/e2e/287_*.sh \ tests/e2e/288_*.sh tests/e2e/289_*.sh tests/e2e/291_*.sh \ tests/e2e/292_*.sh tests/e2e/293_*.sh tests/e2e/294_*.sh \ - tests/e2e/738_*.sh; do + tests/e2e/738_*.sh tests/e2e/778_*.sh; do echo "=== $t ===" bash "$t" 2>&1 | tee "$(basename "$t").log" || true rc=${PIPESTATUS[0]} @@ -630,4 +630,11 @@ jobs: "OK: the list answers what can be built, not what has a payload" || fail=1 check 738_a_graph_supplied_target_closes_the_hosts_own_search.sh \ "PASS: 738 a graph-supplied target closes the host's own search" || fail=1 + # mcpp#696, the link-side twin of 738: both legs, because the one + # that separates the engines is the refusal, and a skip of either + # prints the final line all the same. + check 778_a_graph_link_searches_no_host_directory.sh \ + "ok: -lm is answered by openkal-musl's own archive, and the program runs" || fail=1 + check 778_a_graph_link_searches_no_host_directory.sh \ + "ok: an unanswered -lm fails, and the note names openkal-musl 0.19.2" || fail=1 [ "$fail" = 0 ] || exit 1 diff --git a/CHANGELOG.md b/CHANGELOG.md index bd3426eb5..a26656663 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,57 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 +## [2026.9.26.1] - 2026-09-26 + +### 路径与文本统一为 UTF-8(#693) + +mcpp 此前在 Windows 上以进程 ANSI 代码页持有路径,而它写出的 `compile_commands.json` 只能容纳 +UTF-8。于是工程目录或 mcpp 主目录只要含非 ASCII 字符,每次构建都以 +`internal: unhandled exception: [json.exception.type_error.316]` 失败,系统代码页能拼出的名字也 +不例外(代码页 936 上的中文目录名、代码页 1252 上的 `café`);代码页拼不出的名字则以窄化异常失败。 +Linux 上名字不是 UTF-8 的目录以同样的 JSON 异常失败。现在 mcpp 在所有平台上以 UTF-8 持有文本: + +- `mcpp.exe` 以应用程序清单声明 UTF-8 代码页(`res/mcpp.rc`),在 Windows 10 1903 及以后的版本上 + 它与 Windows 交换的窄字符串都是 UTF-8。`build.mcpp` 链接同一份清单;作为 host tool 构建的程序 + 默认也带它,除非其目标写 `windows_code_page = "legacy"` 或包自己嵌入了清单。 +- 新的目标键 `windows_code_page`(`"utf-8"` 或 `"legacy"`)为工程自己的可执行文件声明代码页; + 不写时保持系统代码页。 +- 路径没有 UTF-8 拼法的工程目录与 `MCPP_HOME` 在构建之前被拒绝,消息以转义拼出该路径并给出 + 本机的原因;工程内部这样的文件名被跳过并按目录汇报一次;文本不是 UTF-8 的 `build.mcpp` 指令 + 按键名被拒绝。`compile_commands.json` 的序列化失败报告为该文件的失败,不再成为内部异常。 +- `build.ninja` 含非 ASCII 字节时,mcpp 以 `ninja -t wincodepage` 核对 Ninja 读取它的编码, + 不是 UTF-8 就拒绝并点名该 Ninja。`cl.exe`、`link.exe`、`lib.exe` 只在响应文件以字节顺序标记开头时 + 按 UTF-8 读取它(在 windows-latest 上实测),msvc 方言的响应文件因此以字节顺序标记开头。 +- 在忽略清单的 Windows(早于 1903)上,受影响路径的诊断会写出进程所在的 ANSI 代码页。 +- Windows CI 新增回归任务:llvm、MSVC、MinGW 三行分别在 ASCII、`café` 与中文目录中构建并运行, + 并核对 `build.ninja` 与 `compile_commands.json` 中该目录名的 UTF-8 字节。 + +报告中的静默 `0xC0000409` 来自 xlings 的 shim:它在代码页之外的工作目录中启动时抛出未捕获的异常。 +xlings 2026.9.26.2 声明同样的 UTF-8 代码页并在 `main` 中设异常边界,mcpp 内置的 xlings 版本随之更新。 + +### `c_standard` 作用于声明它的包(#695) + +`[build] c_standard` 此前写在文件级 `$cflags` 上,图中每个 C 编译单元都读它:根包的值到达每个依赖, +依赖自己的声明被解析、被哈希进缓存键,却没有被施加。现在每个包的 C 单元以该包自己的标准编译, +未声明的包以 `c11` 编译,C++ 单元不接收 C 标准。`cl.exe` 以其默认模式编译 C,构建会用一行列出 +声明未被施加的包。`#690` 设计记录 §3.1 中把 `c_standard` 列为「只读根包」的一行已附更正说明。 + +### 链接图提供的 C 库时不搜索宿主目录(#696) + +C 库来自依赖图的链接此前没有 `--sysroot`,clang 会从宿主推导 `/usr/lib` 等库目录: +`x86_64-linux-musl` 上的 `-lm` 把宿主 glibc 的目标文件静默链接进 musl 镜像,`aarch64-linux-musl` +上则以一条不相干的报错失败。现在由 clang 链接的 ELF 目标带上指向构建目录内空目录的 `--sysroot`; +密封检查拒绝指向工具链存储、构建目录与图中各包之外目录的 `-L`(`[build] allow_host_libs = true` +取消这一拒绝)。图中无人应答的 `-l` 以链接器自己的消息失败,mcpp 另附说明;缺失的名字全属于 musl 以 +`libc.a` 应答的八个名字时,说明会点名带有这八个空归档的 openkal-musl 0.19.2。 + +### musl 目标按声明的优化级别编译(#694) + +引擎此前在 `*-linux-musl` 上把每个非零优化级别替换为 `-Og`(一个 musl-gcc 15.1.0 内部编译器错误的 +变通,也作用于 clang 与 mcpp 自己的 Linux 发布二进制),而 `Finished` 一行按声明的级别报告 +`optimized`。该分支已删除:编译与报告读取同一个值(`realised_opt_level`),空级别即 `0`。 +原先的触发条件在任何可得的输入上都不再复现。 + ## [2026.9.25.1] - 2026-09-25 ### 工作空间成员在每种位置上以相同方式编译(#690) diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index f084455ee..cbdd032a5 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -335,6 +335,7 @@ required_features = ["gui"] # only built when feature `gui` is | `required_features` | The target is emitted only when **every** listed feature is active in the build; otherwise it is silently skipped. A gate only — it does not activate features (use `--features` / `[features].default`). **One exception, and it is not a second rule:** when this target is requested as a host tool (`tools = [...]`, §2.14), the target is what was *asked for*, so its `required_features` become the sub-build's *inputs*. Same field, one meaning — the resolution just runs in the opposite direction. | | `windows_subsystem` *(2026.9.12.2+)* | The PE subsystem of an executable: `"console"` (the default) or `"windows"`, a GUI program that starts without a console. Reaches this target's link and no other, and renders nothing on a target that is not PE. See the section above. | | `windows_entry` *(2026.9.12.2+)* | The entry function the program defines: `"main"` (the default), `"wmain"`, `"WinMain"` or `"wWinMain"`. See the section above. | +| `windows_code_page` *(2026.9.26.1+)* | The ANSI code page of an executable on Windows: `"utf-8"`, embedded as an application manifest that Windows 10 version 1903 and later honour, or `"legacy"`, the system's code page. A program target that writes neither runs in the system's code page, except a program built as a host tool (§2.14), which defaults to `"utf-8"`. Refused on a library target; renders nothing on a target that is not PE. See *Paths and text encoding* in §2.3. | | `linkage` *(2026.9.15.2+)* | A library target's **default** link form, `"static"` or `"shared"`: the form a consumer that writes no `linkage` receives. Unlike `kind = "shared"` it is not a constraint, so a consumer's explicit statement is honoured. Refused beside `kind = "shared"` and on a program target. See [`dependency_linkage`](#dependency_linkage--static-or-shared-is-the-consumers-decision). | > **Scope (important):** `defines` / `cxxflags` / `cflags` on a target apply **only to that @@ -403,7 +404,7 @@ build_program_timeout = 1800 # Seconds a build.mcpp may run; 0 = no limit ( include_dirs = ["include", "third_party/include"] # Header search paths of this package (§ below) include_dirs_after = ["*"] # Header dirs searched AFTER system dirs (-idirafter) private_include_dirs = ["vendor/src/include"] # Of `include_dirs`, the ones a consumer must NOT get -c_standard = "c11" # Standard for C source files (default c11) +c_standard = "c11" # Standard for this package's C sources (default c11; § below) cflags = ["-DFOO=1"] # Extra C compile flags cxxflags = ["-DBAR=2"] # Extra C++ compile flags (do not put -std=... here) ldflags = ["-lfoo"] # Extra link flags @@ -494,6 +495,23 @@ When a dependency's compile reports a missing header that exists in the consumer's include directories, mcpp names that directory after the compiler's message. +#### `c_standard` applies to the package that declares it *(mcpp 2026.9.26.1+)* + +`c_standard` sets the C standard of the declaring package's own C units. A +package that declares none compiles its C units at `c11`, and a consumer's value +never reaches a dependency: a dependency that declares `gnu11` compiles at +`gnu11` inside a project that declares `c99`, and one that declares nothing +compiles at `c11` there. C++ units receive no C standard. The value is part of +the package's build key, so the dependency's cached objects serve every consumer. + +`cl.exe` compiles C in its default mode and receives no C standard from mcpp. +When a package in the build declares one, the build reports in one line each +package whose declared standard was not applied. + +Before mcpp 2026.9.26.1 the root package's value reached every C unit in the +graph, and a dependency's own declaration was read, hashed into its cache key +and not applied (mcpp#695). + #### `dependency_linkage` — static or shared is the consumer's decision ```toml @@ -896,33 +914,56 @@ The **compile** phase is not bounded, only the build *program*. See Moved to [20 — Toolchain Management](20-toolchains.md). -### File names outside the host code page - -Globs are narrow strings, and so are compile commands and `build.ninja`. On -Windows those strings are produced in the process's **ANSI code page**, so a -file whose name has no spelling in that code page cannot be matched by a glob, -named on a compile command, or written into a build file. - -Such entries are skipped, and the skip is reported once per directory: - -```text -warning: 'C:/.../pkg/test/www' contains names this system's active code page cannot represent - impact: those files take no part in the build - hint: Windows only: this is the process ANSI code page, which `chcp` does not change. ... -``` - -The reported path is the nearest ancestor whose name the code page *can* spell, -in generic (`/`) spelling. The offending name itself is never printed: rendering -it would throw the same exception the message is reporting. - -`chcp` sets the *console* code page and has no effect here. Names that are only -test data or documentation are harmless — an upstream tarball carrying a -Japanese-named fixture directory builds fine on an en-US host. Sources are not: -they need renaming, or a host whose code page covers them. - -Linux and macOS perform no such conversion, so nothing is skipped there. A -package that builds on one and not the other, with an -`internal: unhandled exception` from a code-page message, was mcpp#516. +### Paths and text encoding *(mcpp 2026.9.26.1+)* + +mcpp holds every path, argument and file content as UTF-8 text, on every +platform. Every file it writes for another tool is UTF-8: `build.ninja`, +`compile_commands.json`, and the response files of the compile and link edges. +A path enters that text only through its UTF-8 spelling. + +**Windows.** `mcpp.exe` declares the UTF-8 code page in an application manifest. +On Windows 10 version 1903 and later its narrow strings, and every path it +exchanges with Windows, are therefore UTF-8, whatever the system's ANSI code page +is. The programs mcpp runs as part of a build share that code page: `build.mcpp` +is linked with the same manifest, and a program built as a host tool (§2.14) +receives it unless its target says `windows_code_page = "legacy"` or the package +embeds a manifest of its own. The programs a project builds for itself keep the +system's code page unless their target says `windows_code_page = "utf-8"`. + +Ninja 1.11 and later read `build.ninja` as UTF-8 under the same declaration. +When the file holds a non-ASCII byte, mcpp asks Ninja which encoding it reads +(`ninja -t wincodepage`) and refuses the build, naming that Ninja, if the answer +is not UTF-8. `cl.exe`, `link.exe` and `lib.exe` read a response file as UTF-8 +only when it begins with a byte order mark, so the response files of the msvc +dialect begin with one. + +**Paths with no UTF-8 spelling.** On Linux and macOS a file name is a sequence of +bytes, which need not be UTF-8. On a Windows host older than version 1903 the +manifest is ignored, and the process runs in the system's ANSI code page, which +spells only part of Unicode. On either, some paths have no UTF-8 spelling: + +- A project directory or an mcpp home (`MCPP_HOME`) whose path has none is + refused before anything is built. The message spells the path with escapes + (`\xE9` for a byte that is not UTF-8) and gives the reason for this host. +- A file or directory inside a project whose name has none is skipped, and the + skip is reported once per directory: + + ```text + warning: '/home/user/pkg/test/data' contains names that have no UTF-8 spelling + impact: those files take no part in the build + hint: The name's bytes are not UTF-8, and build.ninja and compile_commands.json hold UTF-8 text. ... + ``` + + The reported path is the nearest ancestor that has a UTF-8 spelling. Names of + test data or documentation are harmless; sources need renaming. +- A `build.mcpp` directive whose text is not UTF-8 is refused by its key + ([30-build-mcpp.md](30-build-mcpp.md)). + +Before mcpp 2026.9.26.1 such a path failed the build with `internal: unhandled +exception: [json.exception.type_error.316]`, and on Windows every non-ASCII +project path or home failed that way, including names the system's code page +could spell (mcpp#693). The skip of names outside the Windows ANSI code page +dates from mcpp#516. ### 2.3.1 `[build] accel` — the accelerator this build targets @@ -1576,6 +1617,14 @@ build directory (`target///res/.mcpp.rc`) and list it in `files`. The result is byte-identical, so moving from generated to hand-written never changes what ships. +A script may embed an application manifest (`1 24 "app.manifest"`, type 24 being +`RT_MANIFEST`). `windows_code_page = "utf-8"` embeds one at the same ordinal, so a +package that declares both is refused, with the two ways to keep one: add the +`activeCodePage` element to the script's manifest and set +`windows_code_page = "legacy"`, or remove the manifest from the script. A program +built as a host tool, which receives `utf-8` by default, keeps its own manifest +instead. + > **`VS_VERSION_INFO` needs ``.** In a hand-written script, > `VS_VERSION_INFO VERSIONINFO` without `#include ` files the version > resource under a *string* name instead of ordinal 1. Every tool still reports @@ -1587,8 +1636,10 @@ never changes what ships. #### Tracked inputs mcpp reads the `.rc` for quoted `#include`s and for the files named by resource -statements (`ICON`, `RCDATA`, `MANIFEST`, …), and makes them build inputs, so -editing the icon relinks. Angled includes (``) are the toolchain's +statements (`ICON`, `RCDATA`, `MANIFEST`, the numeric type `24`, …), and makes +them build inputs, so editing the icon relinks. A relative file name is resolved +against the script's own directory, which is also where rc.exe, llvm-rc and +windres look for it. Angled includes (``) are the toolchain's and are covered by the toolchain fingerprint instead. A file name reached through a macro (`1 ICON APP_ICON`) is invisible to that diff --git a/docs/22-target-side.md b/docs/22-target-side.md index a9f37759f..5ac0e7b37 100644 --- a/docs/22-target-side.md +++ b/docs/22-target-side.md @@ -107,6 +107,29 @@ Moving a layer from a prebuilt payload into the dependency graph therefore removes engine work rather than adding it. This is the mechanism by which one source reaches several platforms without an engine change. +### A Link Over A Graph-Supplied C Library Searches No Host Directory (mcpp 2026.9.26.1+) + +When the `c-abi` layer comes from the graph, the library search of the link is +the graph's as well. On an ELF target linked by clang, the link receives +`--sysroot` naming an empty directory inside the build directory, which removes +every library directory the driver would otherwise derive from the host +(`/usr/lib`, `/lib` and their multiarch forms). The hermetic check then refuses +a `-L` that names a directory outside the toolchain store, the build directory +and the packages of the graph; `[build] allow_host_libs = true` lifts the +refusal. + +A `-l` that nothing in the graph answers therefore fails with the linker's own +message, and mcpp adds a note naming the libraries. musl answers `m`, `rt`, +`pthread`, `crypt`, `util`, `xnet`, `resolv` and `dl` from `libc.a` itself and +installs an empty archive under each of those names; openkal-musl 0.19.2 ships +the eight archives, and the note names that release when every missing name is +one of them. + +Before mcpp 2026.9.26.1 such a link searched the host's directories after the +graph's, so a `-lm` on `x86_64-linux-musl` linked the host's glibc objects into a +musl image without a diagnostic, and failed on other targets with an unrelated +message (mcpp#696). + ### Layer Names Are Fixed, Implementations Are Not The five layer names are a closed set compiled into the engine. The diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index a76771373..f19fa9ffb 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -946,6 +946,21 @@ These values are folded into the re-run key **unconditionally** — changing the target, profile, or feature set re-runs the program without any `rerun-if-env-changed` declaration. +### Text is UTF-8 in both directions (mcpp 2026.9.26.1+) + +Every path and value mcpp passes to the program is UTF-8, and mcpp reads every +directive the program prints as UTF-8. On Windows the program is linked with the +application manifest that `mcpp.exe` itself carries, which sets its ANSI code page +to UTF-8 on Windows 10 version 1903 and later: its environment, its arguments and +the narrow strings it prints are UTF-8 there without any conversion in the +program. On Linux and macOS a file name is bytes, and a program that lists a +directory prints whatever bytes it finds. + +A directive line whose text is not UTF-8 is refused, whatever protocol the +program announces, and the refusal names the directive's key. Its value would +reach `build.ninja` as bytes that name a different file, so nothing of it is +applied. Lines that are not directives are not examined. + ### `PATH` — the environment the project declared (mcpp 2026.8.25.1+) A project that declares `[xlings].subos` runs its build programs with that diff --git a/docs/91-toolchain-internals.md b/docs/91-toolchain-internals.md index 41c91288d..d1037b996 100644 --- a/docs/91-toolchain-internals.md +++ b/docs/91-toolchain-internals.md @@ -430,6 +430,14 @@ cached per flag-set (`.mcpp-hermetic-ok`); escape hatches: `[build] allow_host_libs = true` or `MCPP_ALLOW_HOST_LIBS=1`. System/PATH compilers are exempt — using the host world explicitly is the user's choice. +A link over a graph-supplied C library is held to one more rule (mcpp#696, +2026.9.26.1+). Such a link carries `--sysroot=/graph-sysroot`, an +empty directory, so the driver derives no library directory from the host, and +the check also reads the `-L` directories of the dry run: each one must lie in +the toolchain store, the build directory or a package of the graph. The +verdict's cache key records whether the link is of this kind and which package +roots it allows. + CI keeps this honest with a job that has **no host toolchain at all** (`debian:stable-slim`, no gcc, no host `Scrt1.o`) — the only environment class that faithfully reproduces the clean-machine failure mode, plus e2e diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index dc820266a..4408d0d49 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -339,6 +339,7 @@ required_features = ["gui"] # only built when feature `gui` is | `required_features` | 只有当构建中**每一个**列出的 feature 都被激活时，这个目标才会被产出；否则被静默跳过。它只是一道闸——不会激活 feature（用 `--features` / `[features].default`）。**一个例外，但它不是第二条规则：** 当这个目标作为 host 工具被请求时（`tools = [...]`，§2.14），这个目标就是被**请求**的那一个，于是它的 `required_features` 变成子构建的**输入**。同一个字段、同一个含义——只是解析的方向反过来了。 | | `windows_subsystem` *（2026.9.12.2+）* | 可执行文件的 PE subsystem：`"console"`（默认）或 `"windows"`，一个启动时没有控制台的 GUI 程序。只到达这个目标的链接，别处不受影响，在非 PE 的目标上不渲染任何东西。见上一节。 | | `windows_entry` *（2026.9.12.2+）* | 程序定义的入口函数：`"main"`（默认）、`"wmain"`、`"WinMain"` 或 `"wWinMain"`。见上一节。 | +| `windows_code_page` *（2026.9.26.1+）* | 可执行文件在 Windows 上的 ANSI 代码页：`"utf-8"`，以 Windows 10 1903 及以后版本会遵从的应用程序清单嵌入；或 `"legacy"`，即系统代码页。两者都不写的程序目标运行在系统代码页里，作为 host 工具构建的程序（§2.14）除外，它默认为 `"utf-8"`。写在库目标上会被拒绝；在非 PE 目标上不产生任何东西。见 §2.3 的*路径与文本编码*。 | | `linkage` *（2026.9.15.2+）* | 一个库目标的**默认**链接形态，`"static"` 或 `"shared"`：不写 `linkage` 的消费者得到的形态。与 `kind = "shared"` 不同，它不是约束，因此消费者的显式陈述会被遵从。与 `kind = "shared"` 同写，或写在程序目标上，都会被拒绝。见[`dependency_linkage`](#dependency_linkage--静态还是动态由消费者决定)。 | > **范围（重要）：** 目标上的 `defines` / `cxxflags` / `cflags` **只** @@ -405,7 +406,7 @@ build_program_timeout = 1800 # Seconds a build.mcpp may run; 0 = no limit ( include_dirs = ["include", "third_party/include"] # 本包的头文件搜索路径（见下文） include_dirs_after = ["*"] # Header dirs searched AFTER system dirs (-idirafter) private_include_dirs = ["vendor/src/include"] # Of `include_dirs`, the ones a consumer must NOT get -c_standard = "c11" # Standard for C source files (default c11) +c_standard = "c11" # Standard for this package's C sources (default c11; § below) cflags = ["-DFOO=1"] # Extra C compile flags cxxflags = ["-DBAR=2"] # Extra C++ compile flags (do not put -std=... here) ldflags = ["-lfoo"] # Extra link flags @@ -488,6 +489,20 @@ defines = ["LEVEL=2", "!TRACE"] # Windows 上：-DLEVEL=2，且不定义 T 编译报告缺少某个头文件、而该文件存在于消费者的头文件目录中时，mcpp 会在编译器的 报错之后指出那个目录。 +#### `c_standard` 作用于声明它的包 *(mcpp 2026.9.26.1+)* + +`c_standard` 设定声明它的那个包自己的 C 编译单元所用的 C 标准。没有声明的包 +以 `c11` 编译其 C 单元，消费者的值永远不会到达依赖：声明了 `gnu11` 的依赖在 +一个声明 `c99` 的工程里仍以 `gnu11` 编译，什么都没声明的依赖在那里以 `c11` +编译。C++ 单元不接收 C 标准。这个值是包构建键的一部分，所以依赖的缓存对象 +服务于每一个消费者。 + +`cl.exe` 以其默认模式编译 C，不从 mcpp 接收 C 标准。当构建中有包声明了 C 标准 +时，构建会用一行汇报每个声明未被施加的包。 + +在 mcpp 2026.9.26.1 之前，根包的值会到达图中每一个 C 单元，而依赖自己的声明 +被读取、被哈希进它的缓存键，却没有被施加（mcpp#695）。 + #### `dependency_linkage` —— 静态还是动态由消费者决定 ```toml @@ -859,33 +874,51 @@ MCPP_BUILD_PROGRAM_TIMEOUT= (this invocation; highest) 已移至 [20 —— 工具链管理](20-toolchains.md)。 -### 宿主代码页之外的文件名 - -glob 是窄字符串，编译命令与 `build.ninja` 也是。在 Windows 上，这些 -字符串以进程的 **ANSI 代码页**产生，所以一个文件名在那个代码页里没有 -拼法的文件，无法被 glob 匹配，无法出现在编译命令里，也无法写进构建 -文件。 - -这样的条目会被跳过，跳过信息按目录汇报一次： - -```text -warning: 'C:/.../pkg/test/www' contains names this system's active code page cannot represent - impact: those files take no part in the build - hint: Windows only: this is the process ANSI code page, which `chcp` does not change. ... -``` - -报出的路径是最近的、其名字**能**被该代码页拼出的祖先目录，采用通用 -（`/`）拼法。出问题的名字本身永远不会被打印：渲染它会抛出与这条消息 -正在报告的同一个异常。 - -`chcp` 设置的是**控制台**代码页，在这里没有作用。只是测试数据或文档 -的文件名是无害的——一个携带日语命名测试夹具目录的上游压缩包，在 -en-US 宿主上照常能构建。源文件则不然：它们需要改名，或者需要一个 -代码页能覆盖它们的宿主。 - -Linux 与 macOS 不做这种转换，所以那里没有任何东西被跳过。一个能在 -其中一个上构建、在另一个上不能、并报出一条来自代码页消息的 -`internal: unhandled exception` 的包，就是 mcpp#516。 +### 路径与文本编码 *(mcpp 2026.9.26.1+)* + +mcpp 在所有平台上都以 UTF-8 文本持有每一个路径、参数与文件内容。它为其他 +工具写出的每个文件都是 UTF-8：`build.ninja`、`compile_commands.json`，以及 +编译边与链接边的响应文件。一个路径只能以它的 UTF-8 拼法进入这些文本。 + +**Windows。** `mcpp.exe` 在应用程序清单里声明 UTF-8 代码页。因此在 +Windows 10 1903 及以后的版本上，无论系统的 ANSI 代码页是什么，它的窄字符串 +以及它与 Windows 交换的每个路径都是 UTF-8。mcpp 在构建中运行的程序共用这个 +代码页：`build.mcpp` 链接时带同一份清单；作为 host 工具构建的程序（§2.14） +也会得到它，除非其目标写了 `windows_code_page = "legacy"`，或者该包自己嵌入 +了清单。工程为自己构建的程序保持系统代码页，除非其目标写了 +`windows_code_page = "utf-8"`。 + +Ninja 1.11 及以后的版本在同样的声明下以 UTF-8 读取 `build.ninja`。当文件中 +含有非 ASCII 字节时，mcpp 会询问 Ninja 以哪种编码读取（`ninja -t wincodepage`）， +若回答不是 UTF-8，就拒绝这次构建并点名那个 Ninja。`cl.exe`、`link.exe` 与 +`lib.exe` 只有在响应文件以字节顺序标记开头时才按 UTF-8 读取它，所以 msvc +方言的响应文件以字节顺序标记开头。 + +**没有 UTF-8 拼法的路径。** 在 Linux 与 macOS 上，文件名是一串字节，不必是 +UTF-8。在早于 1903 的 Windows 宿主上，清单会被忽略，进程运行在系统的 ANSI +代码页里，而它只能拼出 Unicode 的一部分。在这两种情况下，有些路径没有 UTF-8 +拼法： + +- 路径没有 UTF-8 拼法的工程目录或 mcpp 主目录（`MCPP_HOME`）在构建任何东西 + 之前就被拒绝。消息用转义拼出该路径（不是 UTF-8 的字节写作 `\xE9`），并给出 + 这台宿主上的原因。 +- 工程内部名字没有 UTF-8 拼法的文件或目录会被跳过，跳过信息按目录汇报一次： + + ```text + warning: '/home/user/pkg/test/data' contains names that have no UTF-8 spelling + impact: those files take no part in the build + hint: The name's bytes are not UTF-8, and build.ninja and compile_commands.json hold UTF-8 text. ... + ``` + + 报出的路径是最近的、有 UTF-8 拼法的祖先目录。测试数据或文档的名字是无害的； + 源文件需要改名。 +- 文本不是 UTF-8 的 `build.mcpp` 指令按其键名被拒绝 + （[30-build-mcpp.md](30-build-mcpp.md)）。 + +在 mcpp 2026.9.26.1 之前，这样的路径会让构建以 `internal: unhandled +exception: [json.exception.type_error.316]` 失败；在 Windows 上，每个非 ASCII +的工程路径或主目录都会这样失败，包括系统代码页能拼出的名字（mcpp#693）。 +跳过 Windows ANSI 代码页之外的名字始于 mcpp#516。 ### 2.3.1 `[build] accel` —— 本次构建面向的加速器 @@ -1510,6 +1543,12 @@ files = ["res/app.rc"] （`target///res/.mcpp.rc`）并列进 `files`。结果 逐字节相同，所以从生成切换到手写，不会改变实际发布的内容。 +脚本可以嵌入应用程序清单（`1 24 "app.manifest"`，类型 24 即 +`RT_MANIFEST`）。`windows_code_page = "utf-8"` 会在同一个序号上嵌入另一份， +所以同时声明两者的包会被拒绝，并给出保留其一的两种做法：把 `activeCodePage` +元素加进脚本的清单并设 `windows_code_page = "legacy"`，或者从脚本里去掉清单。 +作为 host 工具构建、默认得到 `utf-8` 的程序则保留它自己的清单。 + > **`VS_VERSION_INFO` 需要 ``。** 在一份手写脚本里， > `VS_VERSION_INFO VERSIONINFO` 若没有 `#include `，会把 > 版本资源归档到一个*字符串*名字下，而不是序号 1。每个工具仍会报告 @@ -1521,8 +1560,9 @@ files = ["res/app.rc"] #### 被跟踪的输入 mcpp 读取 `.rc` 里带引号的 `#include`，以及资源语句（`ICON`、 -`RCDATA`、`MANIFEST` 等）命名的文件，并把它们变成构建输入，所以修改 -图标会触发重新链接。尖括号 include（``）属于工具链，由 +`RCDATA`、`MANIFEST`、数字类型 `24` 等）命名的文件，并把它们变成构建输入， +所以修改图标会触发重新链接。相对文件名按脚本自己所在的目录解析，rc.exe、 +llvm-rc 与 windres 也在那里查找它。尖括号 include（``）属于工具链，由 工具链指纹覆盖，而不是这项扫描。 一个经由宏到达的文件名（`1 ICON APP_ICON`）对这项扫描是不可见的。 diff --git a/docs/zh/22-target-side.md b/docs/zh/22-target-side.md index 86b47c0fa..a34032d65 100644 --- a/docs/zh/22-target-side.md +++ b/docs/zh/22-target-side.md @@ -93,6 +93,23 @@ C 库、平台接口与 C++ 运行时是互斥的选择，不是可叠加的贡 因此，把一层从预制载荷移入依赖图，减少的是引擎的工作，而不是增加。这正是 一份源码能够在引擎不作任何改动的情况下到达多个平台的机制所在。 +### 链接图提供的 C 库时不搜索任何宿主目录（mcpp 2026.9.26.1+） + +当 `c-abi` 层来自依赖图时，链接的库搜索也属于依赖图。在由 clang 链接的 ELF 目标上， +链接会收到指向构建目录内一个空目录的 `--sysroot`，它去掉了驱动器原本会从宿主推导出的 +每一个库目录（`/usr/lib`、`/lib` 及其 multiarch 形式）。随后密封检查会拒绝指向工具链 +存储、构建目录与依赖图中各包之外目录的 `-L`；`[build] allow_host_libs = true` 取消这个 +拒绝。 + +因此，图中没有任何东西应答的 `-l` 会以链接器自己的消息失败，mcpp 另附一条点名这些库的 +说明。musl 以 `libc.a` 本身应答 `m`、`rt`、`pthread`、`crypt`、`util`、`xnet`、 +`resolv` 与 `dl`，并在这些名字下各安装一个空归档；openkal-musl 0.19.2 带有这八个归档， +当缺失的名字全都属于它们时，说明会点名这个版本。 + +在 mcpp 2026.9.26.1 之前，这样的链接会在图的目录之后继续搜索宿主目录，因此 +`x86_64-linux-musl` 上的 `-lm` 会把宿主的 glibc 目标文件链接进 musl 镜像而没有任何诊断， +在其他目标上则以一条不相干的消息失败（mcpp#696）。 + ### 层名固定，实现不固定 五个层名是编译进引擎的一个闭集。填充它们的具体实现出现在包清单与索引中， diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index 80c2d41c7..c27a7b46d 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -811,6 +811,18 @@ mcpp 会把它自己构建时用的**同一份** std 模块暂存过来，缓存 这些契约值**无条件**折入重跑键——换 target、换 profile、开关 feature 都会触发重跑， 不需要任何 `rerun-if-env-changed` 声明。 +### 两个方向上的文本都是 UTF-8（mcpp 2026.9.26.1+） + +mcpp 传给程序的每个路径与值都是 UTF-8，程序打印的每条指令也按 UTF-8 读取。 +在 Windows 上，程序链接时带有 `mcpp.exe` 自身携带的应用程序清单，它在 +Windows 10 1903 及以后的版本上把程序的 ANSI 代码页设为 UTF-8：程序的环境、参数 +以及它打印的窄字符串在那里都是 UTF-8，程序内无需任何转换。在 Linux 与 macOS 上， +文件名是字节，列出目录的程序打印的就是它找到的那些字节。 + +文本不是 UTF-8 的指令行会被拒绝，无论程序声明了哪个协议，拒绝信息会点名该指令的 +键。它的值会以指向另一个文件的字节进入 `build.ninja`，所以它的任何部分都不会被 +施加。不是指令的行不会被检查。 + ### `PATH` —— 项目声明的那个环境（mcpp 2026.8.25.1+） 声明了 `[xlings].subos` 的项目，其构建程序运行时，该环境的 `bin` 在 `PATH` 的 diff --git a/docs/zh/91-toolchain-internals.md b/docs/zh/91-toolchain-internals.md index 4a4ec2159..1d9c37764 100644 --- a/docs/zh/91-toolchain-internals.md +++ b/docs/zh/91-toolchain-internals.md @@ -398,6 +398,11 @@ CRT 名字（干净机器上的 #195 症状），以及静默的宿主 CRT 污 `MCPP_ALLOW_HOST_LIBS=1`。系统 / PATH 编译器豁免此项检查 —— 显式选择 宿主世界是用户自己的决定。 +链接依赖图提供的 C 库时还要满足一条规则（mcpp#696，2026.9.26.1+）。这样的链接 +带有 `--sysroot=<构建目录>/graph-sysroot`，一个空目录，驱动器因此不会从宿主推导 +任何库目录；检查还会读取试运行中的 `-L` 目录：每一个都必须位于工具链存储、构建目录 +或依赖图中的某个包之内。判定结果的缓存键记录链接是否属于这一类，以及它允许的包根。 + CI 用一个**完全没有宿主工具链**的 job（`debian:stable-slim`，没有 gcc， 也没有宿主的 `Scrt1.o`）守住这一点 —— 它是唯一能忠实复现干净机器故障 模式的环境类别，另外还有 e2e `86_llvm_hermetic_link.sh`，在任何机器上 diff --git a/mcpp.toml b/mcpp.toml index d34c38e34..96c19fc01 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.9.25.1" +version = "2026.9.26.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] @@ -35,6 +35,11 @@ default-profile = "release" # Revert is this one line if anything here is ever unexplained. bmi_schedule = "on" +# mcpp.exe declares the UTF-8 code page (mcpp#693); see res/mcpp.rc. Only a PE +# target compiles a resource script, so this changes nothing elsewhere. +[resources] +files = ["res/mcpp.rc"] + [toolchain] default = "gcc@16.1.0" macos = "llvm@22.1.8" @@ -49,11 +54,13 @@ windows = "llvm@20.1.7" toolchain = "gcc@16.1.0-musl" linkage = "static" -# aarch64 (ARM64) static target. No explicit toolchain: mcpp's host-aware -# convention picks it — on an aarch64 host this is a NATIVE build (xim:musl-gcc), -# on an x86_64 host a CROSS build (xim:aarch64-linux-musl-gcc, gcc 15.1.0 from -# musl-cross-make). linkage=static is also implied by the -musl suffix; kept -# explicit for clarity. See .agents/docs/2026-06-22-aarch64-android-mvp-design.md. +# aarch64 (ARM64) static target. No explicit toolchain: the target table pins +# gcc@16.1.0 for this row, and mcpp's host-aware convention picks the payload — +# on an aarch64 host a NATIVE build (xim:musl-gcc), on an x86_64 host a CROSS +# build (xim:aarch64-linux-musl-gcc 16.1.0; measured for #694, which found this +# comment still naming 15.1.0). linkage=static is also implied by the -musl +# suffix; kept explicit for clarity. See +# .agents/docs/2026-06-22-aarch64-android-mvp-design.md. [target.aarch64-linux-musl] linkage = "static" diff --git a/modules/buildmcpp/src/directives.cppm b/modules/buildmcpp/src/directives.cppm index 44d97293a..2328774bb 100644 --- a/modules/buildmcpp/src/directives.cppm +++ b/modules/buildmcpp/src/directives.cppm @@ -425,6 +425,11 @@ struct Directives { // `mcpp:` keys this engine does not know. Whether that is fatal depends on // `protocol` — see unknown_directive_error(). std::vector unknownKeys; + // Directive lines whose text is not UTF-8, as the `mcpp:` each one + // began with (#693). Every path and flag mcpp holds is UTF-8, and the + // value of such a line would reach build.ninja as bytes Ninja reads in + // another encoding, naming a different file. + std::vector nonUtf8Keys; std::vector& at(Slot s) { return slots[static_cast(s)]; } const std::vector& at(Slot s) const { return slots[static_cast(s)]; } @@ -446,6 +451,7 @@ enum class LineResult { Accepted, Protocol, // `mcpp:protocol=` Unknown, // a `mcpp:` key this engine does not know + NotUtf8, // a directive whose text is not UTF-8 }; // Parse ONE stdout line into `d`. `root` resolves relative paths for the @@ -466,6 +472,11 @@ void accept_output(Directives& d, const mcpp::toolchain::CommandDialect& dial, // protocol 1, and its unknown keys are typos rather than future syntax. std::optional protocol_error(const Directives& d); +// Non-empty when the program printed a directive whose text is not UTF-8. +// Always an error: a value in another encoding is a different path, whatever +// protocol the program speaks. +std::optional encoding_error(const Directives& d); + // ── Advisories ───────────────────────────────────────────────────────────── // // The `mcpp:warning=` lines a program emitted, each already prefixed with the @@ -707,6 +718,17 @@ LineResult accept_line(Directives& d, const mcpp::toolchain::CommandDialect& dia std::string_view body = std::string_view(line).substr(kPfx.size()); auto eq = body.find('='); std::string key = std::string(body.substr(0, eq)); + + // Before the key is looked up: a line in another encoding is refused as + // such, whether or not its key is known. + if (!mcpp::modgraph::is_valid_utf8(line)) { + auto shown = "mcpp:" + key; + if (!mcpp::modgraph::is_valid_utf8(shown)) shown = "mcpp:"; + if (std::find(d.nonUtf8Keys.begin(), d.nonUtf8Keys.end(), shown) + == d.nonUtf8Keys.end()) + d.nonUtf8Keys.push_back(std::move(shown)); + return LineResult::NotUtf8; + } std::string val = eq == std::string_view::npos ? std::string() : std::string(body.substr(eq + 1)); @@ -782,6 +804,21 @@ std::optional protocol_error(const Directives& d) { return std::nullopt; } +std::optional encoding_error(const Directives& d) { + if (d.nonUtf8Keys.empty()) return std::nullopt; + std::string list; + for (auto const& k : d.nonUtf8Keys) list += (list.empty() ? "" : ", ") + k; + return std::format( + "build.mcpp printed directive(s) whose text is not UTF-8: {}.\n" + " mcpp reads a build program's output as UTF-8, and a path in " + "another encoding names a different file\n" + " in build.ninja and compile_commands.json. Print UTF-8: on " + "Windows, the program mcpp compiles runs in\n" + " the UTF-8 code page on Windows 10 version 1903 and later; on " + "POSIX, convert the name before printing it.", + list); +} + void serialize(std::ostream& os, const Directives& d) { // Table order, and one pass per row rather than per slot: rows sharing a // slot (link-lib / link-search) share a tag, so emitting per row would diff --git a/modules/manifest/src/glob.cppm b/modules/manifest/src/glob.cppm index 3cea1cdb7..e4a4e1b57 100644 --- a/modules/manifest/src/glob.cppm +++ b/modules/manifest/src/glob.cppm @@ -9,6 +9,8 @@ export module mcpp.modgraph.glob; import std; +import mcpp.platform.common; // is_windows +import mcpp.platform.windows; // active_code_page export namespace mcpp::modgraph { @@ -32,6 +34,20 @@ std::filesystem::path native_path_from_generic(std::string_view s) { return p; } +// Whether `s` is well-formed UTF-8: no stray continuation byte, no truncated or +// overlong sequence, no surrogate, nothing above U+10FFFF. +bool is_valid_utf8(std::string_view s); + +// A printable spelling of a path that has no UTF-8 spelling, for a diagnostic +// that must name it. Well-formed UTF-8 (POSIX) and well-formed UTF-16 +// (Windows) pass through; any other byte appears as `\xNN` and an unpaired +// surrogate as `\u{NNNN}`. The result is UTF-8 whatever the input. +std::string escaped_spelling(const std::filesystem::path& p); + +// Why a path can have no UTF-8 spelling on this host, as one sentence. The +// three places that refuse or skip such a path give the same reason. +std::string no_utf8_spelling_reason(); + // ─── narrowing a walk-derived path ──────────────────────────────────────── // // THE ONE PLACE a path that came out of a directory walk becomes a narrow @@ -66,9 +82,21 @@ std::filesystem::path native_path_from_generic(std::string_view s) { // nullopt means: this path cannot be named in any string we hand to a // compiler, a build file, or a glob. Skip it — and record it, because // "silently not built" is exactly where this class of bug hides. +// +// A NAME THAT IS NOT UTF-8 CANNOT BE NAMED EITHER (#693). mcpp's strings are +// UTF-8 on every platform, and the documents it writes are UTF-8 by definition +// (JSON) or by declaration (Ninja on Windows). Two inputs used to pass the +// narrowing and fail later, inside a JSON writer, as `internal: unhandled +// exception: [json.exception.type_error.316]`: a POSIX file name made of bytes +// that are not UTF-8 (measured on Linux), and, on a Windows host that ignores +// the UTF-8 code page mcpp declares, any non-ASCII name the ANSI code page can +// spell (measured on cp1252 before the manifest). Both are now skipped and +// reported here, at the point of entry. std::optional try_narrow(const std::filesystem::path& p) { try { - return p.generic_string(); + auto s = p.generic_string(); + if (!is_valid_utf8(s)) return std::nullopt; + return s; } catch (const std::exception&) { return std::nullopt; } @@ -173,6 +201,84 @@ namespace { std::mutex g_unnarrowableMu; std::set g_unnarrowable; +// The length of the well-formed UTF-8 sequence starting at `s[i]`, or 0 when +// the bytes there are not one. +std::size_t utf8_sequence_length(std::string_view s, std::size_t i) { + const auto c = static_cast(s[i]); + if (c < 0x80) return 1; + std::size_t len = 0; + std::uint32_t cp = 0; + if ((c & 0xE0) == 0xC0) { len = 2; cp = c & 0x1Fu; } + else if ((c & 0xF0) == 0xE0) { len = 3; cp = c & 0x0Fu; } + else if ((c & 0xF8) == 0xF0) { len = 4; cp = c & 0x07u; } + else return 0; + if (i + len > s.size()) return 0; + for (std::size_t k = 1; k < len; ++k) { + const auto cc = static_cast(s[i + k]); + if ((cc & 0xC0) != 0x80) return 0; + cp = (cp << 6) | (cc & 0x3Fu); + } + if ((len == 2 && cp < 0x80) || (len == 3 && cp < 0x800) + || (len == 4 && cp < 0x10000) || cp > 0x10FFFF + || (cp >= 0xD800 && cp <= 0xDFFF)) + return 0; + return len; +} + +void append_utf8(std::string& out, std::uint32_t cp) { + if (cp < 0x80) { + out += static_cast(cp); + } else if (cp < 0x800) { + out += static_cast(0xC0 | (cp >> 6)); + out += static_cast(0x80 | (cp & 0x3F)); + } else if (cp < 0x10000) { + out += static_cast(0xE0 | (cp >> 12)); + out += static_cast(0x80 | ((cp >> 6) & 0x3F)); + out += static_cast(0x80 | (cp & 0x3F)); + } else { + out += static_cast(0xF0 | (cp >> 18)); + out += static_cast(0x80 | ((cp >> 12) & 0x3F)); + out += static_cast(0x80 | ((cp >> 6) & 0x3F)); + out += static_cast(0x80 | (cp & 0x3F)); + } +} + +// POSIX: a path is bytes. Well-formed sequences are kept, other bytes escaped. +[[maybe_unused]] std::string escape_native(const std::string& s) { + std::string out; + std::size_t i = 0; + while (i < s.size()) { + if (const std::size_t len = utf8_sequence_length(s, i)) { + out.append(s, i, len); + i += len; + } else { + out += std::format("\\x{:02X}", static_cast(s[i])); + ++i; + } + } + return out; +} + +// Windows: a path is UTF-16 code units. Pairs are combined, and a surrogate +// without its partner is escaped. +[[maybe_unused]] std::string escape_native(const std::wstring& s) { + std::string out; + for (std::size_t i = 0; i < s.size(); ++i) { + std::uint32_t u = static_cast(s[i]); + if (u >= 0xD800 && u <= 0xDBFF && i + 1 < s.size()) { + const std::uint32_t v = static_cast(s[i + 1]); + if (v >= 0xDC00 && v <= 0xDFFF) { + append_utf8(out, 0x10000 + ((u - 0xD800) << 10) + (v - 0xDC00)); + ++i; + continue; + } + } + if (u >= 0xD800 && u <= 0xDFFF) out += std::format("\\u{{{:04X}}}", u); + else append_utf8(out, u); + } + return out; +} + } // namespace void note_unnarrowable_path(const std::filesystem::path& p) { @@ -192,6 +298,36 @@ void note_unnarrowable_path(const std::filesystem::path& p) { g_unnarrowable.insert(std::move(anchor)); } +bool is_valid_utf8(std::string_view s) { + std::size_t i = 0; + while (i < s.size()) { + const std::size_t len = utf8_sequence_length(s, i); + if (len == 0) return false; + i += len; + } + return true; +} + +std::string escaped_spelling(const std::filesystem::path& p) { + return escape_native(p.native()); +} + +std::string no_utf8_spelling_reason() { + if constexpr (mcpp::platform::is_windows) { + const unsigned acp = mcpp::platform::windows::active_code_page(); + if (acp != 65001) + return std::format( + "This process runs in the ANSI code page {} rather than UTF-8: " + "mcpp.exe declares UTF-8, which Windows 10 version 1903 and " + "later honour, and `chcp` changes neither.", acp); + return "On Windows only a name that is not valid Unicode (an unpaired " + "surrogate) has none."; + } else { + return "The name's bytes are not UTF-8, and build.ninja and " + "compile_commands.json hold UTF-8 text."; + } +} + std::vector take_unnarrowable_paths() { std::lock_guard lk(g_unnarrowableMu); std::vector out(g_unnarrowable.begin(), g_unnarrowable.end()); diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index ce75d9780..0eac98d79 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -1742,6 +1742,18 @@ std::expected parse_string(std::string_view content, return std::unexpected(r.error()); if (auto r = read_choice("windows_entry", t.windowsEntry, false); !r) return std::unexpected(r.error()); + // `windows_code_page` (#693): the same closed-set treatment. + if (auto it = tt.find("windows_code_page"); it != tt.end()) { + if (!it->second.is_string()) + return std::unexpected(error(origin, std::format( + "targets.{}.windows_code_page must be a string", tname))); + const std::string v = it->second.as_string(); + if (auto list = windows_code_page_problem(v); !list.empty()) + return std::unexpected(error(origin, std::format( + "targets.{}.windows_code_page = \"{}\" is not one of {}", + tname, v, list))); + t.windowsCodePage = v; + } // An executable's property. A library has no subsystem, and a GUI // subsystem on anything a test runner executes is the defect #618 // describes, so both are refused naming the key. `app` is accepted @@ -1749,12 +1761,15 @@ std::expected parse_string(std::string_view content, // never meets `application_form`'s SharedObject form in practice) -- // `is_program()` is "is this the program", which is what the PE // subsystem attaches to; `TestBinary` stays refused on purpose. - if ((!t.windowsSubsystem.empty() || !t.windowsEntry.empty()) + if ((!t.windowsSubsystem.empty() || !t.windowsEntry.empty() + || !t.windowsCodePage.empty()) && !t.is_program()) return std::unexpected(error(origin, std::format( "targets.{}.{} applies to an executable (`kind = \"bin\"` or " "`\"app\"`), and this target is not one", tname, - t.windowsSubsystem.empty() ? "windows_entry" : "windows_subsystem"))); + !t.windowsSubsystem.empty() ? "windows_subsystem" + : !t.windowsEntry.empty() ? "windows_entry" + : "windows_code_page"))); // Guard: -std=... belongs to [package].standard, not per-target flags // (same rule as [build].cxxflags). Reject early with a clear message. for (auto const& flag : t.cxxflags) { @@ -1773,7 +1788,7 @@ std::expected parse_string(std::string_view content, static constexpr std::string_view kKnownTargetKeys[] = { "kind", "linkage", "main", "soname", "exports", "cflags", "cxxflags", "defines", "required_features", - "windows_entry", "windows_subsystem", + "windows_entry", "windows_subsystem", "windows_code_page", }; for (auto& [key, _] : tt) { bool known = false; diff --git a/modules/manifest/src/types.cppm b/modules/manifest/src/types.cppm index b9f48b695..e04aa713b 100644 --- a/modules/manifest/src/types.cppm +++ b/modules/manifest/src/types.cppm @@ -133,6 +133,17 @@ inline std::string windows_choice_problem(bool subsystem, std::string_view value return accepted ? std::string{} : list; } +// The accepted values of `windows_code_page` (#693), which are the vocabulary of +// the `activeCodePage` element of a Windows application manifest. `utf-8` embeds +// a manifest that makes the process ANSI code page UTF-8 on Windows 10 1903 and +// later, so the narrow C and Win32 APIs take and return UTF-8; `legacy` embeds +// none. Returns an empty string when `value` is accepted, and otherwise the +// accepted values, quoted and comma-separated, for the refusal. +inline std::string windows_code_page_problem(std::string_view value) { + if (value == "utf-8" || value == "legacy") return {}; + return "\"utf-8\", \"legacy\""; +} + struct Target { std::string name; // `Application` (#622 A3, `kind = "app"`) is "the thing a user launches", @@ -194,6 +205,13 @@ struct Target { // inert on every object format that is not PE. std::string windowsSubsystem; std::string windowsEntry; + // `windows_code_page` (#693): "utf-8" embeds an application manifest that + // makes this PE executable's process ANSI code page UTF-8; "legacy" embeds + // none. Empty = the default of the build role: `legacy` for an ordinary + // target, `utf-8` for a target built as a host tool (the tool receives + // mcpp's UTF-8 paths on its command line). Inert on every object format + // that is not PE. + std::string windowsCodePage; // Where `kind` was stated, as the manifest line that states it: // `[targets.fw] kind = "shared"`, or, when a row states it, // `[target.'cfg(os = "android")'.targets.fw] kind = "shared"`. Read by @@ -639,6 +657,22 @@ struct NamedRunner { bool longLived = false; }; +// THE C STANDARD OF A PACKAGE THAT DECLARES NONE (#695). +// +// It is the only C standard the file-level `$cflags` carries, and so it is a +// graph-wide constant rather than the root's value. A package's own +// `[build] c_standard` reaches that package's C units as a per-unit flag, and a +// consumer's value never reaches a dependency: C translation units produce no +// BMI, so nothing requires one standard across a graph, and a program that +// links C objects compiled under different standards is ordinary. +inline constexpr std::string_view kDefaultCStandard = "c11"; + +// The C standard a package's own C units compile at: its declared value, or +// `kDefaultCStandard`. +inline std::string effective_c_standard(std::string_view declared) { + return declared.empty() ? std::string(kDefaultCStandard) : std::string(declared); +} + struct BuildConfig : BuildInputs { // How `mcpp run` / `mcpp test` execute an artifact this host cannot run, // as an argv template (the artifact path is appended, or substituted for @@ -854,8 +888,9 @@ struct BuildConfig : BuildInputs { // "default to fully-static musl" belongs here, not in a toolchain name // (static output is a product property, not a compiler-family property). std::string target; - // M5.x C-language support: `cStandard` controls -std= for the C compile - // rule (.c files); empty → backend default ("c11" today). The cflags / + // `cStandard` is this package's own C standard (`[build] c_standard`): + // empty means `kDefaultCStandard`, and the value reaches this package's C + // units only, never a consumer's or a dependency's (#695). The cflags / // cxxflags / ldflags vectors themselves live in BuildInputs above. // Dialect-class C++ flags: flags that change what the standard library's // headers DECLARE or participate in module dialect checks (issue #210's diff --git a/modules/platform/src/windows/windows.cppm b/modules/platform/src/windows/windows.cppm index 265e72703..684e393b9 100644 --- a/modules/platform/src/windows/windows.cppm +++ b/modules/platform/src/windows/windows.cppm @@ -1,7 +1,8 @@ // mcpp.platform.windows — Windows-specific platform capabilities. // // Provides: -// prepend_path() — add a directory to the front of %PATH% +// prepend_path() — add a directory to the front of %PATH% +// active_code_page() — the process ANSI code page (GetACP) // // Note: Visual Studio / MSVC discovery is in mcpp.toolchain.msvc, which is // the authoritative module for MSVC toolchain detection. This module @@ -11,6 +12,13 @@ module; #include #if defined(_WIN32) #include // _putenv_s +#ifndef NOMINMAX +#define NOMINMAX +#endif +#ifndef WIN32_LEAN_AND_MEAN +#define WIN32_LEAN_AND_MEAN +#endif +#include // GetACP #endif export module mcpp.platform.windows; @@ -22,6 +30,13 @@ export namespace mcpp::platform::windows { // Prepend a directory to the %PATH% environment variable. void prepend_path(const std::filesystem::path& dir); +// The ANSI code page of this process: 65001 when the UTF-8 code page that +// mcpp.exe declares is in effect (Windows 10 version 1903 and later), the +// system's legacy code page otherwise. 0 on every other platform, which has +// no process code page. Every narrow string mcpp exchanges with Win32 is in +// this code page (#693). +unsigned active_code_page(); + } // namespace mcpp::platform::windows // ─── Implementation ────────────────────────────────────────────────────── @@ -38,4 +53,12 @@ void prepend_path(const std::filesystem::path& dir) { #endif } +unsigned active_code_page() { +#if defined(_WIN32) + return static_cast(GetACP()); +#else + return 0; +#endif +} + } // namespace mcpp::platform::windows diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index e7dba7a6a..8723bc1f1 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.9.25.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.9.26.1"; } // namespace mcpp diff --git a/res/mcpp.exe.manifest b/res/mcpp.exe.manifest new file mode 100644 index 000000000..bddb2d96e --- /dev/null +++ b/res/mcpp.exe.manifest @@ -0,0 +1,8 @@ + + + + + UTF-8 + + + diff --git a/res/mcpp.rc b/res/mcpp.rc new file mode 100644 index 000000000..fdbc13737 --- /dev/null +++ b/res/mcpp.rc @@ -0,0 +1,12 @@ +// mcpp.exe runs in the UTF-8 code page (mcpp#693). +// +// Every string mcpp holds is UTF-8, and a Windows process exchanges narrow +// strings with the system in its ANSI code page. The manifest beside this +// script sets that code page to UTF-8 on Windows 10 version 1903 and later. +// +// It is declared here, through `[resources] files`, and not with the +// `windows_code_page` target key: a release is built by the previous release, +// and an engine that does not know the key ignores it. The file is named +// relative to this script, which is where rc.exe, llvm-rc and windres all look +// for it (measured) and where mcpp resolves it as a build input. +1 24 "mcpp.exe.manifest" diff --git a/src/build/build_program.cppm b/src/build/build_program.cppm index 90095f2eb..5e917239a 100644 --- a/src/build/build_program.cppm +++ b/src/build/build_program.cppm @@ -23,6 +23,7 @@ import mcpp.toolchain.fingerprint; // hash_file / hash_string (FNV-1a, 16 hex) import mcpp.build.directives; // the directive definition table (own module: see its header) import mcpp.build.refusal; // the machine-readable identity of a refusal import mcpp.build.hostprogram; // bundled `mcpp` module compile (own module: see its header) +import mcpp.build.resources; // compile_utf8_manifest — the build program speaks UTF-8 (#693) import mcpp.toolchain.hostflags; // the shared host-compile flag producer import mcpp.toolchain.linkmodel; // shared C-library / clang-cfg-bypass model import mcpp.toolchain.model; // Toolchain, PayloadPaths, is_clang/is_musl_target/is_mingw_target @@ -1463,6 +1464,30 @@ std::expected run_build_program( for (auto& hmo : hostModuleObjects) compileArgv.push_back(hmo.string()); for (auto& so : stdObjects) compileArgv.push_back(so); } + // THE BUILD PROGRAM SPEAKS THE ENCODING mcpp SPEAKS (#693, D4). + // + // mcpp hands a build program its paths through the environment and reads + // its directives from stdout, and on Windows mcpp runs with a UTF-8 process + // code page. A build program without the same application manifest reads + // the environment through the machine's ANSI code page instead. Measured on + // a cp1252 runner: a narrow build.mcpp in `C:\w\caf` printed its + // path in cp1252 bytes that mcpp could not decode, and in a directory + // outside the code page it received `??` in place of the name. So the + // program carries the manifest; where no resource compiler stands beside + // the compiler, it is built without one and the build says so. + if constexpr (mcpp::platform::is_windows) { + auto manifestRes = mcpp::build::resources::compile_utf8_manifest( + tc, dial.id, bdir, "build.mcpp.utf8"); + if (manifestRes) { + if (!msvcHost) { compileArgv.push_back("-x"); compileArgv.push_back("none"); } + compileArgv.push_back(manifestRes->string()); + } else { + mcpp::ui::warning(std::format( + "build.mcpp: built without the UTF-8 code page ({}); a path outside " + "this machine's ANSI code page will not reach it intact", + manifestRes.error())); + } + } // Self-contained helper link — see the staticHostHelper doctrine above. // Deliberately NOT in `base`: that also feeds the bundled module's // compile/precompile commands, where a link flag has no business (and for @@ -1590,6 +1615,11 @@ std::expected run_build_program( if (auto perr = dirs::protocol_error(d)) { return std::unexpected(*perr); } + // A directive in another encoding names a different file (#693). Refused + // before anything is applied, for the same reason as the checks below. + if (auto eerr = dirs::encoding_error(d)) { + return std::unexpected(*eerr); + } // Refuse a malformed action BEFORE applying anything: a half-applied // action set is worse than none, and an action that silently does not // exist surfaces as a missing generated source three edges away. diff --git a/src/build/cache_key.cppm b/src/build/cache_key.cppm index e641b551f..984074f0f 100644 --- a/src/build/cache_key.cppm +++ b/src/build/cache_key.cppm @@ -526,7 +526,11 @@ BuildAxes build_axes(const mcpp::toolchain::Toolchain& tc, b.cppStandard = rootManifest.package.standard; b.cppStandardFlag = std::string(cppStandardFlag); b.dialectFlags = dialectFlags; - b.cStandard = rootManifest.buildConfig.cStandard; + // The file-level C standard, which is the engine's constant and not the + // root's value: a consumer's `c_standard` no longer reaches a dependency's + // commands (#695), so it is no longer part of a dependency's key. A + // package's own standard is keyed in its PackageAxes (`__c_standard`). + b.cStandard = std::string(mcpp::manifest::kDefaultCStandard); b.minPlatformVersion = std::string(minPlatformVersion); b.optLevel = rootManifest.buildConfig.optLevel; @@ -578,9 +582,12 @@ void fill_package_config(PackageAxes& out, out.asmflags = pkg.privateBuild.asmflags; } - if (!bc.cStandard.empty()) { - // A package may pin its own C standard; it reaches its own C units. - out.cflags.push_back("__c_standard=" + bc.cStandard); + // A package's own C standard reaches its own C units as a per-unit flag + // when it differs from the default (`make_plan`, #695). Keyed only then, so + // two packages whose C commands are identical share one key. + if (auto own = mcpp::manifest::effective_c_standard(bc.cStandard); + own != mcpp::manifest::kDefaultCStandard) { + out.cflags.push_back("__c_standard=" + own); } // A C++-layer provider's own C++ level reaches its implementation units // (`make_plan`), and the graph's level alone does not say which level diff --git a/src/build/compile_commands.cppm b/src/build/compile_commands.cppm index 8bc73878d..93afe61ca 100644 --- a/src/build/compile_commands.cppm +++ b/src/build/compile_commands.cppm @@ -450,8 +450,22 @@ write_compile_commands(const BuildPlan& plan, const CompileFlags& flags) { auto path = plan.compileDbPath.empty() ? plan.projectRoot / "compile_commands.json" : plan.compileDbPath; + // A JSON document holds UTF-8 text only, and the serialiser throws on any + // other byte (`type_error.316`). The entry points refuse or skip a path + // with no UTF-8 spelling (#693), so a string reaching this point in another + // encoding entered some other way. It fails this document, which the + // caller reports as a warning or, when the database is required, as an + // error, and never as an internal exception. + std::string fresh; + try { + fresh = emit_compile_commands(plan, flags); + } catch (const nlohmann::json::exception& e) { + return std::unexpected(write_error(std::format( + "it cannot be written as JSON, which holds UTF-8 text only ({})", + e.what()))); + } return publish_compile_commands( - path, emit_compile_commands(plan, flags), + path, fresh, [](const std::filesystem::path& candidate) { return std::filesystem::exists(candidate); }); diff --git a/src/build/execute.cppm b/src/build/execute.cppm index a9f1edfbf..6694eec60 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -28,6 +28,7 @@ import mcpp.freestanding.linkline; import mcpp.build.graph_shape; // #407: which mode wrote this build.ninja import mcpp.build.backend; import mcpp.build.ninja; +import mcpp.build.flags; // realises_optimization — one answer for the level (#694) import mcpp.build.runtime_validation; import mcpp.bmi_cache; import mcpp.bmi_cache.maintenance; // dir_size + human_bytes, for `clean --stale` @@ -1031,13 +1032,13 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, // exact failure mode it exists to prevent. if (!mcpp::diag::flush(ctx.strict)) return 1; - // The descriptor comes from the knobs this build actually resolved, so it - // cannot disagree with the compiler flags the way the old hardcoded - // "release [optimized]" did. + // The descriptor reads the level the compile realised, from the one + // function `compute_flags` spells it from (#694). It once read the declared + // level while the compile used another, and said `[optimized]` over `-Og`. { const auto& bc = ctx.manifest.buildConfig; std::string descriptor = - (bc.optLevel.empty() || bc.optLevel == "0") ? "unoptimized" : "optimized"; + mcpp::build::realises_optimization(bc) ? "optimized" : "unoptimized"; if (bc.debug) descriptor += " + debuginfo"; if (bc.lto) descriptor += " + lto"; mcpp::ui::finished(ctx.profile, r->elapsed, descriptor); @@ -1258,6 +1259,9 @@ std::optional run_ninja_fast(const std::string& ninjaProgram, if (auto advice = mcpp::build::graph_c_library_isolation_advice(out); !advice.empty()) std::fputs(advice.c_str(), stderr); + // #696, the fast-path form of the same unnamed shape. + if (auto advice = mcpp::build::graph_link_library_advice(out); !advice.empty()) + std::fputs(advice.c_str(), stderr); // #690: the consumer-include note, from the list the plan wrote beside // build.ninja (`write_consumer_include_sidecar`). if (auto advice = mcpp::build::consumer_include_scope_advice( diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 17854cbe0..90673ed4d 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -41,6 +41,11 @@ struct CompileFlags { std::string as; // asm-safe subset for .S/.s via the C driver std::string nasm; // NASM global flags (.asm; own spelling) std::string ld; // ldflags string + // True when the link line carries the empty graph sysroot (#696): an ELF + // link over a graph-supplied C library, by clang. The backend creates the + // directory, and the hermetic check then holds every `-L` on that link to + // the store, the build directory and the graph's own package roots. + bool graphLinkIsolated = false; // The same link line for a unit with NO C++ in it (mcpp#426). Linking a // pure-C library with the C++ driver gave it `NEEDED libstdc++.so.6`, // `libm.so.6` and `libgcc_s.so.1` with not one symbol referencing them — @@ -140,6 +145,31 @@ std::string render_link_intent_flags( CompileFlags compute_flags(const BuildPlan& plan); +// THE OPTIMIZATION LEVEL A BUILD REALISES, STATED ONCE (#694). +// +// `compute_flags` spells it and the `Finished` line names it. There used to be +// two answerers: the line read the declared level, while the compile took a +// branch keyed on the target triple that replaced every non-zero level with +// `-Og` on `*-linux-musl`. That branch was a workaround for one musl-gcc 15.1.0 +// internal compiler error, it reached clang as well, and it set the level of +// mcpp's own Linux release binaries; its trigger no longer reproduces on any +// available input (.agents/docs/2026-09-25-issues-693-696-triage-and-repair-plan.md +// §3.2). A defect of one toolchain version is answered by the version pin or +// by a project's own per-file flags, never by the engine replacing a declared +// value. An empty level is `0`: the old spelling rendered it as a bare `-O`, +// which GCC reads as `-O1`, while the line called it unoptimized. +std::string realised_opt_level(const mcpp::manifest::BuildConfig& bc); + +// Whether `realised_opt_level` names an optimizing level. +bool realises_optimization(const mcpp::manifest::BuildConfig& bc); + +// The sysroot of an ELF link over a graph-supplied C library: an empty +// directory inside the build directory, so the driver derives no library +// search from the host (#696). `compute_flags` names it and the backend creates +// it; the link-failure advice recognises it by `kGraphLinkSysrootDir`. +inline constexpr std::string_view kGraphLinkSysrootDir = "graph-sysroot"; +std::filesystem::path graph_link_sysroot(const std::filesystem::path& outputDir); + // ── Which link line a (host, target) pair takes (#647 E3) ───────────────── // // THE HOST DOES NOT DECIDE THIS ALONE. Three of the link branches describe a @@ -522,6 +552,18 @@ LinkShape link_shape(LinkHost host, mcpp::build::dist::Format targetFormat, return LinkShape::Generic; } +std::string realised_opt_level(const mcpp::manifest::BuildConfig& bc) { + return bc.optLevel.empty() ? std::string("0") : bc.optLevel; +} + +bool realises_optimization(const mcpp::manifest::BuildConfig& bc) { + return realised_opt_level(bc) != "0"; +} + +std::filesystem::path graph_link_sysroot(const std::filesystem::path& outputDir) { + return outputDir / kGraphLinkSysrootDir; +} + CompileFlags compute_flags(const BuildPlan& plan) { CompileFlags f; @@ -978,14 +1020,14 @@ CompileFlags compute_flags(const BuildPlan& plan) { f.arBinary = mcpp::toolchain::archive_tool(plan.toolchain); // Opt level + debug come from the resolved build profile - // ([profile.] → buildConfig). musl keeps -Og as an ICE workaround - // unless the profile pins -O0. + // ([profile.] → buildConfig). The level is the one the profile + // declares, on every target and with every compiler; see + // `realised_opt_level` for why nothing here may replace it. auto& prof = plan.manifest.buildConfig; - std::string opt_flag = isMuslTc && prof.optLevel != "0" - ? " -Og" - : (isMsvcDialect && prof.optLevel == "0") + const std::string optLevel = realised_opt_level(prof); + std::string opt_flag = (isMsvcDialect && optLevel == "0") ? " /Od" // MSVC's no-opt spelling (there is no /O0) - : std::format(" {}{}", d.optPrefix, prof.optLevel); + : std::format(" {}{}", d.optPrefix, optLevel); if (prof.debug) opt_flag += std::format(" {}", d.debugFlags); if (prof.lto && !isMsvcDialect) opt_flag += " -flto"; @@ -1011,9 +1053,13 @@ CompileFlags compute_flags(const BuildPlan& plan) { user_ldflags += normalize_ldflag(plan.projectRoot, flag); } - // C standard - std::string c_std = - plan.manifest.buildConfig.cStandard.empty() ? "c11" : plan.manifest.buildConfig.cStandard; + // C standard. The file-level `$cflags` carries the engine default, a + // graph-wide constant, and never the root's own value: a package's + // `[build] c_standard` reaches its own C units as a per-unit flag + // (`make_plan`), the root's included. When this line carried the root's + // value, every dependency's C units compiled at the consumer's standard and + // their own declarations were parsed, hashed and never applied (#695). + std::string c_std(mcpp::manifest::kDefaultCStandard); // Assemble // Module-flag spellings come from BmiTraits: GCC needs -fmodules on every @@ -2000,6 +2046,30 @@ CompileFlags compute_flags(const BuildPlan& plan) { if (plan.toolchain.compiler == mcpp::toolchain::CompilerId::Clang) graphLd += " -fuse-ld=lld"; + // AND NO SYSROOT IS THE HOST'S ROOT (#696). + // + // The payload link above passes `--sysroot=`; this replacement + // passes none, and a clang with no sysroot derives its library search + // from `/` and, on x86_64, from the host's GCC installation. `-nostdlib` + // removes the startup files and default libraries but not those + // directories, so a `-l` the graph did not answer was looked up on the + // build machine. Measured with clang 22.1.8: `-lm` linked glibc's + // `s_fmaximum.o` into an x86_64-linux-musl static image on Ubuntu 24.04, + // and failed on aarch64-linux-musl because glibc's `libm.a` is an x86_64 + // linker script. An empty directory owned by the build is the sysroot + // that holds nothing of the host's; the C library package answers the + // names musl answers from libc with its own empty archives. + // + // ELF only, and clang only: that is the measured case. PE links through + // the MinGW driver have the same shape and wait for an inventory of the + // import libraries they resolve from a host MinGW today; Mach-O keeps + // the SDK as its declared platform anchor. + if (plan.toolchain.compiler == mcpp::toolchain::CompilerId::Clang + && targetObjectFormat == mcpp::build::dist::Format::Elf) { + graphLd += " --sysroot=" + escape_path(graph_link_sysroot(plan.outputDir)); + f.graphLinkIsolated = true; + } + // AND THE TARGET'S OWN ANCHOR SURVIVES THE REPLACEMENT. // // Everything this branch rebuilds describes the PAYLOAD — its `-B`, its diff --git a/src/build/hermetic.cppm b/src/build/hermetic.cppm index 43e7eb0fb..4f21abdab 100644 --- a/src/build/hermetic.cppm +++ b/src/build/hermetic.cppm @@ -36,11 +36,20 @@ export namespace mcpp::build { // string from flags.cppm (un-escaped internally). Returns an error message // naming the leaked/bare paths, or empty success. `outputDir` caches the // verdict per flag-set so unchanged builds don't re-spawn the driver. +// +// `isolatedGraphLink` marks an ELF link over a graph-supplied C library that +// carries the empty graph sysroot (#696). On such a link every `-L` directory +// must lie in the store, in the build directory, or in one of `graphRoots` (a +// package may add a package-relative `-L`, and a path dependency's root can be +// anywhere): the C library comes from the graph, so a host library directory +// would answer a `-l` the graph does not. std::expected verify_hermetic_link( const mcpp::toolchain::Toolchain& tc, const std::string& ldflagsNinja, const std::filesystem::path& outputDir, - bool allowHostLibs); + bool allowHostLibs, + bool isolatedGraphLink = false, + const std::vector& graphRoots = {}); } // namespace mcpp::build @@ -105,7 +114,9 @@ std::expected verify_hermetic_link( const mcpp::toolchain::Toolchain& tc, const std::string& ldflagsNinja, const std::filesystem::path& outputDir, - bool allowHostLibs) + bool allowHostLibs, + bool isolatedGraphLink, + const std::vector& graphRoots) { if constexpr (!mcpp::platform::is_linux) return {}; @@ -137,9 +148,13 @@ std::expected verify_hermetic_link( // Verdict cache: same driver + flags ⇒ same resolution; skip the spawn. auto marker = outputDir / ".mcpp-hermetic-ok"; + std::string rootsKey; + if (isolatedGraphLink) + for (auto const& r : graphRoots) rootsKey += "\x1e" + r.generic_string(); auto key = mcpp::toolchain::hash_string( tc.binaryPath.string() + "\x1f" + ldflags - + "\x1f" + (allowHostLibs ? "1" : "0")); + + "\x1f" + (allowHostLibs ? "1" : "0") + + "\x1f" + (isolatedGraphLink ? "graph" : "payload") + rootsKey); { std::ifstream is(marker); std::string prev; @@ -213,6 +228,47 @@ std::expected verify_hermetic_link( check(effectiveLoader); } + // AN ISOLATED GRAPH LINK SEARCHES NO HOST DIRECTORY (#696). + // + // The CRT objects and the loader above are what the driver adds for a C + // library it believes the host provides; on a graph link there are none, + // and until the empty graph sysroot the library search directories were + // the host's all the same. Every `-L` of the linker invocation is held to + // the store, the build directory and the graph's own package roots, so a + // later change to the link line that brings a host directory back fails + // here instead of linking the host's objects in silence. + std::vector searchLeaks; + if (isolatedGraphLink) { + std::vector graphAllowed = allowed; + graphAllowed.push_back(outputDir); + for (auto const& r : graphRoots) graphAllowed.push_back(r); + auto checkDir = [&](std::string_view dir) { + std::filesystem::path p{std::string(dir)}; + if (p.is_absolute() && !under_any(p, graphAllowed)) + searchLeaks.push_back(std::string(dir)); + }; + for (std::size_t i = 0; i < toks.size(); ++i) { + std::string_view t = toks[i]; + if (t == "-L" && i + 1 < toks.size()) { checkDir(toks[++i]); continue; } + if (t.starts_with("-L") && t.size() > 2) checkDir(t.substr(2)); + } + } + if (!searchLeaks.empty()) { + std::string list; + for (auto& l : searchLeaks) list += "\n " + l; + auto msg = std::format( + "hermetic link check failed — this link takes its C library from the " + "dependency graph, and its linker would search directories outside " + "the store, the build directory and the graph's packages:{}\n" + " A library found there would answer a `-l` the graph does not " + "supply (mcpp#696).\n" + " To deliberately link against host libraries set " + "[build] allow_host_libs = true (or MCPP_ALLOW_HOST_LIBS=1).", + list); + if (!allowHostLibs) return std::unexpected(msg); + mcpp::log::verbose("hermetic", "allow_host_libs set — " + msg); + } + if (!leaks.empty()) { std::string list; for (auto& l : leaks) list += "\n " + l; diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index a43a14c4f..2d260ee75 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -167,6 +167,21 @@ std::string graph_c_library_isolation_advice(std::string_view output, std::string_view cAbiName = {}, std::string_view cAbiCoordinate = {}); +// #696: the note for a link over a graph-supplied C library that asked for a +// library the graph does not supply. +// +// Such a link carries an empty sysroot (`kGraphLinkSysrootDir`), so the host's +// library directories no longer answer a `-l`: a build that used to link +// because `/usr/lib` held a `libm.a` now fails with the linker's own +// `unable to find library -lm`, which read on its own names neither the graph +// nor the change. Two conditions, both read from `output`: the failing command +// carries the graph sysroot, and the linker reported an unanswered `-l`. For +// the names musl answers from libc itself, the note names the openkal-musl +// release that ships musl's empty archives for them. +std::string graph_link_library_advice(std::string_view output, + std::string_view cAbiName = {}, + std::string_view cAbiCoordinate = {}); + // #690 (design record F7): the note for a dependency that compiled only because // its consumer's include directories used to reach it. // @@ -198,12 +213,28 @@ void write_consumer_include_sidecar(const std::filesystem::path& outputDir, std::vector read_consumer_include_sidecar(const std::filesystem::path& outputDir); +// #693 (M4): whether a Ninja reads build.ninja in the encoding mcpp writes it. +// +// mcpp writes the file in UTF-8: every path in it has a UTF-8 spelling +// (`try_narrow`), and every other string is UTF-8 by construction. Ninja 1.11 +// and later read it in their process code page and report which with +// `-t wincodepage` (`Build file encoding: UTF-8` or `ANSI`). `reported` is that +// output; `processCodePage` is mcpp's own (65001 for UTF-8), which decides the +// advice. Empty when Ninja reads UTF-8 or when `reported` names no encoding; +// otherwise the refusal, naming the Ninja. +std::optional ninja_encoding_mismatch(std::string_view reported, + unsigned processCodePage, + std::string_view ninjaProgram); + } // namespace mcpp::build namespace mcpp::build { namespace { +// U+FEFF in UTF-8. The response files of the MSVC tools begin with it (#693). +constexpr std::string_view kUtf8ByteOrderMark = "\xEF\xBB\xBF"; + std::string escape_ninja_path(const std::filesystem::path& p) { // Ninja escapes: $ → $$, : → $:, space → $ (with leading space). // For simplicity we wrap in case-by-case. @@ -977,6 +1008,62 @@ std::string graph_c_library_isolation_advice(std::string_view output, library, predicateAbi); } +std::string graph_link_library_advice(std::string_view output, + std::string_view cAbiName, + std::string_view cAbiCoordinate) { + if (output.find(kGraphLinkSysrootDir) == std::string_view::npos) return {}; + constexpr std::string_view kUnfound = "unable to find library -l"; + std::vector names; + for (std::size_t at = output.find(kUnfound); at != std::string_view::npos; + at = output.find(kUnfound, at + kUnfound.size())) { + auto begin = at + kUnfound.size(); + auto end = begin; + while (end < output.size() && !std::isspace(static_cast(output[end]))) + ++end; + std::string name(output.substr(begin, end - begin)); + if (!name.empty() && std::ranges::find(names, name) == names.end()) + names.push_back(std::move(name)); + } + if (names.empty()) return {}; + + std::string library = !cAbiName.empty() + ? (!cAbiCoordinate.empty() ? std::format("{} ({})", cAbiName, cAbiCoordinate) + : std::string(cAbiName)) + : std::string("this target's C library"); + std::string listed; + for (auto const& n : names) { + if (!listed.empty()) listed += ", "; + listed += "-l" + n; + } + // The names musl's own `make install` answers with empty archives, because + // libc holds everything they would hold. + static constexpr std::array kMuslSubsumed = { + "m", "rt", "pthread", "crypt", "util", "xnet", "resolv", "dl"}; + const bool allMusl = std::ranges::all_of(names, [](const std::string& n) { + return std::ranges::find(kMuslSubsumed, n) != kMuslSubsumed.end(); + }); + const bool muslOrUnknown = cAbiName.empty() || cAbiName == "musl"; + + std::string out = std::format( + "\n" + "note: {} comes from the dependency graph, and this link searches none of " + "the host's\n" + " library directories (mcpp#696). The graph supplies no library for: " + "{}\n" + " A library a package links has to reach the link through the graph, " + "as a dependency\n" + " of the package that names it.\n", + library, listed); + if (allMusl && muslOrUnknown) + out += " These are names musl answers from libc itself: openkal-musl " + "0.19.2 and later ship\n" + " the empty archives musl installs for them, so a graph with an " + "older openkal-musl\n" + " needs its pin moved (openkal-llvm-runtime 0.15.2 and later carry " + "it).\n"; + return out; +} + std::string filter_ninja_output(std::string_view output, std::span commandPrefixes) { std::string filtered; @@ -1396,6 +1483,17 @@ std::string emit_ninja_string(const BuildPlan& plan) { // the same over-approximation the link rules use (`separateLinker || // is_windows`). const bool useCompileRsp = mcpp::platform::is_windows || msvcDeps; + // THE BYTE ORDER MARK, FOR THE TOOLS THAT NEED ONE TO READ UTF-8 (#693). + // + // build.ninja is UTF-8, and Ninja copies `rspfile_content` into the + // response file byte for byte. cl.exe, link.exe and lib.exe read a response + // file as UTF-8 only when it begins with a byte order mark, and in the ANSI + // code page otherwise. Measured on windows-latest (code page 1252, MSVC + // 14.51): a UTF-8 file without the mark failed every non-ASCII case, with + // it every case passed, including the same file written by Ninja, and ASCII + // content passed both ways. The GNU drivers would read the mark as part of + // the first argument, so it is written for the msvc dialect only; the link + // rules below do the same for link.exe and lib.exe. // Both take the payload with its leading space, so callers read as // `command = $cxx{payload} ...` exactly like the inline form did. auto rsp_ref = [&](const std::string& payload) { @@ -1404,7 +1502,8 @@ std::string emit_ninja_string(const BuildPlan& plan) { auto append_rspfile = [&](const std::string& payload) { if (!useCompileRsp) return; append(" rspfile = $out.rsp\n"); - append(std::format(" rspfile_content ={}\n", payload)); + append(std::format(" rspfile_content ={}{}\n", + msvcDeps ? " " + std::string(kUtf8ByteOrderMark) : "", payload)); }; // Tell the driver, every time, that this TU is a module interface. @@ -1713,7 +1812,11 @@ std::string emit_ninja_string(const BuildPlan& plan) { // LLVM response-file parsing treat any whitespace as a // separator, newline included, and link.exe/lib.exe want // exactly this form. - append(" rspfile_content = $in_newline\n"); + // + // link.exe and lib.exe take the byte order mark that cl.exe + // takes above (#693), and for the same reason. + append(std::format(" rspfile_content = {}$in_newline\n", + separateLinker ? kUtf8ByteOrderMark : "")); } else { append(std::format(" command = {}\n", cmd)); } @@ -3276,6 +3379,29 @@ std::optional check_inline_command_lengths(const std::string& manif return std::nullopt; } +std::optional ninja_encoding_mismatch(std::string_view reported, + unsigned processCodePage, + std::string_view ninjaProgram) { + constexpr std::string_view kKey = "Build file encoding: "; + const auto at = reported.find(kKey); + if (at == std::string_view::npos) return std::nullopt; + auto theirs = reported.substr(at + kKey.size()); + theirs = theirs.substr(0, theirs.find_first_of("\r\n")); + if (theirs == "UTF-8") return std::nullopt; + return std::format( + "'{}' reads build.ninja as {} text, and mcpp writes it in UTF-8: a " + "non-ASCII path or argument in it would reach the tools as different " + "characters.\n {}", + ninjaProgram, theirs, + processCodePage == 65001 + ? "Use a Ninja 1.11 or later that declares the UTF-8 code page, as " + "the one mcpp installs does." + : std::format("This process runs in the ANSI code page {}: this " + "Windows ignores the UTF-8 code page that mcpp.exe and " + "Ninja declare (Windows 10 version 1903 and later honour " + "it), so a build here must be ASCII.", processCodePage)); +} + std::expected NinjaBackend::build(const BuildPlan& plan, const BuildOptions& opts) { auto t0 = std::chrono::steady_clock::now(); @@ -3394,6 +3520,27 @@ std::expected NinjaBackend::build(const BuildPlan& plan for (auto const& d : flags.diagnostics) mcpp::ui::warning(std::format("cxx_runtime: {}", d)); + // A declared C standard the compiler does not apply is said once, never + // dropped without a word (#695, W3b): cl.exe compiles C in its default + // mode, and mapping `c_standard` onto `/std:` waits for a measurement. + if (!plan.cStandardsNotApplied.empty()) { + std::string list; + const std::size_t shown = std::min(plan.cStandardsNotApplied.size(), 4); + for (std::size_t i = 0; i < shown; ++i) { + if (i) list += ", "; + list += plan.cStandardsNotApplied[i]; + } + if (plan.cStandardsNotApplied.size() > shown) + list += std::format(" and {} more", plan.cStandardsNotApplied.size() - shown); + mcpp::ui::warning(std::format( + "c_standard: cl.exe compiles C in its default mode and does not apply " + "the C standard {} package{} declare{}: {}", + plan.cStandardsNotApplied.size(), + plan.cStandardsNotApplied.size() == 1 ? "" : "s", + plan.cStandardsNotApplied.size() == 1 ? "s" : "", + list)); + } + // #336: the generated initializer-ordering TU. Written before ninja runs, // since the link edge lists its object as an input. if (flags.needsStreamInitShim) { @@ -3416,8 +3563,15 @@ std::expected NinjaBackend::build(const BuildPlan& plan // objects + dynamic linker inside the sandbox BEFORE running the build — // catches both the bare-CRT link failure (#195) and silent host-library // contamination, cached per flag-set. + // The empty graph sysroot is created here, where the build writes its + // files, because `compute_flags` runs twice per build and stays pure (#696). + if (flags.graphLinkIsolated) { + std::error_code sec; + std::filesystem::create_directories(graph_link_sysroot(plan.outputDir), sec); + } if (auto h = verify_hermetic_link(plan.toolchain, flags.ld, plan.outputDir, - plan.manifest.buildConfig.allowHostLibs); !h) { + plan.manifest.buildConfig.allowHostLibs, + flags.graphLinkIsolated, plan.packageRoots); !h) { return std::unexpected(BuildError{h.error(), {}}); } stage("hermetic-check"); @@ -3442,6 +3596,27 @@ std::expected NinjaBackend::build(const BuildPlan& plan std::string ninjaProgram = ninjaBin.empty() ? std::string("ninja") : ninjaBin.string(); + // THE BUILD FILE IN THE ENCODING NINJA READS IT IN (#693, M4). Ninja reads + // UTF-8 when it declares the UTF-8 code page and the host honours it, and + // the ANSI code page otherwise: a ninja.exe that declares none, or a host + // older than Windows 10 version 1903. Asked only when the file holds a + // non-ASCII byte, because ASCII is the same text in both encodings; a + // refusal here comes before the fast-path record is written, so the next + // run asks again. + if constexpr (mcpp::platform::is_windows) { + const bool nonAscii = std::ranges::any_of(manifest, [](char c) { + return static_cast(c) >= 0x80; + }); + if (nonAscii) { + auto probe = mcpp::platform::process::capture_stdout( + {ninjaProgram, "-t", "wincodepage"}); + if (auto bad = ninja_encoding_mismatch( + probe.output, mcpp::platform::windows::active_code_page(), + ninjaProgram)) + return std::unexpected(BuildError{*bad, ninja_path}); + } + } + // Record ninja binary for P0 fast-path cache. BuildResult r; r.compileCommands = cdb ? cdb->commandCount : 0; @@ -3679,6 +3854,9 @@ std::expected NinjaBackend::build(const BuildPlan& plan // and gets the degraded-but-still-correct form. diagnostics += graph_c_library_isolation_advice( out, plan.targetSide.cAbi.interfaceName, plan.targetSide.cAbi.impl); + // #696: an unanswered `-l` on a link with the empty graph sysroot. + diagnostics += graph_link_library_advice( + out, plan.targetSide.cAbi.interfaceName, plan.targetSide.cAbi.impl); // #690: read from the plan here and from the sidecar on the fast path. diagnostics += consumer_include_scope_advice(out, root_include_dirs_of(plan)); if (plan.targetSide.cAbiDecl) diff --git a/src/build/plan.cppm b/src/build/plan.cppm index cec7ecfeb..85d82d252 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -226,6 +226,16 @@ StaticPlacement place_static_packages( struct BuildPlan { mcpp::manifest::Manifest manifest; + // Packages whose declared `[build] c_standard` the compiler does not apply, + // spelt ` ()`, in the order their first C unit appears. + // Filled only for cl.exe (W3b of the #693-#696 record) and reported once by + // the backend, so a declared value is never dropped without a word. + std::vector cStandardsNotApplied; + // Every package root of the graph, the root's first. The hermetic check + // reads it on an isolated graph link: a package may add a package-relative + // `-L` (openkal-musl's empty archives), and a path dependency's root can + // be anywhere on the machine, not only under the store. + std::vector packageRoots; mcpp::toolchain::Toolchain toolchain; mcpp::toolchain::Fingerprint fingerprint; // Where the target's platform interface, C library and C++ runtime come @@ -1238,6 +1248,7 @@ make_plan(const mcpp::manifest::Manifest& manifest, plan.manifest = manifest; plan.toolchain = tc; plan.fingerprint = fp; + for (auto const& p : packages) plan.packageRoots.push_back(p.root); // The ROOT package's extension table. Only the synthesized entry main // needs it — every scanned unit arrives with its kind already set by the @@ -1735,6 +1746,53 @@ make_plan(const mcpp::manifest::Manifest& manifest, } } + // EVERY PACKAGE'S C UNITS COMPILE AT THAT PACKAGE'S OWN C STANDARD (#695). + // + // The file-level `$cflags` carries `kDefaultCStandard`. A package whose + // effective standard differs gets the dialect's spelling appended to its C + // units' own flags, which the `c_object` rule reads after `$cflags`, so the + // later `-std=` is the one the driver takes. The root is one of `packages`, + // so its value reaches its own units and no one else's. This is the + // mechanism the C++ layer's implementation standard uses just above. + // + // cl.exe is left as it is. `/std:c11` and `/std:c17` also switch on the + // conforming preprocessor, and no CI row builds the index with cl, so the + // mapping waits for a measurement (W3b in the #693-#696 record). What cl + // does not apply is recorded here and reported once by the backend, rather + // than dropped in silence. + std::map> cStandardFlag; + std::map> cStandardNotApplied; + { + const auto& cd = mcpp::toolchain::dialect_for(tc); + const bool clDialect = cd.id == "msvc"; + for (auto const& p : packages) { + const auto& declared = p.manifest.buildConfig.cStandard; + if (clDialect) { + if (!declared.empty()) + cStandardNotApplied[qualified_package_name(p.manifest)] = declared; + continue; + } + const auto own = mcpp::manifest::effective_c_standard(declared); + if (own == mcpp::manifest::kDefaultCStandard) continue; + cStandardFlag[qualified_package_name(p.manifest)] = + std::format("{}{}", cd.stdPrefix, own); + } + } + std::set> cStandardNotAppliedSeen; + // One application for every C unit, the scanned ones below and the entry + // `main` a target synthesizes further down: a C entry is a unit of its + // package like any other. + auto apply_c_standard = [&](CompileUnit& cu) { + if (cu.kind != mcpp::SourceKind::C) return; + if (auto it = cStandardFlag.find(cu.packageName); it != cStandardFlag.end()) + cu.packageCflags.push_back(it->second); + if (auto it = cStandardNotApplied.find(cu.packageName); + it != cStandardNotApplied.end() + && cStandardNotAppliedSeen.insert(it->first).second) + plan.cStandardsNotApplied.push_back( + std::format("{} ({})", it->first, it->second)); + }; + // 1. Compile units in topological order for (auto idx : topoOrder) { auto& u = graph.units[idx]; @@ -1768,6 +1826,7 @@ make_plan(const mcpp::manifest::Manifest& manifest, && cu.declaration == mcpp::modgraph::ModuleDeclaration::None) { cu.packageCxxflags.push_back(it->second); } + apply_c_standard(cu); plan.compileUnits.push_back(std::move(cu)); } @@ -2238,6 +2297,7 @@ make_plan(const mcpp::manifest::Manifest& manifest, // treatment of every scanned unit. mcpp::modgraph::normalize_include_flags(projectRoot, main_cu.packageCflags); mcpp::modgraph::normalize_include_flags(projectRoot, main_cu.packageCxxflags); + apply_c_standard(main_cu); // The entry is in the package scan only when a `sources` glob // matched it; otherwise it is scanned here. The unit is built as one diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index e020bb30d..e40927bfe 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -2353,6 +2353,21 @@ prepare_build(bool print_fingerprint, if (!root) { return std::unexpected("no mcpp.toml found in current directory or any parent"); } + // THE PROJECT'S PATH IS PART OF EVERY DOCUMENT A BUILD WRITES, and those + // documents are UTF-8 text (build.ninja, compile_commands.json). A + // directory whose path has no UTF-8 spelling used to fail the first build + // with `internal: unhandled exception: [json.exception.type_error.316]` + // (#693, measured on Linux with a Latin-1 name and on a Windows code page + // 1252 host with a name that page can spell). It is refused here, by name. + if (!mcpp::modgraph::try_narrow(*root)) { + return std::unexpected(std::format( + "the project directory '{}' has no UTF-8 spelling.\n" + " {}\n" + " Every file a build writes names this directory in UTF-8; " + "rename or move it.", + mcpp::modgraph::escaped_spelling(*root), + mcpp::modgraph::no_utf8_spelling_reason())); + } // NOTE: `workRoot` is deliberately NOT derived here. `root` is not final // yet — the workspace block below reassigns it to the selected member // (`root = memberDir`), and anchoring the write root to the pre-switch @@ -14356,7 +14371,32 @@ prepare_build(bool print_fingerprint, // (and it could not be used anyway: the conditional channel carries // BuildInputs only). // 4. Nothing to embed into (an archive-only package) → say so and stop. - if (m->resources.declared()) { + // + // The same pipeline carries the application manifest of `windows_code_page` + // (#693). A PE executable embeds one that makes its process ANSI code page + // UTF-8 when its target says `windows_code_page = "utf-8"`, or, with nothing + // said, when it is built as a host tool (D6): such a tool receives mcpp's + // UTF-8 paths on its command line. `legacy` opts out, and an ordinary target + // that says nothing embeds nothing (M6: the program's encoding is its own). + // + // The host-tool default yields to a manifest the package embeds itself + // through `[resources] files`: both would sit at ordinal 1, the package + // said nothing about code pages, and its own manifest is the one it ships. + // A DECLARED `utf-8` beside such a manifest is refused below instead. + const bool hostToolBuild = overrides.tool_depth > 0; + const bool ownManifest = hostToolBuild + && std::ranges::any_of(m->resources.files, [&](const auto& f) { + const auto abs = (f.is_absolute() ? f : (*root / f)).lexically_normal(); + return mcpp::build::resources::scan_rc(abs).declaresManifest; + }); + auto codePageOf = [&](const mcpp::manifest::Target& t) -> std::string_view { + if (!t.windowsCodePage.empty()) return t.windowsCodePage; + return (hostToolBuild && t.is_program() && !ownManifest) ? "utf-8" : "legacy"; + }; + const bool anyUtf8Image = std::ranges::any_of(m->targets, [&](const auto& t) { + return t.is_program() && codePageOf(t) == "utf-8"; + }); + if (m->resources.declared() || anyUtf8Image) { namespace rsrc = mcpp::build::resources; const auto& R = m->resources; @@ -14500,6 +14540,15 @@ prepare_build(bool print_fingerprint, "editing that file will not trigger a rebuild", "list it in [resources] extra-inputs = [...]"); } + if (scan.declaresManifest && anyUtf8Image) + return std::unexpected(std::format( + "[resources] {} embeds an application manifest, and " + "`windows_code_page = \"utf-8\"` embeds another at the same " + "ordinal (1).\n Keep one: add `" + "UTF-8` to your manifest and set " + "`windows_code_page = \"legacy\"`, or drop your manifest.", + rcSrc.filename().generic_string())); auto inputs = std::move(scan.inputs); inputs.insert(inputs.end(), extraInputs.begin(), extraInputs.end()); if (auto a = add_unit(rcSrc, rcSrc.stem().string(), @@ -14509,14 +14558,23 @@ prepare_build(bool print_fingerprint, } // The synthesised script: per image, because OriginalFilename and - // the version block belong to a specific artifact. - if (!iconAbs.empty() || R.synthesize_version_info()) { + // the version block belong to a specific artifact, and the + // manifest to a specific executable. + const bool synthVersion = R.declared() && R.synthesize_version_info(); + auto wantsUtf8 = [&](const mcpp::build::LinkUnit& lu) { + if (lu.kind != mcpp::build::LinkUnit::Binary) return false; + for (auto const& t : m->targets) + if (t.name == lu.targetName) + return t.is_program() && codePageOf(t) == "utf-8"; + return false; + }; + if (!iconAbs.empty() || synthVersion || anyUtf8Image) { // A version key mcpp cannot order (an upstream build number) // leaves FILEVERSION's four numeric fields at zero while the // string fields keep the real text. Say so — the properties // dialog will disagree with `[package].version` and nothing // else would explain why. - if (R.synthesize_version_info() && !m->package.version.empty() + if (synthVersion && !m->package.version.empty() && !mcpp::version_req::parse_version(m->package.version)) { mcpp::diag::degraded("resources/version", std::format("[package].version = \"{}\" has no numeric " @@ -14528,8 +14586,31 @@ prepare_build(bool print_fingerprint, } for (auto i : peUnits) { const auto& lu = ctx.plan.linkUnits[i]; + const bool utf8 = wantsUtf8(lu); + if (iconAbs.empty() && !synthVersion && !utf8) continue; + std::filesystem::path manifestAbs; + if (utf8) { + manifestAbs = resDir / (lu.targetName + ".mcpp.manifest"); + const auto manifestText = rsrc::utf8_code_page_manifest(); + std::string had; + if (std::ifstream in(manifestAbs, std::ios::binary); in) + had.assign(std::istreambuf_iterator(in), {}); + if (had != manifestText) { + std::ofstream os(manifestAbs, std::ios::binary); + if (!os) return std::unexpected(std::format( + "cannot write the application manifest '{}'", + manifestAbs.string())); + os << manifestText; + } + } + // A script synthesised for the manifest alone carries + // nothing else: a package that declares no [resources] + // asked for no version resource. + mcpp::manifest::Resources forScript = R; + if (!synthVersion) forScript.versionInfo = false; auto text = rsrc::synthesize_rc( - m->package, R, lu.output.filename().string(), iconAbs); + m->package, forScript, lu.output.filename().string(), + iconAbs, manifestAbs); if (!text) return std::unexpected(text.error()); // A stable path, so `cp` + `files = [...]` reproduces the // same resource byte for byte (the L0→L1 escape hatch). @@ -14548,6 +14629,7 @@ prepare_build(bool print_fingerprint, } std::vector inputs; if (!iconAbs.empty()) inputs.push_back(iconAbs); + if (!manifestAbs.empty()) inputs.push_back(manifestAbs); inputs.insert(inputs.end(), extraInputs.begin(), extraInputs.end()); if (auto a = add_unit(rcPath, lu.targetName + ".mcpp", std::move(inputs), i); !a) diff --git a/src/build/resources.cppm b/src/build/resources.cppm index 5bc8a1c79..9546eac56 100644 --- a/src/build/resources.cppm +++ b/src/build/resources.cppm @@ -49,6 +49,7 @@ import mcpp.manifest; import mcpp.toolchain.detect; import mcpp.toolchain.triple; import mcpp.version_req; +import mcpp.platform.process; // capture_exec: the build program's manifest resource export namespace mcpp::build::resources { @@ -110,6 +111,10 @@ struct ScanResult { // nothing that could. See the module header. bool versionInfoNamedByString = false; std::string versionInfoName; + // The script embeds an application manifest of its own: a statement whose + // type is `24`, `RT_MANIFEST` or `MANIFEST`. A second one from + // `windows_code_page` would collide with it at ordinal 1 (#693). + bool declaresManifest = false; }; ScanResult scan_rc(const std::filesystem::path& rc); @@ -124,11 +129,34 @@ ScanResult scan_rc(const std::filesystem::path& rc); // package version cannot be expressed as FILEVERSION's four 16-bit fields — // clamping silently would put a version in the binary that is not the version // that was built. +// +// `manifestAbs`, when not empty, is an application manifest embedded as +// `RT_MANIFEST` at ordinal 1 (#693). A VERSIONINFO is synthesized when +// `res.synthesize_version_info()`; a caller writing a script for the manifest +// alone passes `versionInfo = false`, so that script carries nothing else. std::expected synthesize_rc(const mcpp::manifest::Package& pkg, const mcpp::manifest::Resources& res, std::string_view outputFileName, - const std::filesystem::path& iconAbs); + const std::filesystem::path& iconAbs, + const std::filesystem::path& manifestAbs = {}); + +// The application manifest that makes a process's ANSI code page UTF-8 on +// Windows 10 1903 and later (#693), as Microsoft documents it. ASCII text; the +// same bytes for every image that carries it. +std::string utf8_code_page_manifest(); + +// A compiled resource holding `utf8_code_page_manifest()`, for an executable +// the engine links outside the ninja graph: the build program (#693, D4). +// Written into `dir` as `.manifest`, `.rc` and `.res` +// (rc.exe, llvm-rc) or `.o` (windres), reused while the script is +// unchanged, and returned for the link. Fails, naming the reason, when `tc` has +// no resource compiler beside it or the compiler refuses the script. +std::expected +compile_utf8_manifest(const mcpp::toolchain::Toolchain& tc, + std::string_view dialectId, + const std::filesystem::path& dir, + std::string_view stem); } // namespace mcpp::build::resources @@ -244,6 +272,69 @@ std::optional find_rc_tool(const mcpp::toolchain::Toolchain& tc, return std::nullopt; } +std::string utf8_code_page_manifest() { + return + "\n" + "\n" + " \n" + " \n" + " " + "UTF-8\n" + " \n" + " \n" + "\n"; +} + +std::expected +compile_utf8_manifest(const mcpp::toolchain::Toolchain& tc, + std::string_view dialectId, + const std::filesystem::path& dir, + std::string_view stem) { + auto tool = find_rc_tool(tc, dialectId); + if (!tool) + return std::unexpected(std::format( + "no Windows resource compiler beside {}", tc.binaryPath.string())); + std::error_code ec; + std::filesystem::create_directories(dir, ec); + const bool msvcStyle = tool->style == "msvc"; + const auto manifest = dir / (std::string(stem) + ".manifest"); + const auto script = dir / (std::string(stem) + ".rc"); + const auto out = dir / (std::string(stem) + (msvcStyle ? ".res" : ".o")); + + auto write_if_changed = [](const std::filesystem::path& p, const std::string& text) { + std::string had; + if (std::ifstream in(p, std::ios::binary); in) + had.assign(std::istreambuf_iterator(in), {}); + if (had == text) return false; + std::ofstream os(p, std::ios::binary); + os << text; + return true; + }; + bool changed = write_if_changed(manifest, utf8_code_page_manifest()); + changed = write_if_changed(script, std::format( + "1 24 \"{}\"\n", escape_rc_string(manifest.generic_string()))) || changed; + if (!changed && std::filesystem::is_regular_file(out, ec)) return out; + + // The same spelling as the `rc_object` rule of the ninja backend. + std::vector argv = {tool->path.string()}; + if (msvcStyle) + for (auto a : {"/nologo", "/C", "65001", "/fo"}) argv.emplace_back(a); + else + for (auto a : {"-O", "coff", "--codepage=65001", "-o"}) argv.emplace_back(a); + argv.push_back(out.string()); + argv.push_back(script.string()); + // rc.exe resolves through the SDK PATH the toolchain states for itself. + std::vector> env; + for (auto const& ev : tc.envOverrides) env.emplace_back(ev.key, ev.value); + auto r = mcpp::platform::process::capture_exec(argv, env, dir.string()); + if (r.exit_code != 0) { + std::filesystem::remove(out, ec); + return std::unexpected(std::format("{} failed (exit {}): {}", + tool->name(), r.exit_code, r.output)); + } + return out; +} + ScanResult scan_rc(const std::filesystem::path& rc) { ScanResult out; std::ifstream is(rc, std::ios::binary); @@ -303,12 +394,30 @@ ScanResult scan_rc(const std::filesystem::path& rc) { // ` ` — find the type keyword by scanning words. std::size_t i = 0; + std::size_t wordIndex = 0; std::string_view prevWord; while (i < line.size()) { if (!(std::isalnum(static_cast(line[i])) || line[i] == '_')) { ++i; continue; } auto w = word_at(line, i); if (w.empty()) { ++i; continue; } + // An application manifest: ` 24 ""`, the TYPE position of + // a statement holding `24` (RT_MANIFEST) or its name, followed by a + // file name. Both conditions count: the numbers of a VERSIONINFO + // block contain `24` as well (`FILEVERSION 24,1,0,0` puts it at the + // same position), and no file name follows them. The numeric form + // is no keyword, so its file is tracked here. + if (wordIndex == 1 && (w == "24" || w == "RT_MANIFEST" || w == "MANIFEST")) { + auto rest = line.substr(i + w.size()); + while (!rest.empty() && (rest.front() == ' ' || rest.front() == '\t')) + rest.remove_prefix(1); + if (rest.starts_with('"')) { + out.declaresManifest = true; + auto e = rest.find('"', 1); + if (e != std::string_view::npos) add_input(rest.substr(1, e - 1)); + } + } + if (w == "VERSIONINFO" && !prevWord.empty()) { // The mcpp#365 shape: an identifier name that is not `1`. const bool numeric = std::all_of(prevWord.begin(), prevWord.end(), @@ -335,6 +444,7 @@ ScanResult scan_rc(const std::filesystem::path& rc) { } prevWord = w; i += w.size(); + ++wordIndex; } } return out; @@ -344,7 +454,8 @@ std::expected synthesize_rc(const mcpp::manifest::Package& pkg, const mcpp::manifest::Resources& res, std::string_view outputFileName, - const std::filesystem::path& iconAbs) { + const std::filesystem::path& iconAbs, + const std::filesystem::path& manifestAbs) { std::string out; // ASCII throughout, including this banner: see the LegalCopyright note // below for why generated text must not lean on the codepage flag. @@ -353,6 +464,13 @@ synthesize_rc(const mcpp::manifest::Package& pkg, out += "// [resources] files = [...] to take it over; the result is\n"; out += "// byte-identical.\n\n"; + if (!manifestAbs.empty()) { + // CREATEPROCESS_MANIFEST_RESOURCE_ID (1) of type RT_MANIFEST (24), + // spelt numerically so the script needs no . + out += std::format("1 24 \"{}\"\n\n", + escape_rc_string(manifestAbs.generic_string())); + } + if (!iconAbs.empty()) { // Ordinal 1: Explorer and the shell show the LOWEST-numbered icon group. out += std::format("1 ICON \"{}\"\n\n", diff --git a/src/cli.cppm b/src/cli.cppm index a84982a61..16d0ed29c 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -101,8 +101,9 @@ void print_usage() { std::println("Docs: https://github.com/mcpp-community/mcpp/tree/main/docs"); } -// The ONE place this run's "could not be named in the active code page" -// records are reported. +// The ONE place this run's "has no UTF-8 spelling" records are reported: a +// name that is not UTF-8 on POSIX, or one the process code page cannot spell +// on a Windows host that ignores the UTF-8 code page mcpp.exe declares (#693). // // `src/modgraph/` and `src/manifest/` are leaf layers — not one module in // either imports `mcpp.ui` or `mcpp.diag` — so the glob walk RECORDS @@ -130,13 +131,12 @@ struct ReportUnnarrowablePaths { for (auto const& anchor : mcpp::modgraph::take_unnarrowable_paths()) { mcpp::diag::degraded( "path/codepage", - std::format("'{}' contains names this system's active code " - "page cannot represent", anchor), + std::format("'{}' contains names that have no UTF-8 spelling", + anchor), "those files take no part in the build", - "Windows only: this is the process ANSI code page, which " - "`chcp` does not change. Harmless when the names are test " - "data or docs; if they are sources, rename them or build on a " - "system whose code page covers them."); + std::format("{} Harmless when the names are test data or " + "documentation; sources need renaming.", + mcpp::modgraph::no_utf8_spelling_reason())); } } catch (...) { // Losing the report is bad; terminating instead of it is worse. diff --git a/src/config.cppm b/src/config.cppm index 7d370d757..f3e7d878c 100644 --- a/src/config.cppm +++ b/src/config.cppm @@ -22,6 +22,7 @@ export module mcpp.config; import std; import mcpp.home; +import mcpp.modgraph.glob; // try_narrow: the one UTF-8 spelling of a path import mcpp.libs.toml; import mcpp.libs.json; import mcpp.pm.index_spec; @@ -659,6 +660,21 @@ std::expected load_or_init( cfg.logDir = cfg.mcppHome / "log"; cfg.configFile = cfg.mcppHome / "config.toml"; + // 1b. The home is part of every toolchain path a build writes into + // build.ninja and compile_commands.json, which are UTF-8 text. A home with + // no UTF-8 spelling -- a Windows account name the ANSI code page of an + // older host renders in its own bytes, or a POSIX name that is not UTF-8 -- + // let toolchains install and then failed every build with an internal JSON + // exception (#693, F-693f). It is refused before anything is written there. + if (!mcpp::modgraph::try_narrow(cfg.mcppHome)) { + return std::unexpected(ConfigError{std::format( + "the mcpp home '{}' has no UTF-8 spelling.\n" + " {}\n" + " Set MCPP_HOME to a directory whose path has one.", + mcpp::modgraph::escaped_spelling(cfg.mcppHome), + mcpp::modgraph::no_utf8_spelling_reason())}); + } + // 2. Create directory tree std::error_code ec; for (auto& d : { cfg.binDir, cfg.registryDir, cfg.bmiCacheDir, diff --git a/tests/e2e/190_link_rspfile_newlines.sh b/tests/e2e/190_link_rspfile_newlines.sh index ecee7cc39..821a63b4d 100755 --- a/tests/e2e/190_link_rspfile_newlines.sh +++ b/tests/e2e/190_link_rspfile_newlines.sh @@ -49,13 +49,16 @@ cd multi ninja_file=$(find target -name build.ninja | head -1) [ -n "$ninja_file" ] || { echo "FAIL: no build.ninja"; exit 1; } -# 1. Structural: no link rule may write its response file on one line. -if grep -qE '^[[:space:]]*rspfile_content = \$in[[:space:]]*$' "$ninja_file"; then +# 1. Structural: no link rule may write its response file on one line. Under +# the msvc dialect the content begins with a UTF-8 byte order mark, which +# link.exe and lib.exe need to read the file as UTF-8 (#693). +BOM=$'\xef\xbb\xbf' +if grep -qE "^[[:space:]]*rspfile_content = (${BOM})?\\\$in[[:space:]]*\$" "$ninja_file"; then grep -nE '^[[:space:]]*rspfile_content' "$ninja_file" echo "FAIL: a link rule still writes its response file on ONE line (\$in)" exit 1 fi -grep -qE '^[[:space:]]*rspfile_content = \$in_newline[[:space:]]*$' "$ninja_file" || { +grep -qE "^[[:space:]]*rspfile_content = (${BOM})?\\\$in_newline[[:space:]]*\$" "$ninja_file" || { grep -nE '^[[:space:]]*rspfile_content' "$ninja_file" echo "FAIL: no link rule uses \$in_newline"; exit 1; } echo " ok: link rules declare rspfile_content = \$in_newline" diff --git a/tests/e2e/776_a_path_with_no_utf8_spelling_is_named.sh b/tests/e2e/776_a_path_with_no_utf8_spelling_is_named.sh new file mode 100755 index 000000000..26737c3b8 --- /dev/null +++ b/tests/e2e/776_a_path_with_no_utf8_spelling_is_named.sh @@ -0,0 +1,107 @@ +#!/usr/bin/env bash +# requires: elf gcc +# mcpp#693, the POSIX half. build.ninja and compile_commands.json are UTF-8 +# text, and a Linux file name is bytes that need not be UTF-8. Before the fix a +# project directory named in Latin-1 failed every build with +# +# error: internal: unhandled exception: [json.exception.type_error.316] +# invalid UTF-8 byte at index 123: 0x2F +# +# Four entry points, four answers: +# 1. a project directory with no UTF-8 spelling is refused before anything is +# built, and the refusal names it with escapes; +# 2. a file with no UTF-8 spelling inside a project is skipped and reported by +# its nearest nameable ancestor, and the rest of the project builds; +# 3. a build.mcpp directive whose text is not UTF-8 is refused by its key; +# 4. an MCPP_HOME with no UTF-8 spelling is refused before anything is written +# into it. +# `elf` stands for Linux here: macOS refuses such a name at creation. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +BAD=$(printf 'caf\xe9') # Latin-1 for U+00E9; not UTF-8 + +fail() { [ -n "$2" ] && cat "$2"; echo "FAIL: $1"; exit 1; } + +# ── 1. the project directory ──────────────────────────────────────────────── +mkdir -p "$TMP/$BAD/src" +cat > "$TMP/$BAD/mcpp.toml" <<'EOF' +[package] +name = "latin1dir" +version = "0.1.0" +EOF +printf 'int main() { return 0; }\n' > "$TMP/$BAD/src/main.cpp" +cd "$TMP/$BAD" +if "$MCPP" build > "$TMP/b1.log" 2>&1; then + fail "a project directory with no UTF-8 spelling built" "$TMP/b1.log" +fi +grep -q 'internal: unhandled exception' "$TMP/b1.log" \ + && fail "the directory still reaches a JSON writer" "$TMP/b1.log" +grep -q 'has no UTF-8 spelling' "$TMP/b1.log" \ + || fail "the refusal does not say what is wrong" "$TMP/b1.log" +grep -qF 'caf\xE9' "$TMP/b1.log" \ + || fail "the refusal does not name the directory" "$TMP/b1.log" +echo " ok: the project directory is refused, by name" + +# ── 2. a file inside an ordinary project ──────────────────────────────────── +mkdir -p "$TMP/ok/src" +cat > "$TMP/ok/mcpp.toml" <<'EOF' +[package] +name = "latin1file" +version = "0.1.0" +EOF +printf 'int main() { return 0; }\n' > "$TMP/ok/src/main.cpp" +# Matched by the default source glob, and never compiled: it defines main too. +printf 'int main() { return 7; }\n' > "$TMP/ok/src/$BAD.cpp" +cd "$TMP/ok" +"$MCPP" build > "$TMP/b2.log" 2>&1 \ + || fail "a project holding such a name must still build" "$TMP/b2.log" +grep -q 'contains names that have no UTF-8 spelling' "$TMP/b2.log" \ + || fail "the skipped file is not reported" "$TMP/b2.log" +grep -qF "'$TMP/ok/src'" "$TMP/b2.log" \ + || fail "the report does not name the nearest nameable directory" "$TMP/b2.log" +"$MCPP" run > "$TMP/r2.log" 2>&1 || fail "the program did not run" "$TMP/r2.log" +cdb=$(find "$TMP/ok" -name compile_commands.json | head -1) +[ -n "$cdb" ] || fail "no compile_commands.json was written" "$TMP/b2.log" +grep -q 'main.cpp' "$cdb" || fail "the compile database lost the ordinary source" "$cdb" +echo " ok: the file is skipped and reported, and the rest builds and runs" + +# ── 3. a build.mcpp directive ─────────────────────────────────────────────── +mkdir -p "$TMP/bp/src" +cat > "$TMP/bp/mcpp.toml" <<'EOF' +[package] +name = "latin1directive" +version = "0.1.0" +EOF +printf 'int main() { return 0; }\n' > "$TMP/bp/src/main.cpp" +cat > "$TMP/bp/build.mcpp" <<'EOF' +#include +int main() { + std::printf("mcpp:cfg=FROM_BUILD_PROGRAM\n"); + std::printf("mcpp:include-dir=inc/caf\xE9\n"); + return 0; +} +EOF +cd "$TMP/bp" +if "$MCPP" build > "$TMP/b3.log" 2>&1; then + fail "a directive with no UTF-8 spelling was applied" "$TMP/b3.log" +fi +grep -q 'internal: unhandled exception' "$TMP/b3.log" \ + && fail "the directive still reaches a JSON writer" "$TMP/b3.log" +grep -q 'whose text is not UTF-8: mcpp:include-dir' "$TMP/b3.log" \ + || fail "the refusal does not name the directive" "$TMP/b3.log" +echo " ok: the directive is refused by its key" + +# ── 4. the mcpp home ──────────────────────────────────────────────────────── +cd "$TMP/ok" +if MCPP_HOME="$TMP/home-$BAD" "$MCPP" build > "$TMP/b4.log" 2>&1; then + fail "an MCPP_HOME with no UTF-8 spelling was used" "$TMP/b4.log" +fi +grep -q 'the mcpp home .* has no UTF-8 spelling' "$TMP/b4.log" \ + || fail "the refusal does not name the home" "$TMP/b4.log" +[ ! -e "$TMP/home-$BAD/registry" ] \ + || fail "the refused home was written into" "$TMP/b4.log" +echo " ok: the home is refused before anything is written into it" + +echo "PASS: every path with no UTF-8 spelling is named where it enters" diff --git a/tests/e2e/777_c_standard_applies_to_the_package_that_declares_it.sh b/tests/e2e/777_c_standard_applies_to_the_package_that_declares_it.sh new file mode 100755 index 000000000..7285d1398 --- /dev/null +++ b/tests/e2e/777_c_standard_applies_to_the_package_that_declares_it.sh @@ -0,0 +1,98 @@ +#!/usr/bin/env bash +# requires: gcc +# mcpp#695: `[build] c_standard` applies to the C units of the package that +# declares it, and a consumer's value never reaches a dependency. +# +# Before the fix the root's value was on the file-level `$cflags` line of every +# C compile in the graph, so a dependency's own declaration was parsed, hashed +# into its cache key and never applied. Each C file below states, in the +# preprocessor, the standard its package declares; a unit compiled at any other +# standard fails with `#error` and names what it received. +# +# app declares c99 its C unit must see __STDC_VERSION__ 199901L +# cdep declares gnu11 201112L, and no __STRICT_ANSI__ (a GNU dialect) +# plain declares none 201112L with __STRICT_ANSI__: the default, c11 +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +fail() { [ -n "$2" ] && cat "$2"; echo "FAIL: $1"; exit 1; } + +mkdir -p app/src cdep/src plain/src + +cat > cdep/mcpp.toml <<'EOF' +[package] +name = "cdep" +version = "0.1.0" + +[build] +c_standard = "gnu11" + +[targets.cdep] +kind = "lib" +EOF +cat > cdep/src/cdep.c <<'EOF' +#if __STDC_VERSION__ != 201112L +#error "cdep declares gnu11 and was compiled at another standard" +#endif +#ifdef __STRICT_ANSI__ +#error "cdep declares gnu11 and was compiled at a strict standard" +#endif +int cdep_value(void) { return 11; } +EOF + +cat > plain/mcpp.toml <<'EOF' +[package] +name = "plain" +version = "0.1.0" + +[targets.plain] +kind = "lib" +EOF +cat > plain/src/plain.c <<'EOF' +#if __STDC_VERSION__ != 201112L || !defined(__STRICT_ANSI__) +#error "plain declares no C standard and was not compiled at the default, c11" +#endif +int plain_value(void) { return 1; } +EOF + +cat > app/mcpp.toml <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[build] +c_standard = "c99" + +[dependencies] +cdep = { path = "../cdep" } +plain = { path = "../plain" } +EOF +cat > app/src/root.c <<'EOF' +#if __STDC_VERSION__ != 199901L +#error "app declares c99 and was compiled at another standard" +#endif +int root_value(void) { return 99; } +EOF +cat > app/src/main.cpp <<'EOF' +extern "C" int cdep_value(); +extern "C" int plain_value(); +extern "C" int root_value(); +int main() { return cdep_value() == 11 && plain_value() == 1 && root_value() == 99 ? 0 : 1; } +EOF + +cd app +"$MCPP" build > b.log 2>&1 || fail "each package's C units must compile at its own standard" b.log +"$MCPP" run > r.log 2>&1 || fail "the program did not run" r.log +echo " ok: app at c99, cdep at gnu11, plain at c11" + +# The compile database records the same per-unit standard the build used. +cdb=$(find . -name compile_commands.json | head -1) +[ -n "$cdb" ] || fail "no compile_commands.json" b.log +grep -q -- '-std=gnu11' "$cdb" || fail "the database does not carry cdep's standard" "$cdb" +grep -q -- '-std=c99' "$cdb" || fail "the database does not carry app's standard" "$cdb" +echo " ok: compile_commands.json carries the same standards" + +echo "PASS: c_standard applies to the package that declares it" diff --git a/tests/e2e/778_a_graph_link_searches_no_host_directory.sh b/tests/e2e/778_a_graph_link_searches_no_host_directory.sh new file mode 100644 index 000000000..8c172fba3 --- /dev/null +++ b/tests/e2e/778_a_graph_link_searches_no_host_directory.sh @@ -0,0 +1,96 @@ +#!/usr/bin/env bash +# requires: llvm elf unix-shell +# mcpp#696: a link whose C library comes from the dependency graph searches no +# library directory of the host. +# +# Before the fix clang derived `/usr/lib/x86_64-linux-gnu` and its siblings for +# a `x86_64-linux-musl` link over openkal-musl, so `-lm` was answered by the +# HOST's glibc archive and its objects were linked into a musl image, with no +# diagnostic. The graph link now carries `--sysroot` naming an empty directory +# in the build directory, which removes every such directory. +# +# Two legs, and the second is the one that separates the engines: +# +# A. openkal-musl 0.19.2 ships musl's eight empty archives (`libm.a` among +# them). `-lm` is answered by the graph, the link carries the empty +# sysroot, and the program runs. +# B. openkal-musl 0.19.1 has no `libm.a`. With the host's directories gone, +# `-lm` is unanswered: the build fails with the linker's own message and +# mcpp's note naming the release that answers it. The released engine +# links this leg successfully, from the host's libm. +set -e + +MCPP="${MCPP:-mcpp}" +work="$(mktemp -d)" +trap 'rm -rf "$work"' EXIT +TARGET=x86_64-linux-musl + +fail() { [ -n "$2" ] && cat "$2"; echo "FAIL: $1"; exit 1; } + +make_project() { + local dir="$1" musl="$2" + mkdir -p "$dir/src" + cat > "$dir/mcpp.toml" < "$dir/src/main.c" <<'C' +#include +#include +int main(void) { + volatile double a = 1.0, b = 2.0; + printf("%g\n", fmax(a, b)); + return fmax(a, b) == 2.0 ? 0 : 1; +} +C +} + +skip_if_unreachable() { + if grep -qE 'not found in the synced index|install_packages failed' "$1"; then + echo "SKIP: the openkal packages are not reachable from here" + exit 0 + fi +} + +# ── A. the graph answers -lm ──────────────────────────────────────────────── +make_project "$work/a" "0.19.2" +cd "$work/a" +if ! "$MCPP" build --target "$TARGET" --verbose > a.log 2>&1; then + skip_if_unreachable a.log + fail "with openkal-musl 0.19.2, -lm must be answered by the graph" a.log +fi +grep -q -- '--sysroot=[^ ]*graph-sysroot' a.log \ + || fail "the graph link does not carry the empty sysroot" a.log +out=$("$MCPP" run --target "$TARGET" 2>&1) || fail "the program did not run: $out" +printf '%s\n' "$out" | grep -qx '2' || fail "unexpected output: $out" +echo " ok: -lm is answered by openkal-musl's own archive, and the program runs" + +# ── B. nothing in the graph answers -lm ───────────────────────────────────── +make_project "$work/b" "0.19.1" +cd "$work/b" +if "$MCPP" build --target "$TARGET" > b.log 2>&1; then + fail "-lm was answered although nothing in the graph provides it: the host's library directories are still searched" b.log +fi +skip_if_unreachable b.log +grep -q 'unable to find library -lm' b.log \ + || fail "the failure is not the unanswered -lm" b.log +grep -q 'openkal-musl 0.19.2' b.log \ + || fail "the note does not name the release that answers -lm" b.log +echo " ok: an unanswered -lm fails, and the note names openkal-musl 0.19.2" + +echo "PASS: a graph link searches no host directory" diff --git a/tests/unit/test_build_directives.cpp b/tests/unit/test_build_directives.cpp index ac837f848..f1c2a04ec 100644 --- a/tests/unit/test_build_directives.cpp +++ b/tests/unit/test_build_directives.cpp @@ -195,6 +195,31 @@ TEST(BuildDirectives, UnknownKeysAreDeduplicated) { EXPECT_EQ(d.unknownKeys, (std::vector{"zzz", "yyy"})); } +// #693: a directive whose text is not UTF-8 names a different file once it is +// in build.ninja, so it is refused by key, whatever protocol the program +// speaks, and nothing of it is applied. +TEST(BuildDirectives, ADirectiveThatIsNotUtf8IsRefusedByKey) { + // "caf" + 0xE9: Latin-1, the shape a narrow program in a code page 1252 + // process prints. + auto d = parse("mcpp:include-dir=inc/caf\xE9\nmcpp:cxxflag=-Wall\n"); + EXPECT_TRUE(d.at(dirs::Slot::IncludeDirs).empty()); + EXPECT_EQ(d.at(dirs::Slot::CxxFlags), (std::vector{"-Wall"})); + EXPECT_EQ(d.nonUtf8Keys, (std::vector{"mcpp:include-dir"})); + auto err = dirs::encoding_error(d); + ASSERT_TRUE(err.has_value()); + EXPECT_NE(err->find("mcpp:include-dir"), std::string::npos) << *err; + EXPECT_NE(err->find("UTF-8"), std::string::npos) << *err; +} + +TEST(BuildDirectives, UtfEightDirectivesAndOtherChatterPass) { + // U+00E9 in UTF-8 is accepted; a line that is not a directive is not + // examined at all. + auto d = parse("mcpp:include-dir=inc/caf\xC3\xA9\nnoise \xE9\n"); + EXPECT_EQ(d.at(dirs::Slot::IncludeDirs).size(), 1u); + EXPECT_TRUE(d.nonUtf8Keys.empty()); + EXPECT_FALSE(dirs::encoding_error(d).has_value()); +} + // ── Cache round-trip ─────────────────────────────────────────────────────── TEST(BuildDirectives, SerializeDeserializeRoundTrip) { diff --git a/tests/unit/test_build_resources.cpp b/tests/unit/test_build_resources.cpp index 38ca9bf47..c4dd78958 100644 --- a/tests/unit/test_build_resources.cpp +++ b/tests/unit/test_build_resources.cpp @@ -170,11 +170,46 @@ END EXPECT_TRUE(has("ids.h")); EXPECT_TRUE(has("app.ico")); EXPECT_TRUE(has("blob.bin")); + // An application manifest is a file like the icon (#693): type 24 at the + // TYPE position, spelt numerically, which no keyword names. + EXPECT_TRUE(has("app.manifest")); + EXPECT_TRUE(s.declaresManifest); // Angled includes belong to the toolchain: immutable for the life of a // build directory and already folded into the fingerprint. EXPECT_FALSE(has("windows.h")); // STRINGTABLE carries its data inline — nothing to track. - EXPECT_EQ(s.inputs.size(), 3u); + EXPECT_EQ(s.inputs.size(), 4u); +} + +// The numbers of a VERSIONINFO block contain 24 as well; only the TYPE +// position of a statement declares a manifest. +TEST(BuildResources, ATwentyFourOutsideTheTypePositionIsNotAManifest) { + TempDir d; + auto rc = d.write("app.rc", R"(1 VERSIONINFO + FILEVERSION 24,1,0,0 + PRODUCTVERSION 1,24,0,0 +BEGIN +END +)"); + auto s = res::scan_rc(rc); + EXPECT_FALSE(s.declaresManifest); + EXPECT_TRUE(s.inputs.empty()); +} + +// The manifest `windows_code_page = "utf-8"` embeds sits at ordinal 1 of type +// 24, and nothing else is added to a script synthesised for it alone. +TEST(BuildResources, AManifestOnlyScriptCarriesTheManifestAndNothingElse) { + mcpp::manifest::Resources r; + r.versionInfo = false; + auto rc = res::synthesize_rc(sample_package(), r, "tool.exe", {}, + fs::path("/b/res/tool.mcpp.manifest")); + ASSERT_TRUE(rc) << rc.error(); + EXPECT_NE(rc->find("1 24 \"/b/res/tool.mcpp.manifest\""), std::string::npos) << *rc; + EXPECT_EQ(rc->find("VERSIONINFO"), std::string::npos) << *rc; + EXPECT_EQ(rc->find("ICON"), std::string::npos) << *rc; + EXPECT_NE(res::utf8_code_page_manifest().find( + "UTF-8"), + std::string::npos); } TEST(BuildResources, ScanNamesWhatItCouldNotResolve) { diff --git a/tests/unit/test_c_standard_per_package.cpp b/tests/unit/test_c_standard_per_package.cpp new file mode 100644 index 000000000..a8bc1f41e --- /dev/null +++ b/tests/unit/test_c_standard_per_package.cpp @@ -0,0 +1,243 @@ +// #695 -- every package's C units compile at that package's own C standard. +// +// `[build] c_standard` used to take effect only on the package being built: the +// file-level `$cflags` carried the root's value, so a dependency's C units +// compiled at the consumer's standard while its own declaration was parsed, +// hashed into its cache key and never applied. The file-level line now carries +// `kDefaultCStandard`, and `make_plan` appends a package's own standard to that +// package's C units when it differs. These assertions are on the plan's +// per-unit C flags, which the compile edge, the scan edge and both build +// databases read after the file-level flags. + +#include + +import std; +import mcpp.build.plan; +import mcpp.manifest; +import mcpp.modgraph.graph; +import mcpp.modgraph.scanner; +import mcpp.source_kind; +import mcpp.toolchain.model; + +using namespace mcpp::build; + +namespace { + +struct Tmp { + std::filesystem::path path; + Tmp() { + path = std::filesystem::temp_directory_path() + / std::format("mcpp_c_standard_{}", std::random_device{}()); + std::filesystem::create_directories(path); + } + ~Tmp() { + std::error_code ec; + std::filesystem::remove_all(path, ec); + } +}; + +mcpp::toolchain::Toolchain clang_like() { + mcpp::toolchain::Toolchain tc; + tc.compiler = mcpp::toolchain::CompilerId::Clang; + tc.version = "22.1.8"; + tc.binaryPath = "/usr/bin/clang++"; + tc.targetTriple = "x86_64-linux-gnu"; + return tc; +} + +mcpp::toolchain::Toolchain cl_like() { + mcpp::toolchain::Toolchain tc; + tc.compiler = mcpp::toolchain::CompilerId::MSVC; + tc.version = "19.51"; + tc.binaryPath = "cl.exe"; + tc.targetTriple = "x86_64-pc-windows-msvc"; + return tc; +} + +mcpp::modgraph::SourceUnit unit(const std::filesystem::path& root, + const std::filesystem::path& rel, + std::string_view pkg, mcpp::SourceKind kind) { + mcpp::modgraph::SourceUnit u; + u.path = root / rel; + u.relPath = rel; + u.packageName = std::string(pkg); + u.kind = kind; + std::filesystem::create_directories(u.path.parent_path()); + std::ofstream(u.path) << "/* test */\n"; + return u; +} + +struct Fixture { + std::string rootStandard; // the consumer's `c_standard` + std::string depStandard = "gnu11"; // the dependency's `c_standard` + std::string quietStandard; // a dependency with no C units + mcpp::toolchain::Toolchain tc = clang_like(); + // The root as an executable whose entry `main` is a C file. The entry is + // synthesized by the link-unit pass, outside the scanned units. + bool cEntry = false; +}; + +struct Result { + // The C flags the plan gave each unit, by source file name. + std::map> cflags; + std::map> cxxflags; + std::vector notApplied; +}; + +Result plan_for(const Fixture& f) { + Tmp t; + const auto appRoot = t.path / "app"; + const auto depRoot = t.path / "cdep"; + const auto plainRoot = t.path / "plain"; + const auto quietRoot = t.path / "quiet"; + + mcpp::manifest::Manifest app; + app.package.name = "app"; + app.package.version = "0.1.0"; + app.buildConfig.cStandard = f.rootStandard; + mcpp::manifest::Target bin; + bin.name = "app"; + bin.kind = mcpp::manifest::Target::Library; + if (f.cEntry) { + bin.kind = mcpp::manifest::Target::Binary; + bin.main = "src/entry.c"; + std::filesystem::create_directories(appRoot / "src"); + std::ofstream(appRoot / "src" / "entry.c") << "int main(void) { return 0; }\n"; + } + app.targets.push_back(bin); + + mcpp::manifest::Manifest dep; + dep.package.name = "cdep"; + dep.package.version = "0.1.0"; + dep.buildConfig.cStandard = f.depStandard; + + // A dependency that declares nothing: it compiles at the default, whatever + // its consumer declares. + mcpp::manifest::Manifest plain; + plain.package.name = "plain"; + plain.package.version = "0.1.0"; + + // A dependency that declares a standard and has no C units. + mcpp::manifest::Manifest quiet; + quiet.package.name = "quiet"; + quiet.package.version = "0.1.0"; + quiet.buildConfig.cStandard = f.quietStandard; + + std::vector packages; + packages.push_back({appRoot, app}); + packages.push_back({depRoot, dep}); + packages.push_back({plainRoot, plain}); + packages.push_back({quietRoot, quiet}); + + using K = mcpp::SourceKind; + mcpp::modgraph::Graph graph; + graph.units.push_back(unit(appRoot, "src/app.c", "app", K::C)); + graph.units.push_back(unit(appRoot, "src/main.cpp", "app", K::Cxx)); + graph.units.push_back(unit(depRoot, "src/cdep.c", "cdep", K::C)); + graph.units.push_back(unit(depRoot, "src/more.c", "cdep", K::C)); + graph.units.push_back(unit(plainRoot, "src/plain.c", "plain", K::C)); + graph.units.push_back(unit(quietRoot, "src/quiet.cpp","quiet", K::Cxx)); + + std::vector topo; + for (std::size_t i = 0; i < graph.units.size(); ++i) topo.push_back(i); + + auto plan = make_plan(app, f.tc, {}, graph, topo, packages, + appRoot, appRoot / "target" / "t", {}, {}, {}); + EXPECT_TRUE(plan.has_value()) << (plan ? "" : plan.error()); + Result r; + if (!plan) return r; + for (auto const& cu : plan->compileUnits) { + r.cflags[cu.source.filename().string()] = cu.packageCflags; + r.cxxflags[cu.source.filename().string()] = cu.packageCxxflags; + } + r.notApplied = plan->cStandardsNotApplied; + return r; +} + +bool has(const std::vector& v, std::string_view flag) { + return std::ranges::find(v, flag) != v.end(); +} + +bool has_std(const std::vector& v) { + return std::ranges::any_of(v, [](const std::string& s) { return s.starts_with("-std="); }); +} + +} // namespace + +// The report's case: a dependency that declares `gnu11` compiles its C units at +// `gnu11`, and not at the consumer's standard. +TEST(CStandardPerPackage, ADependencyCompilesAtItsOwnStandard) { + auto r = plan_for(Fixture{}); + ASSERT_FALSE(r.cflags["cdep.c"].empty()); + EXPECT_EQ(r.cflags["cdep.c"].back(), "-std=gnu11"); + EXPECT_EQ(r.cflags["more.c"].back(), "-std=gnu11"); +} + +// The consumer's standard reaches the consumer's own C units and no one else's. +TEST(CStandardPerPackage, TheRootsStandardStaysInTheRoot) { + Fixture f; + f.rootStandard = "c99"; + auto r = plan_for(f); + ASSERT_FALSE(r.cflags["app.c"].empty()); + EXPECT_EQ(r.cflags["app.c"].back(), "-std=c99"); + EXPECT_FALSE(has(r.cflags["plain.c"], "-std=c99")); + EXPECT_FALSE(has_std(r.cflags["plain.c"])) + << "a package that declares nothing compiles at the file-level default"; + EXPECT_EQ(r.cflags["cdep.c"].back(), "-std=gnu11"); +} + +// A declaration equal to the default adds nothing: the file-level line already +// carries it, so the command line is unchanged. +TEST(CStandardPerPackage, DeclaringTheDefaultAddsNothing) { + Fixture f; + f.depStandard = "c11"; + auto r = plan_for(f); + EXPECT_FALSE(has_std(r.cflags["cdep.c"])); +} + +// A C standard is a C setting: no C++ unit receives one. +TEST(CStandardPerPackage, NoCxxUnitReceivesACStandard) { + Fixture f; + f.rootStandard = "c99"; + auto r = plan_for(f); + EXPECT_FALSE(has_std(r.cxxflags["main.cpp"])); + EXPECT_FALSE(has_std(r.cflags["main.cpp"])); +} + +// cl.exe compiles C in its default mode and takes no C `/std:` from mcpp yet +// (W3b). Nothing is added to a unit, and every declaring package that has C +// units is recorded once, so the backend can say so in one line. +TEST(CStandardPerPackage, ClRecordsWhatItDoesNotApply) { + Fixture f; + f.tc = cl_like(); + f.rootStandard = "c11"; + f.quietStandard = "c17"; + auto r = plan_for(f); + for (auto const& [file, v] : r.cflags) + for (auto const& flag : v) + EXPECT_FALSE(flag.starts_with("-std=") || flag.starts_with("/std:c")) + << file << ": " << flag; + EXPECT_EQ(r.notApplied, (std::vector{"app (c11)", "cdep (gnu11)"})) + << "a package with no C units (quiet) is not reported"; +} + +// With no cl.exe in the build, nothing is recorded as unapplied. +TEST(CStandardPerPackage, GnuDriversApplyEverythingTheyAreGiven) { + Fixture f; + f.rootStandard = "c99"; + auto r = plan_for(f); + EXPECT_TRUE(r.notApplied.empty()); +} + +// A target's entry `main` written in C is a unit of the root package: it +// compiles at the root's standard, as a scanned C unit does. +TEST(CStandardPerPackage, ACEntryMainTakesItsPackagesStandard) { + Fixture f; + f.rootStandard = "c99"; + f.cEntry = true; + auto r = plan_for(f); + ASSERT_TRUE(r.cflags.contains("entry.c")) + << "the plan has no unit for the entry main"; + ASSERT_FALSE(r.cflags["entry.c"].empty()); + EXPECT_EQ(r.cflags["entry.c"].back(), "-std=c99"); +} diff --git a/tests/unit/test_hermetic_graph_link.cpp b/tests/unit/test_hermetic_graph_link.cpp new file mode 100644 index 000000000..06e4eac96 --- /dev/null +++ b/tests/unit/test_hermetic_graph_link.cpp @@ -0,0 +1,101 @@ +// #696 (W2c) -- on an isolated graph link, the hermetic check holds every `-L` +// to the store, the build directory and the graph's own package roots. +// +// The check dry-runs the driver (`-###`) and reads the linker invocation it +// prints. A driver stands in here: a script under a path that names `xpkgs` +// (the check covers sandbox toolchains only) printing one linker line with the +// search directories each case needs. The check itself is Linux-only. + +#include + +import std; +import mcpp.build.hermetic; +import mcpp.platform; +import mcpp.toolchain.model; + +namespace { + +struct Tmp { + std::filesystem::path path; + Tmp() { + path = std::filesystem::temp_directory_path() + / std::format("mcpp_hermetic_graph_{}", std::random_device{}()); + std::filesystem::create_directories(path); + } + ~Tmp() { + std::error_code ec; + std::filesystem::remove_all(path, ec); + } +}; + +// A driver that answers `-###` with one linker line carrying `searchDirs`. +mcpp::toolchain::Toolchain fake_driver(const std::filesystem::path& root, + const std::vector& searchDirs) { + const auto bin = root / "xpkgs" / "fake-llvm" / "bin"; + std::filesystem::create_directories(bin); + const auto driver = bin / "clang++"; + std::string line = " \"/usr/bin/ld.lld\" \"-static\" \"-o\" \"/dev/null\""; + for (auto const& d : searchDirs) line += std::format(" \"-L{}\"", d); + line += " \"/tmp/nothing.o\""; + { + std::ofstream os(driver); + os << "#!/bin/sh\ncat <<'EOF'\n" << line << "\nEOF\n"; + } + std::filesystem::permissions(driver, std::filesystem::perms::owner_all, + std::filesystem::perm_options::add); + mcpp::toolchain::Toolchain tc; + tc.compiler = mcpp::toolchain::CompilerId::Clang; + tc.binaryPath = driver; + tc.targetTriple = "x86_64-linux-musl"; + return tc; +} + +} // namespace + +TEST(HermeticGraphLink, AHostSearchDirectoryIsRefusedAndNamed) { + if constexpr (!mcpp::platform::is_linux) GTEST_SKIP() << "the check runs on Linux hosts"; + Tmp t; + auto tc = fake_driver(t.path, {"/usr/lib", (t.path / "xpkgs" / "musl" / "lib").string()}); + auto r = mcpp::build::verify_hermetic_link(tc, "", t.path / "out", false, + /*isolatedGraphLink=*/true, {}); + ASSERT_FALSE(r.has_value()); + EXPECT_NE(r.error().find("/usr/lib"), std::string::npos) << r.error(); + EXPECT_NE(r.error().find("mcpp#696"), std::string::npos) << r.error(); + EXPECT_EQ(r.error().find("xpkgs/musl"), std::string::npos) + << "a store directory is allowed and must not be named: " << r.error(); +} + +TEST(HermeticGraphLink, AllowHostLibsDowngradesTheRefusal) { + if constexpr (!mcpp::platform::is_linux) GTEST_SKIP() << "the check runs on Linux hosts"; + Tmp t; + auto tc = fake_driver(t.path, {"/usr/lib"}); + auto r = mcpp::build::verify_hermetic_link(tc, "", t.path / "out", true, + /*isolatedGraphLink=*/true, {}); + EXPECT_TRUE(r.has_value()) << (r ? "" : r.error()); +} + +// The build directory holds the empty graph sysroot, and a package may add a +// package-relative `-L` whose root is anywhere (a path dependency). +TEST(HermeticGraphLink, TheBuildAndTheGraphsPackagesAreAllowed) { + if constexpr (!mcpp::platform::is_linux) GTEST_SKIP() << "the check runs on Linux hosts"; + Tmp t; + const auto out = t.path / "out"; + const auto pathDep = t.path / "elsewhere" / "openkal-musl"; + auto tc = fake_driver(t.path, {(out / "graph-sysroot" / "lib").string(), + (pathDep / "lib" / "empty").string(), + (t.path / "xpkgs" / "musl" / "lib").string()}); + auto r = mcpp::build::verify_hermetic_link(tc, "", out, false, + /*isolatedGraphLink=*/true, {pathDep}); + EXPECT_TRUE(r.has_value()) << (r ? "" : r.error()); +} + +// A payload link keeps its payload's directories; the `-L` rule is for the +// isolated graph link only. +TEST(HermeticGraphLink, APayloadLinkIsNotHeldToTheGraphRule) { + if constexpr (!mcpp::platform::is_linux) GTEST_SKIP() << "the check runs on Linux hosts"; + Tmp t; + auto tc = fake_driver(t.path, {"/usr/lib"}); + auto r = mcpp::build::verify_hermetic_link(tc, "", t.path / "out", false, + /*isolatedGraphLink=*/false, {}); + EXPECT_TRUE(r.has_value()) << (r ? "" : r.error()); +} diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index c8c2fe90b..21021d267 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -2435,9 +2435,47 @@ windows_entry = "wWinMain" EXPECT_TRUE(m->schemaWarnings.empty()); } +// #693: `windows_code_page` states the process code page of an executable, in +// the vocabulary of the manifest's `activeCodePage` element. +TEST(Manifest, ParsesWindowsCodePageOnABinaryTarget) { + for (std::string_view value : {"utf-8", "legacy"}) { + const auto src = std::format(R"( +[package] +name = "app" +version = "0.1.0" +[targets.app] +kind = "bin" +main = "src/main.cpp" +windows_code_page = "{}" +)", value); + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << m.error().format(); + ASSERT_EQ(m->targets.size(), 1u); + EXPECT_EQ(m->targets[0].windowsCodePage, value); + EXPECT_TRUE(m->schemaWarnings.empty()); + } +} + +TEST(Manifest, RefusesAnUnknownWindowsCodePageNamingTheAcceptedOnes) { + constexpr auto src = R"( +[package] +name = "app" +version = "0.1.0" +[targets.app] +kind = "bin" +main = "src/main.cpp" +windows_code_page = "65001" +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_FALSE(m.has_value()); + EXPECT_NE(m.error().message.find("\"65001\" is not one of \"utf-8\", \"legacy\""), + std::string::npos) << m.error().message; +} + TEST(Manifest, RefusesWindowsKeysOnALibraryNamingTheTargetAndTheKey) { const std::pair keys[] = { - {"windows_subsystem", "windows"}, {"windows_entry", "wmain"}}; + {"windows_subsystem", "windows"}, {"windows_entry", "wmain"}, + {"windows_code_page", "utf-8"}}; for (std::string_view kind : {"lib", "shared"}) { for (auto [key, value] : keys) { const auto src = std::format(R"( diff --git a/tests/unit/test_modgraph.cpp b/tests/unit/test_modgraph.cpp index 293bb5687..a75c761ea 100644 --- a/tests/unit/test_modgraph.cpp +++ b/tests/unit/test_modgraph.cpp @@ -1143,6 +1143,71 @@ TEST(Scanner, GlobWalkSurvivesNamesTheCodePageCannotSpell) { #endif } +// ─── #693: the UTF-8 spelling a path must have ───────────────────────────── +// +// build.ninja and compile_commands.json are UTF-8 text, so a path enters them +// only through a UTF-8 spelling (`try_narrow`). These state the check itself, +// and the POSIX half of the walk: a name whose bytes are not UTF-8 is skipped +// and recorded, exactly as the Windows test above states for a name the code +// page cannot spell. + +TEST(Glob, Utf8ValidityIsDecidedByteByByte) { + EXPECT_TRUE(is_valid_utf8("")); + EXPECT_TRUE(is_valid_utf8("plain/ascii")); + EXPECT_TRUE(is_valid_utf8("caf\xC3\xA9")); // U+00E9 + EXPECT_TRUE(is_valid_utf8("\xE6\xB5\x8B\xE8\xAF\x95")); // U+6D4B U+8BD5 + EXPECT_TRUE(is_valid_utf8("\xF0\x9F\x98\x80")); // U+1F600 + EXPECT_FALSE(is_valid_utf8("caf\xE9")); // Latin-1 + EXPECT_FALSE(is_valid_utf8("\xB2\xE2\xCA\xD4")); // GBK for U+6D4B U+8BD5 + EXPECT_FALSE(is_valid_utf8("\xC0\xAF")); // an overlong '/' + EXPECT_FALSE(is_valid_utf8("\xED\xA0\x80")); // a surrogate + EXPECT_FALSE(is_valid_utf8("\xF4\x90\x80\x80")); // above U+10FFFF + EXPECT_FALSE(is_valid_utf8("\xE6\xB5")); // truncated + EXPECT_FALSE(is_valid_utf8("\x80")); // a stray continuation +} + +// A diagnostic names a path that has no UTF-8 spelling through an escaped one. +TEST(Glob, EscapedSpellingIsUtf8WhateverTheName) { +#ifdef _WIN32 + const std::wstring lone{L'a', wchar_t(0xD800), L'b'}; + EXPECT_EQ(escaped_spelling(std::filesystem::path(lone)), "a\\u{D800}b"); + const std::wstring cafe{L'c', L'a', L'f', wchar_t(0x00E9)}; + EXPECT_EQ(escaped_spelling(std::filesystem::path(cafe)), "caf\xC3\xA9"); +#else + EXPECT_EQ(escaped_spelling("/x/caf\xE9"), "/x/caf\\xE9"); + EXPECT_EQ(escaped_spelling("/x/caf\xC3\xA9"), "/x/caf\xC3\xA9"); +#endif + EXPECT_TRUE(is_valid_utf8(escaped_spelling(std::filesystem::path("caf\xE9")))); +} + +TEST(Scanner, GlobWalkSkipsNamesThatAreNotUtf8) { +#ifdef _WIN32 + GTEST_SKIP() << "a Windows name is UTF-16; the test above states the Windows case"; +#else + auto dir = make_tempdir("mcpp-scanner-latin1"); + std::error_code ec; + std::filesystem::create_directories(dir / "caf\xE9", ec); // Latin-1 + if (ec) { + std::filesystem::remove_all(dir); + GTEST_SKIP() << "this file system refuses a name that is not UTF-8: " + << ec.message(); + } + write(dir / "caf\xE9" / "x.h", "#pragma once\n"); + write(dir / "zzz_ascii" / "x.h", "#pragma once\n"); + (void)take_unnarrowable_paths(); + + std::vector files; + ASSERT_NO_THROW({ files = expand_glob(dir, "**/*.h"); }); + EXPECT_EQ(files, (std::vector{dir / "zzz_ascii" / "x.h"})); + + auto notes = take_unnarrowable_paths(); + ASSERT_EQ(notes.size(), 1u); + EXPECT_EQ(notes[0], dir.generic_string()); + + std::filesystem::remove_all(dir); +#endif +} + // `module : private;` is a THIRD production, not a spelling of the two the // scanner already knew ([module.private.frag]). It declares nothing: the unit // still provides what its `export module` line said, and requires nothing new. diff --git a/tests/unit/test_ninja_backend.cpp b/tests/unit/test_ninja_backend.cpp index ab370a97d..e4a496908 100644 --- a/tests/unit/test_ninja_backend.cpp +++ b/tests/unit/test_ninja_backend.cpp @@ -10,6 +10,7 @@ import mcpp.libs.json; import mcpp.manifest; import mcpp.toolchain.dialect; import mcpp.toolchain.model; +import mcpp.toolchain.triple; import mcpp.platform; import mcpp.platform.runtime_search; import mcpp.targetside; @@ -853,8 +854,9 @@ TEST(NinjaBackend, CompileAndScanRulesRouteFlagsThroughRspfileUnderMsvcDialect) // only paths this file forward-slashes itself. $cxxflags carries // native-separated paths from flags.cppm and must stay inline — // routing it through the rsp ate the separators of the std.pcm path - // and broke every `import std;` on Windows. - EXPECT_NE(body.find("rspfile_content = $local_includes\n"), + // and broke every `import std;` on Windows. The byte order mark in + // front is how cl.exe learns the file is UTF-8 (#693). + EXPECT_NE(body.find("rspfile_content = \xEF\xBB\xBF $local_includes\n"), std::string::npos) << body; // The payload must not ALSO remain inline, or the ceiling stands. auto cmdStart = body.find("command = "); @@ -2274,3 +2276,279 @@ TEST(NinjaBackend, WindowsSubsystemReachesOnlyTheDeclaringExecutable) { EXPECT_GT(flag, guiEdge) << ninja; EXPECT_LT(flag, nextEdge) << ninja; } + +// #694 (W1d): THE LEVEL A PROFILE DECLARES IS THE LEVEL EVERY ROW REALISES. +// +// A branch keyed on the target triple once replaced every non-zero level with +// `-Og` on `*-linux-musl`, for GCC and clang alike, while the `Finished` line +// still said `[optimized]`; it set the level of mcpp's own Linux release +// binaries. The walk covers every row of the target table with both GNU-dialect +// compilers, so a later branch that overrides a declared level fails here and +// not in a published binary. `realised_opt_level` is the answer both the flags +// and the `Finished` line read, so it is asserted against the same rows. +TEST(NinjaBackendOptimization, EveryTargetRowRealisesTheDeclaredLevel) { + using mcpp::toolchain::CompilerId; + for (auto const& row : mcpp::toolchain::triple::kKnownTargets) { + for (auto compiler : {CompilerId::GCC, CompilerId::Clang}) { + for (std::string level : {"0", "1", "2", "3", "s"}) { + auto plan = minimal_plan(); + plan.toolchain.targetTriple = std::string(row.canonical); + plan.toolchain.compiler = compiler; + plan.toolchain.binaryPath = compiler == CompilerId::Clang + ? "/usr/bin/clang++" : "/usr/bin/g++"; + plan.manifest.buildConfig.optLevel = level; + + auto f = compute_flags(plan); + const std::string want = " -O" + level; + const std::string where = std::format( + "target={} compiler={} opt={}", row.canonical, + static_cast(compiler), level); + EXPECT_NE(f.cxx.find(want), std::string::npos) << where << "\n" << f.cxx; + EXPECT_NE(f.cc.find(want), std::string::npos) << where << "\n" << f.cc; + EXPECT_EQ(f.cxx.find(" -Og"), std::string::npos) << where << "\n" << f.cxx; + EXPECT_EQ(f.cc.find(" -Og"), std::string::npos) << where << "\n" << f.cc; + EXPECT_EQ(realised_opt_level(plan.manifest.buildConfig), level) << where; + EXPECT_EQ(realises_optimization(plan.manifest.buildConfig), level != "0") + << where; + } + } + } +} + +// The MSVC dialect spells the same levels its own way: `/Od` for zero (there is +// no `/O0`) and `/O` otherwise. +TEST(NinjaBackendOptimization, MsvcDialectRealisesTheDeclaredLevel) { + for (std::string level : {"0", "1", "2"}) { + auto plan = minimal_plan(); + plan.toolchain.compiler = mcpp::toolchain::CompilerId::MSVC; + plan.toolchain.binaryPath = "cl.exe"; + plan.toolchain.targetTriple = "x86_64-pc-windows-msvc"; + plan.manifest.buildConfig.optLevel = level; + auto f = compute_flags(plan); + const std::string want = level == "0" ? " /Od" : " /O" + level; + EXPECT_NE(f.cxx.find(want), std::string::npos) << level << "\n" << f.cxx; + EXPECT_NE(f.cc.find(want), std::string::npos) << level << "\n" << f.cc; + } +} + +// An empty level is zero, for the flags and for the `Finished` line alike. The +// old spelling rendered it as a bare `-O`, which GCC reads as `-O1`, while the +// line called the build unoptimized. +TEST(NinjaBackendOptimization, AnEmptyLevelIsZeroForFlagsAndDescriptor) { + auto plan = minimal_plan(); + plan.manifest.buildConfig.optLevel.clear(); + auto f = compute_flags(plan); + EXPECT_NE(f.cxx.find(" -O0"), std::string::npos) << f.cxx; + EXPECT_EQ(f.cxx.find(" -O "), std::string::npos) << f.cxx; + EXPECT_EQ(realised_opt_level(plan.manifest.buildConfig), "0"); + EXPECT_FALSE(realises_optimization(plan.manifest.buildConfig)); +} + +// #695 and #690 (C2): THE FILE-LEVEL FLAGS CARRY NOTHING A PACKAGE DECLARES FOR +// ITSELF. +// +// Every unit of every package reads `$cflags` and `$cxxflags`, so a value placed +// there is broadcast to the whole graph. The root's include directories rode +// that channel until #691, and the root's C standard until #695. The root here +// declares every package-private key; none of those values may appear in the +// file-level flags. The C standard there is the engine's constant. +TEST(NinjaBackendScope, TheFileLevelFlagsCarryNoPackagePrivateValue) { + auto plan = minimal_plan(); + auto& bc = plan.manifest.buildConfig; + bc.cStandard = "gnu11"; + bc.includeDirs = {"/root-private/include"}; + bc.privateIncludeDirs = {"/root-private/include"}; + bc.cflags = {"-DROOT_ONLY_CFLAG=1"}; + bc.cxxflags = {"-DROOT_ONLY_CXXFLAG=1"}; + bc.defines = {"ROOT_ONLY_DEFINE=1"}; + + auto f = compute_flags(plan); + for (auto const* line : {&f.cc, &f.cxx}) { + EXPECT_EQ(line->find("gnu11"), std::string::npos) << *line; + EXPECT_EQ(line->find("/root-private/include"), std::string::npos) << *line; + EXPECT_EQ(line->find("ROOT_ONLY"), std::string::npos) << *line; + } + EXPECT_NE(f.cc.find("-std=c11"), std::string::npos) << f.cc; +} + +// #696 (W2b): AN ELF LINK OVER A GRAPH-SUPPLIED C LIBRARY SEARCHES NOTHING OF +// THE HOST'S. With no sysroot, clang derived its library search from `/` and, +// on x86_64, from the host's GCC; the link now names an empty directory inside +// the build as its sysroot. +namespace { +BuildPlan graph_c_library_plan(std::string_view triple, + mcpp::toolchain::CompilerId compiler) { + auto plan = minimal_plan(); + plan.toolchain.targetTriple = std::string(triple); + plan.toolchain.compiler = compiler; + plan.toolchain.binaryPath = compiler == mcpp::toolchain::CompilerId::Clang + ? "/usr/bin/clang++" : "/usr/bin/g++"; + plan.targetSide.kernelAbi = { mcpp::targetside::Origin::Graph, "openkal", + "openkal-linux@0.15.0", false }; + plan.targetSide.cAbi = { mcpp::targetside::Origin::Graph, "musl", + "openkal-musl@0.19.2", false }; + plan.targetSide.cxx = { mcpp::targetside::Origin::Graph, "libc++", + "openkal-llvm-runtime@0.15.2", false }; + return plan; +} +} // namespace + +TEST(NinjaBackendGraphLink, AnElfGraphLinkByClangCarriesTheEmptySysroot) { + for (auto triple : {"x86_64-linux-musl", "aarch64-linux-musl"}) { + auto plan = graph_c_library_plan(triple, mcpp::toolchain::CompilerId::Clang); + auto f = compute_flags(plan); + EXPECT_TRUE(f.graphLinkIsolated) << triple; + EXPECT_NE(f.ld.find("--sysroot="), std::string::npos) << triple << "\n" << f.ld; + EXPECT_NE(f.ld.find(std::string(kGraphLinkSysrootDir)), std::string::npos) + << triple << "\n" << f.ld; + } +} + +// The measured case is clang on ELF; GCC over the graph and PE links keep their +// line until they are measured, and a payload C library keeps its own sysroot. +TEST(NinjaBackendGraphLink, OtherLinksDoNotCarryTheEmptySysroot) { + { + auto plan = graph_c_library_plan("x86_64-linux-musl", mcpp::toolchain::CompilerId::GCC); + auto f = compute_flags(plan); + EXPECT_FALSE(f.graphLinkIsolated); + EXPECT_EQ(f.ld.find(std::string(kGraphLinkSysrootDir)), std::string::npos) << f.ld; + } + { + auto plan = graph_c_library_plan("x86_64-windows-musl", mcpp::toolchain::CompilerId::Clang); + auto f = compute_flags(plan); + EXPECT_FALSE(f.graphLinkIsolated); + EXPECT_EQ(f.ld.find(std::string(kGraphLinkSysrootDir)), std::string::npos) << f.ld; + } + { + auto plan = minimal_plan(); + auto f = compute_flags(plan); + EXPECT_FALSE(f.graphLinkIsolated); + EXPECT_EQ(f.ld.find(std::string(kGraphLinkSysrootDir)), std::string::npos) << f.ld; + } +} + +// #696: the note for an unanswered `-l` on an isolated graph link. +TEST(GraphLinkLibraryAdvice, NamesTheLibraryAndTheMuslRelease) { + const std::string out = + "FAILED: bin/app\n" + "clang++ @bin/app.rsp -o bin/app --sysroot=/p/target/x/graph-sysroot -lm -lpthread\n" + "ld.lld: error: unable to find library -lm\n" + "ld.lld: error: unable to find library -lpthread\n" + "ld.lld: error: unable to find library -lm\n"; + auto advice = graph_link_library_advice(out, "musl", "openkal-musl@0.19.1"); + EXPECT_NE(advice.find("musl (openkal-musl@0.19.1)"), std::string::npos) << advice; + EXPECT_NE(advice.find("-lm, -lpthread"), std::string::npos) << advice; + EXPECT_NE(advice.find("openkal-musl 0.19.2"), std::string::npos) << advice; + EXPECT_NE(advice.find("mcpp#696"), std::string::npos) << advice; +} + +TEST(GraphLinkLibraryAdvice, AnotherLibraryGetsNoMuslParagraph) { + const std::string out = + "clang++ -o bin/app --sysroot=/p/target/x/graph-sysroot -lz\n" + "ld.lld: error: unable to find library -lz\n"; + auto advice = graph_link_library_advice(out, "musl", "openkal-musl@0.19.2"); + EXPECT_NE(advice.find("-lz"), std::string::npos) << advice; + EXPECT_EQ(advice.find("openkal-musl 0.19.2 and later"), std::string::npos) << advice; +} + +TEST(GraphLinkLibraryAdvice, StaysSilentWithoutTheGraphSysroot) { + const std::string out = + "clang++ -o bin/app -lm\n" + "ld.lld: error: unable to find library -lm\n"; + EXPECT_TRUE(graph_link_library_advice(out, "musl", "openkal-musl@0.19.1").empty()); + EXPECT_TRUE(graph_link_library_advice( + "clang++ --sysroot=/p/graph-sysroot\nld.lld: error: undefined symbol: foo\n").empty()); +} + +// ─── #693 (W4d): the encodings the tools read ─────────────────────────────── +// +// cl.exe, link.exe and lib.exe read a response file as UTF-8 only when it +// begins with a byte order mark, and in the ANSI code page otherwise (measured +// on windows-latest; see the emitter). Every response file of the msvc dialect +// begins with one; a GNU driver would read the mark as part of its first +// argument, so no response file of the gnu dialect has one. + +namespace { + +// The `rspfile_content` line of `rule`, or empty when the rule has none. +std::string rsp_line(const std::string& ninja, std::string_view rule) { + auto start = ninja.find(rule); + if (start == std::string::npos) return {}; + auto end = ninja.find("\n\n", start); + auto body = ninja.substr(start, end - start); + auto at = body.find("rspfile_content ="); + if (at == std::string::npos) return {}; + return body.substr(at, body.find('\n', at) - at); +} + +} // namespace + +TEST(NinjaBackendEncoding, MsvcResponseFilesBeginWithTheByteOrderMark) { + auto plan = minimal_plan(); + plan.toolchain.compiler = mcpp::toolchain::CompilerId::MSVC; + plan.toolchain.binaryPath = "cl.exe"; + plan.toolchain.targetTriple = "x86_64-pc-windows-msvc"; + plan.compileUnits.push_back({ + .source = "src/m.cppm", + .kind = mcpp::SourceKind::ModuleInterface, + .object = "obj/m.o", + .packageName = "objc_rule_test", + .providesModule = "m", + }); + plan.compileUnits.push_back({ + .source = "src/a.c", + .kind = mcpp::SourceKind::C, + .object = "obj/a.o", + .packageName = "objc_rule_test", + }); + auto ninja = emit_ninja_string(plan); + for (std::string_view rule : {"rule c_object\n", "rule cxx_scan\n", + "rule cxx_link\n", "rule cxx_archive\n", + "rule cxx_shared\n"}) { + auto line = rsp_line(ninja, rule); + ASSERT_FALSE(line.empty()) << rule << ninja; + EXPECT_NE(line.find("= \xEF\xBB\xBF"), std::string::npos) << rule << line; + } +} + +TEST(NinjaBackendEncoding, GnuResponseFilesCarryNoByteOrderMark) { + auto plan = minimal_plan(); + plan.compileUnits.push_back({ + .source = "src/a.c", + .kind = mcpp::SourceKind::C, + .object = "obj/a.o", + .packageName = "objc_rule_test", + }); + auto ninja = emit_ninja_string(plan); + EXPECT_FALSE(rsp_line(ninja, "rule cxx_link\n").empty()) << ninja; + EXPECT_EQ(ninja.find("\xEF\xBB\xBF"), std::string::npos) << ninja; +} + +// Ninja reports the encoding it reads build.ninja in; mcpp writes UTF-8. +TEST(NinjaBackendEncoding, ANinjaThatReadsUtf8Passes) { + EXPECT_FALSE(ninja_encoding_mismatch("Build file encoding: UTF-8\r\n", 65001, + "ninja.exe").has_value()); + EXPECT_FALSE(ninja_encoding_mismatch("Build file encoding: UTF-8\n", 1252, + "ninja.exe").has_value()); +} + +TEST(NinjaBackendEncoding, ANinjaThatReadsAnsiIsRefusedByName) { + auto modern = ninja_encoding_mismatch("Build file encoding: ANSI\r\n", 65001, + "C:/tools/ninja.exe"); + ASSERT_TRUE(modern.has_value()); + EXPECT_NE(modern->find("C:/tools/ninja.exe"), std::string::npos) << *modern; + EXPECT_NE(modern->find("ANSI"), std::string::npos) << *modern; + EXPECT_NE(modern->find("1.11"), std::string::npos) << *modern; + + // On a host that ignores the declaration the advice names the code page. + auto legacy = ninja_encoding_mismatch("Build file encoding: ANSI\n", 936, + "ninja.exe"); + ASSERT_TRUE(legacy.has_value()); + EXPECT_NE(legacy->find("936"), std::string::npos) << *legacy; +} + +TEST(NinjaBackendEncoding, OutputThatNamesNoEncodingDecidesNothing) { + EXPECT_FALSE(ninja_encoding_mismatch("", 65001, "ninja").has_value()); + EXPECT_FALSE(ninja_encoding_mismatch("ninja: error: unknown tool 'wincodepage'", + 65001, "ninja").has_value()); +} From 9697b21d8dbded7915231d6836b36b13c7e13ef7 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Sat, 26 Sep 2026 03:36:31 +0800 Subject: [PATCH 2/3] xlings 2026.9.26.2, the key rule a test still stated, and the graph link measured on four legs - kXlingsVersion and every xlings pin in .github move to 2026.9.26.2 (openxlings/xlings#613): xlings.exe declares the UTF-8 code page and its main has an exception boundary, so a working directory or an MCPP_HOME outside the ANSI code page no longer ends it with 0xC0000409 and no output. - test_cache_key stated the old rule, that a package's `c_standard = "c11"` is keyed; a package that spells the default and one that says nothing compile identically and now share a key. CI run 1 failed that test on every platform, and nothing else in the unit suites. - e2e 778 pins each leg to the openkal-musl and openkal-llvm-runtime pair that belongs together (the runtime pins openkal-musl exactly, and a root that pins another is refused as irreconcilable), and adds the report's aarch64 case under qemu-aarch64 and a host `-L` refused by the hermetic check. The openkal job asserts each leg ran. - docs: a macOS file name is UTF-8; `allow_host_libs` lifts the graph-link refusal rather than turning it into a warning. - The plan's implementation record carries CI run 1: every Windows row of the regression job passes, and the two xcode-27 legs fail as on main (#669). --- ...5-issues-693-696-triage-and-repair-plan.md | 32 +++++++++- .github/actions/bootstrap-mcpp/action.yml | 2 +- .github/actions/setup-macos-llvm/action.yml | 2 +- .github/workflows/bootstrap-macos.yml | 2 +- .github/workflows/ci-fresh-install.yml | 6 +- .github/workflows/ci-linux-e2e.yml | 2 +- .github/workflows/cross-build-test.yml | 4 +- .github/workflows/openkal-cross.yml | 4 ++ .github/workflows/release.yml | 14 ++--- docs/04-mcpp-toml.md | 5 +- docs/30-build-mcpp.md | 4 +- docs/zh/04-mcpp-toml.md | 4 +- docs/zh/30-build-mcpp.md | 4 +- src/xlings/xlings.cppm | 9 ++- ...a_graph_link_searches_no_host_directory.sh | 60 +++++++++++++++---- tests/unit/test_cache_key.cpp | 22 ++++++- 16 files changed, 133 insertions(+), 43 deletions(-) diff --git a/.agents/docs/2026-09-25-issues-693-696-triage-and-repair-plan.md b/.agents/docs/2026-09-25-issues-693-696-triage-and-repair-plan.md index 7cc3d17c4..8c3a08998 100644 --- a/.agents/docs/2026-09-25-issues-693-696-triage-and-repair-plan.md +++ b/.agents/docs/2026-09-25-issues-693-696-triage-and-repair-plan.md @@ -1049,8 +1049,11 @@ Three items are split out, each for a stated reason. |---|---|---| | 1 | openxlings/xlings#613 | open, CI running; released as 2026.9.26.2, because 2026.9.26.1 had been released separately earlier the same day | | 2 | mcpplibs/openkal-musl#43 | merged as `20b92683`; tag `0.19.2`; the GitHub archive (1157319 bytes, sha256 `e8043bcd...c82238`) and the GitCode copy compared byte for byte; the archive carries `port/lib/lib*.a`, eight files of eight bytes | -| 3 | mcpplibs/mcpp-index#467 | open, CI running | -| 4 to 7 | | follow in the order of §12.1 | +| 3 | mcpplibs/mcpp-index#467 | merged as `1529f5f3`; the index artifact `1529f5f` is served by both hosts with the pointer's digest | +| 4 | mcpplibs/openkal-llvm-runtime#30 | merged as `79671f6b`; tag `0.15.2`; archive 15631743 bytes, sha256 `c2219ad8...c0d0d9`, GitCode copy byte-identical; its mcpp.toml reads openkal-musl `0.19.2` | +| 5 | mcpplibs/mcpp-index#468 | registration of runtime 0.15.2, CI running | +| 6 | mcpp-community/mcpp#698 | CI run 1: every Windows row of the regression job passes (below); follow-up commit pending the xlings release | +| 7 | | after the mcpp release | **Where the implementation departs from §12.2, and why.** @@ -1081,6 +1084,18 @@ Three items are split out, each for a stated reason. there when run from another directory; mcpp resolves it there as a build input. - **The rc scanner** tracks a manifest named by the numeric type `24` and treats a statement as a manifest only when a file name follows the type, so `FILEVERSION 24,1,0,0` is not one. +- **A target's entry `main` written in C takes its package's standard.** The entry is + synthesized after `make_plan`'s unit loop, so a flag added only inside the loop missed it; + found in self-review, one helper now serves both sites (unit test + `ACEntryMainTakesItsPackagesStandard`). +- **The cache key records a package's C standard only when it differs from `c11`.** A package + that spells the default and one that says nothing compile identically, so they share a key. + The first CI run failed the unit test that encoded the old rule; the test now states the new + one (`DeclaringTheDefaultCStandardKeysNothing`). +- **openkal-llvm-runtime pins openkal-musl exactly.** A root that pins openkal-musl 0.19.2 + beside runtime 0.15.1 is refused as irreconcilable (measured with e2e 778), so a consumer + receives the archives by moving the runtime pin, which is what the #696 note says; the + fixture pairs musl 0.19.2 with runtime 0.15.2. **Test results on Linux** (the worktree at `fix/693-696`): @@ -1090,11 +1105,22 @@ Three items are split out, each for a stated reason. | e2e 776, a path with no UTF-8 spelling is named (new) | passes; released 2026.9.25.1 fails at its first criterion with the JSON exception | | e2e 777, `c_standard` applies to the package that declares it (new) | passes; 2026.9.25.1 compiles `cdep` and `plain` at the consumer's `c99` | | e2e 778, a graph link searches no host directory (new), leg B | passes; 2026.9.25.1 links leg B from the host's `libm` | -| e2e 778, leg A | runs once openkal-musl 0.19.2 is in the index | +| e2e 778, legs A, C (aarch64 under qemu) and D (`-L/usr/lib` refused) | run once openkal-llvm-runtime 0.15.2 is in the index | +| CI run 1 of #698 | the two xcode-27 legs fail as on main (#669, lld 22 rejects `arm64e.x1` in the SDK stubs); `test_cache_key` failed on the old key rule (fixed) | | e2e 190 | accepts the byte order mark before `$in_newline` | The Windows rows run in CI through `.github/tools/check_unicode_paths.sh` (W4f): llvm, MSVC and MinGW in an ASCII directory, `caf` + U+00E9 and U+6D4B U+8BD5, plus a path through `build.mcpp`. +CI run 1 of #698 (windows-latest, code page 1252; `mcpp.exe` self-hosted by the bootstrap +2026.9.24.1 with `res/mcpp.rc`): + +``` + ok llvm in 'ascii' ok msvc in 'ascii' ok mingw in 'ascii' + ok llvm in 'café' ok msvc in 'café' ok mingw in 'café' + ok llvm in '测试' ok msvc in '测试' ok mingw in '测试' + ok a path through build.mcpp in '测试' +OK: every row builds in every directory +``` --- diff --git a/.github/actions/bootstrap-mcpp/action.yml b/.github/actions/bootstrap-mcpp/action.yml index d69f3fbff..46cd95402 100644 --- a/.github/actions/bootstrap-mcpp/action.yml +++ b/.github/actions/bootstrap-mcpp/action.yml @@ -25,7 +25,7 @@ inputs: # `package.name`, so one of the two was simply unreachable — and which one # depended on the machine, which is why CI failed on `compat:lua` on # Windows and `mcpplibs.capi:lua` on Linux. Never pin below that. - default: '2026.9.20.1' + default: '2026.9.26.2' cache-target: description: also restore/save target/ (build artifacts + BMIs) required: false diff --git a/.github/actions/setup-macos-llvm/action.yml b/.github/actions/setup-macos-llvm/action.yml index 8feb50804..d0e556393 100644 --- a/.github/actions/setup-macos-llvm/action.yml +++ b/.github/actions/setup-macos-llvm/action.yml @@ -15,7 +15,7 @@ inputs: # Floor imposed by the index, not a routine bump — see # .github/actions/bootstrap-mcpp/action.yml for why 0.4.69 is required # (two packages named `lua` in one repo need openxlings/xlings#381). - default: '2026.9.20.1' + default: '2026.9.26.2' image: description: > The runner label the job runs on (macos-15, xcode-27). It is part of the diff --git a/.github/workflows/bootstrap-macos.yml b/.github/workflows/bootstrap-macos.yml index aef5c7e4b..4c76ba350 100644 --- a/.github/workflows/bootstrap-macos.yml +++ b/.github/workflows/bootstrap-macos.yml @@ -17,7 +17,7 @@ jobs: # Dormant (workflow_dispatch only), but kept in step with the rest — # check_version_pins.sh holds it there. Floor: 0.4.69, below which the # index cannot resolve two packages that share a short name. - XLINGS_VERSION: '2026.9.20.1' + XLINGS_VERSION: '2026.9.26.2' steps: - uses: actions/checkout@v4 diff --git a/.github/workflows/ci-fresh-install.yml b/.github/workflows/ci-fresh-install.yml index 352c77dde..75d97be04 100644 --- a/.github/workflows/ci-fresh-install.yml +++ b/.github/workflows/ci-fresh-install.yml @@ -152,7 +152,7 @@ jobs: env: XLINGS_NON_INTERACTIVE: '1' run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.20.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.26.2 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror @@ -312,7 +312,7 @@ jobs: - name: Install xlings + mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.20.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.26.2 # Deliberately NOT writing to $GITHUB_PATH here. On container # images that declare no PATH in their config (opensuse/ # tumbleweed), appending a single dir to GITHUB_PATH makes the @@ -403,7 +403,7 @@ jobs: # (older ones carry minos=15 and refuse to start). # v0.4.51+: in-process sha256 — this image has no sha256sum # binary, so pinned fetches failed before it. - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.20.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.26.2 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index 18c3398a7..f7ad1f041 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -384,7 +384,7 @@ jobs: - name: Bootstrap xlings + released mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.20.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.9.26.2 export PATH="$HOME/.xlings/subos/current/bin:$PATH" xlings update xlings install mcpp -y -g diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index 8a0ff8071..0fabc96e3 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -135,7 +135,7 @@ jobs: # release assets were uploaded in a broken state (records present, # blobs missing → 404 on GET); re-uploaded clean. The stale-INDEX # half is handled by the marker-clear below. - XLINGS_VERSION: '2026.9.20.1' + XLINGS_VERSION: '2026.9.26.2' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ @@ -289,7 +289,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.9.20.1' + XLINGS_VERSION: '2026.9.26.2' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ diff --git a/.github/workflows/openkal-cross.yml b/.github/workflows/openkal-cross.yml index 6e6bdaa2a..dcd4923b0 100644 --- a/.github/workflows/openkal-cross.yml +++ b/.github/workflows/openkal-cross.yml @@ -637,4 +637,8 @@ jobs: "ok: -lm is answered by openkal-musl's own archive, and the program runs" || fail=1 check 778_a_graph_link_searches_no_host_directory.sh \ "ok: an unanswered -lm fails, and the note names openkal-musl 0.19.2" || fail=1 + check 778_a_graph_link_searches_no_host_directory.sh \ + "ok: aarch64-linux-musl links -lm from the graph and runs under qemu-aarch64" || fail=1 + check 778_a_graph_link_searches_no_host_directory.sh \ + "ok: a host directory in ldflags is refused, by name" || fail=1 [ "$fail" = 0 ] || exit 1 diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 624637304..b5f7e3c45 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -96,7 +96,7 @@ jobs: # Pin xlings to a known-good version. The upstream install # script always grabs `latest` (no version override), so we # download + self-install manually to avoid broken releases. - XLINGS_VERSION: '2026.9.20.1' + XLINGS_VERSION: '2026.9.26.2' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" @@ -314,7 +314,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.9.20.1' + XLINGS_VERSION: '2026.9.26.2' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" bash "$GITHUB_WORKSPACE/.github/tools/fetch_release.sh" \ @@ -385,7 +385,7 @@ jobs: # below are pinned to the same version as XLINGS_VERSION; they are # NOT interpolated from it, so check_version_pins.sh scans for them # explicitly (they were absent from the old lock-step comment). - XLA="xlings-2026.9.20.1-linux-aarch64.tar.gz" + XLA="xlings-2026.9.26.2-linux-aarch64.tar.gz" # NOT fetch_release.sh: this asset is OPTIONAL and the `if` is the # point — an arch with no prebuilt xlings must fall through quietly, # while the helper retries a 404 five times before giving up. The one @@ -394,9 +394,9 @@ jobs: # cover it. if curl -fsSL --retry 3 --retry-delay 2 --retry-all-errors \ --connect-timeout 20 --max-time 600 -o "/tmp/$XLA" \ - "https://github.com/openxlings/xlings/releases/download/v2026.9.20.1/$XLA"; then + "https://github.com/openxlings/xlings/releases/download/v2026.9.26.2/$XLA"; then tar -xzf "/tmp/$XLA" -C /tmp - XLBIN=$(find /tmp/xlings-2026.9.20.1-linux-aarch64 -path '*/bin/xlings' -type f | head -1) + XLBIN=$(find /tmp/xlings-2026.9.26.2-linux-aarch64 -path '*/bin/xlings' -type f | head -1) if [ -n "$XLBIN" ]; then mkdir -p "$STAGING/$WRAPPER/registry/bin" cp "$XLBIN" "$STAGING/$WRAPPER/registry/bin/xlings" @@ -474,7 +474,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.9.20.1' + XLINGS_VERSION: '2026.9.26.2' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then WORK=$(mktemp -d) @@ -657,7 +657,7 @@ jobs: shell: bash env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.9.20.1' + XLINGS_VERSION: '2026.9.26.2' run: | # Captured before the `cd` below, in POSIX form: this step never # returns to the workspace, and GITHUB_WORKSPACE is a backslash diff --git a/docs/04-mcpp-toml.md b/docs/04-mcpp-toml.md index cbdd032a5..df94f6896 100644 --- a/docs/04-mcpp-toml.md +++ b/docs/04-mcpp-toml.md @@ -937,8 +937,9 @@ is not UTF-8. `cl.exe`, `link.exe` and `lib.exe` read a response file as UTF-8 only when it begins with a byte order mark, so the response files of the msvc dialect begin with one. -**Paths with no UTF-8 spelling.** On Linux and macOS a file name is a sequence of -bytes, which need not be UTF-8. On a Windows host older than version 1903 the +**Paths with no UTF-8 spelling.** On Linux a file name is a sequence of bytes, +which need not be UTF-8 (macOS file systems store names in UTF-8). On a Windows +host older than version 1903 the manifest is ignored, and the process runs in the system's ANSI code page, which spells only part of Unicode. On either, some paths have no UTF-8 spelling: diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index f19fa9ffb..a0b365e5c 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -953,8 +953,8 @@ directive the program prints as UTF-8. On Windows the program is linked with the application manifest that `mcpp.exe` itself carries, which sets its ANSI code page to UTF-8 on Windows 10 version 1903 and later: its environment, its arguments and the narrow strings it prints are UTF-8 there without any conversion in the -program. On Linux and macOS a file name is bytes, and a program that lists a -directory prints whatever bytes it finds. +program. On Linux a file name is bytes, and a program that lists a directory +prints whatever bytes it finds. A directive line whose text is not UTF-8 is refused, whatever protocol the program announces, and the refusal names the directive's key. Its value would diff --git a/docs/zh/04-mcpp-toml.md b/docs/zh/04-mcpp-toml.md index 4408d0d49..73057d18e 100644 --- a/docs/zh/04-mcpp-toml.md +++ b/docs/zh/04-mcpp-toml.md @@ -894,8 +894,8 @@ Ninja 1.11 及以后的版本在同样的声明下以 UTF-8 读取 `build.ninja` `lib.exe` 只有在响应文件以字节顺序标记开头时才按 UTF-8 读取它，所以 msvc 方言的响应文件以字节顺序标记开头。 -**没有 UTF-8 拼法的路径。** 在 Linux 与 macOS 上，文件名是一串字节，不必是 -UTF-8。在早于 1903 的 Windows 宿主上，清单会被忽略，进程运行在系统的 ANSI +**没有 UTF-8 拼法的路径。** 在 Linux 上，文件名是一串字节，不必是 UTF-8 +（macOS 的文件系统以 UTF-8 存储名字）。在早于 1903 的 Windows 宿主上，清单会被忽略，进程运行在系统的 ANSI 代码页里，而它只能拼出 Unicode 的一部分。在这两种情况下，有些路径没有 UTF-8 拼法： diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index c27a7b46d..16fab4d63 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -816,8 +816,8 @@ mcpp 会把它自己构建时用的**同一份** std 模块暂存过来，缓存 mcpp 传给程序的每个路径与值都是 UTF-8，程序打印的每条指令也按 UTF-8 读取。 在 Windows 上，程序链接时带有 `mcpp.exe` 自身携带的应用程序清单，它在 Windows 10 1903 及以后的版本上把程序的 ANSI 代码页设为 UTF-8：程序的环境、参数 -以及它打印的窄字符串在那里都是 UTF-8，程序内无需任何转换。在 Linux 与 macOS 上， -文件名是字节，列出目录的程序打印的就是它找到的那些字节。 +以及它打印的窄字符串在那里都是 UTF-8，程序内无需任何转换。在 Linux 上，文件名是 +字节，列出目录的程序打印的就是它找到的那些字节。 文本不是 UTF-8 的指令行会被拒绝，无论程序声明了哪个协议，拒绝信息会点名该指令的 键。它的值会以指向另一个文件的字节进入 `build.ninja`，所以它的任何部分都不会被 diff --git a/src/xlings/xlings.cppm b/src/xlings/xlings.cppm index ab6ea539f..a80079a7c 100644 --- a/src/xlings/xlings.cppm +++ b/src/xlings/xlings.cppm @@ -103,7 +103,14 @@ namespace pinned { // install that failed to download could still print `installed`. mcpp // drives xlings from inside project and sandbox subos, which is the // position where the first misread applied. - inline constexpr std::string_view kXlingsVersion = "2026.9.20.1"; + // + // Sixth, at 2026.9.26.2 (openxlings/xlings#613). Below it, xlings.exe ran + // in the system's ANSI code page and its `main` had no exception boundary: + // a working directory outside that code page ended it with 0xC0000409 and + // no output (mcpp#693), and under an MCPP_HOME outside it the xlings mcpp + // vendors could not initialise its sandbox. It now declares the UTF-8 code + // page, as mcpp.exe does. + inline constexpr std::string_view kXlingsVersion = "2026.9.26.2"; inline constexpr std::string_view kNasmVersion = "3.02"; } diff --git a/tests/e2e/778_a_graph_link_searches_no_host_directory.sh b/tests/e2e/778_a_graph_link_searches_no_host_directory.sh index 8c172fba3..60b2ac817 100644 --- a/tests/e2e/778_a_graph_link_searches_no_host_directory.sh +++ b/tests/e2e/778_a_graph_link_searches_no_host_directory.sh @@ -9,15 +9,24 @@ # diagnostic. The graph link now carries `--sysroot` naming an empty directory # in the build directory, which removes every such directory. # -# Two legs, and the second is the one that separates the engines: +# Four legs; the second is the one that separates the engines: # # A. openkal-musl 0.19.2 ships musl's eight empty archives (`libm.a` among -# them). `-lm` is answered by the graph, the link carries the empty -# sysroot, and the program runs. -# B. openkal-musl 0.19.1 has no `libm.a`. With the host's directories gone, -# `-lm` is unanswered: the build fails with the linker's own message and -# mcpp's note naming the release that answers it. The released engine -# links this leg successfully, from the host's libm. +# them), through openkal-llvm-runtime 0.15.2, which pins it. `-lm` is +# answered by the graph, the link carries the empty sysroot, and the +# program runs. +# B. openkal-musl 0.19.1 (runtime 0.15.1) has no `libm.a`. With the host's +# directories gone, `-lm` is unanswered: the build fails with the linker's +# own message and mcpp's note naming the release that answers it. The +# released engine links this leg successfully, from the host's libm. +# C. The report's own case: aarch64-linux-musl, where the host's `libm.a` +# is an x86_64 linker script. It links, and runs under qemu-aarch64. +# D. A host directory given in `ldflags` is refused by the hermetic check, +# which names it. +# +# The runtime pins openkal-musl exactly, so each leg names the pair that +# belongs together: a root that pins another openkal-musl is refused as +# irreconcilable before anything is linked. set -e MCPP="${MCPP:-mcpp}" @@ -28,7 +37,7 @@ TARGET=x86_64-linux-musl fail() { [ -n "$2" ] && cat "$2"; echo "FAIL: $1"; exit 1; } make_project() { - local dir="$1" musl="$2" + local dir="$1" musl="$2" runtime="$3" ldflags="${4:-\"-lm\"}" mkdir -p "$dir/src" cat > "$dir/mcpp.toml" < "$dir/src/main.c" <<'C' #include @@ -68,7 +77,7 @@ skip_if_unreachable() { } # ── A. the graph answers -lm ──────────────────────────────────────────────── -make_project "$work/a" "0.19.2" +make_project "$work/a" "0.19.2" "0.15.2" cd "$work/a" if ! "$MCPP" build --target "$TARGET" --verbose > a.log 2>&1; then skip_if_unreachable a.log @@ -81,7 +90,7 @@ printf '%s\n' "$out" | grep -qx '2' || fail "unexpected output: $out" echo " ok: -lm is answered by openkal-musl's own archive, and the program runs" # ── B. nothing in the graph answers -lm ───────────────────────────────────── -make_project "$work/b" "0.19.1" +make_project "$work/b" "0.19.1" "0.15.1" cd "$work/b" if "$MCPP" build --target "$TARGET" > b.log 2>&1; then fail "-lm was answered although nothing in the graph provides it: the host's library directories are still searched" b.log @@ -93,4 +102,31 @@ grep -q 'openkal-musl 0.19.2' b.log \ || fail "the note does not name the release that answers -lm" b.log echo " ok: an unanswered -lm fails, and the note names openkal-musl 0.19.2" +# ── C. the report's case, on aarch64 ──────────────────────────────────────── +make_project "$work/c" "0.19.2" "0.15.2" +cd "$work/c" +"$MCPP" build --target aarch64-linux-musl > c.log 2>&1 \ + || fail "with openkal-musl 0.19.2, -lm must link on aarch64-linux-musl" c.log +bin=$(find target -path '*aarch64*' -name lmprobe -type f | head -1) +[ -n "$bin" ] || fail "no aarch64 lmprobe was produced" c.log +runner="$(command -v qemu-aarch64 || command -v qemu-aarch64-static || true)" +if [ -z "$runner" ]; then + echo " SKIP no aarch64 emulator here; the link was checked, not the run" +else + ran=$("$runner" "$bin" 2>&1) || fail "non-zero exit under $(basename "$runner"): $ran" + [ "$ran" = "2" ] || fail "unexpected output under emulation: $ran" + echo " ok: aarch64-linux-musl links -lm from the graph and runs under $(basename "$runner")" +fi + +# ── D. a host directory given in ldflags ──────────────────────────────────── +make_project "$work/d" "0.19.2" "0.15.2" '"-L/usr/lib", "-lm"' +cd "$work/d" +if "$MCPP" build --target "$TARGET" > d.log 2>&1; then + fail "a graph link was allowed to search /usr/lib" d.log +fi +grep -q 'hermetic link check failed' d.log \ + || fail "the refusal is not the hermetic check's" d.log +grep -q '/usr/lib' d.log || fail "the refusal does not name the directory" d.log +echo " ok: a host directory in ldflags is refused, by name" + echo "PASS: a graph link searches no host directory" diff --git a/tests/unit/test_cache_key.cpp b/tests/unit/test_cache_key.cpp index 01ba883b7..bcec3a7df 100644 --- a/tests/unit/test_cache_key.cpp +++ b/tests/unit/test_cache_key.cpp @@ -273,7 +273,7 @@ TEST(CacheKey, FillPackageConfigCarriesFlagsAndGeneratedFiles) { pkgRoot.manifest.buildConfig.cflags = {"-D_GNU_SOURCE"}; pkgRoot.manifest.buildConfig.cxxflags = {"-fno-exceptions"}; pkgRoot.manifest.buildConfig.defines = {"ZLIB_CONST"}; - pkgRoot.manifest.buildConfig.cStandard = "c11"; + pkgRoot.manifest.buildConfig.cStandard = "gnu11"; pkgRoot.manifest.buildConfig.generatedFiles = {{"cfg.h", "#define A 1"}}; ck::PackageAxes p; @@ -284,12 +284,28 @@ TEST(CacheKey, FillPackageConfigCarriesFlagsAndGeneratedFiles) { ASSERT_EQ(p.generatedFiles.size(), 1u); EXPECT_EQ(p.generatedFiles.front(), "cfg.h=#define A 1"); // A package's own C standard reaches its own C units, so it must be in the - // key even though the whole-graph C standard is on the B axis. + // key; the B axis carries only the engine default (#695). bool sawCStd = false; - for (auto& f : p.cflags) if (f.find("c_standard=c11") != std::string::npos) sawCStd = true; + for (auto& f : p.cflags) if (f.find("c_standard=gnu11") != std::string::npos) sawCStd = true; EXPECT_TRUE(sawCStd); } +// Declaring the default is declaring nothing: the unit's command is the same, +// so the key is the same, and a package that spells `c11` shares its cached +// objects with one that leaves the key out (#695). +TEST(CacheKey, DeclaringTheDefaultCStandardKeysNothing) { + std::filesystem::path store = "/home/u/.mcpp/registry/data/xpkgs"; + auto declared = rootAt(store / "compat-x-compat.zlib" / "1.3.2"); + declared.manifest.buildConfig.cStandard = "c11"; + auto silent = rootAt(store / "compat-x-compat.zlib" / "1.3.2"); + + ck::PackageAxes a, b; + ck::fill_package_config(a, declared, store); + ck::fill_package_config(b, silent, store); + EXPECT_EQ(a.cflags, b.cflags); + for (auto& f : a.cflags) EXPECT_EQ(f.find("c_standard="), std::string::npos) << f; +} + // Generated files are a map; iteration order must not leak into the key. TEST(CacheKey, GeneratedFilesAreOrderIndependent) { std::filesystem::path store = "/home/u/.mcpp/registry/data/xpkgs"; From 15317aab4462f0aaea74d522a9317c293f095264 Mon Sep 17 00:00:00 2001 From: speak-agent <248744407+speak-agent@users.noreply.github.com> Date: Sat, 26 Sep 2026 03:39:25 +0800 Subject: [PATCH 3/3] SPEC-002 rule three: a link over a graph-supplied C library searches no host directory (v1.1) --- docs/specs/target-side.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/specs/target-side.md b/docs/specs/target-side.md index b4a837ebf..9ddfb59aa 100644 --- a/docs/specs/target-side.md +++ b/docs/specs/target-side.md @@ -132,6 +132,12 @@ mcpp:<层名>[=<实现名>] 当前实现覆盖「两层均来自图」与「两层均来自载荷」。 「一层预制、一层来自图」的接线尚不完整。 +`c-abi` 层来自图时,链接的库搜索同样属于图:引擎**禁止**让驱动器从宿主推导库目录, +这样的链接上每一个 `-L` **必须**位于工具链存储、构建目录或图中某个包之内; +`[build] allow_host_libs = true` 解除这一约束。由 clang 链接的 ELF 目标已实现 +(mcpp 2026.9.26.1,链接带指向构建目录内空目录的 `--sysroot`,mcpp#696);经 MinGW +驱动器的 PE 链接尚未实现,待其从宿主 MinGW 解析的导入库清点完成。 + ### 3.4 规则四:三元组是请求 已实现 三元组的 env 段**必须**被当作对 `c-abi` 的请求,而非其答案。 @@ -218,3 +224,4 @@ mcpp:<层名>[=<实现名>] | 版本 | 日期 | 变更 | |---|---|---| | v1.0 | 2026-08-24 | 初版。五层闭集、`provides`/`requires` 语法、三条规则、报告与兼容性条款。 | +| v1.1 | 2026-09-26 | 规则三增补:`c-abi` 来自图时,链接不搜索宿主库目录(ELF 已实现,mcpp#696)。 |