By: Lucien Huber <im@lcnbr.ch>
benchmark FeynKit CI before efficiency changes PLEASE IMPLEMENT THIS PLAN: # Improve CI on main while preserving FeynKit’s caching benefits ## Findings and approach `feynkit` adds nine crates, increasing the workspace from **18 to 27**. The extracted libraries do not depend on GammaLoop, so a GammaLoop implementation change should leave their compiled artifacts reusable. This is a useful improvement for incremental builds; it does not guarantee faster clean builds. There is evidence of that reuse already: one [FeynKit test job](https://nix-ci.com/gh:alphal00p:gammaloop/feynkit/2cf9ab9e64e4ea06441fba9e4d6c4501be801f60/0d666ad0-34a2-421d-b1a0-9f2f11236c3b) ran **207 tests from cached binaries in approximately 35 seconds**. However, another [Python-binding build](https://nix-ci.com/gh:alphal00p:gammaloop/feynkit/2cf9ab9e64e4ea06441fba9e4d6c4501be801f60/a06b7a33-e25c-47f4-b047-6057bd402f65) restored **2.4 GiB in 29 seconds** before compiling only `feynkit-py` for approximately **20 seconds**. Therefore, preserve per-crate compilation and prioritize reducing artifact transfers. Treat broader job grouping as an experiment, rather than a prerequisite. ## Implementation ### Clean up the flake - Branch from current `main` in an isolated `codex/ci-efficiency` worktree. Keep the FeynKit extraction and physics changes out of the main PR. - Extract existing Rust build machinery into `nix/rust-workspace.nix` and CI group/selection definitions into `nix/ci.nix`. Keep `flake.nix` focused on composing outputs. - Preserve public output names, existing helpers, comments, source filtering, features, and profiles. Pass the repository root explicitly so moving code does not change relative paths. - Generate the self-contained `nix-ci.nix` from the authoritative CI definition and workspace graph. Extend existing checks to detect stale generated configuration and omitted test groups. - Add `just ci-update` for regeneration and correct `just ci-checks` to invoke supported primary checks. ### Reduce NixCI work - Select tests, Clippy, doctests, formatting, graph validation, necessary producers, and final success explicitly. - Keep packaging, documentation, WASM, and development-shell outputs available without making them automatic NixCI targets. - Preserve package-specific compilation artifacts and archive commands. Do not replace them with one workspace-wide compilation. - Change merged Cargo artifacts from expanded directories to self-contained compressed archives, using existing Crane support. Preserve recursive inheritance, writable extraction, and epoch-1 timestamps. - In a separate commit, trial one producer for final test archives and the Python module. Retain internal per-crate cache boundaries and separately reported test groups. - Keep that grouping only if measurements on both main and FeynKit meet the acceptance criteria. Otherwise remove the grouping commit and retain independently validated improvements. ### Support FeynKit correctly - Validate the CI changes on a temporary integration branch containing the existing FeynKit code; do not change the shared `feynkit` branch during experimentation. - Preserve its eighth test group, all nine extracted crates, boundary checks, Python stub checks, embedded fixtures, and optional Python dependencies. - Preserve the extracted libraries’ independence from GammaLoop and its private workspace-hack dependency. - Regenerate scheduling metadata instead of carrying forward FeynKit’s stale embedded dependency edges. - Keep current manifest/lockfile cache boundaries initially. They invalidate broadly and must be measured separately from ordinary source edits. - Report Clippy/doctest compilation separately: their current workspace-wide inputs mean they do not automatically gain the same isolation as package test artifacts. ## Measurement Extend the existing log collector/parser into `just ci-report MANIFEST OUTPUT_DIR`, producing JSON, CSV, and a readable comparison. Record required-check and whole-suite latency, individual group completion, worker minutes, compilation by crate/context, repeated identical builds, restored content sizes, upload/download intervals, and executed versus reused test results. Keep waiting separate from execution; do not infer CHF costs or network bytes from these proxies. Use **12 paired NixCI suites**: | Application layout | Baseline and candidate scenarios | Suites | |---|---|---:| | Main | Initial run; unchanged follow-up commit | 4 | | FeynKit | Initial run; unchanged follow-up; GammaLoop-only source edit; FeynKit CFF source edit | 8 | Each source-edit scenario independently starts from its clean application revision. Compare CI implementations within each layout; do not attribute differences between main and FeynKit directly to the crate split. Allow one ambiguous pair to be repeated, for a maximum of 14 suites. Run controlled project-cold builds locally for both variants of both layouts, using disposable stores and identical resource limits. Shared remote caches mean an initial NixCI run cannot automatically be called cold. Verify these cache boundaries through derivation inspection and targeted builds: - GammaLoop-only edit: extracted FeynKit package artifacts remain unchanged. - `feynkit-py` edit: sibling package artifacts remain reusable. - CFF edit: rebuild CFF and its actual consumers, preserving unaffected ancestors and siblings. - Model edit: broader rebuilding follows the dependency graph. - Embedded fixture edit: affected test binaries change; production artifacts remain stable. - Manifest/lockfile edit: report broad invalidation explicitly. Require matching test inventories, filters, features, Python behavior, doctests, and Clippy results. Resolve any still-expired license before accepting FeynKit runtime comparisons. ## Acceptance and rollout - Warm unchanged runs: **zero real Cargo recompilation**, targeting **80% less intermediate restored content** and **50% fewer worker minutes**. - Changed-source runs: target **30% fewer worker minutes**, with unaffected per-crate artifacts preserved and no repeated identical test-artifact builds across workers. - Cold completion: **no more than 10% slower**. Check changed-source and individual-group latency too; grouping must not substantially delay otherwise independent cached checks. - Final success: within **60 seconds** of the last intended check, with no unrelated preparation continuing. - Correctness and coverage are mandatory; performance targets are goals to verify, not promised results. Retain all existing Actions workflows. Add draft guards, ready-for-review triggering, manual execution, and cancellation of superseded PR runs while preserving branch filters and merge-queue coverage. NixCI draft suppression remains a separate integration question. Keep mechanical cleanup and performance changes in separate commits. Attach measurements to the main PR, merge the validated changes, then have active branches rebase and regenerate their branch-specific CI definitions.