By: ValentinHirschi <valentin.hirschi@gmail.com>
monitor componentwise absolute integrands User request: PLEASE IMPLEMENT THIS PLAN: # Componentwise absolute-integrand monitoring ## Summary - Replace the RMS proposal entirely with always-on monitoring of `⟨|Re I|⟩` and `⟨|Im I|⟩`; do not compute the complex modulus or `√⟨I²⟩`. - Preserve the existing signed integration, sampling, targets, stopping rules, PDFs, and RNG sequence. - Track two extra overall accumulators plus shallow accumulators for only the existing monitored discrete level—never duplicate the full continuous/nested grid. ## Implementation changes - Extend the existing complex accumulator with absolute real/imaginary `StatisticsAccumulator`s. For each fully weighted sample, add `x` to the signed accumulator and `|x|` to the absolute accumulator; merge, finalize, preview, serialize, and resume all four together. - For discrete monitoring, resolve the existing `monitored_discrete_path` and create ephemeral shallow Havana grids containing only that level’s bins, with no children or continuous dimensions. - Copy the production grid’s current bin PDFs before each iteration. - Traverse each production sample to the monitored level and call `add_training_sample(local_sample, abs(effective_component))`; do not pre-apply the local sample weight. - Merge shallow worker grids into persisted one-level real/imaginary absolute summaries before updating the production grid. - Never sample from, adapt, or persist these shallow grids; rebuild them from current production PDFs each iteration. - Treat grid/sample/path mismatches as internal errors rather than silently dropping samples. - Absolute `All` statistics come from the overall accumulators; `Bin` statistics come from the shallow summaries; `Sum` adds bin means and combines errors in quadrature. Sample fractions, counts, PDFs, labels, and fixed coordinates remain shared. - Keep signed and absolute values as an orthogonal view dimension in the renderer-neutral status model. Each result cell carries signed and absolute value/error/%error/χ²/MWI data, while sampling metadata remains single-copy. Absolute max-weight sections contain one unsigned maximum per component with the winning coordinates. - Update the living Ratatui design document with the new mode and accumulation constraints. ## UI and result interfaces - Add a global Ratatui `a`/`A` toggle, defaulting to `⟨I⟩`, and retain it across tabs and dashboard suspend/resume. - Switch the ribbon, chart and complete history, focused results, summaries, discrete values/sorting/details, χ², MWI, and max-weight panels. - Label modes explicitly as `⟨I⟩` and `⟨|I|⟩`; use `re`/`im` versus `|re|`/`|im|`. - Preserve the full convergence plot and uncertainty bands in absolute mode, but omit its yellow target line. Hide target values and delta columns while retaining an explicitly signed `ETA to ⟨I⟩ target`, since that still controls stopping. - Keep timing, precision, stability, event statistics, and training behavior unchanged. - Keep canonical tabled output signed except for one continuation row after each overall component row: `All/re`, then blank-contribution `|re|`; likewise for imaginary. - Populate only each slot’s formatted absolute value-with-error cell. - Leave absolute `%err`, χ², MWI, target, sample/PDF, Sum/Bin, max-weight, and statistics output blank or absent. - Add an additive `absolute: AbsoluteIntegrationResult` field to each persisted slot: - `integral: IntegralEstimate` - `table_results` - `max_weight_info` - one-level `grid_breakdown` - Keep targets and integration statistics at slot level and all existing signed fields unchanged. Absolute target deltas are always `None`; max-weight entries contain at most one positive entry per component. - Mirror the bundle in the Python API as `slot.absolute`, retaining existing attributes and signed `Display` behavior. Both latest and per-iteration JSON snapshots receive the full absolute bundle automatically. ## Test plan - Validate signed versus absolute mean, error, χ², MWI, zeros, cancellation, negative-only samples, max coordinates, core merging, live previews, and uninterrupted versus resumed runs. - Cover monitored paths `[]`, nested singleton prefixes such as `[0]`, explicit one-bin roots, nonuniform and changing PDFs, correlated/uncorrelated slots, additive absolute `Sum`, and absence of descendant absolute summaries. - Ratatui buffer tests cover `a`/`A`, every tab, full-history switching, active-view sorting, stable logical bin selection, signless maxima, target hiding, retained signed ETA, and toggling back. - Tabled tests assert row order, phase filtering, value/error-only absolute cells, and no duplicated panels or discrete rows. - Verify JSON latest/archive round trips, Python conversion, and that absolute monitoring does not change signed results, production grids, PDFs, or RNG sequences. Run formatting, targeted integration/UI tests, broader workspace tests, and clippy. ## Assumptions - The feature applies to the current `havana_integrate` path used by the `integrate` CLI and Python integration command; the legacy distributed `MasterNode` path remains unchanged. - Absolute monitoring is always enabled and has no new CLI/configuration flag. - No Numerica/Havana dependency change is required. - Existing binary integration checkpoints are incompatible with the expanded accumulator state and require `--restart`; no migration fallback will be added.