metrics: typed report_result_invalid for overflow; annualised metrics under one day are null with a reason (rev 3.1, CR-202)

docker/report_annualization.def

@@ -0,0 +1,5 @@
+// Revision 3.1 policy, shared by the kernel and the public report writer.
+// Ship beside run_json.py, like its other companion resources.
+PF_ANNUALIZATION_SPAN_MS(86400000)
+PF_SHORT_SPAN_ANNUALIZED_FIELD(cagr)
+PF_SHORT_SPAN_ANNUALIZED_FIELD(calmar)

docker/run_json.py

@@ -54,7 +54,7 @@
         "longs":  { ...pf_trade_stats_t... },   # long trades only
         "shorts": { ...pf_trade_stats_t... },   # short trades only
         "equity": { ...pf_equity_stats_t... }   # sharpe/sortino/cagr/calmar/...
-      },                               # any NaN statistic -> null (see _num)
+      },                               # defined metric absence -> null (see _num)
       "equity_curve": [                # ABI v2: one point per script bar
         { "time_ms": int, "equity": float, "open_profit": float },
         ...
@@ -129,7 +129,8 @@
 
 NaN convention: any metric with an empty/zero denominator is null (JSON has no
 NaN); a real computed 0 stays 0. See the report-schema + metrics reference docs
-for the per-field meaning of every metrics.* key.
+for the per-field meaning of every metrics.* key. Overflow and other computed
+nonfinite report values fail as report_result_invalid, reason nonfinite.
 
 Selected-window mode (selected-window wire v1.3, full/v1 report, no digest). Opt in
 with --report-policy selected-window/v1, --window-start-ms T, --window-end-ms E,
@@ -2210,6 +2211,13 @@ def failure_line(text, code=None, code_args=None) -> str:
             (_cut_utf8(_no_surrogates(value), ERROR_ARG_TEXT_MAX)
              if isinstance(value, str) else value)
             for name, value in (code_args or {}).items()}
+        if code == "report_result_invalid":
+            # Translate native metric fields at the public wire boundary.
+            field = doc["args"].get("field")
+            if isinstance(field, str):
+                prefix, separator, name = field.rpartition(".")
+                if prefix == "metrics.equity":
+                    doc["args"]["field"] = prefix + separator + EQUITY_REPORT_KEYS.get(name, name)
     line = _dump_line(doc)
     if len(line) > ERROR_LINE_MAX:
         if code is not None and len(_dump_line({**doc, "error": ""})) > ERROR_LINE_MAX:
@@ -3543,12 +3551,13 @@ def fmt_utc(ms: int) -> str:
         ms / 1000, tz=timezone.utc).strftime("%Y-%m-%d %H:%M UTC")
 
 
-def _num(x):
-    """JSON-safe float. The engine's metric NaN convention (empty / zero
-    denominator -> NaN, never 0) cannot survive JSON: json.dump emits a bare
-    `NaN` token that a strict downstream JSON.parse (the MCP layer) rejects.
-    Collapse every non-finite double to null so the report stays valid JSON."""
+def _num(x, *, report_metric=False):
+    """Native metric NaN is the producer's explicit defined-absence sentinel.
+    Preserve infinity until _validate_report_result can raise the typed R.
+    Recorded script outputs have their separate nonfinite-to-null convention."""
     f = float(x)
+    if report_metric:
+        return None if math.isnan(f) else f
     return f if math.isfinite(f) else None
 
 
@@ -3559,16 +3568,54 @@ def _num(x):
 EQUITY_REPORT_KEYS = {"sharpe_monthly": "sharpe_tv", "sortino_monthly": "sortino_tv"}
 
 
+# Revision 3.1 policy. docker/report_annualization.def holds the same two
+# facts for the C++ metric walk; run_json.py ships as a single file, so it
+# keeps its own copy and tests/test_run_json_nonfinite_report.py pins the two.
+MIN_ANNUALIZATION_SPAN_MS = 86_400_000
+SHORT_SPAN_ANNUALIZED_FIELDS = ("cagr", "calmar")
+
+
+def _annualization_diagnostics(curve, metrics):
+    """Explain only the revision 3.1 absence, from native report observations.
+
+    net_profit_pct's existing domain witnesses positive initial capital even
+    when the first equity mark differs from it or provenance is unresolved.
+    Python's integer subtraction is exact across the full int64 timestamp
+    domain; convert the difference to double just as the kernel does.
+    """
+    if (not curve or metrics["all"]["net_profit_pct"] is None
+            or not math.isfinite(metrics["all"]["net_profit_pct"])
+            or not math.isfinite(curve[-1]["equity"]) or curve[-1]["equity"] <= 0):
+        return {}
+    first, last = curve[0]["time_ms"], curve[-1]["time_ms"]
+    span_ms = float(last - first) if last >= first else -float(first - last)
+    if not 0 < span_ms < MIN_ANNUALIZATION_SPAN_MS:
+        return {}
+    return {
+        EQUITY_REPORT_KEYS.get(field, field): "span_too_short_to_annualize"
+        for field in SHORT_SPAN_ANNUALIZED_FIELDS
+        if metrics["equity"][EQUITY_REPORT_KEYS.get(field, field)] is None
+    }
+
+
+def _annualization_block(curve, metrics) -> dict:
+    """The diagnostics entry for revision 3.1: present only when the rule fired."""
+    reasons = _annualization_diagnostics(curve, metrics)
+    return {"annualization": reasons} if reasons else {}
+
+
 def _stats_dict(s) -> dict:
     """Serialize a pf_trade_stats_t / pf_equity_stats_t ctypes struct to a dict,
     keying off each field's ctype: integer counters stay ints, every double is
-    sanitized through _num. Driven by _fields_ so it tracks the struct verbatim;
+    normalized through _num's metric mode. Computed nonfinite values must
+    already have failed native M checks; infinities also fail the C backstop.
+    Driven by _fields_ so it tracks the struct verbatim;
     an equity struct's two monthly ratios take their EQUITY_REPORT_KEYS."""
     keys = EQUITY_REPORT_KEYS if isinstance(s, EquityStatsC) else {}
     out = {}
     for name, ctype in s._fields_:
         v = getattr(s, name)
-        out[keys.get(name, name)] = _num(v) if ctype is ctypes.c_double else int(v)
+        out[keys.get(name, name)] = _num(v, report_metric=True) if ctype is ctypes.c_double else int(v)
     return out
 
 
@@ -3613,9 +3660,18 @@ def build_report_dict(report: ReportC, ohlcv_path: Path,
 
     cum, peak, max_dd = 0.0, 0.0, 0.0
     for p in pnls:
+        if not math.isfinite(p):
+            raise RunFailure("the report result trades.pnl is not finite",
+                             "report_result_invalid",
+                             {"field": "trades.pnl", "reason": "nonfinite"})
         cum += p
         peak = max(peak, cum)
-        max_dd = min(max_dd, cum - peak)
+        drawdown = cum - peak
+        if not math.isfinite(cum) or not math.isfinite(drawdown):
+            raise RunFailure("the report result summary.max_drawdown is not finite",
+                             "report_result_invalid",
+                             {"field": "summary.max_drawdown", "reason": "nonfinite"})
+        max_dd = min(max_dd, drawdown)
 
     # Computed trading metrics (ABI v2): all/longs/shorts trade stats + the
     # equity-curve-derived block (sharpe/sortino/cagr/calmar/...). See the
@@ -3636,8 +3692,8 @@ def build_report_dict(report: ReportC, ohlcv_path: Path,
             p = report.equity_curve[i]
             equity_curve.append({
                 "time_ms":     int(p.time_ms),
-                "equity":      _num(p.equity),
-                "open_profit": _num(p.open_profit),
+                "equity":      float(p.equity),
+                "open_profit": float(p.open_profit),
             })
 
     return {
@@ -3674,6 +3730,7 @@ def build_report_dict(report: ReportC, ohlcv_path: Path,
             "magnifier_sub_bars_total":     int(report.magnifier_sub_bars_total),
             "magnifier_sample_ticks_total": int(report.magnifier_sample_ticks_total),
             "bar_magnifier_enabled":        bool(report.bar_magnifier_enabled),
+            **_annualization_block(equity_curve, metrics),
         },
         "trades": trades,
         "metrics": metrics,
@@ -3682,9 +3739,9 @@ def build_report_dict(report: ReportC, ohlcv_path: Path,
 
 
 def _validate_report_result(out: dict) -> None:
-    """Canonical trade and summary doubles have no defined non-finite absence.
-    Keep the failure fields as engine-owned literals; metrics, equity and
-    recorded outputs retain their existing _num-to-null convention."""
+    """Required results are finite; only native metric absence is nullable.
+    Metric keys come from the producer's ctypes descriptors and alias map,
+    after native computation has distinguished absence from computed NaN."""
     for trade in out["trades"]:
         for key, field in (
             ("entry_price", "trades.entry_price"),
@@ -3701,6 +3758,7 @@ def _validate_report_result(out: dict) -> None:
                                  "report_result_invalid",
                                  {"field": field, "reason": "nonfinite"})
     for key, field in (
+        ("win_rate_pct", "summary.win_rate_pct"),
         ("net_pnl", "summary.net_pnl"),
         ("avg_trade", "summary.avg_trade"),
         ("best_trade", "summary.best_trade"),
@@ -3711,6 +3769,22 @@ def _validate_report_result(out: dict) -> None:
             raise RunFailure(f"the report result {field} is not finite",
                              "report_result_invalid",
                              {"field": field, "reason": "nonfinite"})
+    for block in ("all", "longs", "shorts", "equity"):
+        for name, value in out["metrics"][block].items():
+            if value is not None and not math.isfinite(value):
+                metric_field = f"metrics.{block}.{name}"
+                raise RunFailure(f"the report result {metric_field} is not finite",
+                                 "report_result_invalid",
+                                 {"field": metric_field, "reason": "nonfinite"})
+    for point in out["equity_curve"]:
+        for key, field in (
+            ("equity", "equity_curve.equity"),
+            ("open_profit", "equity_curve.open_profit"),
+        ):
+            if not math.isfinite(point[key]):
+                raise RunFailure(f"the report result {field} is not finite",
+                                 "report_result_invalid",
+                                 {"field": field, "reason": "nonfinite"})
 
 
 def _request_invalid(text: str, option: str, exit_status: int = 1) -> RunFailure:

docs/adr/0001-kernel-adapter-boundary.md

@@ -86,7 +86,7 @@ summary of it.
   `NativeCoordinate::interval_index` (`market_driver.hpp:74`) is the sequential
   SCRIPT-bar index even when `input_tf` is finer than `script_tf`. The same
   script coordinate books `PyramidEntry::entry_bar_index`, trade-row indices and
-  the inclusive bar metrics (`engine_orders.cpp:113-114`, `engine_metrics.cpp:140`).
+  the inclusive bar metrics (`engine_orders.cpp:113-114`, `engine_metrics.cpp:164`).
   `NativeCoordinate::input_interval_index` (`market_driver.hpp:79`) names the
   input slot that supplied the point for a host that needs input cadence. Both
   coordinate values are folded by `hash_coordinate`
@@ -172,7 +172,7 @@ summary of it.
   (`guard_native_mutation` `engine_aux_security.cpp:83`).
   Indicators are `ta_*.cpp`, session/calendar `session_time.cpp` / `native_calendar.cpp`, FX
   `configure_native_fx_curve` (`native_host.hpp:1202`).
-- **Reporting.** `fill_report` (`engine.hpp:1859`, `src/engine_report.cpp:56`) gives a bare host
+- **Reporting.** `fill_report` (`engine.hpp:1859`, `src/engine_report.cpp:58`) gives a bare host
   trade stats, and the equity curve is a run-spec choice rather than a host chore.
   `NativeReportPolicy::HostRecorded` (the default) leaves the series to the host, whose
   `record_equity_point` / `update_equity_extremes` remain protected
@@ -181,8 +181,8 @@ summary of it.
   `KernelRecordedAtHostMarks` records the same series at the host's own marks.
   `report_open_position_at_end` books the still-open position as a mark-to-market row under
   `KernelRecorded`. Under `HostRecorded` an unrecorded curve still degenerates —
-  `compute_equity_stats` answers its all-NaN value with no error (`src/engine_metrics.cpp:203`,
-  consumed by `fill_metrics_section`, `src/engine_report.cpp:138-145`) — which is why the policy exists.
+  `compute_equity_stats` answers its defined empty-curve metrics (`src/engine_metrics.cpp:228`,
+  consumed by `fill_metrics_section`, `src/engine_report.cpp:203`) — which is why the policy exists.
 - **Reproducibility.** Two digests, one wrapping the other. `native_continuation_hash()`
   (`native_host.hpp:1376`) returns the consumer's event/continuation hash. **A digest names
   the run's inputs** (rule 2): the timezone identity enters it as the zone's *content* —
@@ -761,10 +761,10 @@ number names its symbol on this tree.
 | `NativeReportPolicy::KernelRecordedAtHostMarks` (`native_run_spec.hpp:64`) | **C++-only** (lane E22's ruling, restated) | Under it the host names each report point from inside its own callbacks, and the C table has no call that marks one: a C host could only declare a series nobody records. `pf_native_report_policy_e` (`native_c_api.h:661`) leaves its value unnamed, and the enum guard holds that exclusion. A C++ host now reaches the same kernel producer through `NativeStrategyHost::mark_native_report_point`; a mark outside a running HostMarks run returns false. |
 | `strategy_set_path_order` (`pineforge.h:948`) clamps an out-of-range mode to AUTO | **retained** | A `void` ABI v4 setter cannot answer a refusal. Its values are now documented as `pf_native_path_order_e`'s (`native_c_api.h:904`); a C host that wants an unknown word refused declares `path_order` in `pf_native_run_spec_ext_v1`, which answers `PF_NATIVE_E_TAG`. |
 | the enumerated `int32` fields of `pf_pending_order_v1_t` (`strategy_pending_order_get` `pineforge.h:1096`) | **no C names in the header; generator-owned** | The mirror is generated from the Pine source host's resting-order record by `scripts/gen_pending_order_mirror.py` (held by its `--check` source guard) and self-described at run time by `strategy_pending_order_layout`. Its enumerated fields are the source layer's order vocabulary, not a kernel enumeration. A C name for them belongs in that generator, which owns the struct, so that it cannot drift from the mirror — never hand-written beside it. |
-| `strategy_configure_native_v1` (`c_abi.cpp:980`) versus `strategy_configure_native_ext_result_v1` (`native_c_host.cpp:4066`) | **retained legacy base plus typed additive route** | A base specification the kernel rejects still returns `-1` and latches `Failed`, preserving its frozen caller contract; a Ready misuse retains that Contract latch. The C boundary refuses arguments, a Running phase and a Completed hash hook without entering kernel configure, so a read cannot reconfigure its host. The extended result call validates before its first configure, writes `pf_native_spec_error_t` / `pf_native_spec_field_t`, and names reused-host session-key and run-number refusals; no old signature or layout moved. |
+| `strategy_configure_native_v1` (`c_abi.cpp:980`) versus `strategy_configure_native_ext_result_v1` (`native_c_host.cpp:4091`) | **retained legacy base plus typed additive route** | A base specification the kernel rejects still returns `-1` and latches `Failed`, preserving its frozen caller contract; a Ready misuse retains that Contract latch. The C boundary refuses arguments, a Running phase and a Completed hash hook without entering kernel configure, so a read cannot reconfigure its host. The extended result call validates before its first configure, writes `pf_native_spec_error_t` / `pf_native_spec_field_t`, and names reused-host session-key and run-number refusals; no old signature or layout moved. |
 | `strategy_configure_native_fx_curve_v1` (`pineforge.h:546`) | **typed additive route** | `strategy_configure_native_fx_curve_ext_v1` writes the C twin of `NativeFxCurveValidation` and its offending index. The old `-1` route remains unchanged, while a C caller can now distinguish a decreasing timestamp, non-positive rate, allocation failure or wrong phase. |
 | `on_native_timeframe_bar` (`native_host.hpp:885`) and `NativeTimeframeBarContext::interval` (`native_host.hpp:809`) | **query routed** | The callback signature stays frozen. `strategy_native_timeframe_bar_interval_v1` is valid only during that callback and copies the kernel calendar interval without deriving it from the delivered bar or delivery timestamp. |
-| `magnifier_sub_bars_total` / `magnifier_sample_ticks_total` (`engine_report.cpp:75-76`) | **kernel-owned report facts** | The generic intrabar driver already increments `NativeDriverStatistics::sub_bars_processed` / `sample_ticks_processed` for every retained path (`native_execution_consumer.cpp:8681-8693`); the kernel now publishes those same counters for bare hosts. They are report fields, not new durable state: the existing `native_continuation_hash` fold of `NativeDriverStatistics` (`native_execution_consumer.cpp:1125-1130`) sees the same values, so no hash contract moves. |
+| `magnifier_sub_bars_total` / `magnifier_sample_ticks_total` (`engine_report.cpp:77-78`) | **kernel-owned report facts** | The generic intrabar driver already increments `NativeDriverStatistics::sub_bars_processed` / `sample_ticks_processed` for every retained path (`native_execution_consumer.cpp:8681-8693`); the kernel now publishes those same counters for bare hosts. They are report fields, not new durable state: the existing `native_continuation_hash` fold of `NativeDriverStatistics` (`native_execution_consumer.cpp:1125-1130`) sees the same values, so no hash contract moves. |
 | `pf_native_lot_excursion_v1` (`native_c_api.h:1912`) | **additive fee tail** | `entry_commission` carries the C++ `ClosedLotExcursionFacts::entry_commission` in account currency. The callback table publishes a trailing layout marker so F4 callers still receive the base facts length; current callers receive the fee tail without changing `PF_NATIVE_API_VERSION`. |
 | `strategy_outputs_set_enabled` and the eight `strategy_outputs_*` readers (`pineforge.h`, group `pf_outputs`) | **additive runtime exports** | Recording is off by default, and nothing a run computes reads the record. The event copy is size-prefixed like `strategy_pending_order_get`, and `PF_ABI_VERSION` stays 4. The writers are C++ only in this version (the BASE-CLASS SEAMS rows of `native_c_api.h`): a C host declares no outputs, so its handle answers every reader as empty. |