PDL-JSON-2: Strict-JSON Migration Plan for Public summarize_* Functions#

Status: OPEN debt ticket. Filed in v0.31b0. Companion to PDL-JSON-1 which lists the inventory. Individual migrations execute per the schedule below; there is no bulk-migration release.

Migration rules#

  1. Composed summarizers migrate WITH the release that adds fields to them. When a release lands new fields on a summarizer that embeds sub-summaries, that summarizer migrates to strict validation as part of the same PR. Example: summarize_generator_confidence migrates in v0.32b, when the additive method_scores, uncertainty_report, and calibration_report fields land. The rationale is that new-field PRs are already touching the payload shape and its tests, so folding the strict-validation flip into that PR minimises churn and keeps the strict boundary aligned with the point at which the schema actually changes.

  2. Leaf summarizers migrate on their own rolling schedule — one per release maximum — to avoid regression flooding. Leaf summarizers (no embedded sub-summaries) can migrate independently, but the queue is throttled to at most one leaf migration per minor release so that any regression introduced by a strict-flip is easy to bisect and revert without dragging unrelated migrations along with it.

  3. Every migration PR must ship a per-summary NaN adversarial test. The PR must include a regression test that injects float("nan") into a nested field of the summarizer’s payload and asserts the strict-validated version raises SchemaValidationError. See the “Regression-test template” section below. The test lives beside the summarizer’s existing unit tests and is named test_<summary_name>_rejects_nan_strict.

  4. Every migration PR must document a “known-emit-NaN” audit. The PR description must enumerate every place in the current codebase that could emit NaN into the migrated summary’s inputs (e.g. divisions by zero, np.nanmean over empty slices, missing-value sentinels, upstream fit residuals). For each emitter the PR must either (a) fix the emitter so it cannot produce NaN on the migrated payload’s path, or (b) document a specific, referenced reason the migration is safe despite the emitter (guarded upstream, filtered before summarisation, structurally impossible for this call site, etc.). “No known emitters” is an acceptable audit outcome only if the PR author has actually looked.

  5. No migration may weaken any existing v0.NN release-gate test. If a v0.NN release-gate test relies on the permissive contract for a payload it constructs — e.g. it hand-builds a summary with a NaN in it and expects the summariser to swallow it — that is a bug in the release-gate test and must be fixed in the migration PR. The correct fix is to update the test to construct a non-NaN payload (or to assert SchemaValidationError on the NaN path); the wrong fix is to make the summariser permissive again to keep the old test green.

Per-summary migration schedule#

Function

Target release

Trigger

Est. size

summarize_downstream_discovery_workflow

v0.31b1

Embedded in TaskResult via v0.31 task-bridge composition; must be strict since underlying_discovery_result is strict.

S

summarize_generator_confidence

v0.32b

Additive method_scores / uncertainty_report / calibration_report fields land under strict validation.

M

summarize_vertical_slice

v0.32.5 or v0.33

Composed summary embedding 3 sub-summaries; migrates after its embedded producers.

L

summarize_residual_batch

v0.32.5

Leaf; independently migratable.

S

summarize_generator_family

v0.33

Leaf.

S

summarize_verification_report

v0.33

Leaf.

S

summarize_generator_fit_diagnostics

v0.33

Leaf.

S

summarize_field_batch_readiness

v0.34a

Leaf.

S

summarize_invariant_workflow

v0.34a

Composed; embeds invariant sub-payloads.

M

summarize_split_leakage_provenance

v0.34a

Leaf.

S

summarize_formula_generator_family

v0.35a

Leaf.

S

summarize_weak_residual_report

v0.35a

Leaf.

S

Size legend: S = single-file summariser + its unit tests; M = summariser plus one or two adjacent producers touched by the field-addition; L = composed summariser that cannot migrate until all embedded producers have migrated first.

Regression-test template#

Every migration PR adapts the following skeleton. Substitute <summary_name> with the function being migrated, <payload_fixture> with a valid fixture the summariser accepts today, and <nested_field_path> with a real nested numeric field in that payload.

# tests/summarizers/test_<summary_name>_strict.py

import math
import copy

import pytest

from pdelie.serialization.errors import SchemaValidationError
from pdelie.summarizers import <summary_name>


def _valid_payload():
    """Return a fresh, fully-populated payload the summariser accepts today."""
    return copy.deepcopy(<payload_fixture>)


def test_<summary_name>_accepts_valid_payload():
    """Baseline: strict validation must still pass on a clean payload."""
    result = <summary_name>(_valid_payload())
    assert result is not None


def test_<summary_name>_rejects_nan_strict():
    """
    Adversarial: inject NaN into a nested numeric field and assert the
    strict-validated summariser raises SchemaValidationError rather than
    silently passing NaN through into downstream JSON.
    """
    payload = _valid_payload()

    # Walk to the injection point. Keep this explicit — do not hide it behind
    # a helper — so future readers can see exactly which field is being
    # poisoned and why the summariser must reject it.
    target = payload
    for key in <nested_field_path>[:-1]:
        target = target[key]
    target[<nested_field_path>[-1]] = float("nan")

    with pytest.raises(SchemaValidationError) as excinfo:
        <summary_name>(payload)

    # Assert the error message mentions the offending field so operators can
    # trace it back to the emitter without re-running under a debugger.
    assert <nested_field_path>[-1] in str(excinfo.value)


@pytest.mark.parametrize("bad_value", [float("inf"), float("-inf")])
def test_<summary_name>_rejects_non_finite_strict(bad_value):
    """
    Optional companion: strict JSON also disallows +/-inf. Include this
    parametrisation if the migrated payload has any field where a runaway
    fit or division could plausibly produce inf.
    """
    payload = _valid_payload()
    target = payload
    for key in <nested_field_path>[:-1]:
        target = target[key]
    target[<nested_field_path>[-1]] = bad_value

    with pytest.raises(SchemaValidationError):
        <summary_name>(payload)

Notes on adapting the template:

  • The <nested_field_path> MUST be a genuinely nested field, not a top-level key. The whole point of the strict flip is to catch NaN that escapes through deep sub-payloads, so a top-level injection would under-test the migration.

  • If the summariser is composed (embeds sub-summaries), add a second injection test that poisons a field inside an embedded sub-summary to prove strict validation recurses.

  • If the summariser accepts an optional field, do not use that field as the injection site — poison a required field so the failure cannot be attributed to the value being dropped.

Cross-references#