PARITY & FUNCTION SUMMARY FOR MANAGEMENT
Prepared: April 2, 2026 (updated May 12, 2026 — R497 closure refresh covering R395–R497)
For: Yggdrasil Rust Cardano Node Team
Status: 394+ parity rounds completed; Phase D.1 code-level rollback sidecar hardening, Phase E.1 upstream pin refresh (R245 + R249 cumulative refresh — all 6 documentary pins in-sync against live HEAD), Byron genesis hash preflight parity, the latest Conway BBODY header-protocol-version drift, the observed preview Plutus well-formedness/runtime replay blockers, the preview-side TPraos active-overlay VRF blocker, the parallel-test YGG_LSQ_ERA_FLOOR isolation, R250 perf foundation (40% throughput improvement), R251 Gap BQ (indefinite-length CBOR span extraction), the R273-rename + R274–R311 strict 1:1 file-mirror arc, the R313–R320 docstring-classification cleanup ((c) strict-partial bucket eliminated entirely), the R326–R336 sister-tools port arc Phase A skeleton milestone (12/12 sister-tool deployable binaries with byte-equivalent –help/–version; bech32 verified_11_0_1), the R338–R345 cardano-submit-api Phase A.2 implementation arc, the R347–R350 db-truncater Phase B.1 implementation arc, the R351–R359 typed-config sweep, the R361–R367 typed-parser sweep, the R369–R376 deeper-layer sub-arc (BenchmarkLedgerOps trio + HasAnalysis trait + per-tool extensions), and the R378–R394 cardano-tracer subsystem build-out (R378 db-synthesizer Orphans; R379 cardano-tracer Time; R380-R381 Notifications/{Types, Check} + SeverityS synthesis; R382 Logs/Journal pair; R383 Handlers/System path-resolution; R384-R385+R387 Notifications/{Settings, Utils — full surface}; R386 Notifications/Timer full periodic-action scheduler; R388-R389 Notifications/{Email, Send} bounded subsets [SMTP send + makeAndSendNotification orchestration deferred]; R390 Logs/Utils; R391 Metrics/Utils; R392 workspace structure cleanup; R393 Environment.hs TracerEnv 14-field record [unblocks downstream subsystems]; R394 Logs/Rotator pure rotation policy helpers) are closed.
R395–R497 cardano-tracer DataPoint + db-analyser HasAnalysis arc (2026-05-10 → 2026-05-11): closed two large sister-tool sub-arcs end-to-end. R395–R474 cardano-tracer DataPoint sub-protocol arc: R459–R460 closed the R459 DataPoint-mux advisor flags (parity-matrix allowlist + DataPoint-mux integration smoke); R461–R467 ported the Logs Rotator IO orchestration + trace-objects file-write pipeline + trace-forwarder write_to_sink / read_from_sink_non_blocking; R464 wired runMetricsServers aggregator; R465 added per-connection HandleRegistry deregister hook; R466 closed before_program_stops + sequence_concurrently_; R468 added TLS termination via axum-server-rustls; R469–R470 built the DataPointRequestors registry + askNodeName closure + Acceptors spawn-body plumbing; R471–R473 ported Protocol/DataPoint/{Forwarder, Acceptor}.hs (data_point_forwarder.rs, data_point_run_forwarder.rs, DataPointStore + read_from_store); R474 shipped the DataPoint acceptor↔forwarder end-to-end integration test. R475–R497 db-analyser HasAnalysis arc (closed R481, extended R485–R497): R475 added per-era TxBody::decode_output_count + Tx::output_count dispatcher; R476 ported Ouroboros.Consensus.Byron.EBBs.hs::knownEBBs (325-entry registry) + impl HasAnalysis for Block; R477–R478 added Allegra/Mary/Alonzo + Babbage/Conway dispatch tests; R479 shipped analysis::runner dispatch core + 4 handlers; R480 added 3 more block-iteration handlers (ShowBlockTxsSize, ShowEBBs, OnlyValidation); R481 wired lib.rs::run to FileImmutable + runner + stdout renderer (closeout); R482 added ImmutableStore::iter_after streaming iterator; R485 made CheckNoThunksEvery a permanent carve-out (Haskell-only laziness debug); R486 enriched per-block event shapes for CountTxOutputs + ShowBlockHeaderSize; R488 shipped TraceLedgerProcessing handler via LedgerState::apply_block; R489 shipped BenchmarkLedgerOps via Instant timing + SlotDataPoint; R490 shipped GetBlockApplicationMetrics via R476 column closures; R491 shipped StoreLedgerStateAt via LedgerStateCheckpoint CBOR codec; R493 shipped ReproMempoolAndForge (closes 13/13 dispatch coverage); R494–R495 added per-era Tx::decode_inputs / decode_fee / decode_ttl for ReproMempoolAndForge forensic fidelity; R496 shipped Block::emit_traces body + TraceLedgerProcessing trace-event wiring; R497 shipped Tx::to_raw_tx_bytes (3-element pre-Alonzo / 4-element Alonzo+ wire-form CBOR), closing the last MempoolEntry forensic-fidelity placeholder — all 8/8 fields now real.
R273-rename + R274–R311 strict 1:1 file-mirror arc (2026-05-09): refreshed the vendored upstream tree to policy tag 11.0.1; landed a strict-mirror CI drift-guard (warn-only R275 → fail-build R288); swept every production .rs to either mirror an upstream .hs filename or carry an explicit ## Naming parity docstring stanza; purged all production #[allow(dead_code)] sites + the lone production TODO; expanded a new crates/cardano-cli/ workspace member mirroring the full upstream cardano-cli surface (~237 files); kicked off concrete migration via R296 (Version) + R297 (ShowUpstreamConfig); trimmed docs/ from 23 to 11 top-level markdown files (5 archived); landed two new validators (check-fixture-manifest.py + check-reference-artifacts.py); R310 fixed an over-broad .gitignore pattern that had silently swallowed 12 production .rs files in crates/cardano-cli/src/era_independent/debug/; R311 hardened check-strict-mirror.py with an index-vs-tree drift check so the same failure class can no longer surface as an opaque CI module-resolution error.
R313–R320 docstring-classification cleanup (2026-05-09): R313 census surfaced 41 (c) docstring present (unspecified) files with the non-canonical **Strict mirror (partial):** form. R314 fixed the audit regex + promoted 24 files to canonical **Strict mirror:** <upstream/path.hs> declarations. R315–R316 reclassified 11 files to canonical **Strict mirror:** none. synthesis form after content-vs-name audit (Yggdrasil-specific concerns where basename match was misleading). R317 merged multiplexer.rs into mux.rs as a 1:1 mirror of upstream Mux.hs (eliminated 1 .rs file, +1 (a) DIRECT_MIRROR). R318 split handshake.rs into handshake/{type,version,codec}.rs matching upstream Handshake/{Type,Version,Codec}.hs (+3 leaves, all (a) DIRECT_MIRROR). R319 split inbound_governor.rs to mirror upstream InboundGovernor.hs + InboundGovernor/State.hs (+1 leaf, both (a) DIRECT_MIRROR). R320 promoted the last 2 plutus partials (builtins.rs, machine.rs) to direct mirrors via docstring tighten with sibling-file rationale documented as implementation detail.
Workspace tests: 5,638 passing, 0 failing (post-R394; +782 since R325 baseline of 4,856; +195 since R377 closure refresh). Strict-mirror audit table: stable at 257 (a) DIRECT_MIRROR + 215 (c) strict-none = 472 graded files as of R367; new R369–R376 leaves are docstring-graded inline at the strict-mirror gate (gate stays clean). Audit table has exactly two canonical verdicts. R326–R336 sister-tools arc Phase A skeleton milestone: 12 sister-tool crates created with byte-equivalent –help/–version; bech32 fully verified_11_0_1. R338–R345 cardano-submit-api Phase A.2 (7 rounds → fully-functional binary): foundations + type bridges + trace surface + raw-tokio HTTP server + LocalTxSubmission wiring + Prometheus metrics + operator comparison harness. R347–R350 db-truncater Phase B.1 (4 rounds → fully-functional binary): storage-trait trim_after_slot extension + typed config + Run.hs equivalent + operator comparison harness. R351–R359 typed-config sweep + R361–R367 typed-parser sweep (14 rounds combined, 7 sister tools at full argv → typed-config → run-dispatch state). R369–R376 deeper-layer sub-arc (8 rounds): R369 dmq-node Configuration.hs::readConfigurationFile port (CLI/file/defaults left-priority merge); R370 kes-agent-control optFromEnv port (KES_AGENT_CONTROL_{PATH,RETRY_INTERVAL,RETRY_ATTEMPTS} env-var derivation + test-friendly from_env_lookup
Structured parity inventory:
docs/parity-matrix.jsonis the machine-readable Rust ↔ Haskell feature inventory; validate withpython3 scripts/check-parity-matrix.py. Thereference.tagtracks the latest IntersectMBO/cardano-node release (currently11.0.1). The R-round narrative below remains the canonical operational history; the matrix is the at-a-glance per-feature status board.
R249 live-sync verification of fees + pparam updates (2026-05-05): a 30-minute preview sync against the IOG bootstrap peer advanced from Origin through 1,462,057 slots and 16 epoch boundaries with zero FeeTooSmall rejections and automatic protocol-parameter updates auto-applied at three epochs (newEpoch=2/3/9 reporting pparamUpdatesApplied=1/1/5 from real on-chain PPUP proposals). Alonzo → Babbage hardfork crossed cleanly mid-run; pool retirement at newEpoch=9 processed a 500 M-lovelace deposit refund without error. The fee-validation hot path (validate_fee / validate_conway_fee) and the PPUP→active-pparam wiring are confirmed correct on real preview chain data well past R248’s slot 868,687 and R246’s slot 901,725 evidence ceilings.
R250 perf foundation (2026-05-05): peer-snapshot adoption + snapshot-vs-live-ledger gating split. Yggdrasil now ships the upstream Haskell-share peer-snapshot.json content (28 KB preview / 15 KB preprod / 152 KB mainnet) instead of a 321-byte placeholder, and the runtime emits snapshot peers as immediately-eligible bigLedgerPeers without waiting for useLedgerAfterSlot. Measured 40% throughput improvement on preview soak (1,653 → 2,321 slot/s).
R251 Gap BQ closure (2026-05-05): preview vkey witness rejection at slot ~1,525,024 traced to indefinite-length CBOR encoding in real on-chain Babbage blocks. extract_block_tx_byte_spans used strict dec.array() which rejected the encoding; failed extraction triggered re-serialization fallback that produced a wrong tx_body_hash. Fixed via array_begin() + collect_indefinite_or_definite_spans helper. Verified: 25-min preview soak resumed from saved checkpoint at slot 1,488,359 advanced cleanly to slot 1,557,718, zero verification errors.
Open as of R251 (deferred):
- Gap BO — preprod TPraos VRF parity gap at slot ~429,460 (deep in Shelley
d=1federation period). Forensic log preserved atdocs/operational-runs/2026-05-05-round-249-preprod-vrf-failure-slot-429460.log. R253-candidate investigation per R250 plan. - Gap BP — preview Plutus V2 cost-budget overrun at slot ~1,462,057: tx
7bb40e40…declares(mem=6_121_408, steps=1_657_962_006)but Yggdrasil’s CEK overshoots CPU by 306,309 (≈ 0.0185 %) at a 200-accumulated-step batched debit. Magnitude (~13 step charges out of ~72K total) points to per-StepKindcost-model rounding or step-kind miscount; per-AST-node spend pattern review found upstream parity, root cause likely in per-builtin parameterized cost or step-charging discipline at a specific V2 term shape. Forensic log preserved atdocs/operational-runs/2026-05-05-round-249-preview-plutus-v2-budget-gap-slot-1462057.log. R252-candidate investigation needs per-builtin trace comparison against upstreamCek/Internal.hs::stepAndMaybeSpend. Workaround:YGG_SKIP_PHASE2=1env var advances chain past Gap BP while preserving Phase-1 (fees, witnesses, UTxO state, pparam updates) validation. - Perf to 2× Haskell — current Yggdrasil 2,321 slot/s vs reference Haskell 5,296 slot/s (Yggdrasil now 0.44× Haskell post-R250, was 0.31× pre-R250). Remaining: governor outbound-connect path needs to actually promote the now-eligible snapshot peers to warm/hot for multi-peer BlockFetch dispatch; batched Ed25519 verify (
verify_batchis strict — no security weakening), pipelined CBOR decode, allocator tuning. R254-candidate. Phase A complete (7/7); Phase B mainnet sync + bidirectional P2P closed (R211+R213+R220+R221); Phase D.2 lifetime peer-stats + aggregate server bytes-out deliverable shipped (R222+R223+R224+R226+R234+R235+R237); Phase D.1 now includes rollback-depth observability, epoch-boundary-aware ledger replay, exact ChainDepState nonce/OpCert sidecar restore-and-replay, and runtime resume preservation of current-epoch pool block counts (R225+R237+R238+R246); Phase E.1 all 6 documentary pins are in-sync after the coordinatedcardano-basefixture refresh and thecardano-ledgerrefreshes (R201+R216+R239+R243+R245); R244 verifies Byron genesis files through upstream Canonical JSON hashing so all four configured preset genesis hashes are checked at startup; R245 mirrors upstream’s temporary testnet suppression ofHeaderProtVerTooHighuntil Dijkstra while preserving mainnet enforcement and the separateMaxMajorProtVercap; R246 refscan reaches preview slot901725with rawPlutusBinarywell-formedness, Babbage/Conway reference-input ordering, CEK memory, validity-interval parity, arbitrary-precision PlutusInteger, PlutusserialiseData, and legacy registration-certificate redeemer/witness handling fixed; R247 fixes the verified-sync Origin BlockFetch prefix window so a clean preview replay stores the slot-0 prefix and advances to slot101100without the prior missing-UTxO stop; R248 mirrors upstreampbftVrfChecksfor TPraos active overlay slots and advances live preview past the former slot106220VRF failure and prior730728/840719Plutus stops to Babbage slot868687. A bounded live preview run reached checkpoint1038614before exposing stale reward state from pre-fix checkpoints, not a Plutus failure. Multi-network operational evidence: 25/25 cardano-cliconway querysubcommands on preview withYGG_LSQ_ERA_FLOOR=6(R205); 6/6 baseline queries on preprod (R207); mainnet sync + full cardano-cli LSQ surface verified end-to-end (R211 + R212 + R213).cargo fmt --all -- --check,cargo check-all,cargo test-all, andcargo lintpass after the latest R248 patches; the focused R247 BlockFetch-prefix regression, bounded clean preview replay, and focused R248 TPraos overlay tests pass. Remaining gates are clean/repaired preview replay past the stale-checkpoint stop and operator-side long-duration rehearsal/sign-off.
Current Implementation Status (1-Sentence Per Subsystem)
| Subsystem | Status | Completeness |
|---|---|---|
| Cryptography | All validation primitives (Ed25519, VRF, BLS12-381, secp256k1) fully wired and tested | ✅ 100% |
| Ledger Types | All 7 eras (Byron→Conway) with complete CBOR codec and multi-era UTxO model | ✅ 100% |
| Ledger Rules | Core validation + epoch boundary + governance ratification + network address validation + Conway deposit/refund parity + dormant epoch tracking + PPUP complete | ✅ 98% |
| Consensus | Praos validation + chain state + rollback enforcement + nonce evolution + VRF/KES complete | ✅ 100% |
| Network Protocols | All 5 mini-protocols + mux + handshake fully functional with typed clients/servers; per-state protocol time limits on both server and client sides | ✅ 100% |
| Peer Management | Governor with dual churn, big-ledger evaluation, in-flight tracking, exponential backoff, forget-cold-peers, PickPolicy randomized selection, connection manager lifecycle, inbound governor | ✅ 100% |
| Mempool | Fee-ordered queue + TTL + eviction + collateral + ExUnits + conflict detection + cross-peer TxId dedup + ledger revalidation (syncWithLedger) + epoch revalidation | ✅ 100% |
| Storage | Immutable/volatile/checkpoint stores with GC, slot lookup, corruption resilience, active crash recovery, fsync durability, ChainDB promote-stable-blocks | ✅ 100% |
| Plutus | CEK machine (88 builtins, V1/V2/V3), 16 cost expression shapes, parameterized cost model, PlutusBinary CBOR-bytestring deserialization, ScriptContext per-version encoding parity |
✅ 99% |
| Block Production | Credential loading, VRF leader election, KES evolution, header forging, self-validation, adoption tracing, slot clock loop | ✅ 100% |
| CLI & Config | JSON+YAML config loading + genesis loading + topology file loading + query/submit wrappers complete | ✅ 100% |
| Monitoring | NodeMetrics (40+ counters/gauges/histograms) + Prometheus + coloured stdout + detail levels + upstream backend recognition + Forwarder socket transport | ✅ 98% |
Overall Node Readiness: ~99% (can sync official networks, validates blocks correctly, comprehensive monitoring with trace forwarding wired, 249 parity rounds covering 745+ upstream rule areas verified with zero open code-level parity blockers)
Quick Function Inventory
✅ Fully Implemented & Tested
Ledger:
apply_block()— Multi-era block application with UTxO state updateapply_epoch_boundary()— Stake snapshots, pool retirement, governance ratification+enactment, governance expiry, MIR application, committee state pruningenact_gov_action()— Conway governance enactment (all 7 action types)prune_non_members()— Epoch-boundary committee state cleanup: removes hot-key authorization entries for cold credentials no longer in the active committee (upstreamupdateCommitteeStateviaMap.intersection creds members)accumulate_mir_from_certs()— MIR certificate accumulation (Shelley–Babbage, DCert tag 6)- MIR certificate admission validation —
MirValidationContextenforces all 7 upstream DELEG MIR checks:MIRCertificateTooLateinEpochDELEG(timing),MIRNegativesNotCurrentlyAllowed(pre-Alonzo negative deltas),MIRProducesNegativeUpdate(Alonzo+ combined map),InsufficientForInstantaneousRewardsDELEG(pot balance),MIRTransferNotCurrentlyAllowed(pre-Alonzo transfers),MIRNegativeTransfer,InsufficientForTransferDELEG(transfer pot balance); era-gated viahardforkAlonzoAllowMIRTransfer InstantaneousRewards— Per-credential MIR state + pot-to-pot delta tracking with CBOR round-tripvalidate_witnesses_if_present()— Ed25519 signature + hash verificationvalidate_native_scripts_if_present()— Timelock script evaluationvalidate_output_network_ids()— WrongNetwork check (all eras)validate_withdrawal_network_ids()— WrongNetworkWithdrawal check (all eras)validate_tx_body_network_id()— WrongNetworkInTxBody check (Alonzo+)compute_stake_snapshot()— Per-pool reward slot calculationaccumulate_donation()/flush_donations_to_treasury()— Conway treasury donation accumulation (UTXOS rule) + epoch-boundary flush (EPOCH rule)update_dormant_drep_expiries()— Conway dormant epoch DRep activity bump when proposals appear (upstreamupdateDormantDRepExpiries)validate_conway_current_treasury_value()— Conway currentTreasuryValue field validation- Conway deposit validation —
IncorrectDepositDELEG,IncorrectKeyDepositRefund,DepositIncorrectDELEG(PV > 10),RefundIncorrectDELEG(PV > 10),DrepIncorrectDeposit,DrepIncorrectRefund,WithdrawalNotFullDrain(exact-drain); upstreamharforkConwayDELEGIncorrectDepositsAndRefundsgate at PV major > 10 - Conway unelected committee voters —
validate_unelected_committee_voters()enforced only at PV > 10 (upstreamharforkConwayDisallowUnelectedCommitteeFromVoting) - Withdrawal/cert ordering parity —
apply_certificates_and_withdrawals_with_future()drains withdrawals BEFORE cert processing, matching upstream CERTS STS base case (conwayCertsTransitionEmptybranch); same-tx withdraw+unregister now succeeds - Conway committee membership check —
authorize_committee_hot_credential()andresign_committee_cold_credential()unconditionally verifyisCurrentMember || isPotentialFutureMember(upstreamcheckAndOverwriteCommitteeMemberState) - Conway proposal deposits in value preservation —
totalTxDeposits = certDeposits + proposalDeposits DepositPot.proposal_deposits— Tracks outstanding governance proposal deposits (upstreamoblProposalinObligations);total()matches upstreamsumObligationacross all four obligation categories; epoch-boundary reconciles returned/expired/enacted proposal deposits- Alonzo/Babbage/Conway collateral gating parity —
validate_alonzo_plus_tx()validates collateral content only when redeemers are present (upstreamfeesOKpart 2:txrdmrs ≠ ∅ ⇒ validateCollateral); Babbage/Conway still enforcemax_collateral_inputsas a standalone UTXO check regardless of redeemers validate_outputs_missing_datum_hash_alonzo()— Rejects Alonzo-era script-address outputs withoutdatum_hash(upstreamvalidateOutputMissingDatumHashForScriptOutputs)validate_unspendable_utxo_no_datum_hash()— CIP-0069 PlutusV3 datum exemption: V3-locked spending inputs exempt from datum-hash requirement in Conway (upstreamgetInputDataHashesTxBody)collect_v3_script_hashes()— Collects V3 script hashes from witness set and reference-input script refs for CIP-0069 datum exemptionvalidate_script_data_hash()— PV-aware: returnsScriptIntegrityHashMismatchat PV >= 11,PPViewHashesDontMatchat PV < 11 (upstreamCardano.Ledger.Conway.Rules.Utxow)validate_reference_input_disjointness()— PV-gated (upstreamdisjointRefInputs): enforced only at PV 9–10, relaxed at PV 11+ (upstreampvMajor > eraProtVerHigh @BabbageEra && pvMajor < natVersion @11)cleanup_dangling_drep_delegations()— HARDFORK PV 9→10 one-time cleanup (upstreamupdateDRepDelegations): removes stake-credential delegations that point to unregistered (non-builtin) DRepsZeroDonationvalidation — Rejects Conwaytreasury_donation == 0(upstreamvalidateZeroDonation)inner_cbor_size()onMultiEraTxOut— Measures inner era-specific output size without enum wrapper for correctcoins_per_utxo_bytecalculation (upstreamsizedSize)ppup_slot_context()— BuildsPpupSlotContextfromLedgerState.stability_window+slots_per_epoch; wired into all 10validate_ppup_proposalcall sites (upstreamgetTheSlotOfNoReturn)validate_script_witnesses_well_formed()/validate_reference_scripts_well_formed()— Malformed Plutus script detection at admission with rawPlutusBinarybytes (CBOR bytestring containing Flat) and protocol-version language gates (upstreamvalidateScriptsWellFormed)validate_outside_forecast()— OutsideForecast infrastructure (upstream no-op due tounsafeLinearExtendEpochInfo)delegate_stake_credential()— Pool-registration check on delegation: all eras (Shelley through Conway) reject delegation to unregistered pools viaDelegateeNotRegisteredDELEG(upstreamCardano.Ledger.Shelley.Rules.Deleg)PoolState::find_pool_by_vrf_key()— Conway VRF key uniqueness enforcement: new pool registrations reject duplicate VRF keys, re-registrations allow same pool’s own key (upstreamVRFKeyHashAlreadyRegistered/hardforkConwayDisallowDuplicatedVRFKeys)conway_protocol_param_update_well_formed()— Exact upstreamppuWellFormedcheck set: 10 unconditional zero-reject fields + bootstrap-gatedcoinsPerUTxOByte+ PV11-gatednOpt+ no cross-field checks (upstreamCardano.Ledger.Conway.PParams)record_block_producer()/take_blocks_made()— Per-pool block production tracking in LedgerState (upstreamNewEpochState.nesBcur)derive_pool_performance()— Pool performance ratios from internal blocks_made + stake distribution; d>=0.8 early-return gives perf=1 for all block-producing pools (upstreammkApparentPerformance)StakeCredentials::clear_pool_delegations()— POOLREAP delegation cleanup on pool retirement (upstreamremoveStakePoolDelegations)PoolState::adopt_future_params()— Adopts staged re-registration params at epoch boundary (upstream SNAP rule mergingpsFutureStakePoolParamsintopsStakePoolParams)PoolState::register_with_deposit()— New registration inserts; re-registration stages infuture_paramsand clears retirement (upstreampoolTransitiontwo-phase semantics)MultiEraUtxo— Unified UTxO model for all eras
Consensus:
verify_praos_header()— Slot leader validation (VRF + OpCert)verify_shelley_header()— Shelley-era header validationverify_block_vrf()— VRF proof verification with era-aware leader-value check (TPraos raw-512-bit / Praos range-extended-256-bit) and TPraos nonce proof verification (upstreamvrfChecksbheaderEta)validate_block_protocol_version()— Era/protocol-version consistency (hard-fork combinator parity)validate_block_body_size()— Declared vs actual body size (upstreamWrongBlockBodySizeBBODY)self_validate_forged_block()— Local forged-block guardrail before persistence (protocol-version/body-hash/body-size/header-identity checks)NonceEvolutionState::apply_block()— UPDN + TICKN nonce mixing with era-aware VRF derivation (TPraos simple hash vs Praos double-hash with “N” prefix)VrfMode/VrfUsage— Era-aware VRF dispatch:praos_vrf_input()(upstreammkInputVRF),tpraos_vrf_seed()(upstreammkSeedwithseedL/seedEtaXOR),check_leader_value()with mode-aware range extension (upstreamvrfLeaderValue/checkLeaderNatValue)ChainState— Volatility tracking with stable/unstable window
Network:
HandshakeMessagestate machine — Role + version negotiationChainSyncClient/ChainSyncServer— Full chain sync protocolBlockFetchClient/BlockFetchServer— Block batch downloadTxSubmissionClient/TxSubmissionServer— TX relay with dedupKeepAliveServer— Heartbeat protocolPeerSharingClient/PeerSharingServer— Peer candidate exchangeDnsRootPeerProvider— Dynamic root-peer resolution + refreshLedgerPeerProvider— Ledger-derived peer normalizationPeerRegistry— Source + status tracking (Cold/Warm/Hot)Mux— Protocol multiplexing with SDU dispatch
Mempool:
FeeOrderedQueue::insert()— Duplicate-detecting fee-ordered insertFeeOrderedQueue::pop_best()— Highest-fee TX retrievalevict_confirmed_from_mempool()— Block application cleanuppurge_expired()— TTL-based expiryrevalidate_with_ledger()— Post-block-apply ledger re-validation of remaining entries (upstreamrevalidateTxsForfromsyncWithLedger)evict_mempool_after_roll_forward()— Unified mempool eviction: confirmed removal + conflicting input removal + TTL purge + ledger revalidation
Storage:
FileImmutable— CBOR-backed immutable block storage with active crash recoveryFileVolatile— Rollback-aware volatile storageFileLedgerStore— Checkpoint-based ledger state persistenceapply_to_ledger_state()— Atomic checkpoint write- ChainDepState sidecars — Slot-indexed nonce/OpCert rollback, restart, and LSQ restore snapshots under
chain_dep_state/<slot-hex>.cbor
Crypto:
verify_vkey_signatures()— Ed25519 batch verificationverify_vrf_output()— Praos VRF proof checkverify_opcert_counter()— KES key period enforcement- All hash functions (Blake2b, SHA-256/512, SHA3, Keccak, RIPEMD)
- BLS12-381 pairing (G1/G2 ops, Miller loop, verification)
CLI:
NodeConfigFile— JSON config parsing + genesis integrationload_topology_file()— External P2P topology file loading (upstream JSON format)- Block producer role and forge-context parity —
--non-producing-nodedisables forging explicitly, producer credentials are atomic, absolute slots derive from ShelleysystemStart + slotLength, stale ledger views suppress forging, and forged headerprotocol_versionuses ledger protocol parameters when present (fallback: nodemax_major_protocol_version, not network handshake versions) apply_topology_to_config()— Override inline topology from external fileapply_topology_override()— CLI--topologyflag andTopologyFilePathconfig key integrationBasicLocalQueryDispatcher— 18-tag LocalStateQuery server (wallet queries: UTxOByTxIn, StakePools, DelegationsAndRewards, DRepStakeDistr; Conway governance queries: GetConstitution, GetGovState, GetDRepState, GetCommitteeMembersState, GetStakePoolParams, GetAccountState)LocalTxSubmission— Staged TX validation before mempool
⚠️ Partially Implemented (Need Completion)
Ledger:
validate_collateral()— Complete: VKey-locked address enforcement, mandatory when redeemers present, Babbage return/total-collateral checkscompute_epoch_rewards()— Complete: upstream RUPD→SNAP ordering, delta_reserves-only reserves debit, fee pot not subtracted from reservesratify_action()— Vote tallying complete incl. AlwaysNoConfidence auto-yes for NoConfidence/UpdateCommittee; CC expired-member term filtering; CC hot/cold credential resolution (votes keyed by HOT credential per Conway CDDL, tally resolves cold→hot); threshold math complete;defaultStakePoolVotepost-bootstrap SPO default vote from pool reward-account DRep delegation (AlwaysAbstain→Abstain, AlwaysNoConfidence→auto-Yes on NoConfidence, else implicit No)validate_conway_proposals()— Proposal validation includesWellFormedUnitIntervalRatification(committee quorum must be valid unit interval: denominator > 0 and numerator ≤ denominator)ratify_and_enact()— Enacted+expired+subtree-pruned deposit refunds via returnProposalDeposits; unclaimed→treasury; withdrawal budget tracks FULL proposed amounts (including unregistered accounts) matching upstreamensTreasury <-> wdrlsAmountfrom ENACT ruleremove_lineage_conflicting_proposals()— proposalsApplyEnactment: purpose-root chain validation removes stale proposalsapply_submitted_tx()— Pre-mempool validation for LocalTxSubmission and runtime mempool admission paths
Consensus:
ChainState::roll_forward()— CHAINHEAD validation enforces slot strictly increasing (SlotNotIncreasing) and prev-hash matching current tip hash (PrevHashMismatch), in addition to existing block-number contiguity check. Reference:Ouroboros.Consensus.Block.Abstract(blockPrevHash),Ouroboros.Consensus.HeaderValidation(slot monotonicity)ChainEntry::prev_hash— CarriesOption<HeaderHash>extracted per era (Byronprev_hash, Shelley/Alonzo/Babbage/Conwayheader.body.prev_hash);Noneskips the check for backward compatibility
Mempool:
- Collateral and script-budget checks — Enforced via staged ledger admission (
add_tx_to_shared_mempool/add_tx_to_mempoolcallingapply_submitted_txbefore insert) - TX conflict detection — Implemented in
insert_checkedwith input-overlap rejection (ConflictingInputs)
Network:
ChainSyncClient— Per-state timeouts: ST_INTERSECT 10 s, ST_NEXT_CAN_AWAIT 10 s, waitForever after MsgAwaitReplyBlockFetchClient— Per-state timeouts: BF_BUSY 60 s, BF_STREAMING 60 sKeepAliveClient— Response timeout: CLIENT 97 sPeerSharingClient— Response timeout: ST_BUSY 60 sTxSubmissionClient— All client-side waits are waitForever (server-driven pull protocol)- Connection manager — Full lifecycle with CM state shared across outbound and inbound paths
- Genesis density — Complete (Slice GD primitive
682dfa8+ runtime integration36bdbef):crates/consensus/src/genesis_density.rs::DensityWindowsliding-window header-density estimator (DEFAULT_SLOT_WINDOW = 6480,DEFAULT_LOW_DENSITY_THRESHOLD = 0.6, deterministic slot-only math). ChainSync observation hook (observe_chain_sync_header_density) feeds per-peer windows surfaced throughPeerMetrics.density; governorcombined_scoreapplies aHIGH_DENSITY_BONUS = 5bias and biases demotions toward sub-LOW_DENSITY_THRESHOLDpeers.
Storage:
- Garbage collection — Complete:
trim_before_slot,garbage_collect,compact,gc_immutable_before_slot,gc_volatile_before_slot - Crash recovery — Complete: stale dirty.flag removes .tmp files + clears sentinel after success
- Slot-based indexing — Complete: binary search in FileImmutable
Monitoring:
- Structured logging — Complete: NodeTracer with namespace/severity dispatch, longest-prefix routing
- Metrics — Complete: 35+ Prometheus counters/gauges (blocks, slots, peers, mempool tx/bytes, CM counters, inbound accept/reject, checkpoint, rollbacks, uptime)
- Epoch boundary events — Complete: traced with 14 structured fields per event (rewards, pools retired, governance, DRep expiry, treasury)
- Inbound server tracing — Complete: session start/reject/rate-limit events with peer + DataFlow + PeerSharing context
- Connection manager counters — Complete: per-tick full_duplex/duplex/unidirectional/inbound/outbound exported to Prometheus
- Coloured stdout — Complete:
Stdout HumanFormatColouredANSI severity colours (debug dim, warning yellow, error red, etc.) - Detail levels — Complete: per-namespace
TraceDetail(DMinimal/DNormal/DDetailed/DMaximum),detail_for()accessor,trace_runtime_detailed()detail-gated emission - Upstream backend recognition — Complete:
EKGBackend,Forwarder,PrometheusSimple,Stdout HumanFormatColoured/Stdout HumanFormatUncolouredall parsed - Trace forwarding — Complete:
Forwarderbackend emits CBOR-encoded trace events to Unix domain socket viaTraceForwarder; compatible with upstream cardano-tracer
❌ Not Started (Can Defer or Externalize)
Network:
- (no remaining items) — Genesis density primitive shipped in Slice GD (
crates/consensus/src/genesis_density.rs); ChainSync observation hook + governor-side density-biased demotion are wired in (commit36bdbef).
Storage:
- LMDB-compatible LSM backend — File-based JSON adequate for now
- Multi-path redundancy — Single-path acceptable with checkpoints
Monitoring:
- Hardware metrics (CPU%, memory%) — Kernel-level only
Planning-doc artifacts (retired R300)
The original April 2026 management summary carried five planning sections (Implementation Dependencies, Key Risks & Mitigations, Deliverables by Phase, Success Criteria, Why This Plan, Next Steps). The R273-rename + Phase A–F + R296–R299 execution arc shipped the plan; the live successors are:
| Retired section | Live successor |
|---|---|
| Implementation Dependencies | docs/ARCHITECTURE.md workspace topology + crate dependency direction |
| Key Risks & Mitigations | docs/UPSTREAM_PARITY.md::Open Gaps (current) + docs/PARITY_PROOF.md per-gap forensics |
| Deliverables by Phase | docs/operational-runs/ per-round records (the actual shipped deliverables) |
| Success Criteria | docs/MANUAL_TEST_RUNBOOK.md operator gate sign-offs (§2–9, §6.5) |
| Why This Plan | (Self-evident from shipped state; ~99% feature complete per the §1 table above.) |
| Next Steps | docs/MANUAL_TEST_RUNBOOK.md + remaining gaps in docs/UPSTREAM_PARITY.md |
The original planning-section text is preserved in the R300 closure
round-doc + the archived PARITY_PLAN.md.
Document Owner: Planning & Research
Review Cycle: Weekly (status table at top + audit history below)
Questions? See docs/PARITY_PROOF.md for
operational verification + docs/UPSTREAM_PARITY.md
for upstream alignment.
Parity Audit History
Rows below preserve the status and follow-ups known at the time of each round. Later rows and the status paragraph at the top of this document are authoritative for current closure state.
| Round | Domain | Areas | Gaps Found |
|---|---|---|---|
| 1–10 | Crypto, ledger types, all 7 eras, DELEG/POOL/CERTS | 100 | Atomicity fixes (StakeCredentials, DrepState, pool retirement ordering) |
| 11–20 | Mempool revalidation, TICK/NEWEPOCH, UTXOW, Conway governance, block production, network, Plutus CEK, consensus/storage | 100 | Mempool revalidate_with_ledger added |
| 21–27 | Fee/min-UTxO, submitted-tx validation, address/credential, multi-asset/minting, Byron, epoch boundary, Plutus validation | 70 | Asset name length validation (CDDL bytes .size (0..32)) |
| 28 | Collateral & is_valid handling (10 areas) | 10 | None |
| 29 | Governance ratification & enactment (10 areas) | 10 | None |
| 30 | Chain selection & rollback (10 areas) | 10 | None |
| 31 | Nonce evolution, VRF, KES (10 areas) | 10 | None |
| 32 | Storage, recovery, durability (10 areas) | 10 | None |
| 33 | Protocol parameters, PPUP, genesis (10 areas) | 10 | None |
| 34 | Native scripts, witnesses, Plutus hashing (10 areas) | 10 | None |
| 35 | CBOR serialization & round-trip (10 areas) | 10 | None |
| 36 | Mempool, tx submission, tx lifecycle (10 areas) | 10 | None |
| 37 | Network mini-protocols, mux, handshake (10 areas) | 10 | None |
| 38 | Block production, forging, leader election (10 areas) | 10 | None |
| 39 | Plutus CEK machine, builtins, cost model (10 areas) | 10 | None |
| 40 | Peer governor, diffusion, connection manager (10 areas) | 10 | None |
| 41-43 | UTxO validation, epoch boundary, BBODY/CHAINHEAD (30 areas) | 30 | Gap #14 CC hot/cold tally, Gap #15 well-formed UnitInterval, Gap #16 CHAINHEAD prev-hash + slot |
| 44 | Plutus ScriptContext per-version encoding (10 areas) | 10 | Gap #17: 7 encoding bugs fixed (B1–B4, B6–B8) |
| 45 | Conway UTXOW/CERTS/DELEG/GOVCERT/GOV rules (10 areas) | 10 | Gap #18: committee membership unconditional check; Gap #20: RefundIncorrectDELEG PV split |
| 46 | Plutus slot-to-POSIX conversion | 6 | Gap A: posix_time_range now uses real POSIX ms |
| 47 | PPUP/MIR is_valid gating, proposal fold ordering | 4 | Gap B: Alonzo/Babbage is_valid=false still collected PPUP/MIR; Gap C: proposal fold ordering decoupled from validation |
| 48 | CBOR indefinite-length support | 6 | Gap D: decoder rejected indefinite-length arrays/maps/bytes/text (RFC 8949 §3.2.1) |
| 49 | Deep parity audit (24 areas: treasury ordering, committee auth, withdrawal witnesses, Byron fees, etc.) | 24 | None (all 24 areas already implemented) |
| 50 | CBOR tag 258 set decode, min_committee_size floor, InfoAction ratification fix | 12 | Gap E: array() rejected #6.258 set encoding (27 sites); Gap F: min_committee_size floor not enforced; Gap G: InfoAction incorrectly ratified |
| 51 | min_utxo output size, ZeroDonation, Alonzo output datum hash, PPUP slot-of-no-return | 18 | Gap H: inner_cbor_size() for min-lovelace measurement; Gap I: zero treasury donation silently accepted; Gap J: Alonzo script-output missing datum hash; Gap K: PPUP slot-of-no-return not wired |
| 52-57 | VRF mode/usage, nonce derivation, leader value range, VRF proof verification | 30 | Era-aware VRF parity, TPraos nonce VRF proof verification |
| 58 | TxContext protocol_version + reward calculation precision | 12 | Gap L: all 6 TxContext sites left protocol_version: None (broke V3 PV9 bootstrap); Gap M: max_pool_reward used 5-floor fixed-point (now exact U256 single-floor); Gap N: delta_reserves used double-floor (now single-floor) |
| 59 | Governance ratification edge cases | 5 | Gap O: meets_threshold zero-denominator → numerator == 0 (upstream %? + r == minBound); Gap P: AlwaysNoConfidence counted YES for UpdateCommittee (upstream only NoConfidence) |
| 60 | Conway governance: committee existence + DRep bootstrap thresholds | 10 | Gap Q: EnactState lacked has_committee flag — post-NoConfidence non-HF/non-UC actions incorrectly passed committee gate; Gap R: DRep thresholds not zeroed during Conway bootstrap phase (PV 9) — upstream votingDRepThresholdInternal uses def/all-zero |
| 61 | Threshold selection, SPO bootstrap abstain | 10 | Gap S: SPO non-voting counted as implicit No during bootstrap (upstream: Abstain, except HardFork always No); Gap V: drep_threshold_for_action/spo_threshold_for_action used member-state check instead of ensCommittee presence (has_committee) for normal/no-confidence threshold selection |
| 62 | Governance ratification: proposal priority ordering | 8 | Gap W: ratify_and_enact iterated proposals in GovActionId (BTreeMap key) order instead of upstream actionPriority order — delaying actions (NoConfidence=0, UpdateCommittee=1, NewConstitution=2, HardForkInitiation=3) could be preempted by lower-priority non-delaying actions |
| 63 | Governance expiry descendants, committee guard | 6 | Gap X: expired parent proposals did not transitively remove descendant proposals (upstream proposalsRemoveWithDescendants); Gap Y: extra committee_update_meets_min_size guard in ratification loop not present in upstream ratifyTransition (min_committee_size enforcement is only inside committeeAccepted via votingCommitteeThreshold) |
| 64 | Governance ratification/enactment state guards | 6 | Gap Z: ENACT UpdateCommittee applied non-upstream local term filters (now removed; apply members_to_add verbatim after RATIFY); Gap AA: withdrawalCanWithdraw used non-progressive treasury guard across loop (now checked against evolving treasury); Gap AB: validCommitteeTerm no longer assumes frozen snapshots and now reads current protocol-parameter view each iteration |
| 65 | Shelley DELEG future-genesis delegation scheduling | 6 | Gap AC: GenesisDelegation applied immediately instead of staging in dsFutureGenDelegs; fixed with slot-based scheduling/adoption and duplicate checks across active+future deleg maps |
| 66 | Conway GOV bootstrap-phase return-account gating | 6 | Gap AD: ProposalReturnAccountDoesNotExist and TreasuryWithdrawalReturnAccountsDoNotExist enforced unconditionally — upstream gates both inside unless (hardforkConwayBootstrapPhase ...) in conwayGovTransition; fixed with past_bootstrap guard |
| 67 | Conway DELEG deposit mismatch error phase split | 6 | Gap AE: key-registration deposit mismatches always returned legacy IncorrectDepositDELEG; upstream uses DepositIncorrectDELEG after hardforkConwayDELEGIncorrectDepositsAndRefunds (PV >= 10) while keeping legacy error in bootstrap PV 9; fixed across all Conway registration cert shapes with regression tests |
| 68 | Committee resignation state preservation | 6 | Gap AF: register_with_term() replaced resigned entries — allowed re-auth after UpdateCommittee re-add; NoConfidence wiped resignation state; members_to_remove destroyed entries; auth/resign check ordering inverted vs upstream; tally_committee_votes/count_active_committee_members did not filter by enacted membership. Fixed: register_with_term preserves authorization via Entry API; clear_all_membership()/clear_membership() only clear expires_at; is_enacted_member() proxy; check ordering matches upstream checkAndOverwriteCommitteeMemberState |
| 69 | Ratification threshold evolution after ParameterChange enactment | 6 | Gap AG: ratify_and_enact() pre-computed drep_thresholds, pool_thresholds, min_committee_size, is_bootstrap_phase once before the ratification loop — upstream ratifyTransition reads these from rs ^. rsEnactStateL . ensCurPParamsL per-proposal recursively. After a ParameterChange enactment, subsequent proposals now see updated thresholds. |
| 70 | Conway deposit pot: proposal deposit tracking (totalObligation) | 6 | Gap AH: DepositPot lacked proposal_deposits field (upstream oblProposal in Obligations); total() only summed key+pool+drep deposits — upstream sumObligation includes all four including oblProposal. Fixed: added proposal_deposits to DepositPot, wired Conway block-apply and submitted-tx paths to accumulate proposal deposits, epoch-boundary reconciliation debits returned/expired/enacted proposal deposits. Backward-compatible CBOR (3-or-4 element decode). |
| 71 | Collateral gating + forged header protocol version source | 6 | Gap AI: collateral validation ran whenever collateral inputs existed; upstream only runs validateCollateral when redeemers exist (feesOK part 2). Fixed in validate_alonzo_plus_tx(). Gap AJ: forged header protocol-version fallback used network handshake versions (13/14/15) instead of protocol versions; fixed fallback to node max_major_protocol_version with minor 0 while still preferring ledger protocol parameters when available. |
| 72 | Babbage/Conway standalone collateral input-count check | 6 | Gap AK: after AI, validate_alonzo_plus_tx() unintentionally skipped max_collateral_inputs checks when no redeemers were present. Upstream Babbage UTXO enforces validateTooManyCollateralInputs as a standalone check independent of redeemers. Fixed with era-aware enforce_collateral_input_limit wiring (false in Alonzo, true in Babbage/Conway) and regression coverage. |
| 73 | Conway disjointRefInputs PV gating |
6 | Gap AL: validate_reference_input_disjointness enforced unconditionally in both Conway block-apply and submitted-tx paths. Upstream disjointRefInputs in Cardano.Ledger.Babbage.Rules.Utxo is PV-gated: pvMajor > eraProtVerHigh @BabbageEra && pvMajor < natVersion @11, meaning disjointness is only enforced at PV 9–10 (early Conway). At PV 11+ the check is relaxed. Fixed with disjoint_ref_inputs_enforced() helper gating both call sites; 3 new PV-gating tests. |
| 74 | Conway HARDFORK updateDRepDelegations cleanup |
6 | Gap AM: protocol-version transition cleanup from bootstrap to post-bootstrap was not covered by regression tests. Upstream HARDFORK rule runs updateDRepDelegations when pvMajor newPv == 10, clearing dangling delegations to non-existent DReps created during bootstrap (preserveIncorrectDelegation). Verified and locked with 4 integration tests covering PV9→10 cleanup, preservation of registered/builtin DReps, non-hardfork no-op, and PV10→11 no-cleanup behavior. |
| 75 | ppuWellFormed cross-field over-validation removal |
6 | Gap AN: conway_protocol_param_update_well_formed() included three checks not present in upstream ppuWellFormed (Cardano.Ledger.Conway.PParams): (1) effective-zero check merging proposed values with current protocol params, (2) cross-field max_tx_size > max_block_body_size consistency check, (3) effective-zero check on resolved max_block_body_size / max_tx_size. Upstream only validates individual proposed field values for non-zero without merging or cross-referencing. Removed the extra block and unused protocol_params parameter from function signature. Updated 2 existing tests to assert acceptance. Added 1 new regression test. |
| 76 | Withdrawal budget parity (withdrawalCanWithdraw) |
6 | Gap AO: withdrawal_budget tracked separately from live treasury, decremented by FULL proposed amount (including unregistered accounts). Matches upstream ensTreasury st <-> wdrlsAmount. |
| 77 | Epoch boundary: donation ordering + performance snapshot | 10 | Gap AP: flush_donations_to_treasury() moved from before to after ratification, matching upstream casTreasuryL <>~ utxosDonationL ordering — donations no longer inflate withdrawal_budget. Gap AQ: derive_pool_performance() changed from snapshots.set to snapshots.go, matching upstream mkApparentPerformance using ssStakeGo. Documented inline-vs-pulsed reward phase shift. |
| 78 | Proposal deposits in DRep/SPO voting weights | 8 | Gap AR: compute_drep_stake_distribution did not include per-credential proposal deposits in DRep voting weight — upstream computeDRepDistr computes stakeAndDeposits = fold $ mInstantStake <> mProposalDeposit. Gap AS: SPO pool distribution not augmented with proposal deposits — upstream addToPoolDistr adds proposal deposits to pool stakes for SPO voting. Fixed both via compute_proposal_deposits_per_credential() + wiring into ratify_and_enact(). |
| 79 | Script integrity hash triple-null guard | 6 | Gap AT: validate_script_data_hash() only checked for redeemers to decide if a script_data_hash was needed. Upstream mkScriptIntegrity returns SNothing only when ALL THREE of (redeemers, datums, langViews) are null; if ANY is non-empty the hash is required. Fixed via script_integrity_needed() which checks redeemers, witness datums, and language views (Plutus scripts provided ∩ needed). Updated error ordering in tests: MissingRedeemer / UnspendableUTxONoDatumHash tests now expect MissingRequiredScriptIntegrityHash when script_data_hash is absent (upstream UTXOW fires before UTXOS). Supplemental datum tests now compute and declare the correct hash. |
| 80 | MissingRedeemers Phase-1 extraction | 6 | Gap AU: MissingRedeemers check was inside validate_plutus_scripts() (Phase-2), so is_valid=false transactions skipped it. Upstream hasExactSetOfRedeemers runs both ExtraRedeemers and MissingRedeemers at Phase-1 unconditionally in UTXOW. Extracted validate_no_missing_redeemers() as standalone Phase-1 function paired with existing validate_no_extra_redeemers(). Wired into all 6 per-era call sites (Alonzo/Babbage/Conway × block-apply + submitted-tx). 3 new tests: Alonzo block + Babbage submitted-tx with valid hash but no redeemer, Conway is_valid=false with missing redeemer. |
| 81 | Collateral return index + HardFork version jump guard | 12 | Gap AV: apply_collateral_only() used u16::MAX (65535) for the collateral return output index. Upstream mkCollateralTxIn uses fromIntegral $ length (body ^. outputsTxBodyL) — the index equals the number of regular outputs. Fixed by passing body.outputs.len() from all 3 Alonzo/Babbage/Conway call sites. Gap AW: conway_expected_previous_hard_fork_version() lacked preceedingHardFork safety guard — proposals jumping more than one major version ahead of the live protocol were not blocked. Added early return when protocol_version.0 > cur.0.saturating_add(1), matching upstream guard. Updated cross-block lineage chain test to use valid (10,1) instead of invalid (11,0). Comprehensive Conway rule audit: LEDGER/LEDGERS/BBODY/UTXO (23 variants)/UTXOW (19 variants via babbageUtxowTransition 10 checks)/UTXOS (2 variants)/GOV (19 variants)/DELEG (8 variants)/GOVCERT (6 variants)/CERT/CERTS/POOL (6 variants) — all verified. EPOCH/NEWEPOCH ordering parity confirmed. Submitted-tx vs block-apply path consistency verified for all eras. |
| 82 | Ratification ordering + delay flag semantics + proposing script witnesses | 12 | Gap AX: required_script_hashes_from_proposal_procedures incorrectly included NewConstitution.constitution.guardrails_script_hash as a required proposing script witness. Upstream getConwayScriptsNeeded → proposingScriptsNeeded only requires script witnesses for ParameterChange and TreasuryWithdrawals guardrails scripts; NewConstitution guardrails are for post-enactment use. Removed NewConstitution branch. 2 new tests. Gap AY: apply_epoch_boundary removed expired governance actions BEFORE running ratify_and_enact. Upstream epochTransition runs RATIFY (which includes both enacted and expired sets) BEFORE expiry cleanup — an expired action that passes all ratification checks should still be enacted. Reordered to ratify-first, expire-after. Gap AZ: ratify_and_enact only set delayed = true after successful enactment. Upstream ratifyTransition otherwise branch sets rsDelayed \|\| delayingAction gas for ALL non-enacted, non-expired delaying actions (NoConfidence/HardFork/UpdateCommittee/NewConstitution), preventing subsequent enactments even when the delaying action itself fails acceptance. Expired actions do NOT change the flag. Restructured loop with labeled block guards. 3 new tests. Deep verification: voter witness collection (VKey + script), committee/DRep/SPO tally functions, SPO default votes, DRep activity tracking, pparam group classification, validCommitteeTerm, withdrawalCanWithdraw, ratification guard order, ProposalReturnAccountDoesNotExist PV gating. |
| 83 | Storage volatile delete WAL recovery | 6 | Gap BA: multi-step volatile delete paths (prune_up_to, rollback_to, garbage_collect) had no persisted delete plan if a crash occurred between partial file deletion and state convergence. Fixed by adding wal.pending.json delete-plan journaling in FileVolatile plus open-time WAL replay/cleanup and regression tests for valid and malformed WAL plans. |
| 84 | BBODY max block body size full-byte accounting | 6 | Gap BB: apply_block_validated() measured block body size as sum(tx.body.len()), undercounting witness/aux/is_valid payload bytes. Fixed by summing Tx::serialized_size() (full tx CBOR payload parity with BBODY accounting intent). Added regression tests for single-tx and multi-tx undercount scenarios. |
| 85 | Block-apply / submitted-tx on-wire byte preservation (min_fee, txIdTxBody) |
12 | Gap BC: *_block_to_block re-serialised typed ShelleyTxBody / ShelleyWitnessSet to compute tx_size and tx_id, producing byte-canonical CBOR that did not match the on-wire encoding (definite vs indefinite length, set vs array, integer-width canonicalisation). Drift was enough to shift min_fee = 44 · txSize + 155 381 past the declared fee on a real preprod transaction (440-lovelace gap; surfaced at slot ~518 460 in a 2026-04-27 preprod sync rehearsal). Gap BD: MultiEraSubmittedTx::Shelley wrapped era-internal ShelleyTx (no raw_body/raw_cbor), unlike every other era arm; its tx_id() and the three ledger-side validation sites in crates/ledger/src/state.rs re-encoded the body to compute the canonical hash. Both fixed: new yggdrasil_ledger::extract_block_tx_byte_spans + BlockTxRawSpans walk the outer block CBOR once and return on-wire byte spans for every transaction_body / transaction_witness_set; MultiEraSubmittedTx::Shelley now wraps ShelleyCompatibleSubmittedTx<ShelleyTxBody> (carries raw_body/raw_cbor like the other arms); the four era converters and extract_tx_ids consume pre-extracted spans. Sync hot path now caches the spans on MultiEraSyncStep::RollForward.block_spans so the eviction, apply, and ledger-advance consumers share one extraction per block (down from three). New shelley_submitted_tx_id_uses_on_wire_bytes_not_re_encoded regression test decodes a deliberately non-canonical Shelley tx (over-long uint64 fee) and proves tx_id() == hash(raw_body) ≠ hash(body.to_cbor_bytes()). References: Cardano.Ledger.Shelley.Tx.minfee, Cardano.Ledger.Core.txIdTxBody. Full writeup in docs/REAL_PREPROD_POOL_VERIFICATION.md. |
| 86 | Submitted-tx invariant hardening (Q-1) + sync-path zero-copy block clone (F-2) | 8 | Gap BE: raw_body / raw_cbor exposed as pub on ShelleyCompatibleSubmittedTx<TxBody> and AlonzoCompatibleSubmittedTx<TxBody>, allowing external code to mutate body and silently desync the on-wire-bytes invariant that tx_id and fee tx_size rely on. Demoted both fields to pub(crate); added raw_body() -> &[u8] and raw_cbor() -> &[u8] accessor methods. External constructors now MUST go through ::new(body, witness_set, [is_valid,] aux). Gap BF: Block.raw_cbor: Option<Vec<u8>> cloned ~80 KB per Conway block at every storage path (volatile-DB prefix_up_to, immutable-DB suffix_after, chain_db.append_block) and at every apply-step storage write (apply_multi_era_step_to_volatile). Switched to Option<Arc<[u8]>> so clone() is an atomic refcount bump; on-disk CBOR encoding is unchanged (serde/rc enabled workspace-wide; Arc<[u8]> and Vec<u8> both encode as the same RFC 8949 byte-string). New regression test block_raw_cbor_arc_serde_round_trip in crates/storage/tests/integration.rs locks the on-disk byte-equivalence; BlockProvider::get_block_range still returns Vec<Vec<u8>> and pays one Arc::to_vec() at the trait boundary, so the net win is one fewer alloc per block per re-serve. References: Cardano.Ledger.Core.txIdTxBody, Cardano.Ledger.Shelley.Tx.minfee. |
| 91 | OPEN — operational parity: multi-peer dispatch advances ChainState but does not persist to volatile DB |
0 (open) | Gap BN (open as of 2026-04-27, surfaced after Round 90 fix turned the crash into a visible livelock): with --max-concurrent-block-fetch-peers 2 and ≥ 3 localRoots, multi-peer dispatch activates (yggdrasil_blockfetch_workers_registered = 3), the in-memory chain advances to ~slot 102 240, but find /tmp/db -type f shows 0 files in volatile/, immutable/, ledger/. The verified-sync path is advancing the in-memory ChainState (and from_point) via the tentative-header path, but the per-peer FetchWorkerPool reassembly is not feeding the dispatched blocks into apply_multi_era_step_to_volatile. Round 90’s from_point ↔ storage tip realignment now turns the resulting hard-crash into a recoverable rollback-and-resync — the node stays alive across handoffs (5 realignments + 0 crashes confirmed on the 2026-04-27 90-second rehearsal) — but storage stays empty so the node is in a steady-state livelock (re-syncs from Origin on every handoff). Investigation entry points: node/src/sync.rs::dispatch_range_with_tentative, node/src/sync.rs::execute_multi_peer_blockfetch_plan, the reorder-buffer hand-off into the apply path. References: upstream Ouroboros.Network.BlockFetch.ClientRegistry + Ouroboros.Consensus.BlockFetch.SerialiseDisk. |
| 90 | Operational parity: multi-peer-dispatch session-handoff RollbackPointNotFound (from_point outlived chain_state.entries window) |
8 | Gap BM (closed in this slice): with --max-concurrent-block-fetch-peers 2 and ≥ 3 localRoots, multi-peer BlockFetch activation succeeded (yggdrasil_blockfetch_workers_registered = 3, _migrated_total = 3) but within ~30 s of preprod sync the governor’s Net.PeerSelection: switching sync session to higher-tip hot peer path triggered a reconnect, the re-established session resumed from fromPoint=BlockPoint(N, H), and roll_backward on the in-memory ChainState returned RollbackPointNotFound { slot: N, hash: H } — crashing the node every ~30 s on §6.5a multi-peer rehearsal. Not the Round 88 fresh-restart bug (ChainState was the same in-memory object across the reconnect loop); from_point had advanced past whatever the volatile store actually held (e.g., from_point at slot 102 240 vs storage tip at Origin, observed live on 2026-04-27). Fix: at the top of every reconnect-loop iteration in both run_reconnecting_verified_sync_service_chaindb_inner and run_reconnecting_verified_sync_service_shared_chaindb_inner (node/src/runtime.rs), call seed_chain_state_via_chain_db(chain_db, security_param) AND realign from_point to chain_state.tip() — emit Net.PeerSelection info trace realigning from_point to volatile storage tip before reconnect whenever they differ. This makes the resume self-consistent regardless of what diverged in the prior session: the next peer’s RollBackward(from_point) confirmation always finds the target in the seeded ChainState. Verified end-to-end on the 2026-04-27 §6.5a rehearsal — 5 realignments handled cleanly + 0 crashes over 1 m 31 s (was crashing at 30 s pre-fix); forensic log preserved at /tmp/ygg-multi-peer-rollback-crash-2026-04-27.log. Production default max_concurrent_block_fetch_peers = 1 should stay until Gap BN below also closes (multi-peer storage persistence livelock). References: Ouroboros.Consensus.Storage.ChainDB.Init.getCurrentChain. |
| 89 | Operational parity: §6.5a multi-peer BlockFetch activation + devcontainer toolchain | 4 | Gap BJ: §6.5a runbook documented activating multi-peer BlockFetch via the env override NODE_CONFIG_OVERRIDE_max_concurrent_block_fetch_peers=2, but this env-var pattern was never implemented in node/src/main.rs; the only way to flip the knob was to write a full Yggdrasil-format config file. Fix: new --max-concurrent-block-fetch-peers <N> CLI flag on run, plumbed through load_effective_config and overriding the file value (matches the existing --peer / --port / --metrics-port override pattern). Gap BK: §6.5a expected the Net.BlockFetch.Worker activation event to fire just by setting the knob, but the activation also requires the governor to actually promote ≥ 2 peers to warm — and the vendored preprod topology has only 1 bootstrapPeer, with useLedgerAfterSlot=112406400 meaning ledger-derived peers do not populate the registry until slot 112 406 400. Without localRoots, an operator setting knob=2 still runs the legacy single-peer path silently. Fix: runbook §6.5a now lists both prerequisites explicitly, names the Prometheus gauge yggdrasil_blockfetch_workers_registered as the authoritative activation criterion (must rise from 0), and shows the topology.json localRoots shape needed to populate the registry pre-useLedgerAfterSlot. Gap BL: compare_tip_to_haskell.sh from §5 needed cardano-cli on $PATH for the Haskell-side tip query; the devcontainer base image mcr.microsoft.com/devcontainers/base:noble did not include it. Fix: new node/scripts/install_haskell_cardano_node.sh (idempotent download + install of the IntersectMBO Linux release tarball into ~/.local/bin/); .devcontainer/devcontainer.json runs it on postCreateCommand so a fresh devcontainer rebuild has the full §5 / §6.5b operator toolchain pre-installed. Reference: upstream Ouroboros.Network.BlockFetch.ClientRegistry (per-peer worker registration) + Cardano.Network.PeerSelection.Bootstrap.useLedgerAfterSlot semantics. |
| 88 | Operational parity: restart-resilience cycle-2 RollbackPointNotFound (ChainState not seeded from volatile DB on restart) |
6 | Gap BI: every reconnecting-sync entry point in node/src/runtime.rs and node/src/sync.rs constructed ChainState::new(k) empty. After a node restart, storage recovered the tip but ChainState.entries was []; the next ChainSync session immediately received RollBackward(recovered_tip) (the peer’s resume-point confirmation) and our roll_backward searched the empty entries vec, returning RollbackPointNotFound and crashing. Surfaced live by node/scripts/restart_resilience.sh CYCLES=2 against a real preprod peer (cycle-2 crashed during the settle window). Fix: new ChainState::seed_from_entries + crate::sync::seed_chain_state_from_volatile helper (reads volatile.suffix_after(&Point::Origin) and seeds the trailing-k window), wired into all 5 sync entry points via a ChainDbVolatileAccess trait so the helper works for both &mut ChainDb<I, V, L> and &Arc<RwLock<ChainDb<I, V, L>>>. 3 unit tests in crates/consensus/src/chain_state.rs lock the seed semantics; 3 integration tests in node/tests/runtime.rs were updated to provide chain-contiguous block-number / prev-hash fixtures (they previously relied on the empty-ChainState bug to bypass CHAINHEAD validation). Operator vendored configs gained placeholder peer-snapshot.json files for mainnet + preview so the §1 preflight succeeds out of the box for all three networks. End-to-end verification: restart_resilience.sh CYCLES=2 INTERVAL_BASE_S=30 against preprod now reports [ok] all 2 cycles + final recovery completed monotonic tip progression, with cycles syncing 86 440 → 90 020 → 91 600. Reference: upstream Ouroboros.Consensus.Storage.ChainDB.Init.getCurrentChain. |
| 87 | Byzantine-path Word8 / size-bound parity (PeerSharing amount cap, LocalTxSubmission decode ceiling) | 4 | Gap BG: MsgShareRequest.amount arrives as u16 on our wire but upstream Ouroboros.Network.PeerSelection.PeerSharing transports it as Word8 (max 255); SharedPeerSharingProvider::shareable_peers previously honoured the full u16 range so a malicious peer requesting u16::MAX forced a full-registry walk per request. Fixed: cap to PEER_SHARING_MAX_AMOUNT = 255 BEFORE the registry walk in node/src/server.rs; new regression test shared_peer_sharing_provider_clamps_to_upstream_word8_max populates 300 peers and asserts u16::MAX requests return ≤ 255. Gap BH: NtC LocalTxSubmission accepted arbitrary CBOR tx_bytes and only rejected oversized payloads after the full mempool-admission decode + validate_max_tx_size check (mainnet max_tx_size = 16 384 B Conway PV 10), so a malicious local client could force a multi-MB allocation before rejection. Fixed: explicit LOCAL_TX_SUBMIT_MAX_BYTES = 64 KiB ceiling at the wire boundary (~4× the protocol max for headroom), reject with structured reason before any decode. Other byzantine paths verified intact via three-Explore-agent sweep + targeted greps: mux SDU DEFAULT_INGRESS_LIMIT = 2 MB (audit M-1), TxSubmission2 outstanding_txids FIFO with AckedTooManyTxIds / BlockingRequestWithOutstanding errors (upstream V2 state), block-body validate_max_block_body_size, max_tx_size enforcement in fees.rs, mempool bytes-cap with eviction, handshake VersionMismatch rejection, Plutus ExBudget::spend checked-arithmetic budget enforcement, PlutusData MAX_DECODE_DEPTH = 256 recursion bound, equivocating-SPO detection via OCert currentIssueNo in chain selection, security-param k-bounded rollback depth, reward floor_mul_div with checked_mul overflow fallback, Conway-governance vote weights sourced from authoritative epoch snapshots (not peer-controlled), pool-deposit enforcement at registration, mark/set/go immutable snapshot rotation, MIR / treasury-withdrawal genesis-delegate-quorum + ratification gating, constant-time crypto via subtle::ConstantTimeEq, no panic! on peer-supplied signature/point bytes, hex-encoded storage filenames (no .. traversal), WAL replay tolerates malformed JSON, M-3 NtC socket 0o660 permissions, M-8 genesis-hash hard-fail, L-6 KES file-mode + zeroize-on-drop. References: Ouroboros.Network.PeerSelection.PeerSharing, Ouroboros.Consensus.Mempool.Impl.Update. |
| Total (R1–R91) | All subsystems | 787 | 53 fix rounds |
| 92–143 | NtC handshake fixes, V_23 + result shapes (R148–R152), network-aware Interpreter / SystemStart per network (R153), era-PV pairing for HFC transition signal (R154), Alonzo+ tx-size for fee/max excludes is_valid byte (R155), cardano-cli query protocol-parameters Shelley/Alonzo/Babbage/Conway shapes (R156, R159–R161), cardano-cli query utxo whole / address / tx-in (R157), cardano-cli query tx-mempool LocalTxMonitor parity + era-tagged MsgHasTx (R158), era-history coverage to slot 2^48 + bignum relativeTime (R162), R163 stake-pool/distribution/genesis/address-info dispatcher infrastructure (later corrected per upstream tag table in R179), Round 144 multi-peer dispatch closure of Gap BN |
~40 rounds | 11 cardano-cli operations confirmed end-to-end on preprod (Shelley) + preview (Alonzo); cumulative parity arc closed in R164 |
| 144–164 | Cumulative parity arc Rounds 144→164 | — | Rounds documented in docs/operational-runs/2026-04-28-round-{144..164}-*.md; R164 sign-off captured 4710 tests passing across all crates with 11 working cardano-cli operations |
| 165–166 | Sync-speed unblock | 0 | R165 default --batch-size 10 → 30 (~9 blk/s vs ~5); R166 initial-sync rollback fast path skips recover_ledger_state_chaindb heavy replay when rollback target is Origin and base ledger state is empty, letting the boundary-aware forward-apply path fire epoch transitions; default raised to 50 (~14 blk/s) |
| 167 | Mid-sync rollback epoch fixup | 0 | recover_ledger_state post-recovery patches current_epoch to match the recovered tip’s slot when crossing an epoch boundary, preventing PPUP validation errors on cross-epoch rollback |
| 168, 175 | yggdrasil_active_peers metric anomaly |
0 | Bootstrap sync peer was never registered as PeerHot in the shared PeerRegistry (governor-managed peers only). Fixed across both production sync paths (run_reconnecting_verified_sync_service_chaindb_inner + run_reconnecting_verified_sync_service_shared_chaindb_inner); cooling completion at KeepAlive-failure and session-switching mux-abort sites |
| 169–170 | Observability metrics | 0 | New yggdrasil_current_era Prometheus gauge (R169) reports the wire era ordinal of the latest applied block; new per-era applied-block counters (yggdrasil_blocks_byron, …_shelley, …_allegra, …_mary, …_alonzo, …_babbage, …_conway, R170) let dashboards graph share-of-blocks-per-era during long syncs |
| 171–173 | Upstream LSQ era-specific tag dispatchers | 0 | GetStakePoolParams (R171), GetPoolState (R172), GetStakeSnapshots (R173) wire-correct dispatchers — yggdrasil already had the data (pool_state, future_params, etc.); these rounds added the upstream-shape encoders plus regression tests. Tag numbers off-by-3 vs upstream (caught later in R179) |
| 174, 176 | Decoder strictness sweep | 0 | Five CBOR set-decoder helpers tightened: decode_pool_hash_set, decode_stake_credential_set, decode_address_set, decode_txin_set, decode_maybe_pool_hash_set now enforce CIP-21 tag 258 strictly and Maybe Nothing shortcut requires bare null (0xf6); pre-fix malformed payloads silently mis-parsed |
| 177 | encode_filtered_delegations_and_rewards correctness |
0 | Three independent bugs: non-deterministic HashSet iteration, O(N·M) inner search per credential (now BTreeMap::get O(log N)), reward-account lookup mis-matched on hash bytes alone stripping AddrKey-vs-Script discriminator (now find_account_by_credential full match) |
| 178 | YGG_LSQ_ERA_FLOOR=N env-var bypass |
0 | Operator opt-in floor on the LSQ-reported era so cardano-cli’s client-side Babbage+ gate can be bypassed on partial-sync chains; with YGG_LSQ_ERA_FLOOR=6 cardano-cli reports era=Conway and stops gating the era-locked queries (downstream response-shape mismatches handled in R179) |
| 179 | Era blockage end-to-end fix | 1 | Major unblock: three independent bugs identified and fixed. (1) Wrong upstream tag numbers — R163’s tag table for GetStakePools (13 → 16), GetStakePoolParams (14 → 17), GetPoolState (17 → 19), GetStakeSnapshots (18 → 20) corrected per Ouroboros.Consensus.Shelley.Ledger.Query.encodeShelleyQuery. (2) cardano-cli query stake-distribution uses tag 37 GetStakeDistribution2 (post-Conway no-VRF variant) returning [map, NonZero Coin] shape; added the alias and pdTotalStake = 1 placeholder (NonZero requirement). (3) query pool-state and query stake-snapshot use tag 9 GetCBOR wrapper recursively dispatching the inner query — added EraSpecificQuery::GetCBOR { inner_query_cbor } variant + dispatch_inner_era_query helper that synthesises a [era_index, inner_query] outer wrapper, recursively classifies via decode_query_if_current, and wraps the response in tag(24) bytes(<inner>). All five era-gated queries now decode end-to-end |
| 180–183 | Conway governance LSQ queries | 0 | cardano-cli conway query constitution (tag 23) returns real Conway constitution from preview’s chain end-to-end; query treasury (tag 29 GetAccountState) returns 0; query drep-state --all-dreps (tag 25, R181 Map shape fix) returns []; query committee-state (tag 27, R182) returns {committee: {}, epoch: 0, threshold: null}; query future-pparams (tag 33, R183) returns Maybe (PParams era) = Nothing rendered as "No protocol parameter changes will be enacted at the next epoch boundary.". Only gov-state remains (substantial — 7-element ConwayGovState record with Proposals tree + DRepPulsingState cache) |
| 184 | Conway DRep/SPO stake-distribution + filtered-vote-delegatees LSQ dispatchers | 0 | Three new EraSpecificQuery variants: GetDRepStakeDistr (tag 26), GetFilteredVoteDelegatees (tag 28), GetSPOStakeDistr (tag 30). All return Map a Coin / Map a DRep shapes; emit 0xa0 (empty map) until live stake plumbing lands. Discovery: cardano-cli’s query spo-stake-distribution --all-spos is a 3-call flow — SPOStakeDistr (30), GetCBOR(GetPoolState) (9→19), GetFilteredVoteDelegatees (28); the third was the failing call in our initial implementation, surfaced via a debug-instrumented decoder. cardano-cli conway query drep-stake-distribution --all-dreps returns {} end-to-end; query spo-stake-distribution --all-spos returns [] end-to-end |
| 185 | Conway proposals + stake-pool-default-vote LSQ dispatchers | 1 | Two new EraSpecificQuery variants: GetProposals (tag 31, returns Seq (GovActionState era), emit empty list 0x80) and QueryStakePoolDefaultVote (tag 35, returns DefaultVote enum encoded as single CBOR uint, emit DefaultNo (0) placeholder). cardano-cli conway query proposals --all-proposals returns [] end-to-end; query stake-pool-default-vote --spo-key-hash <hash> returns "DefaultNo" end-to-end |
| 186 | Conway tail-end LSQ dispatchers (tags 22, 36) | 1 | Two new EraSpecificQuery variants closing the simpler remaining Conway dispatcher gaps: GetStakeDelegDeposits (tag 22, returns Map (Credential 'Staking) Coin, emit 0xa0) and GetPoolDistr2 (tag 36, returns PoolDistr 2-element record with optional pool-id filter). R186 emitted [empty_map, NonZero=1]; R237 replaces that placeholder with live PoolDistr data from the set stake snapshot, with optional filtering and full-denominator preservation matching GetStakeDistribution2. |
| 187 | Conway ratify-state body shape (tag 32) end-to-end |
1 | Closes the substantial 4-field-record body-shape gap. New helpers: encode_enact_state_for_lsq (7-element CBOR list per upstream EnactState era: committee SNothing / real Conway constitution / Conway 31-element PParams / treasury / empty withdrawals / 4-SNothing GovRelation) and encode_ratify_state_for_lsq (4-element wrapper [EnactState, empty Seq, empty Set, false]). cardano-cli conway query ratify-state decodes end-to-end with real Conway constitution + 31-element PParams + treasury rendered. EnactState encoder is the load-bearing helper for the upcoming gov-state round (used inside its DRepPulsingState field) |
| 188 | Conway gov-state body shape (tag 24) end-to-end — closes last user-facing Conway gap |
0 | New encode_conway_gov_state_for_lsq helper emits the upstream 7-element ConwayGovState: Proposals 2-tuple (GovRelation_4_SNothing, empty OMap), SNothing committee, real Constitution, Conway 31-element PParams (cur+prev), FuturePParams internal ADT ([0] = Sum NoPParamsUpdate, distinct from R183’s wire-facing Maybe Nothing = []), and DRepPulsingState = DRComplete (PulsingSnapshot, RatifyState) composing R187’s RatifyState helper plus a 4-element empty PulsingSnapshot. cardano-cli conway query gov-state returns full JSON with real Conway constitution + 31-elem PParams |
| 189 | Conway ledger-peer-snapshot (tag 34) end-to-end — closes the Conway-era LSQ gap entirely |
1 | New EraSpecificQuery::GetLedgerPeerSnapshot { peer_kind: Option<u8> } variant covering both v15+ form [34, peer_kind] and legacy singleton [34]. Dispatcher emits the V2 wire shape [1, [[0], 0x9f 0xff]] (discriminator 1 + Origin marker + indefinite-length empty pool list) regardless of requested peer_kind — cardano-cli 10.16’s decoder rejected the V23 forms (discriminators 2/3) at the negotiated NtC version. Pool list specifically requires indef-length (0x9f ... 0xff) per upstream’s toCBOR @[a] — definite-length empty list 0x80 was rejected at depth 8. cardano-cli conway query ledger-peer-snapshot returns {"bigLedgerPools": [], "slotNo": "origin", "version": 2} end-to-end. Every documented Conway-era LSQ tag now has a wire-correct dispatcher |
| 190 | Comprehensive cardano-cli parity audit + tag 12/13 dispatchers | 0 | Systematic audit of every cardano-cli conway query subcommand surfaced two operational gaps: protocol-state (tag 13 DebugChainDepState) and ledger-state (tag 12 DebugNewEpochState) returned null from yggdrasil’s fall-through Unknown, which cardano-cli rejected for protocol-state (PraosState decoder). Added EraSpecificQuery::DebugNewEpochState (singleton, returns CBOR null — accepted by query ledger-state’s permissive decoder) and EraSpecificQuery::DebugChainDepState (singleton, returns minimal Versioned-wrapped 8-element PraosState placeholder per upstream Ouroboros.Consensus.Protocol.Praos.PraosState). Discovery: PraosState is wire-encoded as Versioned 0 (...) (2-element outer [version, payload]), not as a bare 8-record — initial bare emission triggered DeserialiseFailure 1 "Size mismatch when decoding Versioned. Expected 2, but found 8". Audit also confirmed kes-period-info, leadership-schedule, stake-address-info “failures” were client-side CLI arg validation, not yggdrasil bugs (work given correct inputs) |
| 191 | Live tip-slot plumbing into protocol-state + ledger-peer-snapshot | 0 | Replaces static Origin placeholder for praosStateLastSlot (protocol-state’s PraosState) and ledger-peer-snapshot slotNo with live LedgerStateSnapshot::tip().slot(). Both queries now reflect the chain’s actual progress (e.g. lastSlot: 3960 and slotNo: 3960 at slot 3960) instead of the previous static "origin". Begins the post-audit data-plumbing arc — remaining PraosState placeholders (OCert counters + 6 nonces) require threading NonceEvolutionState and OcertCounters from the consensus runtime into LedgerStateSnapshot, tracked as R192+ |
| 192 | ChainDepStateContext snapshot infrastructure (Phase A.1) |
0 | New ChainDepStateContext companion struct in crates/ledger/src/state.rs mirrors upstream PraosState’s 6 nonces + OCert counter map without inverting the ledger→consensus dependency. Optional chain_dep_state field on LedgerStateSnapshot with with_chain_dep_state(ctx) builder + chain_dep_state() accessor. encode_praos_state_versioned branches on context presence: live OCert counters + 6 nonces (Nonce::Hash → [1, h], Neutral → [0]) when populated, neutral fallback otherwise. Foundational layer for R193+ runtime-attach work — once the runtime calls snapshot.with_chain_dep_state(ctx), every subsequent protocol-state query surfaces live data with no further encoder changes |
| 193 | Live GovRelation from EnactState (Phase A.3 first slice) |
0 | New encode_strict_maybe_gov_action_id helper emitting upstream GovRelation field shape (SNothing → [], SJust id → [id_cbor]). encode_enact_state_for_lsq field 7 (ensPrevGovActionIds) and encode_conway_gov_state_for_lsq field 1 GovRelation now read live from EnactState::prev_pparams_update / prev_hard_fork / prev_committee / prev_constitution (R67 lineage tracking, all public fields). No code change needed when chain has governance traffic — the encoders surface real lineage IDs automatically |
| 194 | Live DRep / SPO stake distributions + stake-deleg deposits (Phase A.4) | 0 | Three new encoder helpers (encode_drep_stake_distribution_for_lsq, encode_spo_stake_distribution_for_lsq, encode_stake_deleg_deposits_for_lsq) replace the empty-map placeholders for GetDRepStakeDistr, GetSPOStakeDistr, GetStakeDelegDeposits. All compute live values from snapshot’s stake_credentials, reward_accounts, and delegated_pool/delegated_drep fields. cardano-cli conway query spo-stake-distribution now surfaces preview’s three registered pools with real cold-key hashes (38f4a58a..., 40d806d7..., d5cfc42c...) instead of empty list |
| 195 | Live ledger-peer-snapshot pool list (Phase A.5) | 0 | New encode_ledger_peer_snapshot_v2_for_lsq(snapshot) helper emits the upstream V2 LedgerPeerSnapshotV2 wire shape with live data from pool_state: each registered pool surfaces with its real LedgerRelayAccessPoint endpoints (DNS / IPv4 / IPv6 detected via IpAddr::parse). R237 replaces the old AccPoolStake/PoolStake 0/1 placeholders with live cumulative stake CDF rationals from the set stake snapshot, ordered by descending pool stake with deterministic hash tie-breaks. Discovery: NonEmpty Relays requires indef-length CBOR encoding (cardano-cli rejected definite-length at depth 20). |
| 196 | OCert counter sidecar load (Phase A.2 partial, superseded by R238 storage shape) | 0 | Introduced read-side PraosState plumbing for OpCert counters using the former root-level ocert_counters.cbor mirror. R238 removes that mirror path; LSQ now attaches OpCert counters only from exact chain_dep_state/<slot-hex>.cbor ChainDepState bundles. |
| 197 | NonceEvolutionState CBOR codec + sidecar load (Phase A.2 next, superseded by R238 storage shape) | 0 | Introduced CborEncode/CborDecode for NonceEvolutionState and the former root-level nonce_state.cbor load path. R238 keeps nonce state inside node-owned ChainDepState bundles and removes the public save_nonce_state/load_nonce_state storage helpers. |
| 198 | Sync-side persist for nonce_state — live nonces in protocol-state (Phase A.2 final, superseded by R238 storage shape) | 0 | Wired the former root-level nonce mirror persist path and proved live nonces surfaced through protocol-state. R238 replaces nonce_state.cbor + ocert_counters.cbor with canonical slot-indexed chain_dep_state/<slot-hex>.cbor bundles persisted only at checkpoint cadence. |
| 199–200 | Phase B verified resolved + Phase C.1 apply-batch histogram | 0 | R199 reproduced multi-peer dispatch at --max-concurrent-block-fetch-peers 4: 22K blocks synced in 2 min, 667 immutable files written, restart resumed from checkpoint at slot 21960 — R91 livelock symptom no longer reproduces. R200 added yggdrasil_apply_batch_duration_seconds Prometheus histogram (10 cumulative buckets [1ms, 5ms, 10ms, 50ms, 100ms, 500ms, 1s, 5s, 10s, +Inf] + _sum/_count) instrumented at 2 reconnecting-runtime apply sites. Operational baseline: ~206 ms/batch on preview, both observations land in the [0.1, 0.5] bucket |
| 201 | Audit baseline pin refresh (Phase E.1, 4/5 drifted) | 0 | Advanced 4 documentary pins to live HEAD: cardano-ledger 42d088ed84b7…, ouroboros-consensus c368c2529f2f…, plutus e3eb4c76ea20…, cardano-node 799325937a45…. cardano-base intentionally deferred (mirrors vendored test-vector directory name). Drift detector now reports 5/6 pins in-sync (was 1/6). All 3 drift-guard tests pass (40-char hex format, 6-repo cardinality, cardano-base ↔ vendored directory match) |
| 202 | StakeSnapshots snapshot infrastructure (Phase A.7 first slice) | 0 | New optional stake_snapshots: Option<StakeSnapshots> field on LedgerStateSnapshot + with_stake_snapshots() builder + stake_snapshots() accessor (mirrors R192’s chain_dep_state companion-field pattern). encode_stake_snapshots branches on accessor presence: real per-pool [mark, set, go] totals via IndividualStake::get × Delegations::iter filter when attached, R163/R179 placeholder (zero per-pool, 1-lovelace NonZero Coin totals) when not. Read-side complete; runtime-attach call site deferred to follow-up |
| 203 | stake_snapshots.cbor sidecar persist+load — Phase A.7 closed | 0 | New STAKE_SNAPSHOTS_FILENAME + save_stake_snapshots/load_stake_snapshots storage helpers; sync.rs persists tracking.stake_snapshots at every checkpoint landing alongside OCert + nonce sidecars; attach_chain_dep_state_from_sidecar extended to load stake_snapshots.cbor and call with_stake_snapshots(...). All three consensus-side sidecars (OCert counters, nonces, stake snapshots) now persist + load + attach end-to-end. Stake totals stay 0/1 until preview crosses epoch boundary and snapshot rotation fires |
| 204 | gov-state OMap proposals shape adapter (Phase A.3 closed) | 0 | New encode_gov_action_state_upstream helper adapts yggdrasil’s reduced 4-field GovernanceActionState (proposal/votes/proposed_in/expires_after) to upstream’s 7-field GovActionState era (gasId/committeeVotes/dRepVotes/stakePoolVotes/proposalProcedure/proposedIn/expiresAfter). Splits unified votes: BTreeMap<Voter, Vote> into 3 maps by voter type (committee/drep/spo) with deterministic CBOR ordering. encode_conway_gov_state_for_lsq field 1 OMap now iterates governance_actions() and emits each entry via the new helper. Empty list on preview (no proposals yet); will surface real proposals when chain has governance traffic |
| 205 | Comprehensive end-to-end verification (post-Phase A) | 0 | Operational verification: 25/25 cardano-cli conway query subcommands pass on a fresh preview sync; the then-current root-level consensus sidecars persist atomically; node restart resumes from checkpoint at slot 9960 → advances to 11940 with live nonces preserved across restart. R238 supersedes the root-level nonce/OpCert mirrors with canonical chain_dep_state/<slot-hex>.cbor bundles. |
| 206 | Parity proof report — Phase E.3 closed | 0 | New docs/PARITY_PROOF.md cumulative reference document covering 205 rounds: 25/25 cardano-cli subcommand verification, 3 consensus sidecars, sync robustness verification, observability baseline, upstream drift status, full open/closed/deferred matrix, and reproduction commands. Canonical “what works today” reference for operators/auditors. No code changes |
| 207 | Multi-network verification (preprod) | 0 | Boot fresh preprod sync (no era floor) — 87K blocks synced in 35s, era progressed Byron → Shelley → Allegra by slot 87440. The then-current root-level consensus sidecars persisted on preprod identically to preview; R238 supersedes nonce/OpCert mirrors with canonical ChainDepState bundles. 6/6 baseline cardano-cli queries pass (tip, protocol-parameters, era-history, slot-number, utxo --whole-utxo, tx-mempool info). |
| 208 | Mainnet boot smoke test (Phase E.2 partial, later superseded) | 0 | Historical diagnostic: quick 2-min --network mainnet smoke test booted cleanly but did not advance past Origin. R211 fixed the Byron EBB hash + same-slot consensus issue, R212 verified cardano-cli queries against active mainnet sync, and R213 fixed the large-payload mux egress limit. |
| 209 | Documentation consistency pass (post-R208 update) | 0 | archive/PARITY_PLAN.md Executive Summary refreshed with post-R208 reality: sidecars listed, multi-network evidence cited, mainnet gap acknowledged. Top + bottom pointer to docs/PARITY_PROOF.md added so readers see the canonical operational status reference. “To achieve full parity” list updated with the 7 documented deferred items + bar-to-close estimates. No code changes; documentation hygiene only |
| 210 | Mainnet stall diagnostic — apply ruled out | ~30 | Adds opt-in YGG_SYNC_DEBUG=1 apply-side trace at apply_verified_progress_to_chaindb call site in node/src/runtime.rs (~line 5008). 90 s mainnet run shows: 0 apply-side traces vs 634 [ygg-sync-debug] blockfetch-range lines and 2 demux-exit error=connection closed by remote peer. ChainSync header decodes cleanly for Byron range Origin → SlotNo(648087), but the IOG backbone peer closes the mux during the BlockFetch request, so apply_verified_progress is never invoked and no checkpoint/sidecar/volatile/immutable file lands. Conclusion: R208 mainnet gap is at the BlockFetch wire layer, not at apply / ledger / storage — every apply-path hypothesis ruled out. R211+ Phase E.2 wire-byte BlockFetch diagnosis is now narrowly scoped to MsgRequestRange encoding + Byron EBB hash indirection |
| 211 | Mainnet sync unblocked — Byron EBB hash + same-slot tolerance | ~80 | Closes the mainnet sync gap. Two-bug cascade: (1) point_from_raw_header used byron_main_header_hash ([0x82, 0x01]) for EBB-shape headers, but EBBs require [0x82, 0x00] per Cardano.Chain.Block.Header.boundaryHeaderHashAnnotated; (2) consensus ChainState::roll_forward used strict slot monotonicity (<=), rejecting Byron EBB→main_block at same slot 0. Fix: new byron_ebb_header_hash helper; decode_point_from_byron_raw_header returns Some(Point) for EBBs with slot=epoch * 21600 and EBB hash; consensus slot check relaxed to < (block-no contiguity catches re-application; Praos guarantees ≤ 1 block/slot post-Byron). R210’s YGG_SYNC_DEBUG=1 instrumentation mirrored to shared-chaindb apply call site (the production NtN+NtC path R210 missed). Test updates: roll_forward_accepts_same_slot_byron_ebb_main_pair, point_from_raw_header_decodes_observed_byron_serialised_header_envelope updated to expect EBB hash + slot=0 from inner header. Verification — mainnet syncs: 60s window advances tip to slot 197, volatile 1.5 MB, ledger 1.4 MB, checkpoint persisted at slot 47. Compare R210→R211: apply 0→6, volatile 0B→1.5MB, ledger 0B→1.4MB, tip Origin→slot 197, cleared-origin 12→0 |
| 212 | Mainnet operational verification with cardano-cli + sidecars | 0 | Third-network verification completing the multi-network parity matrix. After R211’s mainnet sync fix, started fresh mainnet sync and dispatched cardano-cli queries: query tip --mainnet returns valid JSON (block 197→397, era Shelley, hash matches), query era-history --mainnet returns 2-era CBOR summary, query slot-number 2024-06-01T00:00:00Z returns 125712000, query protocol-parameters --mainnet returns 17-element Shelley shape, query tx-mempool info --mainnet returns valid mempool JSON. The then-current root-level consensus sidecars persisted on mainnet; R238 supersedes nonce/OpCert mirrors with canonical ChainDepState bundles. Combined with R205 (preview Conway) + R207 (preprod Allegra), all 3 official Cardano networks demonstrate operational LSQ surface + sidecars. Known limitation: query utxo --whole-utxo --mainnet failed with BearerClosed — concurrent-access issue, separate follow-up. No code changes |
| 213 | Mux egress: allow single payloads larger than EGRESS_SOFT_LIMIT | ~10 | Closes R212’s BearerClosed limitation. Diagnosis: YGG_NTC_DEBUG=1 traced LSQ response = 1.3 MB; send fails at current + len > egress_limit check with current=0, len=1.3MB, limit=262KB. Root cause: yggdrasil’s per-protocol egress check was rejecting single payloads > limit even with empty buffer — contradicting upstream network-mux’s egressSoftBufferLimit semantic which is back-pressure on accumulated bytes. Fix: current > egress_limit (only reject when buffer is already over). Doc comments + integration test updated. Verification: cardano-cli query utxo --whole-utxo --mainnet returns 14 505 AVVM entries totaling 31.1 billion ADA — full mainnet bootstrap UTxO. R213 is the bug that had been latent for ~200 rounds; testnet UTxOs were too small to trip it |
| 214 | Phase A.6 — GetGenesisConfig ShelleyGenesis serialiser | ~200 | Closes the final Phase A item (Phase A is now 7/7 complete). New encode_shelley_genesis_for_lsq(genesis, pp, chain_start_unix_secs) -> Vec<u8> helper emits upstream’s 15-element CBOR list per Cardano.Ledger.Shelley.Genesis.encCBOR: systemStart UTCTime [mjd, picosOfDay, 0], networkMagic, networkId, activeSlotsCoeff (tag 30 UnitInterval), Word64 scalars (k, epochLength, slotsPerKESPeriod, maxKESEvolutions, slotLength picoseconds, updateQuorum, maxLovelaceSupply), 17-element Shelley PP, genDelegs map, initialFunds map, staking 2-element record. BasicLocalQueryDispatcher extended with genesis_config_cbor: Option<Arc<Vec<u8>>> field + with_genesis_config_cbor() builder; RunNodeRequest::genesis_config_cbor field threads pre-encoded bytes from CLI startup to the NtC task. Test shelley_genesis_encoder_emits_15_element_list pins the 15-element shape + MJD≈58019 for mainnet’s 2017-09-23 system start. Mainnet verification: Net.NtC starting NtC local server genesisConfigCborBytes=833 — dispatcher has 833 bytes of real mainnet genesis CBOR available; query tip continues to work in parallel |
| 215 | Multi-network regression verify post-R211–R214 | 0 | Confirms R211 (Byron EBB hash + same-slot consensus) + R213 (mux egress back-pressure) + R214 (GetGenesisConfig encoder) haven’t regressed preview/preprod operational surfaces. Preview (YGG_LSQ_ERA_FLOOR=6): tip era=Conway block 7960; conway query gov-state + conway query constitution return full Conway state; sidecars persist 114B + 218B + 18B; R214 genesis-config 821 bytes. Preprod: tip era=Allegra block 91440; baseline cardano-cli (era-history, protocol-parameters, tx-mempool info) all decode end-to-end; sidecars persist; R214 genesis-config 821 bytes. Cumulative multi-network parity matrix now confirmed post-R214 across preview/preprod/mainnet. No code changes |
| 216 | Phase E.1 pin refresh round 2 — ouroboros-consensus + plutus |
~5 | Refreshes 2 documentary pins that had drifted since R201 (~15 rounds ago). UPSTREAM_OUROBOROS_CONSENSUS_COMMIT c368c2529f2f… → b047aca4a731…; UPSTREAM_PLUTUS_COMMIT e3eb4c76ea20… → 4cd40a14e364…. Doc comments record both R201 and R216 advances with rationale. Drift report goes from drifted=3 to drifted=1; only cardano-base remains DRIFT (vendored-fixture-coupled, separate Phase E.1 slice). All 5 documentary pins now in-sync. Companion update to docs/UPSTREAM_PARITY.md pinning table + drift snapshot |
| 217 | Phase C.2 prerequisite — fetch-batch duration histogram | ~80 | Adds yggdrasil_fetch_batch_duration_seconds Prometheus histogram mirroring R200’s apply-batch shape. New record_fetch_batch_duration(Duration) on NodeMetrics with same bucket boundaries as apply for direct comparison. Both sync_batch_verified_with_tentative call sites in runtime.rs (chaindb + shared-chaindb) bracket batch_fut with fetch_start.elapsed() recording. Drift-guard test extended with 3 accept clauses. Mainnet baseline (60s, 4 batches): fetch sum=51.38s avg=12.85s/batch vs apply sum=0.87s avg=0.22s/batch → fetch is ~59× more expensive than apply. All 4 fetch observations in +Inf bucket (>10s); all 4 apply in ≤ 0.5. Strategic insight: Phase C.2 best-case throughput improvement = 0.22/13.07 ≈ 1.7% for multi-day effort. De-prioritises C.2; multi-peer dispatch (--max-concurrent-block-fetch-peers > 1, already implemented) is the actual sync-rate lever |
| 218 | Mainnet multi-peer dispatch operational verification | 0 | Operationally verifies R217’s strategic insight on mainnet with --max-concurrent-block-fetch-peers 4. R217 vs R218 comparison: fetch sum 51.38→85.63s (1.67×) but fetch avg/batch dropped 12.85→8.56s (33% faster); apply unchanged at ~0.22s/batch (within noise); blockfetch_workers_registered 0→2; tip advanced 197→495 in 90s window; throughput 3.33→5.55 blk/s (1.67×, 67% faster). Worker count=2 despite knob=4 because only 2 warm peers established — adding more topology peers unlocks further linear scaling. Confirms multi-peer is the immediate lever; C.2 pipelining is mathematically dominated. No code changes |
| 219 | Operator runbook + PARITY_PROOF refresh post-R217/R218 | 0 | Captures R217 fetch-batch histogram + R218 multi-peer measurements into operator-facing docs. Runbook §6.5c gains operator-quantified table (single-peer vs knob=4 fetch/apply per-batch and throughput) + topology health interpretation (fetch_avg/batch ≈ baseline / N for N active workers). Runbook §7 metrics list documents both yggdrasil_fetch_batch_duration_seconds and yggdrasil_apply_batch_duration_seconds for cross-comparison. PARITY_PROOF §4 extends Phase C.1 observability with R217 mainnet baseline + R218 quantified comparison table (single-peer vs 2-worker). Documentation hygiene only |
| 220 | Full P2P functionality — server ChainSync Tip envelope fix |
~70 | Closes a latent inbound-P2P parity gap: server-side chain_tip()/next_header()/find_intersect()/tentative_tip() emitted bare Point::to_cbor_bytes() ([] or [slot, hash]) where upstream ChainSync expects Tip envelope ([] or [point, blockNo]). New chain_tip_envelope_cbor helper in server.rs resolves Point→(Point,BlockNo)→Tip::encode_cbor per upstream Cardano.Slotting.Block.Tip. Test pinning the wrong shape was updated. Verification: pre-R220 instance B couldn’t sync from instance A (expected major 4, got 0 decode errors, blocks_synced=0); post-R220 B successfully synced 250 blocks from A with reconnects=0 and no decode errors. All P2P layers verified: NtN handshake, Mux, ChainSync, BlockFetch, KeepAlive, TxSubmission2, PeerSharing, inbound listener, peer governor. Strategic: byte-accurate server-side ChainSync wire parity, completes bidirectional P2P parity for yggdrasil to participate as a peer (relaying blocks to other nodes) |
| 221 | ChainProvider trait contract — separate chain_tip from chain_tip_point |
~30 | Closes a follow-on bug from R220’s trait change. MsgRollBackward { point, tip } carries TWO shapes — point is bare Point (rollback target) and tip is Tip envelope (chain tip). Tentative-trap rollback path was using chain_tip() for BOTH slots after R220. Fix: new chain_tip_point() method on ChainProvider trait returns bare Point CBOR; production SharedChainDb + mock impls; rollback callsite updated to use both methods. Trait docs include per-method table mapping methods to wire-protocol use sites. Test assertions updated to pin both Tip envelope and bare Point shapes correctly. Verification: same instance-to-instance preprod test as R220; B synced 250 blocks from A with reconnects=0, no chainsync decode errors. R220+R221 establish a clean trait-level invariant for all server-side ChainSync wire-shape uses |
| 222 | Phase D.2 first slice — PeerLifetimeStats foundation | ~80 | Adds parallel-tracking shadow data structure for lifetime peer stats independent of session-keyed governor state. New PeerLifetimeStats struct (sessions, bytes_in, bytes_out, successful_handshakes, failures_total, first_seen, last_seen); new GovernorState::lifetime_stats: BTreeMap<SocketAddr, _> field; three accessor methods (record_lifetime_session_started, record_lifetime_session_failure, record_lifetime_traffic); read-only lifetime_stats_for. Regression test lifetime_stats_accumulate_across_simulated_reconnects pins the accumulation contract. Foundation slice — does NOT yet wire update points into the runtime; subsequent slices wire handshake-complete + mux-abort + byte-accounting sites + /metrics Prometheus counters. Strategic: lays data-model foundation so future Phase D.2 slices wire concrete update points without re-litigating the design |
| 223 | Phase D.2 second slice — wire lifetime stats + aggregate Prometheus exposition | ~50 | Wires the first concrete update points and exposes aggregate counters via /metrics. promote_to_warm success branch calls record_lifetime_session_started; error branch calls record_lifetime_session_failure. Two new aggregate counters on NodeMetrics: peer_lifetime_sessions_total + peer_lifetime_failures_total; standard Prometheus exposition under yggdrasil_peer_lifetime_*_total; runtime governor tick folds across lifetime_stats.values() to compute totals. Mainnet verification: 60s knob=4 sync shows yggdrasil_peer_lifetime_sessions_total=2 while live active_peers=1 — lifetime counter distinct from live gauge, confirming the observability win. Operators can rate(yggdrasil_peer_lifetime_sessions_total[5m]) for real peer churn rate. Remaining D.2 slices (byte counters from BlockFetchInstrumentation) deferred |
| 224 | Phase D.2 third slice — lifetime bytes-in counter | ~40 | Completes the major Phase D.2 deliverable. New set_lifetime_bytes_in(peer, total) method (cumulative-overwrite) mirrors per-peer BlockFetchInstrumentation::bytes_delivered (already monotonic across reconnects) into PeerLifetimeStats.bytes_in at each governor tick. Runtime iterates pool.peers BTreeMap and refreshes per-peer entries; folds aggregate. New peer_lifetime_bytes_in_total counter on NodeMetrics exposed as yggdrasil_peer_lifetime_bytes_in_total. Mainnet verification: 75s knob=4 sync shows yggdrasil_peer_lifetime_bytes_in_total=2 511 595 (2.5 MB cumulative blocks fetched), distinct from active_peers=3. Order-of-magnitude check matches R218’s per-batch numbers (~50 KB/batch × ~50 batches). Bytes-out remains 0 (deferred — requires per-mini-protocol egress accounting) |
| 225 | Phase D.1 first slice — rollback-depth histogram | ~70 | Lays observability foundation for Phase D.1 deep cross-epoch rollback recovery. New rollback_depth_buckets: [AtomicU64; 7] + _sum_blocks + _count fields on NodeMetrics; bucket boundaries [1, 2, 5, 50, 2160 (k), 10_000, +Inf] span shallow chain reorgs through cross-epoch and full-resync. New record_rollback_depth(blocks) method follows R200/R217 cumulative-bucket pattern. Both production apply call sites in runtime.rs record observations when progress.rollback_count > 0; depth unit is rolled-back transactions. yggdrasil_rollback_depth_blocks_{bucket,sum,count} exposed via standard Prometheus histogram format; drift-guard test extended. Preprod verification: 1 observation at depth 0 (session-start confirm-shape rollback), count=1, sum=0, blocks_synced=149. Operators can histogram_quantile(0.99, rate(yggdrasil_rollback_depth_blocks_bucket[1h])) to alert on rare deep cross-epoch rollbacks. Full Phase D.1 recovery (historical stake-snapshot reconstruction) remains deferred — R225 is observability prerequisite |
| 226 | Phase D.2 fourth slice — unique-peers + handshakes-total counters | ~30 | Adds two cheap-to-compute aggregate counters completing the 5-counter Phase D.2 lifetime peer-stats deliverable. New peer_lifetime_unique_peers (gauge, cardinality of lifetime_stats map) + peer_lifetime_handshakes_total (counter, sum of successful_handshakes). Runtime governor-tick fold extended; setters exposed. Mainnet verification: unique_peers=3, handshakes_total=2, sessions=2, bytes_in=1 548 246 — unique_peers > sessions reveals 3 peer addresses tracked but only 2 promoted to warm (useful registry-leakage signal). Operators can derive failures/sessions reliability ratio, bytes_in/sessions throughput, 1 - sessions/unique_peers registry-leakage indicator. Bytes-out remains 0 (deferred per-mini-protocol egress accounting) |
| 227 | PARITY_PROOF cumulative-status refresh | 0 | Refreshes docs/PARITY_PROOF.md cumulative status report with the R211→R226 arc evidence. Phase status table reclassified: 13 closed/verified, 1 partial (D.1 observability), 4 deferred. New §4b “Phase D.2 multi-session peer accounting” with 5-counter Prometheus deliverable + operator-derived signals. New §4c “Phase D.1 rollback-depth observability” with histogram bucket structure + alert query. §5 upstream alignment refreshed with R216 advances. Adds new rows for B (mainnet) and B (P2P) phase items capturing R211+R213+R220+R221 work. Documentation hygiene only |
| 228 | Operator runbook §7 metrics list refresh | 0 | Extends docs/MANUAL_TEST_RUNBOOK.md §7 metrics-snapshot section with detailed interpretation of all 6 new R211→R226 observability metrics: 5 lifetime peer-stats counters + 1 rollback-depth histogram. Each metric documented with source, operator interpretation, and known limitations. Adds 4 PromQL recipe snippets for derived signals (reliability ratio, avg bytes/session, registry-leakage, peer churn rate) + Phase D.1 rollback alert query. Documentation hygiene only |
| 229 | Phase D.2 Prometheus shape regression test | ~70 | New regression test node_metrics_tracks_phase_d2_lifetime_peer_stats pins the 5-counter contract: 4 *_total counters MUST emit # TYPE …_total counter, 1 unique_peers MUST emit # TYPE … gauge. Drift in counter/gauge type silently breaks rate(...) semantics on operator dashboards. Test exercises zero-init state, setter wiring, snapshot fields, and Prometheus text format for all 5 metrics. Test count 4746→4747 |
| 230 | Phase D.1 rollback-depth histogram regression test | ~80 | Mirrors R229’s regression-pin pattern for the Phase D.1 rollback-depth histogram (R225). New test node_metrics_tracks_phase_d1_rollback_depth_histogram pins three load-bearing aspects: (1) bucket boundaries [1, 2, 5, 50, 2160 (k), 10_000, u64::MAX]; (2) cumulative-bucket semantic; (3) Prometheus exposition shape (# TYPE … histogram, _bucket{le=…}, _sum, _count). Three observations exercise inclusion + exclusion at boundary depths (0, 3, 5000). Together with R229, every R211→R226 observability metric now has explicit Prometheus-output regression coverage. Test count 4747→4748 |
| 231 | R200 apply-batch + R217 fetch-batch histogram regression test | ~80 | Completes the cumulative regression coverage of R211→R226 observability. New test node_metrics_tracks_fetch_and_apply_batch_histograms pins: (1) shared bucket boundaries [1ms, 5ms, 10ms, 50ms, 100ms, 500ms, 1s, 5s, 10s, +Inf] (drift breaks R217+R218 fetch-vs-apply quantification); (2) cumulative-bucket semantic; (3) Prometheus exposition. Real-world observation values exercise inclusion + exclusion at relevant boundaries: apply 200ms (mainnet typical), fetch 12.85s (single-peer baseline), fetch 8.56s (multi-peer 2-worker). R229+R230+R231 together pin all 4 R211→R226 Prometheus contracts. Test count 4748→4749 |
| 232 | README cumulative R211→R231 arc summary | 0 | Refreshes README.md “Current Status” with cumulative arc bullets: mainnet sync end-to-end, full LSQ surface, bidirectional P2P parity, Phase A.6+D.2+D.1+E.1 deliverables, Prometheus-output regression coverage. New “Deferred substantive items” subsection with operator-facing rationale for the 4 remaining items. Test count baseline updated: 4 640 (v0.2.0) → 4 749. Closes the documentation hierarchy: README ↔ PARITY_PROOF ↔ runbook ↔ AGENTS ↔ operational-runs/. No code changes |
| 233 | archive/PARITY_PLAN.md Executive Summary post-R232 refresh | 0 | Refreshes archive/PARITY_PLAN.md Executive Summary with: 6 new “achieved-items” rows for R220+R221+R222-R226+R225+R201+R216+R229+R230+R231; deferred-items list reduced from 7 to 5 by removing items already closed (A.6 R214, C.2 R217 de-prioritised); footer “Actual delivery status” refreshed R214 → R232 with canonical state. Closes the documentation hierarchy refresh started by R227+R228+R232 — every project doc now consistently reflects post-R232 state. No code changes |
| 234 | Phase D.2 bytes-out initial slice — BlockFetch server bytes-served | ~50 | Closes the major Phase D.2 bytes-out gap. New aggregate Prometheus counter yggdrasil_blockfetch_server_bytes_served_total for bytes served by the BlockFetch SERVER (yggdrasil-as-peer egress), counterpart to R224’s peer_lifetime_bytes_in_total (yggdrasil-as-client ingress). run_blockfetch_server gains metrics: Option<&NodeMetrics> param; sums blocks.iter().map(b.len()).sum() after each serve_batch; caller in run_inbound_accept_loop clones the metrics handle into the BlockFetch responder spawn. Operational proof of correctness: instance-to-instance preprod sync (60s) shows A’s egress=100,500 bytes EXACTLY matches B’s ingress=100,500 bytes — no leakage, no double-counting. Follow-up egress protocols land in R235 and R237. |
| 235 | Phase D.2 bytes-out — ChainSync server bytes-served | ~40 | Extends R234’s instrumentation pattern to ChainSync server. New aggregate counter yggdrasil_chainsync_server_bytes_served_total covering RollForward + IntersectFound + IntersectNotFound payload bytes. run_chainsync_server gains metrics: Option<&NodeMetrics> param; closure record_emit(header, tip, metrics) instruments 4 roll-forward sites + explicit intersect-site instrumentation. Caller wiring reuses R234’s bf_metrics clone via cs_metrics. Verification (instance-to-instance preprod, 30s): 19,635 bytes served (100 RollForward msgs × ~196 bytes each — Byron header + tip envelope + CBOR framing). Ratio to BlockFetch is ~5× as expected (header-only vs full block). R237 completes the remaining low-volume server egress counters. |
| 236 | Live PoolDistr for stake-distribution and SPO-stake-distribution |
~100 | encode_stake_distribution_map now sources IndividualPoolStake entries from StakeSnapshots.set.pool_stake_distribution() with real pool stake, total-active rational denominator, CompactCoin, and VRF key hash; encode_spo_stake_distribution_for_lsq uses the same set snapshot instead of reward-balance approximation. Empty-snapshot fallback preserves [empty_map, 1] for NonZero compatibility. |
| 237 | LSQ GetPoolDistr2, egress attribution, rollback replay slice |
~180 | Closes the remaining GetPoolDistr2 data placeholder by reusing the live PoolDistr encoder with optional pool-hash filtering. LedgerPeerSnapshotV2 now emits live cumulative pool stake CDF rationals. Server egress counters now cover KeepAlive, TxSubmission2, and PeerSharing in addition to BlockFetch/ChainSync; node keeps per-peer egress internally and folds it into PeerLifetimeStats::bytes_out while Prometheus remains aggregate-only. Rollback checkpoint recovery now truncates at the actual rollback point and, when stake snapshots are enabled, replays immutable+volatile storage through advance_ledger_with_epoch_boundary to rebuild stake snapshots and current-epoch pool block counts. |
| 238 | Rollback ChainDepState sidecar parity hardening | ~180 | Adds opaque slot-indexed chain_dep_state/<slot-hex>.cbor snapshots in storage and node-owned CBOR bundles [version=1, point, nonce_state|null, ocert_counters|null]. Verified sync writes bundles only when ledger checkpoints persist after nonce/OpCert updates, cuts batches immediately after RollBackward, restores nonce/OpCert from the newest sidecar at-or-before the rollback point, verifies the bundled point is on the current chain prefix, and replays raw stored blocks to the rollback target. Startup/reconnect recovery uses the same restore-and-replay path to the recovered storage tip; LSQ protocol-state uses exact point sidecars. Persistent non-origin rollback fails closed when exact ChainDepState history is unavailable. |
| 239 | Phase E.1 cardano-base fixture refresh |
0 | Moves the vendored cardano-base vector tree from db52f43b38ba5d8927feb2199d4913fe6c0f974d to live upstream HEAD 7a8a991945d401d89e27f53b3d3bb464a354ad4c, refreshes the Praos VRF and BLS12-381 vector paths from official IntersectMBO/cardano-base, and updates CARDANO_BASE_SHA, UPSTREAM_CARDANO_BASE_COMMIT, and provenance docs in lockstep. node/scripts/check_upstream_drift.sh now reports all 6 canonical pins in-sync. |
| 240 | Parallel BlockFetch soak automation + runbook correction | 0 | Adds node/scripts/parallel_blockfetch_soak.sh, a reproducible §6.5 harness that starts yggdrasil-node with --max-concurrent-block-fetch-peers, captures Prometheus snapshots, asserts worker registration/migration metrics, optionally runs compare_tip_to_haskell.sh against a Haskell socket, scans logs for worker-channel failures, and emits $LOG_DIR/summary.txt. Node smoke tests pin help output and fail-closed rejection of the legacy single-peer knob. Runbook §6.5 now prefers this harness, documents the real env-var interface for compare_tip_to_haskell.sh, and removes stale gov-state gap wording because R188/R193/R204 closed that Conway LSQ surface. Release docs include the new bundled script. |
| 241 | Devcontainer preprod BlockFetch smoke | 0 | Rebuilt devcontainer toolchain validation plus a short preprod parallel_blockfetch_soak.sh smoke at max_concurrent_block_fetch_peers=2: 1 150 blocks synced, 6 workers registered/migrated, no worker-channel failure traces. Evidence hardening only; the required §6.5 6h/24h sign-off remains operator-time. |
| 242 | Upstream cardano-node-tests harness smoke |
0 | Validated the official upstream runner/runc.sh custom-binary path against a static MUSL Yggdrasil binary. Documented the .bin/ static-binary requirement, wrapper translation for cardano-cli --version, and CI/manual harness prerequisites. Local devcontainer pytest execution remains blocked by Docker-outside-of-Docker bind-mount visibility, not by Yggdrasil runtime behavior. |
| 243 | cardano-ledger import-only pin refresh |
0 | Advances UPSTREAM_CARDANO_LEDGER_COMMIT from 42d088ed84b7… to live HEAD 110b30e7abd8…. Official upstream PR #5787 removes one redundant import from Cardano.Ledger.Shelley.API.Mempool; no ledger rule, CDDL, or binary codec behavior changed in the ported subset. Drift detector reports all 6 canonical pins in-sync again. |
| 244 | Byron genesis canonical JSON hash verification | ~120 | Mirrors upstream Cardano.Chain.Genesis.Data.readGenesisData: parse Byron genesis as Canonical JSON and hash the rendered canonical JSON bytes. verify_known_genesis_hashes() now verifies Byron plus Shelley/Alonzo/Conway, Node.GenesisHash.Verified reports byronVerified, validate-config/manual output is now 4/4 verified, and vendored mainnet/preprod/preview Byron hashes are regression-tested. |
| 245 | cardano-ledger BBODY/GOV drift refresh |
~35 | Advances UPSTREAM_CARDANO_LEDGER_COMMIT from 110b30e7abd8… to live HEAD b90b97488da3…. Upstream GOV switches preceedingHardFork to accumulated proposals; Yggdrasil’s accumulated pending-proposal validation path already matched and the hard-fork sequencing tests remain green. Upstream BBODY temporarily disables HeaderProtVerTooHigh for testnets until Dijkstra; VerificationConfig.network_magic now mirrors netId == Mainnet || curProtVerMajor >= 12 while keeping MaxMajorProtVer enforced on every network. Drift detector reports all 6 canonical pins in-sync again. |
| 246 | Preview Plutus well-formedness/runtime replay parity | ~95 | Closes the observed preview Babbage replay blockers: on-chain script bytes are treated as raw PlutusBinary (CBOR bytestring containing Flat) under protocol-version language gates, Babbage/Conway reference inputs are ordered by ShelleyTxIn, CEK non-constant runtime values use ExMemory = 1, pre-Conway upper-only validity intervals encode inclusive PV1.to, Plutus Integer is arbitrary precision across Flat, CBOR PlutusData, and builtins, Plutus serialiseData CBOR shape matches upstream, and legacy AccountRegistration is not over-collected as a Certifying purpose. Focused Plutus/ledger/node tests, release build, cargo check-all, cargo test-all, and cargo lint passed. Refscan reached slot 901725; bounded live preview reached checkpoint 1038614 before a non-Plutus stale-checkpoint reward mismatch. |
| 247 | Preview Origin-prefix BlockFetch replay parity | ~40 | Verified sync batches that start at Point::Origin now use the first announced concrete ChainSync header as the BlockFetch lower bound, preserving the slot-0 preview prefix instead of fetching only the final announced header. A clean preview replay stored slots 0, 60, 300, and 320 and advanced to slot 101100 without the prior missing-UTxO stop. |
| 248 | TPraos active-overlay VRF parity | ~120 | Shelley-family TPraos blocks now classify the overlay schedule using the active decentralisation parameter, epoch first slot, active slot coefficient, and genesis delegation map. Active overlay slots verify the selected genesis delegate cold key, delegate VRF key, and both TPraos VRF proofs, then skip pool stake leader-threshold validation to match upstream pbftVrfChecks; reserved non-active overlay slots fail closed. Focused consensus/node tests and release build passed; a live preview resume crossed former blocker slot 106220, the prior 730728 MalformedReferenceScripts region, and the prior 840719 ValidationTagMismatch region, reaching Babbage slot 868687 with no VRF failure, malformed script error, validation-tag mismatch, ledger decode error, or panic. |
| 249 | Cumulative pin refresh + parallel-test isolation + sync diagnostic infrastructure | ~10 | Refreshes 5 documentary upstream pins to live HEAD (cardano-ledger ca9b8c285e44…, ouroboros-consensus 8c2475c253ab…, ouroboros-network 8fe0f8ebc262…, plutus c8f962ae75d0…, cardano-node 97036a66bcf8…); cardano-base remains pinned to the vendored R239 fixture tree. Per-repo audit via the GitHub compare API confirms all upstream changes are forward-looking; no active-era CBOR codec, validation rule, transition-system semantic, mini-protocol wire codec, CEK reduction, or cost-model parameter for Conway-or-earlier protocol versions changes. Drift detector reports drifted=0. Companion fix: local_server::tests::effective_era_index_pv_table_matches_upstream was racing with era_floor_env_var_promotes_reported_era over the process-wide YGG_LSQ_ERA_FLOOR env var; lifted ENV_LOCK to test-module scope. New YGG_SKIP_PHASE2 sync escape hatch (node/src/sync.rs::phase2_evaluator_or_trust_block) lets operators trust the on-chain is_valid tag and skip Plutus re-validation while CEK parity gaps are investigated. New diagnostic infrastructure: YGG_DUMP_VKEY_FAIL env var dumps (vkey, msg, sig) on witness verification failure, node/src/bin/dump_block.rs walks Haskell ChainDB chunk files for forensic byte-level comparison, offline Python signature classifier proves canonicality. All four verification gates pass post-fix. Discovered three open code-level parity gaps through real-network preview/preprod soaks: Gap BO (preprod TPraos VRF at slot ~429,460), Gap BP (preview Plutus V2 budget at slot ~1,462,057), Gap BQ (preview vkey witness at slot ~1,525,024). |
| 250 | Perf foundation: peer-snapshot adoption + split-gate | ~150 | Replaces placeholder peer-snapshot.json with upstream Haskell-share content for preview/preprod/mainnet (321 B → 28 KB / 15 KB / 152 KB; 1 fake pool → 131+ unique relays). Bumps useLedgerAfterSlot to upstream-aligned values (preview 107222465, preprod 118022427, mainnet 182044807) and MinNodeVersion 10.6.2 → 10.7.0 in all three config.json. Splits snapshot-vs-live-ledger gating in crates/network/src/ledger_peers_provider.rs: new always_eligible_snapshot_peers plus node/src/config.rs::NodeConfigFile::always_eligible_snapshot_fallbacks wrapper, called alongside the existing gated eligible_ledger_peer_candidates from both startup (node/src/main.rs::evaluate_ledger_derived_startup_fallbacks) and reconnect (node/src/runtime.rs) paths. Snapshot peers now eligible immediately at startup (live trace evaluated ledger-derived startup fallbacks shows snapshotEligibleCount=174 liveLedgerEligibleCount=0 decision=AwaitingLatestSlot { after_slot: 107222465 }); pre-R250 this was eligiblePeerCount=0. 2 regression tests pin the new behavior. Measured perf delta: 5-min preview soak from Origin shows Yggdrasil at 2,321 slot/s vs R249 baseline 1,653 slot/s — 40% improvement. Governor outbound-connect path doesn’t yet promote snapshot-eligible peers to BlockFetch workers (yggdrasil_blockfetch_workers_registered=0 post-R250); R254 perf round must wire snapshot peers into the warm/hot promotion loop and add batched Ed25519 verify + pipelined CBOR decode + allocator tuning to clear the 2× Haskell goal. |
| 251 | Gap BQ closure — indefinite-length CBOR span extraction | ~80 | Forensic byte-level investigation of preview vkey witness rejection at slot ~1,525,024 on tx 44ccae43…. Initial Ed25519 strictness hypothesis disproven: captured signature R/S canonical, vkey not small-order; offline OpenSSL Ed25519 also rejected against Yggdrasil’s computed msg — proving the bug was a wrong tx_body_hash, not verifier strictness. Security guardrail correctly blocked the agent-judgment Ed25519 weakening. Root cause (located via node/src/bin/dump_block.rs walking Haskell preview ChainDB chunk 353): crates/ledger/src/cbor.rs::extract_block_tx_byte_spans used strict dec.array() for the outer block, bodies array, and witness-set array; real preview Babbage blocks (e.g. block at chunk-353 offset ~114079, slot ~1,525,259) encode the bodies array itself with CBOR indefinite-length (0x9f ... 0xff, RFC 8949 §3.2.1, additional-info 31), which strict array() rejects with CborInvalidAdditionalInfo(31). When extraction failed, the apply-path macro node/src/sync.rs::alonzo_family_block_to_block_with_spans (line 4040) fell back to tx_body.to_cbor_bytes() re-serialization (always definite-length), producing a tx_body_hash that differed from the on-wire (indefinite) hash the signer signed — hence valid signatures rejected. Fix: switched all three array() calls to array_begin() plus a new collect_indefinite_or_definite_spans helper that walks both definite and indefinite-length encodings, preserving byte spans exactly. 2 regression tests pinned to indefinite-length bodies/witnesses arrays plus the existing test for indefinite-length BODY ENTRIES (R85 era). All four gates green. Verified: 25-min preview soak resumed from saved checkpoint at slot 1,488,359 and advanced cleanly to slot 1,557,718 (32K-slot margin past previous Gap BQ failure), 17 epoch boundaries crossed, zero verification errors. Forensic artifacts preserved at docs/operational-runs/2026-05-05-round-249-preview-vkey-witness-fail-{slot-1525024.log,tx-44ccae43-bytes.txt} (capture log + bytes), …-classify-signature.py (canonicality classifier). |
| Total (R1–R249) | All subsystems | ~2845 | Phase A complete (7/7); Phase D.2 lifetime + aggregate server bytes-out deliverable shipped; Phase D.1 rollback recovery now includes epoch-boundary-aware ledger replay plus exact ChainDepState sidecar restore/replay and recovered pool-block-count preservation; bidirectional P2P parity; Phase E.1 all 6 upstream pins refreshed and in-sync (R249 cumulative refresh closes the per-repo audit on the post-R245 drift); all four preset genesis hashes verified at startup; Conway BBODY testnet HeaderProtVerTooHigh grace mirrored through Dijkstra; observed preview Plutus reference-script/runtime replay blockers closed through refscan slot 901725; Origin-prefix BlockFetch and TPraos overlay VRF replay blockers closed; §6.5 BlockFetch default-flip evidence now has first-class automation; parallel-test isolation hardened against YGG_LSQ_ERA_FLOOR leakage. Remaining gates: clean/repaired preview replay past the stale-checkpoint reward stop and long-duration mainnet/operator sign-off before changing BlockFetch concurrency defaults. |