# M2 — core state and consensus hash

**Status: complete.** 181 tests passing (32 new, plus M0/M1's 149).

## What was built

| Module | Purpose |
|---|---|
| `ribbit/script.py` | base58check, output classification (P2PKH, P2SH, multisig, OP_RETURN, P2PK), script pushes |
| `ribbit/tx.py` | Class detection, **sender determination**, reference resolution, payload extraction |
| `ribbit/state.py` | Properties, balances, types 0/4/50/54, activation messages |
| `ribbit/consensushash.py` | The consensus hash, plus a per-section breakdown for diagnosing mismatches |
| `ribbit/indexer.py` | `RibbitHandler` — the bridge from `ChainFollower` to state, with a prevout cache |

## Consensus rules implemented, with sources

**Sender determination** (`omnicore.cpp:957-1021`) — two different rules:

| Class | Rule |
|---|---|
| **C** | the owner of the **first input**, full stop |
| **B** | **largest input by sum** — input values summed per source address |

The Class B tie-break is subtle and we reproduce it deliberately: Omni iterates a
`std::map<std::string, int64_t>` (lexicographic by address) and replaces its
running maximum only on a **strictly greater** value, so when two addresses
contribute equally the **lexicographically smaller address wins**.

**Allowed input types** (`rules.cpp:416-430`) — only P2PKH and P2SH. Anything else
invalidates the transaction.

**Reference address** (`omnicore.cpp:1154-1190`) — candidates are outputs with an
extractable destination that are not the marker address. One candidate means that
is the reference; more than one means skip the **first** output back to the sender
as change, and take the **last** of the rest.

**Encoding class** (`omnicore.cpp:758-846`) — Class C wins over Class B when both
shapes are present, because `hasOpReturn` is tested first.

**Consensus hash** (`consensushash.cpp:148-270`) — single SHA-256 over
pipe-delimited records, six sections in fixed order. Two subtleties that are
load-bearing rather than cosmetic:

- A balance whose four buckets are **all zero is skipped**. A fully-spent balance
  must hash identically to one that never existed, or two implementations that
  prune differently will disagree. Pinned by
  `test_fully_spent_balance_hashes_as_if_it_never_existed`.
- `PENDING` is **excluded** from the hash — it is a wallet concept, not consensus
  state.

## The bug the end-to-end test caught

`RegtestNode.params` built a fresh `Params` object listing only name, ports,
activation height and datadir. It **omitted the base58 version bytes**, which
silently fell back to their mainnet defaults — so the indexer encoded regtest
addresses with Pepecoin's mainnet `P` prefix and nothing ever matched.

Every unit test passed throughout, because they all used the same wrong params on
both sides. Only running a real transaction through a real node exposed it. Fixed
by using `dataclasses.replace(REGTEST, ...)` so every field is inherited.

**Lesson recorded:** a config object constructed field-by-field will silently
inherit defaults for anything forgotten. Prefer `replace` over re-construction.

## Two failure modes, deliberately distinguished

| Situation | Behaviour | Why |
|---|---|---|
| **Unsupported message type** | **raise, stop the indexer** | We do not know what the rest of the network did with it, so all subsequent state is untrustworthy (hard rule #2) |
| **Invalid transaction** | record it, change no state | Normal and consensus-relevant: every implementation must agree an underfunded send does nothing |

Invalid transactions are **kept** in `ribbit_tx` with their reason. "This did
nothing, and here is why" is the first thing anyone asks when two implementations
disagree.

## End-to-end coverage

`tests/test_end_to_end.py` builds, signs and broadcasts genuine transactions on a
regtest chain, then reads them back out of mined blocks:

| Test | Proves |
|---|---|
| `test_class_c_issuance_and_send_end_to_end` | issuance → balance → transfer, through real blocks |
| `test_sender_is_the_first_input_for_class_c` | the Class C sender rule against a real transaction |
| `test_class_b_round_trip_end_to_end` | 768-byte payload via marker output + obfuscated multisig, decoded back |
| `test_non_ribbit_transactions_are_ignored` | plain payments are not misread as ours |
| `test_unmarked_op_return_is_ignored` | another protocol's OP_RETURN is not misread as ours |
| `test_reorg_rolls_back_protocol_state` | **real balances roll back**, and replay is deterministic |

The reorg test is worth describing. `invalidateblock` returns the orphaned
transaction to the mempool, so it is inevitably re-mined — the first version of
this test asserted the send stayed gone, which was wrong about the chain, not
about the code. It now asserts at the one observable moment (node tip rolled back,
follower synced) that balances reverted and the consensus hash **matches exactly
what it was before the orphaned block**; then lets the transaction be re-mined at
a different height and asserts the hash returns to its post-send value. That tests
determinism, not merely reversibility.

## Carried into M3

- `ribbit_tx` records a `message_type` for every type, but only 0, 4, 50, 54,
  65533, 65534 and 65535 have handlers. Everything else is recorded as invalid
  with "not yet implemented" — **M3 must replace that for 20/22/25/26/27/28**.
- The consensus-hash sections for DEx offers, DEx accepts and MetaDEx trades are
  written and ordered but read from tables that do not exist yet, so they return
  empty. M3 creates those tables; the hash code needs no change.
- `PrevOutCache` bounds itself by clearing wholesale rather than evicting LRU.
  Fine at current volumes; revisit if a full rescan ever gets slow.