# Ribbit — sidecar Omni application, design plan

Target chain: **Pepecoin** (see `docs/DECISIONS.md` D-002).
Status: **draft for Martin's approval.** No code written yet.

Every source citation below is from `/home/martin/reference/omnicore`
(OmniLayer/omnicore, master, commit `1c0ae8ae`, describes as v0.11.0) unless stated.

---

## 1. What we are actually building, and how it differs from the original brief

The app is a **sidecar**: it runs beside a stock, unmodified `pepecoind`, reads blocks over
RPC/ZMQ, and maintains Omni state in its own database. It is not a fork of the node.

Three consequences of the move to Pepecoin that change the plan materially:

**(a) There is no Omni history on Pepecoin, so we choose the start height.**
This is a large, underappreciated win. On Litecoin the indexer would have had to replay
~1.16 M blocks (2,010,500 → tip) before showing anything. On Pepecoin we declare an
activation height at or near the current tip, and the indexer has **essentially nothing to
replay**. Initial sync becomes seconds, not days — on a 5400 RPM HDD that is the difference
between a usable dev loop and a painful one.

**(b) There is no reference implementation, so there is no oracle.**
Nothing can tell us our consensus engine is right. Mitigations, in priority order:
1. Implement `GetConsensusHash()` exactly (§5) so any second implementation — including a
   future one of ours — can be diffed against us.
2. Port Omni Core's unit tests (`src/omnicore/test/`) and functional tests
   (`test/functional/omni_*.py`) as executable test vectors. They run on regtest and need no
   mainnet chain.
3. Regtest harness from milestone M0 onward, not bolted on at the end.

**(c) We must define our own protocol constants.** Covered in §7 as open decisions.

---

## 2. Architecture

| Layer | Recommendation | Why |
|---|---|---|
| **Node** | stock `pepecoind` v1.1.0, `txindex=1`, ZMQ on | already installed; never forked |
| **Indexer** | **Python 3.12** (pinned) | Martin knows Python; the workload is small (§2.1). Hot loop can move to Rust later behind the same interface if measurement demands it — not before. |
| **Database** | **SQLite + WAL** | single writer, embedded, trivial backup (one file), good enough at this scale. Access goes through a repository layer so Postgres remains possible without touching engine code. |
| **UI** | **local web app** — FastAPI + server-rendered HTML + HTMX, minimal JS, charts via a small JS lib | runs headless, reachable from another machine on the LAN, no GUI toolkit or ARM64/x86 packaging problems. Desktop app buys nothing here. |
| **Content viewer** | **separate origin/port**, sandboxed iframe, strict CSP, no network | inscribed files are untrusted bytes from strangers. Detailed in §6. |

### 2.1 Why Python is sufficient — with the number that would change my mind

Pepecoin has 60-second blocks (*verified in source*, `chainparams.cpp:89`). Because we choose
the activation height, the indexer's backlog starts at zero and grows at **1,440 blocks/day**.
Blocks are currently near-empty. Python handling 1,440 mostly-empty blocks per day is not a
performance question.

The one place Python could bite is a full historical rescan if we ever change the rules and
must replay from activation. Trigger to revisit: **if a full replay exceeds ~30 minutes**,
move the block-parse and payload-decode hot path to Rust. Not before.

### 2.2 Process layout

```
pepecoind (system service, user: pepecoin)
    | JSON-RPC 127.0.0.1:33873   (historical blocks, tx lookup, broadcast)
    | ZMQ     127.0.0.1:28332/3/5 (hashblock, rawblock, rawtx -> live tip + mempool)
    v
omni-indexer  (systemd user service)   ->  omni.sqlite  (state + undo journal)
    v
omni-web      (FastAPI, 127.0.0.1)     ->  browser
    |
    +-- content server on a SEPARATE port/origin, sandboxed, for inscribed files
```

Reorg note: Pepecoin v1.1.0 has **no `zmqpubsequence`** (*verified on host*), so the indexer
tracks `previousblockhash` itself and rolls back via the undo journal.

---

## 3. Inscription capacity — the finding that should drive the v2 format

This is the most consequential thing I found, and it changes how inscriptions should be built.

**Constants** (*verified in source code*):

| Constant | Value | Source |
|---|---|---|
| `PACKET_SIZE` | 31 (1 seq byte + **30 data bytes**) | `src/omnicore/parsing.h:19` |
| `MAX_PACKETS` | 255 | `src/omnicore/parsing.h:20` |
| `SP_STRING_FIELD_LEN` | 256 → **255 usable chars** per text field | `src/omnicore/omnicore.h:48` |
| `MAX_OP_RETURN_RELAY` (Pepecoin) | 83 → **80 usable bytes** | `pepecoin/src/script/standard.h:30` |
| Omni marker | 4 bytes prepended to Class C data | `src/omnicore/encoding.cpp:91-93` |
| Payload header | 4 bytes (version u16 + type u16) | `src/omnicore/createpayload.cpp:575-583` |

**Derived payload capacity per transaction:**

| Vehicle | Data bytes / tx | Arithmetic |
|---|---|---|
| Class C `OP_RETURN`, type 200 | **72** | 80 − 4 marker − 4 header |
| **Class B multisig, type 200 (anydata)** | **7,646** | 255 × 30 − 4 header |
| Class B, type 50 text fields — *Martin's v1* | **1,020** | 4 fields × 255 chars |

### The headline

**Type 200 `anydata` over Class B carries ~7.5× more data per transaction than the type-50
text-field trick used in v1.** A 1 MB file goes from ~1,028 transactions down to ~137.

Rough cost per 1 MB on Pepecoin (*assumption* — needs confirmation against live relay policy):

- ~137 transactions × ~14.7 KB each ≈ 2.0 MB of chain data for 1 MB of payload (~50 % efficiency)
- fee at 0.01 PEP/KB ≈ **~20 PEP per MB**
- plus ~0.13 PEP/tx of dust in multisig outputs — and because Omni Class B always includes the
  **sender's own redeeming pubkey** in each 1-of-3 (`encoding.cpp:37`), those outputs are
  **spendable by the creator** and can be swept later. Good UTXO hygiene, unlike burn-style
  data outputs.

⚠️ **Must verify before relying on this:** a ~14.7 KB transaction with ~128 multisig outputs
must still be standard under Pepecoin's `MAX_STANDARD_TX_SIZE` and output-count policy, and
miners must actually mine it. Test on regtest, then testnet. If it fails, fall back to fewer
packets per transaction — the format must make chunk size a parameter, not a constant.

---

## 4. Feature set

### 4.1 Parity checklist

"Port cost" is my estimate of engine work, not UI.

| Type | Name | In Omni Core | In OmniLite | Plan | Port cost |
|---|---|---|---|---|---|
| 0 | Simple send | ✅ | ✅ | **M2** | S |
| 3 | Send to owners | ✅ | ✅ | M4 | M |
| 4 | Send all | ✅ | ✅ | M2 | S |
| 5 | Send non-fungible | ✅ | ✅ | **M4** | M |
| 20 | DEx sell offer | ✅ | ✅ | M3 | M |
| 22 | DEx accept | ✅ | ✅ | M3 | M |
| **25** | **MetaDEx trade** | ✅ | ❌ | **M3** | **L** |
| **26** | **MetaDEx cancel-price** | ✅ | ❌ | **M3** | M |
| **27** | **MetaDEx cancel-pair** | ✅ | ❌ | **M3** | M |
| **28** | **MetaDEx cancel-ecosystem** | ✅ | ❌ | **M3** | M |
| 50 | Create property (fixed) | ✅ | ✅ | M2 | S |
| 51 | Create property (crowdsale) | ✅ | ✅ | M6 | L |
| 53 | Close crowdsale | ✅ | ✅ | M6 | S |
| 54 | Create property (managed) | ✅ | ✅ | M2 | S |
| 55 / 56 | Grant / revoke | ✅ | ✅ | M4 | M |
| 70 | Change issuer | ✅ | ✅ | M4 | S |
| 71/72/185/186 | Freezing | ✅ | ✅ | M6 | M |
| 73 / 74 | Add / remove delegate | ✅ | ❌ | M6 | M |
| **200** | **AnyData** | ✅ | ✅ | **M5** | S |
| **201** | **Set NFT data** | ✅ | ✅ | **M4** | M |
| 65533/65534/65535 | Deactivation / activation / alert | ✅ | ✅ | M2 | M |

Bold = the three things Martin explicitly asked for.

Property types (*verified in source*, `omnicore.h:89-95`): 1 indivisible, 2 divisible,
**5 non-fungible**, 65/66 replacing, 129/130 appending.

### 4.2 MetaDEx — bids *and* asks

Payload, type 25 — **28 bytes**, all big-endian (*verified in source*,
`createpayload.cpp:486-508`):

```
version           u16
type              u16   = 25
propertyIdForSale u32
amountForSale     u64
propertyIdDesired u32
amountDesired     u64
```

There is no "side" field, and that is the elegant part: **a bid and an ask are the same
message with the pair reversed.** Selling PEP-token A for token B is an ask in A/B and a bid
in B/A. The book is two-sided by construction. Price is the ratio
`amountDesired / amountForSale`, held as an exact rational — **never a float**, or we will
diverge from consensus.

This means MetaDEx gives Martin what he asked for: a genuine two-sided order book with
token↔token pairs, on-chain matching, and partial fills. Charts come from indexed fills,
since every fill has a block height.

⚠️ This is the **most consensus-critical code in the project**. Matching order, price-time
priority, tie-breaking, partial fills and rounding must be bit-exact or our state silently
diverges. It gets the heaviest test-vector coverage (§5).

### 4.3 NFTs

- Property type **5** creates a non-fungible property; tokens are tracked as **ranges**, not
  individual rows (*verified in source*, `nftdb.h:59-67`: `GetRange`, `AddRange`, `DeleteRange`).
- Type **5** send transfers a token range.
- Type **201** sets per-token data, with **two independent slots** (*verified in source*,
  `nftdb.h:17-18`): `IssuerData = 'I'` and `HolderData = 'H'` — the issuer and the current
  holder can each write, without overwriting each other.
- Payload 201 carries `propertyId`, `tokenStart`, `tokenEnd`, an `issuer` flag, and **data
  truncated to 255 bytes** (*verified in source*, `createpayload.cpp:110`).

255 bytes is far too small for a file, which settles a design question: **an NFT does not hold
its artwork — it holds a pointer to it.** The NFT's data slot references an inscription built
from type-200 chunks. That is exactly the "creator binding" the Phase 4 spec called for, and
it is also how we stop name spoofing: the binding is issuer-signed on-chain.

### 4.4 Inscriptions

- **Writer:** chunked type-200 `anydata` over Class B, ~7,646 bytes/tx (§3).
- **Reader:** **v2 only.** The v1 format is out of scope (D-005 dropped Litecoin recovery).
- **Viewer:** §6.

---

## 5. Consensus engine

### 5.1 Consensus hash — implement exactly

`GetConsensusHash()` is a single SHA-256 over pipe-delimited records in a **defined order**
(*verified in source*, `consensushash.cpp:94-234`):

| Section | Record format | Ordering |
|---|---|---|
| Balances | `address\|propertyid\|balance\|selloffer_reserve\|accept_reserve\|metadex_reserve` | address lexicographic, then property id |
| DEx sell offers | `txid\|address\|propertyid\|offeramount\|btcdesired\|minfee\|timelimit` | by txid |
| DEx accepts | `matchedselloffertxid\|buyer\|acceptamount\|acceptamountremaining\|acceptblock` | matched txid, then buyer |
| MetaDEx trades | `txid\|address\|propertyidforsale\|amountforsale\|propertyiddesired\|amountdesired\|amountremaining` | by txid |
| Crowdsales | `propertyid\|propertyiddesired\|deadline\|usertokens\|issuertokens` | by property id |
| Property issuers | `propertyid\|issueraddress` | by property id |

Empty balance records and the pending tally are **skipped** (`consensushash.cpp:111`). Getting
these skip rules and orderings wrong is the single easiest way to produce a subtly wrong hash,
so this gets dedicated tests.

### 5.2 Determinism and reorgs

- Process **block by block**, transactions in block order. No out-of-order work.
- Write an **undo journal** per block; a reorg replays backwards from the journal.
- All amounts are **integers** (`u64`); prices are exact rationals. No floats anywhere in the
  engine. This is a hard rule, enforced by a lint check.
- **On an unknown or unsupported transaction type, stop with a loud error.** Never skip
  silently — a silent skip is a permanent, invisible state divergence.

### 5.3 Testing

1. **Ported vectors** from Omni Core's `src/omnicore/test/` and `test/functional/omni_*.py`.
2. **Regtest harness** from M0 — spin a regtest node, mine, submit, assert state.
3. **Consensus-hash golden files** at fixed heights.
4. **Property-based tests** on the payload codec: encode → decode round-trips for every type.

---

## 6. Safe viewing of inscribed content

Inscribed bytes are untrusted input authored by strangers. Non-negotiable rules:

1. Serve content from a **separate origin** (different port), never the app's own origin.
2. Render inside `<iframe sandbox>` with no `allow-scripts` and no `allow-same-origin`.
3. Strict CSP: `default-src 'none'` plus the single media type being shown; `connect-src 'none'`.
4. **Allowlist** MIME types. Detect type from **magic bytes**, never from a declared header.
5. **Verify the SHA-256** in the v2 header before a single byte is rendered.
6. Enforce a size cap before decode; refuse decompression bombs.
7. Per-item local hide/block, stored client-side.

Note for Martin: Omni's `omni_getcurrentconsensushash` is a **ledger-agreement** check, not a
content checksum — it cannot tell you a recovered JPEG is intact. The per-file SHA-256 in the
v2 header is what does that, and it is ours to design. (Recorded in `docs/DECISIONS.md`.)

---

## 7. Open decisions — I need answers before M1

1. **Protocol name and marker.** Omni's Class C marker is 4 bytes (`omni`). Ours must differ or
   we collide with anything Omni-derived. Proposal: `pepo`. Martin's call.
2. **Exodus-equivalent address.** Class B requires a marker output to a well-known address
   (`omnicore.cpp:81`). We need a Pepecoin address with **no known private key** — deterministically
   derived from a fixed string and documented so anyone can re-derive and verify it.
3. **Activation height.** Proposal: current tip + ~1,440 blocks (~1 day) at launch, so the
   indexer has no backlog and there is a clean, announced start.
4. **Ecosystem model.** Omni has main/test ecosystems. Keep both, or collapse to one?
5. **Crowdsales (51/53).** Real work (bonus math, deadlines). In scope, or defer past v1?
6. **Project name** for the app itself.

---

## 8. Milestones

| # | Deliverable | Proves |
|---|---|---|
| **M0** | Block reader: RPC + ZMQ, regtest harness, schema skeleton, reorg rollback | we can read every block deterministically and survive a reorg |
| **M1** | Payload codec: Class B encode/decode with deobfuscation, Class C, all listed types; round-trip property tests | bytes ↔ structs, provably |
| **M2** | Core state: properties 50/54, balances, sends 0/4, activation messages, **consensus hash** | a real ledger that can be diffed |
| **M3** | **MetaDEx 25–28**: two-sided book, exact-rational matching, partial fills, cancels, OHLCV | the exchange Martin asked for |
| **M4** | **NFTs**: type 5 property, send 5, 201 issuer/holder data, grant/revoke, change issuer | NFT support |
| **M5** | **Inscriptions**: v2 spec, chunked type-200 writer, v1 **and** v2 reader, sandboxed viewer | inscribe + view files |
| **M6** | Web UI: Wallet, Tokens, NFTs, Exchange, Create, Inscriptions, Explorer, Settings | the product |
| **M7** | Transaction building: fund/sign via node wallet or PSBT (**app never holds keys**), regtest → testnet → mainnet | safe broadcast |

Each milestone ends with tests green and a short note in `docs/`.

**Sequencing note:** M0–M2 are prerequisites for everything. M3, M4, M5 are independent of each
other once M2 lands, so their order is Martin's preference. M7's mainnet step is the first time
anything is broadcast, and is gated on explicit confirmation per the ground rules.

---

# APPENDIX — Locked scope (supersedes §4.1 and §7)

Decisions D-003 through D-006. All open questions from §7 are now closed.

## Identity

| | |
|---|---|
| Project | **Ribbit** |
| Class C marker | **`rbit`** — 0x72 0x62 0x69 0x74 |
| Base token | **RBIT**, property 1, divisible (8 dp), uncapped |
| Base token origin | **burn-to-mint, 1 PEP = 1 RBIT**, permanently open, no premine |
| Activation height | set at launch, ~tip + 1,440 blocks |
| Ecosystems | both (main + test) |
| Spec + code | public |
| UI binding | 127.0.0.1 only |
| Signing | node wallet RPC (`fundrawtransaction` / `signrawtransaction`) |

## Transaction types — final list

| Type | Name | Milestone |
|---|---|---|
| 0 | Simple send | M2 |
| 3 | Send to owners | M4 |
| 4 | Send all | M2 |
| 5 | Send non-fungible | M4 |
| 20 / 22 | DEx offer / accept — **extended to accept NFT ranges** | M3 |
| 25 / 26 / 27 / 28 | **MetaDEx trade + 3 cancels** | **M3** |
| 50 / 51 / 53 / 54 | Property: fixed / crowdsale / close crowdsale / managed | M2 (50, 54), M6 (51, 53) |
| 55 / 56 | Grant / revoke | M4 |
| 70 | Change issuer | M4 |
| 71 / 72 / 185 / 186 | Freezing | M6 |
| 73 / 74 | Delegates | M6 |
| 200 | AnyData — inscription payload | M5 |
| 201 | Set NFT data | M4 |
| 65533 / 65534 / 65535 | Deactivation / activation / alert | M2 |
| — | **Burn-to-mint** (Ribbit-specific, no Omni equivalent) | M2 |

**Out of scope:** type 2 restricted send, and types 10/11/12/15/31/40/52 (legacy or never
implemented in Omni Core).

**Deliberate divergences from Omni Core** — each needs its own spec text and test vectors,
because no reference implementation exists for them:
1. **Burn-to-mint** — Exodus address gains a second meaning as a value sink.
2. **NFTs on DEx** — Omni Core rejects non-fungible properties on MetaDEx (`tx.cpp:1653`) and
   offers no NFT trading path at all.

## Revised milestones

| # | Deliverable |
|---|---|
| **M0** | Block reader (RPC + ZMQ), regtest harness, schema, reorg rollback |
| **M1** | Payload codec: Class B (with deobfuscation) + Class C, all in-scope types, round-trip property tests |
| **M2** | Core state: properties 50/54, balances, sends 0/4, **burn-to-mint**, activation messages, **consensus hash** |
| **M3** | **MetaDEx 25-28** two-sided book, exact-rational matching, partial fills, cancels, OHLCV; DEx 20/22 incl. NFT ranges |
| **M4** | NFTs: type 5 property, send 5, 201 issuer/holder data, grant/revoke, change issuer, send-to-owners |
| **M5** | Inscriptions: v2 spec, chunked type-200 writer, reader, sandboxed viewer |
| **M6** | Crowdsales 51/53, freezing, delegates; web UI (Wallet, Tokens, NFTs, Exchange, Create, Inscriptions, Explorer, Settings) |
| **M7** | Transaction building and broadcast; regtest → testnet → mainnet, launch activation height set |

---

# APPENDIX 2 — FINAL LOCKED SCOPE (supersedes Appendix 1)

After D-007 (no base token) and D-008 (no crowdsales).

## Identity

| | |
|---|---|
| Project | **Ribbit** |
| Class C marker | **`rbit`** (0x72 0x62 0x69 0x74) |
| Base / quote token | **none** — not part of the protocol |
| Exodus-equivalent address | **Class B marker only**, exactly as in Omni. No value-sink role. |
| Property IDs 1, 2 | permanently reserved, unassigned. User properties start at **3**. |
| Activation height | set at launch, ~tip + 1,440 blocks |
| Ecosystems | both (main + test) |
| Spec + code | public |
| UI | 127.0.0.1 only |
| Signing | node wallet RPC; app never holds keys |

## Transaction types — final

| Type | Name | Milestone |
|---|---|---|
| 0 | Simple send | M2 |
| 3 | Send to owners | M4 |
| 4 | Send all | M2 |
| 5 | Send non-fungible | M4 |
| 20 / 22 | DEx offer / accept — **extended for NFT ranges** | M3 |
| 25 / 26 / 27 / 28 | **MetaDEx trade + cancels** | **M3** |
| 50 | Create property (fixed) | M2 |
| 54 | Create property (managed) | M2 |
| 55 / 56 | Grant / revoke | M4 |
| 70 | Change issuer | M4 |
| 71 / 72 / 185 / 186 | Freezing | M6 |
| 73 / 74 | Delegates | M6 |
| 200 | AnyData — inscriptions | M5 |
| 201 | Set NFT data | M4 |
| 65533 / 65534 / 65535 | Deactivation / activation / alert | M2 |

**Out of scope:** 2 (restricted send), 10, 11, 12, 15, 31, 40, 51, 52, 53.

**Only one divergence from Omni Core remains:** NFT ranges on DEx (20/22). Omni rejects
non-fungible properties on MetaDEx (`tx.cpp:1653`) and has no NFT trading path. This is the
sole piece of consensus code with no reference implementation, and gets the heaviest bespoke
test coverage.

## Milestones — final

| # | Deliverable | Proves |
|---|---|---|
| **M0** | Block reader (RPC + ZMQ), regtest harness, SQLite schema, reorg rollback | deterministic reads; a reorg actually rolls back |
| **M1** | Payload codec: Class B (with deobfuscation) + Class C, every in-scope type, round-trip property tests | bytes ↔ structs, provably |
| **M2** | Core state: 50, 54, balances, 0, 4, activation messages, **consensus hash** | a ledger that can be diffed |
| **M3** | **MetaDEx 25-28**: two-sided book, exact-rational matching, partial fills, cancels, OHLCV. DEx 20/22 incl. NFT ranges | the exchange |
| **M4** | NFTs: type 5, send 5, 201 issuer/holder slots, 55/56, 70, send-to-owners | NFT support |
| **M5** | Inscriptions: v2 spec, chunked type-200 writer, reader, sandboxed viewer | inscribe + view files |
| **M6** | Freezing, delegates, web UI (Wallet, Tokens, NFTs, Exchange, Create, Inscriptions, Explorer, Settings) | the product |
| **M7** | Transaction building and broadcast; regtest → testnet → mainnet | safe launch |

## Hard engineering rules

1. **No floats in the engine.** All amounts `u64`; all prices exact rationals. Lint-enforced.
2. **Unknown transaction type ⇒ stop with a loud error.** Never skip silently.
3. **Block-by-block, in-order processing**, with a per-block undo journal for reorgs.
4. **App never holds private keys.**
5. **No mainnet broadcast without Martin's explicit per-transaction confirmation**, showing the
   fully decoded transaction, its outputs and the fee.


---

# APPENDIX 3 — M1 findings that change the v2 inscription spec

**Class B payloads are always a multiple of 30 bytes.** The final packet is
NUL-padded and Omni does not strip the padding (`omnicore.cpp:1263`); Ribbit
matches that, because matching is a consensus requirement.

Consequence for the v2 format (M5): a type-200 chunk can carry up to **29 trailing
NUL bytes the sender never wrote**. The v2 header's **content-length field is
therefore load-bearing, not a convenience** — without it, any chunk whose real
content ends in NUL is unrecoverable. Stripping padding in the codec was
considered and rejected: it would silently destroy genuine trailing NULs, which is
a worse failure than carrying padding the header can account for.

Verified capacity (by test, not arithmetic):

| Vehicle | Data per transaction |
|---|---|
| Class C, type 200 | **72 bytes** |
| Class B, type 200 | **7,646 bytes**, in 128 multisig outputs |