Add tunie/docs (full context + open-source roadmap); refresh README

docs/CONTEXT.md captures the entire reverse-engineering effort in one place:
KWP2000 protocol, AES-128 seed/key, the map format (dc decrypt -> flat-ROM
unpack -> directory -> fe table pointers), the fuel/ignition table map, the
validated SAI/O2 device flags, checksums, hardware, and a per-finding confidence
table.

docs/ROADMAP.md covers open-sourcing (legal/IP posture on proprietary maps and
the AES keys, repo hygiene, packaging/CI, community) and the technical path to
tuning the bike (first contact -> ROM dump -> calibration model -> write path ->
editor), with immediate next steps and a risk register.

Top-level README refreshed to describe the whole project (tool + viewer +
research) and point at docs/.
This commit is contained in:
2026-08-11 08:21:16 -05:00
parent 80c9373ded
commit 4debdfa518
4 changed files with 386 additions and 139 deletions

201
tunie/docs/CONTEXT.md Normal file
View File

@@ -0,0 +1,201 @@
# tunie — full technical context
Everything we've reverse-engineered and built for an open-source Triumph Keihin
ECU toolchain. This is the pick-it-up-cold reference; if you read one file, read
this one. Companion docs: [ROADMAP.md](ROADMAP.md) (open-source + next steps),
and the deeper per-topic writeups linked throughout.
---
## 1. The target
- **Bike:** 2010 Triumph Bonneville T100, 865cc air-cooled parallel twin,
**mechanical/analog odometer** variant.
- **ECU:** Keihin, on a Renesas SH7054 (32-bit SuperH RISC). ~384 KB internal
flash (corroborated: the app has a 393216-byte code path; npkern lists SH7054).
- **Goal:** an open, offline toolchain to read, view, edit, and (eventually)
flash the calibration — replacing the closed TuneECU app.
- **Owner's end state for the bike:** 2-1 exhaust, airbox removal, O2 delete, SAI
delete. This matches the community map `20188Map2009AIRBOXBONNY` almost exactly.
## 2. What exists today
| Piece | Path | State |
|---|---|---|
| Read-only KWP2000 diagnostic tool | `src/tunie/` | built, unit-tested; **not yet run on the bike** |
| Map catalogue (1811 maps) | `maps.json` | extracted from the APK |
| Seed/key (AES-128) reference | `research/keihin_seedkey.py` | reversed + FIPS-verified |
| Map decrypt / unpack / table map | `research/reference-maps/` | reversed + validated on real maps |
| Self-contained viewer/editor | `viewer/tunie-viewer.html` | renders real fuel/ignition tables + device toggles |
Nothing has electrically touched the ECU yet. Everything is software + RE against
downloaded map files.
## 3. KWP2000 diagnostic protocol (read path)
Recovered from the TuneECU APK (`m.java`, `MainActivity.java`), validated
byte-for-byte in `tests/verify_protocol.py`:
- **ECU address `0xD5`**, **tester `0xF5`** on K-Line (NOT the common `0xF1`).
CAN-era Triumphs use `0xF1`; K-line uses `0xF5`.
- **Framing:** format byte `0x80 | length`; long form (`0x80` + separate length
byte) when payload > 120 B; **additive mod-256 checksum**.
- **Init:** fast init (`ATTP5`) or 5-baud (`ATTP4` + `ATIIAD5`), address `0xD5`.
- **ECU type char:** `0` = Keihin, `1` = Sagem, `P` = Bosch (from `ecu` array).
- Reference frames: `81 d5 f5 81 cc` (StartComms), `82 d5 f5 1a 80 e6`
(ReadEcuIdentification), `84 d5 f5 18 00 ff 00 65` (ReadDtcByStatus).
The tool (`src/tunie/`) is **read-only by construction**: `safety.assert_read_only`
runs on every outbound request and refuses all write/flash services (0x27, 0x31,
0x34/0x36, 0x35, 0x37, 0x14, 0x11, 0x2E) and programming sessions. A bug can't
brick the ECU because the write path isn't implemented. Details: top-level
`README.md`, `STATUS.md`.
## 4. Security Access seed/key — AES-128
`m.java` `Vb()`. **Standard AES-128**, not the Honda XOR scheme the early brief
guessed. Verified: all 256 T-table entries match AES Te0; the port passes the
FIPS-197 known-answer test. `Vb()` AES-encrypts a 128-bit seed with one of three
embedded keys (selected by an ECU-code index) and returns the first ciphertext
word as the 4-byte key. Reference + keys: `research/keihin_seedkey.py`,
`research/FINDINGS.md`. This is **write-path** material, kept out of the read-only
tool.
Still open (framing, not crypto): which of the 3 keys for this ECU, and the exact
seed-block padding. One captured seed/key pair resolves both.
## 5. Map file format (this was the big reverse)
The full chain, all reversed from `l.java`/`MainActivity.java` and validated
against real stock maps. See `viewer/FORMAT.md` and
`research/reference-maps/README.md`.
### 5a. Download & decryption
- Maps download from `https://www.tuneecu.fr/Maps/<path>` (e.g.
`Triumph/Bonneville/20187Map.hex`), ~300 KB, **encrypted** (entropy ~7.99).
- Decryptor is `l.dc()`: a **self-synchronising CBC-style XOR stream cipher**
seeded by the plaintext 4-byte header. Bytes 0–3 stay plaintext (the magic);
from byte 4: `plain = prev_cipher ^ cipher ^ ((key>>((i%4)*8))&0xFF)`, where
`key = i2 | le32(header)` and `i2` is a constant from `header[3]-24`.
Reversed + round-trip-verified in `research/reference-maps/decode_map.py`.
- Header magic: `le32(bytes[0:4]) & 0xFF00FFE0 == 0x18001360`, i.e.
`byte0 ∈ {0x67,0x68,0x69}` (family), `byte1==0x13`, `byte3==0x18`.
### 5b. Unpack to the flat ROM
The decoded map is **packed** — which is why a naive diff of two decoded maps
showed 96% difference (misaligned chunks). Each decoded map carries an **unpack
directory** (base `le16(dec[28])`, a `0x6F66` marker, a count, then
`(dest_offset, length)` entries) that scatters chunks into a **384 KB (0x60000)
flat ROM** = the real ECU address space. In the flat ROM, production (20187) vs
aftermarket (20188) differ only **1.4%** — the real fuel enrichment.
Reversed in `research/reference-maps/reconstruct_rom.py`.
### 5c. Directory lookup → calibration metadata
Decoded signature at offset 20 → `s.a` directory record → `Qd = field[1]` →
`fe = c.a[Qd*48]` (the 48-int per-calibration metadata that holds all the table
pointers). Validated: sig `0x0187CA84` → `s.a` #541 → Qd 32.
## 6. Calibration tables
`research/reference-maps/TABLES.md` + `table_map.py`. Base `= (fe[0]&0x2F0)<<12 =
0x50000`; **table offset = base + fe[k]**. Tables are **32 RPM rows × 20 throttle
cols, 16-bit big-endian**.
| Table | fe | Offset (20187) | Confidence |
|---|---|---|---|
| Main fuel cyl 1 / 2 | 11 / 12 | 0x56990 / 0x56E90 | high (75/67% aftermarket diff, smooth VE) |
| Low-throttle fuel cyl 1 / 2 | 15 / 16 | 0x55590 / 0x55A90 | high |
| Fuel base/idle | 9 | 0x55500 | high |
| Ignition advance gear 1 / 2–5 / 6 / neutral | 19–22 | 0x587D0 … 0x596D0 | high |
| AFR / target lambda | 2 | 0x52260 | medium (128 = λ1.00 confirmed) |
Axes: RPM = `fe[8]` (32 breakpoints, real RPM); throttle = `fe[27]` (20, 0–100%×10).
**Validated end-to-end:** the main fuel table renders as a clean VE surface, and
the aftermarket map is richer in 284/320 cells — physically correct enrichment.
**Scaling** (relative changes are exact; absolute units still approximate): fuel
values are the ECU's internal VE/fuel-mass units (~0–10800); ignition 13–600 is an
internal advance unit; AFR 128 = λ1.00. Exact fuel/ignition constants: TBD (an XDF
cross-check or more RE).
## 7. Device-enable flags (SAI / O2)
`research/reference-maps/DEVICES.md`. Byte-boolean array at `base + fe[33]`
(=`0x53801`), 1=on/0=off. Validated against the real NO-SAI-NO-O2 map
`20188Map2009AIRBOXBONNY`, which cleared exactly three bytes:
| Offset | Device | Confidence |
|---|---|---|
| `0x53801` | **SAI** | ~90% (code `lc()` → Devices[0] via fe[33], **and** the delete map) |
| `0x53818` / `0x53819` | **O2 sensor 1 / 2** | ~80% (two O2 sensors ↔ two adjacent bytes; not individually isolated) |
**Critical caveat:** correct flags ≠ a good tune. Disabling O2 forces open-loop —
the bike runs on the base fuel map with no live trim; on a stock map that can run
lean/rough. A proper delete pairs flags-off with fuel enrichment (which is why the
reference map changed the fuel tables too). **Flash a complete matching delete map,
not hand-toggled flags on a stock one.** The owner's target = the AIRBOXBONNY map
(SAI/O2 off + airbox fueling done); only the 2-1-vs-2-2 midrange fuel needs
fine-tuning.
## 8. Checksums
Two distinct ones:
- **Table write-back (edit) checksum:** a running 16-bit word, `chk = (chk + old -
new) & 0xFFFF`, used when a cell changes. Location is per-calibration.
- **Distribution `.hex` integrity:** the `dc` cipher + unpack directory. The `caXX`
header/tail bytes are **map-ID metadata, not a cal checksum**, so re-encoding an
edit is complete for the file format.
- **ECU flash checksum:** applied by the flashing tool at write time — out of scope
for the read/edit tool, part of the write path.
## 9. Hardware / cable
Owner has a **VAG KKL 409.1** cable (right class — K-Line) and a Bluetooth
ELM327. The VAG cable is wired for a VW OBD-II socket; the 2010 Bonneville uses
Triumph's proprietary diagnostic connector, so it needs an adapter/re-pin
(K-Line + switched 12V + ground). **Confirm the connector pinout before plugging
in — that's the one genuine electrical risk.** Identify the chip first
(`tunie ports`: FTDI = `/dev/cu.usbserial-*`; CH340 = `/dev/cu.wchusbserial*`).
## 10. The viewer
`viewer/tunie-viewer.html` — one self-contained file, no server, no external refs.
Tabs: Map catalogue (1811, filterable, "my bike" filter) · Tune capabilities
(devices/params/sensors from the APK) · **Triumph tables** (drop a real `.hex` →
decode → unpack → render real fuel/ignition tables with axes + A→B diff + device
checkboxes) · Raw table viewer. Build with `viewer/build_viewer.py`
(`--tuneecu-src` enables the Triumph tab by embedding the `c.a`/`s.a` directory).
`download_maps.py` + `serve.py` add a catalogue dropdown of locally cached maps.
## 11. Validation status & confidence (honest)
| Finding | Confidence | How validated |
|---|---|---|
| KWP2000 addressing/framing | high | byte-identical to TuneECU |
| Seed/key = AES-128 | high | FIPS-197 + T-table match |
| Map decrypt (`dc`) | high | round-trip + readable strings on real maps |
| Flat-ROM unpack | high | sibling diff 96%→1.4%, directory lookup resolves |
| Fuel/ignition table offsets | high | render as real tables; enrichment physically correct |
| Table scaling (absolute units) | low–med | relative exact; absolute TBD |
| SAI flag (0x53801) | high (~90%) | code + real delete map |
| O2 flags (0x53818/9) | probable (~80%) | group-validated, not isolated |
| Anything on the actual bike | **none yet** | no hardware connected |
## 12. What is NOT done
- No comms with the real ECU (blocked on the cable adapter/pinout).
- No ROM dump from the bike.
- No write/flash path (seed/key which-key + padding, flash kernel, ECU checksum).
- Editor exports the distribution `.hex` but re-packing an edited **flat ROM** back
through the unpack directory + `.hex` for arbitrary edits is only partly wired
(device flags work because they live in one chunk).
- Absolute fuel/ignition scaling constants.
- Individual SAI-vs-O2 isolation and the other device flags (exhaust valve, etc.).
## 13. Key source references (in the decompile)
`l.java`: `dc()` decrypt, `zc()`/`Nc()`/`Lb()` map load + tables, `sc()` directory,
`lc()` device list. `m.java`: `Vb()` seed/key, KWP framing (`Ld`/`Ub`), `uc()`
ROM packer. `MainActivity.java`: `p8()` load pipeline, unpack directory build,
`a3`/`W2` init sequences, Maps/ URLs. `c/r/s/t.java`: directory + geometry arrays.
`x.java` `a()`: per-ECU device layout (`x.t/x.s/x.q`) — not fully traced.

14
tunie/docs/README.md Normal file
View File

@@ -0,0 +1,14 @@
# tunie docs
- **[CONTEXT.md](CONTEXT.md)** — full technical context: everything reverse-engineered
and built (protocol, AES seed/key, map decrypt/unpack, table map, device flags,
checksums, validation status & confidence). Read this first.
- **[ROADMAP.md](ROADMAP.md)** — the plan: getting this open-sourced (legal, hygiene,
packaging), the technical path to actually tuning the bike, immediate next steps,
and the risk register.
Deeper per-topic writeups live next to the code:
- `../README.md`, `../STATUS.md` — the read-only tool + project status
- `../research/FINDINGS.md` — AES-128 seed/key
- `../viewer/FORMAT.md` — map file format
- `../research/reference-maps/{README,TABLES,DEVICES}.md` — decrypt, table map, device flags

130
tunie/docs/ROADMAP.md Normal file
View File

@@ -0,0 +1,130 @@
# tunie — roadmap: open-sourcing + what's next
Two tracks: **getting this into a shippable open-source project**, and the
**technical work** still needed to actually tune the bike. Read
[CONTEXT.md](CONTEXT.md) first for the full state.
---
## Track A — open-sourcing
The code is original; the risk in publishing is the **third-party derived data**
and the **flashing capability**. Handle these deliberately before a public repo.
### A1. Legal / IP (do this first)
- **No proprietary binaries in the repo.** Already enforced: `*.hex`, `*.dec.bin`,
`maps_cache/`, `table_map.json` are gitignored. Keep it that way — map files are
Triumph/TuneECU calibration data and must not be redistributed.
- **APK-derived data** (`maps.json`, `mapdefs.json`, embedded `c.a`/`s.a`, the AES
keys, protocol constants) are facts extracted for **interoperability**. Frame the
project explicitly as independent interoperability research on hardware you own.
Add a clear NOTICE describing what's derived and why, and cite that it contains
no TuneECU source.
- **The AES seed/key** enables writing to the ECU. Decide the posture: keep the
key/flashing material in a clearly-separated `research/` area with a warning, or
gate the write path behind an explicit build flag. Do not make bricking a
one-liner.
- **License:** pick one (MIT/Apache-2.0 for the tool; note that embedded
extracted constants are facts, not licensed code). Add `LICENSE` and a top-level
legal/README section.
### A2. Repo hygiene
- Move `tunie/` out of the `samplez` grab-bag into its **own repo** for a real
project (keep samplez as the scratch mirror if you like).
- Add: `CONTRIBUTING.md`, `SECURITY.md` (responsible-disclosure + "don't flash
blindly" warning), issue templates, a CHANGELOG.
- Reproducible builds: pin deps, document `build_viewer.py` inputs (needs a
decompiled APK the repo can't ship — document how to produce it, don't include it).
### A3. Packaging & CI
- `pip install tunie` — the `pyproject.toml` is already there; add a wheel build.
- CI: run `tests/verify_protocol.py` + the reference-map self-tests (`decode_map.py`,
`reconstruct_rom.py`, `table_map.py` — but those need map binaries, so gate them
behind a locally-provided fixtures dir / skip in public CI).
- Lint/format (ruff/black), type-check (mypy) the `src/tunie` package.
### A4. Docs & polish
- A real top-level README with a quickstart, the safety stance, and a screenshot.
- Host the viewer as a static page (it's a single self-contained file).
- Turn `docs/CONTEXT.md` into a short "how the format works" explainer — it's
genuinely novel RE and good for the project's credibility.
### A5. Community
- The SuperH ECU community (npkern/FastECU/ScoobyRom authors, TunerPro forums,
triumphrat) is the natural audience. A working open Keihin decrypt + table map is
a real contribution. Coordinate rather than surprise — some of this overlaps
commercial XDF vendors.
---
## Track B — technical, to actually tune the bike
Ordered by dependency. Phases 1–2 are safe reads; 3+ is the write path.
### B1. First contact (READ) — the current blocker
1. Identify the VAG KKL chip (`tunie ports`); install CH340 driver if needed.
2. **Confirm the Triumph diagnostic-connector pinout** and build the adapter
(K-Line + switched 12V + ground). *This is the one real electrical risk.*
3. `tunie info` — read ECU ID, current map, DTCs. Confirms which stock map is on
the bike (should be 20187/20191 for production silencers, mechanical odo).
### B2. ROM dump (READ, deliberate)
- Enable `0x35` RequestUpload (currently blocked in the read-only tool) to dump the
stock ROM as a recovery image. Do this before any write work.
- Cross-check the dump against our decrypt/unpack: a stock dump should reconstruct
to the same flat ROM as the matching `.hex`, validating the whole pipeline on the
real bike's data.
### B3. Finish the calibration model
- **Absolute scaling** for fuel and ignition (units), via an XDF cross-check (~€70
OldSkull/Tuniverse) or more RE of `Nc`/`Lb`. Relative editing already works.
- **AFR table** dimensions/axes (partly reversed; 128 = λ1.00 confirmed).
- **Remaining device flags** (exhaust valve, air flap…) — needs the `x.a()` device
layout reverse (`x.t/x.s/x.q` for i27=72) or more single-mod reference maps.
- Individual **SAI-vs-O2** isolation — only matters if hand-editing flags; the
owner's path (a complete AIRBOXBONNY-style map) sidesteps it.
### B4. Write / flash path (the risky part — bench first)
- Port the **flash kernel**: `fenugrec/npkern` explicitly lists SH7054 (untested) —
the strongest lead. `miikasyvanen/FastECU` shows a full end-to-end flow.
- Finish **seed/key**: which of the 3 AES keys + the seed-block padding (one
captured pair from the bike/logging APK settles it).
- **ECU flash checksum** at write time.
- **Recovery**: `v-ladimir/audprog` (AUD debugger) for SH705x un-brick via the PCB
debug pins — mandatory backstop when developing a flasher.
- Develop against a **spare/bench ECU**, on a stable supply, never the bike's only
ECU, and validate output against TuneECU before trusting our own writes.
### B5. Editor completion
- Full flat-ROM edit → re-pack → `dc`-encode → `.hex` for **arbitrary** table edits
(device flags already round-trip; generalize to any chunk + fix the edit
checksum).
- In-viewer table editing with axes + a wideband/dyno-oriented diff workflow.
### B6. The owner's actual goal
2-1 exhaust + airbox removal + O2 + SAI delete:
1. Base on `20188Map2009AIRBOXBONNY` (validated: flags off + airbox fueling done).
2. Verify in the viewer (devices off, fuel tables sane vs stock).
3. Flash it — via the tunie write path once it exists, or TuneECU meanwhile.
4. Fine-tune midrange fuel for the 2-1 with a wideband (ideally a dyno).
---
## Immediate next steps (this week, if picking back up)
1. **Cable + pinout** → run `tunie info` on the bike (unblocks everything).
2. Pull `20188Map2009AIRBOXBONNY` into the viewer catalogue and diff it vs stock
20187 so the exact delete-tune changes are visible.
3. Decide the open-source posture on the AES keys / write path (Track A1) before
any public repo.
## Risk register
| Risk | Status |
|---|---|
| Software bricking via the read tool | eliminated (write path not implemented) |
| Electrical (wrong connector pin) | **live** — confirm pinout before plugging in |
| Bad tune from flags-only edit | mitigated by guidance; use a complete map |
| Flashing (comms drop / bad checksum) | future — needs bench ECU + recovery tooling |
| Legal (redistributing proprietary maps) | mitigated — binaries gitignored |
| One ECU, no spare | validate writes against TuneECU; get a spare before B4 |