Locate Keihin tables + render real fuel maps in the viewer

Reversed the inner map format from l.java (Nc/Lb) and validated it against the
stock reference maps:

- reconstruct_rom.py: decoded map -> unpack directory -> 384KB flat ROM. In the
  flat ROM, production (20187) vs aftermarket (20188) differ only 1.4% (vs 96%
  packed), i.e. real fuel enrichment.
- table_map.py + TABLES.md: fe = c.a[Qd*48]; base = (fe[0]&0x2F0)<<12 = 0x50000;
  table = base + fe[k]. Located 9 tables (main/low-throttle fuel per cylinder,
  base/idle fuel, ignition by gear x4), 32 RPM rows x 20 throttle cols, with real
  axes (RPM fe[8], throttle fe[27]) and the AFR curve (fe[2], 128=lambda 1.00).
  Validated: main-fuel delta is uniformly richer in the aftermarket map.

Viewer now has a "Triumph tables" tab: drop a real .hex (or pick two) and it
decodes, unpacks, and renders the fuel/ignition tables as heatmaps with real
axes, plus an A->B difference view. build_viewer.py embeds the c.a/s.a directory
so any map resolves in-browser.

Catalogue dropdown: download_maps.py fetches map .hex files into maps_cache/;
serve.py serves the viewer over http so the dropdown can fetch them (drag-and-drop
still works on file://).

Proprietary map binaries (*.hex, *.dec.bin, maps_cache/) are gitignored — code and
docs only.
This commit is contained in:
2026-08-11 07:00:19 -05:00
parent 5af8825747
commit 173f6fa0c4
12 changed files with 959 additions and 10 deletions

View File

@@ -13,11 +13,16 @@ TunerPro XDF encodes, but pulled straight out of the app.
`zc(byte[])` validates and indexes a loaded map:
- **Magic:** the first 4 bytes, masked `& 0xFF00FF60`, must equal `0x18008060`.
- **Family byte:** `bArr[0]` selects the table directory —
`(bArr[0] & 0x7B) == 0x69` → `r.a`; `bArr[0] == 0x68` → `t.a`; else `s.a`.
(`0x67`/`0x68`/`0x69` are the map "generation" markers.)
- **Calibration signature:** 4 bytes at **offset 20** (big-endian). `sc()`
- **Magic:** the first 4 bytes read as a **little-endian** u32, masked
`& 0xFF00FFE0`, must equal `0x18001360`. Equivalently, by byte:
`byte0 & 0xE0 == 0x60`, `byte1 == 0x13`, `byte3 == 0x18`. (`j5()` is
little-endian; big-endian does not satisfy the family bytes, so this is
settled.)
- **Family byte:** `byte0` selects the table directory —
`(byte0 & 0x7B) == 0x69` → `r.a` (0x69); `byte0 == 0x68` → `t.a`; else `s.a`
(0x67). These are the map "generation" markers, all consistent with the magic.
- **Calibration signature:** 4 bytes at **offset 20**, read **big-endian** (note:
different endianness than the magic — this is how `sc()` reads it). `sc()`
searches the directory for the record whose `field[0]` equals this value.
## Directory → metadata → geometry
@@ -46,6 +51,21 @@ Table **dimensions and axes** come from the runtime (`MainActivity.T8`/`U8` for
rows/cols) and the `title_axis` labels (Throttle %, MAP hPa, RPM, Load %, Temp,
Gear). Cells are **16-bit big-endian** with per-table scaling.
## The body is encrypted (verified against real maps)
Four real stock maps pulled from TuneECU's server
(`research/reference-maps/`) confirmed the **header** decode exactly — but their
bodies are **encrypted**: uniform ~7.91 bit/byte entropy, and two maps that
should differ only in fueling (20187 vs 20188) share just 0.1% of bytes. A
low-entropy footer (last ~5 KB) holds the key/signature material.
The loader reflects this: `zc()` copies 16 bytes at offset 8 into `Kd` (key/IV)
for `byte0=0x67` maps; `p8()` reads key bytes from the file **tail** and derives
a key via `m.Sb()`, then `m.uc()` unpacks using the `c.b` geometry. Likely
AES-128 (same machinery as the ECU seed/key). **Until this is reversed, the table
offsets below describe the *decrypted* map and can't be validated against a real
file.** The four reference maps are the ciphertext test vectors for that work.
## Editing / write-back (this is the whole trick)
TuneECU keeps a **running 16-bit checksum** at a calibration-specific offset and

View File

@@ -62,17 +62,62 @@ def _extract_arrays(arrays_xml: Path) -> dict:
return out
# fe-field -> named table map (from research/reference-maps/table_map.py, validated).
_TABLE_DEFS = [
("Main fuel — cylinder 1", 11, "fuel"),
("Main fuel — cylinder 2", 12, "fuel"),
("Low-throttle fuel — cyl 1", 15, "fuel"),
("Low-throttle fuel — cyl 2", 16, "fuel"),
("Fuel — base/idle", 9, "fuel"),
("Ignition advance — gear 1", 19, "ignition"),
("Ignition advance — gears 2–5", 20, "ignition"),
("Ignition advance — gear 6", 21, "ignition"),
("Ignition advance — neutral", 22, "ignition"),
]
def _extract_romdefs(args) -> dict:
"""Extract the c.a and s.a directory arrays needed to decode a real map in-browser.
Requires --tuneecu-src pointing at the decompiled com/tuneecu sources. If not
given or not found, returns {} and the viewer's Triumph-tables tab is disabled.
"""
src = getattr(args, "tuneecu_src", None)
if not src:
return {}
src = Path(src)
if not (src / "c.java").exists():
return {}
def arr(cls: str, field: str) -> list[int]:
s = (src / f"{cls}.java").read_text()
m = re.search(rf"\b{field} = \{{(.*?)\}};", s, re.S)
return [int(t.strip().rstrip("L")) for t in m.group(1).split(",") if t.strip()]
return {
"ca": arr("c", "a"),
"sa": arr("s", "a"),
"tables": [{"name": n, "fe": f, "kind": k} for n, f, k in _TABLE_DEFS],
"rows": 32, "cols": 20, "rpmFe": 8, "throttleFe": 27,
}
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("--arrays", required=True, type=Path)
ap.add_argument("--maps", required=True, type=Path)
ap.add_argument("--template", required=True, type=Path)
ap.add_argument("--out", required=True, type=Path)
ap.add_argument("--tuneecu-src", default=None,
help="decompiled com/tuneecu dir (enables the Triumph-tables tab)")
args = ap.parse_args()
defs = _extract_arrays(args.arrays)
maps = json.loads(args.maps.read_text(encoding="utf-8"))
# Directory arrays + table map for decoding/rendering real Triumph maps.
romdefs = _extract_romdefs(args)
# Optional: table geometry from extract_mapdefs.py, if it has been run.
mapdefs_path = args.out.parent / "mapdefs.json"
geometry = []
@@ -83,6 +128,7 @@ def main() -> int:
"definitions": defs,
"maps": maps,
"geometry": geometry,
"romdefs": romdefs,
"modTargets": sorted(MOD_TARGETS),
"meta": {
"mapCount": len(maps),

View File

@@ -0,0 +1,99 @@
"""Download TuneECU Triumph map files into a local cache for the viewer.
The viewer can render any of these from a dropdown (when served over HTTP, since
browsers block fetch() on file://). Maps come from the same endpoint the TuneECU
app uses: https://www.tuneecu.fr/Maps/<path> (found in the decompile).
python3 download_maps.py # mechanical-odo Bonneville set
python3 download_maps.py --filter Bonneville # all Bonneville twin maps
python3 download_maps.py 20187 20188 20262 # specific map numbers
python3 download_maps.py --all-twins # every Triumph twin (~big)
Writes maps_cache/<n>Map.hex and maps_cache/index.json (id + description).
"""
from __future__ import annotations
import argparse
import json
import sys
import urllib.request
from pathlib import Path
BASE = "https://www.tuneecu.fr/Maps/"
LIST_URL = BASE + "mapList.dat"
CACHE = Path(__file__).parent / "maps_cache"
MAPS_JSON = Path(__file__).parent / "maps.json"
# Default: the 2010 mechanical-odometer Bonneville candidates.
DEFAULT_IDS = ["20187", "20188", "20191", "20192"]
def _get(url: str) -> bytes:
req = urllib.request.Request(url, headers={"User-Agent": "tunie"})
with urllib.request.urlopen(req, timeout=30) as r:
return r.read()
def _map_list() -> list[str]:
"""Return the '<path>/<n>Map.hex' entries from the server's map list."""
text = _get(LIST_URL).decode("latin1")
return [ln.strip() for ln in text.splitlines() if ln.strip().endswith("Map.hex")]
def _descriptions() -> dict[str, list[str]]:
if not MAPS_JSON.exists():
return {}
return {m["id"]: m["description"] for m in json.loads(MAPS_JSON.read_text())}
def main() -> int:
ap = argparse.ArgumentParser()
ap.add_argument("ids", nargs="*", help="specific map numbers to fetch")
ap.add_argument("--filter", help="download every map whose path contains this substring")
ap.add_argument("--all-twins", action="store_true", help="all Triumph twin maps")
args = ap.parse_args()
entries = _map_list() # e.g. "Triumph/Bonneville/20187Map.hex"
if args.ids:
wanted = [e for e in entries if any(f"/{i}Map.hex" in e for i in args.ids)]
elif args.all_twins:
wanted = [e for e in entries if e.startswith("Triumph/")
and any(k in e for k in ("Bonneville", "Thruxton", "Scrambler",
"America", "Speedmaster"))]
elif args.filter:
wanted = [e for e in entries if args.filter.lower() in e.lower()]
else:
wanted = [e for e in entries if any(f"/{i}Map.hex" in e for i in DEFAULT_IDS)]
if not wanted:
print("Nothing matched.", file=sys.stderr)
return 1
CACHE.mkdir(exist_ok=True)
descs = _descriptions()
index = []
for path in wanted:
fname = path.rsplit("/", 1)[-1] # 20187Map.hex
num = fname.replace("Map.hex", "")
dest = CACHE / fname
if not dest.exists():
try:
data = _get(BASE + path)
except Exception as exc:
print(f" skip {fname}: {exc}", file=sys.stderr)
continue
dest.write_bytes(data)
print(f" got {fname} ({len(data)} bytes)")
index.append({"file": fname, "id": num,
"desc": descs.get(num, [num])})
index.sort(key=lambda x: x["id"])
(CACHE / "index.json").write_text(json.dumps(index, indent=2))
print(f"\n{len(index)} maps in {CACHE}/ (index.json written)")
print("Serve them with: python3 serve.py then use the catalogue dropdown.")
return 0
if __name__ == "__main__":
raise SystemExit(main())

44
tunie/viewer/serve.py Normal file
View File

@@ -0,0 +1,44 @@
"""Serve the tunie viewer locally so the catalogue dropdown can fetch cached maps.
python3 serve.py # serves this folder at http://localhost:8000
python3 serve.py 9000 # custom port
Browsers block fetch() on file:// URLs, so the Triumph-tables catalogue dropdown
only works when the viewer is served over HTTP. Drag-and-drop works either way.
Run download_maps.py first to populate maps_cache/.
"""
from __future__ import annotations
import http.server
import socketserver
import sys
import webbrowser
from pathlib import Path
HERE = Path(__file__).parent
def main() -> int:
port = int(sys.argv[1]) if len(sys.argv) > 1 else 8000
if not (HERE / "maps_cache" / "index.json").exists():
print("note: maps_cache/index.json not found — run download_maps.py first "
"for the catalogue dropdown (drag-and-drop still works).")
handler = lambda *a, **k: http.server.SimpleHTTPRequestHandler(*a, directory=str(HERE), **k)
with socketserver.TCPServer(("127.0.0.1", port), handler) as httpd:
url = f"http://localhost:{port}/tunie-viewer.html"
print(f"serving {HERE} at {url}\nCtrl-C to stop.")
try:
webbrowser.open(url)
except Exception:
pass
try:
httpd.serve_forever()
except KeyboardInterrupt:
print("\nstopped.")
return 0
if __name__ == "__main__":
raise SystemExit(main())

View File

@@ -70,7 +70,8 @@
<nav>
<button data-tab="maps" class="active">Map catalogue</button>
<button data-tab="caps">Tune capabilities</button>
<button data-tab="table">Table viewer</button>
<button data-tab="triumph">Triumph tables</button>
<button data-tab="table">Raw table viewer</button>
</nav>
<main>
@@ -101,6 +102,33 @@
<div class="grid-cards" id="caps"></div>
</section>
<section id="tab-triumph" class="tab">
<p class="hint">Drop a real TuneECU Triumph <span class="mono">.hex</span> map (e.g.
<span class="mono">20187Map.hex</span>). It's decoded, unpacked to the flat ROM, and its
fuel/ignition tables are rendered with real RPM/throttle axes — no XDF needed. Drop a
second map to see the difference (e.g. production vs aftermarket enrichment).</p>
<div class="controls" id="tcatRow" style="display:none">
<label style="color:var(--dim);font-size:12px">Catalogue A
<select id="tcatA"><option value="">— pick a downloaded map —</option></select></label>
<label style="color:var(--dim);font-size:12px">Catalogue B (diff)
<select id="tcatB"><option value="">— none —</option></select></label>
</div>
<div style="display:flex;gap:12px;flex-wrap:wrap">
<div class="drop" id="tdropA" style="flex:1;min-width:260px;padding:22px">Map A — drop <span class="mono">.hex</span>
<input type="file" id="tfileA" style="display:none"></div>
<div class="drop" id="tdropB" style="flex:1;min-width:260px;padding:22px">Map B (optional, for diff)
<input type="file" id="tfileB" style="display:none"></div>
</div>
<div id="tBody" style="display:none">
<div class="controls">
<select id="tsel"></select>
<span class="chip" id="tdiffChip">Δ Show A→B difference</span>
<span class="count" id="tinfo"></span>
</div>
<div style="overflow:auto"><table id="tgrid" class="mono" style="font-size:11px"></table></div>
</div>
</section>
<section id="tab-table" class="tab">
<div class="drop" id="drop">Drop a map file here (<span class="mono">.bin</span> or Intel <span class="mono">.hex</span>) — or click to choose.
<input type="file" id="file" style="display:none">
@@ -211,7 +239,119 @@ Object.values(DATA.definitions).forEach(d=>{
capWrap.appendChild(c);
});
// ---- table viewer ----
// ---- Triumph real-map tables ----
const RD = DATA.romdefs || null;
if(!RD){ document.querySelector('nav button[data-tab=triumph]').style.display='none'; }
else {
const I2={0:0x80808080,1:0x80808080,2:0x84808081,3:0x87848284};
function dcDecode(raw){
const b=new Uint8Array(raw); const i5=b[3]-24;
const i2 = (i5 in I2)?I2[i5] : ((i5+3)<<24)|((i5+1)<<8)|i5|((i5+2)<<16);
const key=((i2 | (b[0]|(b[1]<<8)|(b[2]<<16)|(b[3]<<24)))>>>0);
let prev=0;
for(let i=4;i<b.length;i++){ const c=b[i]; const ks=(key>>>(((i-4)%4)*8))&0xff; b[i]=(prev^c^ks)&0xff; prev=c; }
return b;
}
const le=(b,o,n)=>{let v=0;for(let i=0;i<n;i++)v|=b[o+i]<<(8*i);return v>>>0;};
const be16=(b,o)=>((b[o]<<8)|b[o+1]);
function flatRom(dec){
const i21=le(dec,28,2);
if(le(dec,i21+31,2)!==0x6F66) throw new Error('bad unpack marker');
const cnt=dec[i21+33]; const ents=[];
for(let k=0;k<cnt;k++) ents.push([le(dec,i21+34+k*8,4), le(dec,i21+38+k*8,4)]);
let sz=0; ents.forEach(([o,l])=>sz=Math.max(sz,o+l));
const rom=new Uint8Array(sz).fill(0xff); let p=i21+34+cnt*8;
ents.forEach(([o,l])=>{ rom.set(dec.subarray(p,p+l), o); p+=l; });
return rom;
}
function resolve(dec){
let sig=(dec[20]<<24|dec[21]<<16|dec[22]<<8|dec[23])>>>0;
if(dec[0]===0x67) sig=((sig&0xFFFF0000)|(((sig&0xFFFF)+dec[25])&0xFFFF))>>>0;
let qd=null;
for(let i=0;i<RD.sa.length/8;i++){ if((RD.sa[i*8]>>>0)===sig){ qd=RD.sa[i*8+1]; break; } }
if(qd===null) throw new Error('signature not in directory');
const fe=RD.ca.slice(qd*48, qd*48+48);
return {fe, base:(fe[0]&0x2F0)<<12, qd};
}
function readTable(rom, off){
const g=[]; for(let r=0;r<RD.rows;r++){ const row=[]; for(let c=0;c<RD.cols;c++) row.push(be16(rom, off+(r*RD.cols+c)*2)); g.push(row); } return g;
}
function loadMap(raw){
const dec=dcDecode(raw); const rom=flatRom(dec); const {fe,base}=resolve(dec);
const rpm=[],thr=[];
for(let i=0;i<RD.rows;i++) rpm.push(be16(rom, base+(fe[RD.rpmFe]&0x7FFFF)+i*2));
for(let i=0;i<RD.cols;i++) thr.push(be16(rom, base+(fe[RD.throttleFe]&0x7FFFF)+i*2));
const tables=RD.tables.map(t=>({...t, off:base+(fe[t.fe]&0x7FFFF)}));
return {rom, base, rpm, thr, tables, desc:new TextDecoder().decode(dec.subarray(30,30+le(dec,28,2)))};
}
let TA=null, TB=null, tdiff=false;
function wireDrop(dropId, fileId, setter){
const drop=document.getElementById(dropId), inp=document.getElementById(fileId);
drop.onclick=()=>inp.click();
drop.ondragover=e=>{e.preventDefault();drop.classList.add('hover');};
drop.ondragleave=()=>drop.classList.remove('hover');
const go=f=>{ if(!f)return; const r=new FileReader();
r.onload=()=>{ try{ setter(loadMap(r.result), f.name, drop); }catch(e){ drop.textContent=f.name+' — '+e.message; } };
r.readAsArrayBuffer(f); };
drop.ondrop=e=>{e.preventDefault();drop.classList.remove('hover');go(e.dataTransfer.files[0]);};
inp.onchange=e=>go(e.target.files[0]);
}
wireDrop('tdropA','tfileA',(m,name,drop)=>{ TA=m; drop.textContent=name+' — '+m.desc.split('\n')[0]; initTriumph(); });
wireDrop('tdropB','tfileB',(m,name,drop)=>{ TB=m; drop.textContent=name+' — '+m.desc.split('\n')[0]; renderT(); });
// Catalogue dropdowns (only when served over http, so fetch() works).
if(location.protocol.startsWith('http')){
fetch('maps_cache/index.json').then(r=>r.ok?r.json():null).then(list=>{
if(!list||!list.length) return;
document.getElementById('tcatRow').style.display='flex';
const fill=sel=>list.forEach(m=>{ const o=document.createElement('option');
o.value=m.file; o.textContent=`${m.id} — ${(m.desc[0]||'')}${m.desc[1]?' · '+m.desc[1]:''}`;
sel.appendChild(o); });
const A=document.getElementById('tcatA'), B=document.getElementById('tcatB'); fill(A); fill(B);
const pick=(sel,isB)=>{ if(!sel.value) return;
fetch('maps_cache/'+sel.value).then(r=>r.arrayBuffer()).then(buf=>{
const m=loadMap(buf);
if(isB){ TB=m; renderT(); } else { TA=m; initTriumph(); } }); };
A.onchange=()=>pick(A,false); B.onchange=()=>pick(B,true);
}).catch(()=>{});
}
function initTriumph(){
document.getElementById('tBody').style.display='block';
const sel=document.getElementById('tsel'); sel.innerHTML='';
TA.tables.forEach((t,i)=>{ const o=document.createElement('option'); o.value=i;
o.textContent=`${t.name} (0x${t.off.toString(16)})`; sel.appendChild(o); });
sel.onchange=renderT; renderT();
}
document.getElementById('tdiffChip').onclick=e=>{ tdiff=!tdiff; e.target.classList.toggle('on',tdiff); renderT(); };
function renderT(){
if(!TA) return;
const idx=+document.getElementById('tsel').value;
const t=TA.tables[idx]; const A=readTable(TA.rom,t.off);
const B=(tdiff&&TB)?readTable(TB.rom, TB.tables[idx].off):null;
let vals=[]; for(let r=0;r<RD.rows;r++)for(let c=0;c<RD.cols;c++) vals.push(B?B[r][c]-A[r][c]:A[r][c]);
const mn=Math.min(...vals), mx=Math.max(...vals), span=(mx-mn)||1;
const g=document.getElementById('tgrid');
let h='<tr><th style="position:sticky;left:0">RPM\TP%</th>';
TA.thr.forEach(v=>h+=`<th>${(v/10).toFixed(0)}</th>`); h+='</tr>';
for(let r=0;r<RD.rows;r++){
h+=`<tr><th style="position:sticky;left:0">${TA.rpm[r]}</th>`;
for(let c=0;c<RD.cols;c++){
const v=B?B[r][c]-A[r][c]:A[r][c]; const tt=(v-mn)/span; const hue=(1-tt)*220;
const txt=B?(v>0?'+'+v:v):v;
h+=`<td style="padding:2px 4px;background:hsl(${hue},70%,${30+tt*28}%);color:#0a0a0a">${txt}</td>`;
}
h+='</tr>';
}
g.innerHTML=h;
document.getElementById('tinfo').textContent = (B?`Δ (B−A) `:'')+
`${t.name} · ${RD.rows}×${RD.cols} · range ${mn}…${mx}`+(B&&!TB?'':'');
}
}
// ---- raw table viewer ----
let BYTES=null;
const drop=document.getElementById('drop'), fileIn=document.getElementById('file');
drop.onclick=()=>fileIn.click();

File diff suppressed because one or more lines are too long