B.Eng. candidate in Artificial Intelligence, Wuhan University (2023–2027)
I build measurement tools — benchmarks that can tell a model which did the task from one that happened to guess right.
jiahuili@whu.edu.cn · ecoreal.github.io · Wuhan, China
Benchmark validity for multimodal models. I co-designed a VLM benchmark for
instrument reading and built its data and evaluation pipeline: 1,821 images, 7
subsets, 32 instrument types, every value and box annotated by a human. Two
results shaped how I work. Agent-generated ground truth inflates reading accuracy
(22.4% vs 14.5%) and flattens prompt effects (+0.11 pp vs +4.12 pp); a
same-model label-and-evaluate loop inflates it by 75%. And the metric the
project had planned to headline turned out to be an arithmetic identity — it
equals exactly half the whole-image reading accuracy. What survived is ec_rate,
the share of correct answers whose localization was wrong: 22.8% (50/219).
Self-evolving data engines. A five-stage annotation engine for animal pose (YOLO-World → SAM2.1 → ViTPose++ → CoTracker3 → quad-triage). The design choice that mattered was making tracking abstain rather than propagate unreliable keypoints: Gold clips 56 → 71, worst failure class 36 → 24.
Label quality in remote sensing. Morphological filters for OSM-derived road labels. At an equal budget of 17,802 samples, morphology-based selection raised recall from 72.18% to 80.15% over random sampling. Second author.
CLI tools — one command, zero dependencies, nothing to install.
| Repository | What it is |
|---|---|
ipure |
Local-first IP purity check — itemised scoring, real-traffic exit verification, duplicate exits, native/broadcast via RIR, DNSBL · live demo report |
agentjanitor |
Local-first health check for every AI CLI on your machine — disk hogs, plaintext keys, secrets leaked into transcripts, duplicated MCP configs |
perm-dryrun |
Which permission rule matches your command, from which settings file, allow/ask/deny — before you run it |
hooktestkit |
Replay test harness for Claude Code hooks — fixture payloads in, exit-code and JSON assertions out |
scriptpeek |
What an npm lockfile will run before you install it — lifecycle scripts, real command text, dependency paths |
nexthop |
Turns any Claude Code / Codex session into a HANDOFF.md the next agent picks up |
portafs |
Cross-platform filename audit — case collisions, Windows reserved names, path length, NFC/NFD conflicts |
proxycheckup |
Static health check for proxy environment variables and config files — never opens a socket |
Agent runtime — observing and steering the thing that acts.
| Repository | What it is |
|---|---|
tape |
Time-travel debugger for AI agent sessions — reads the JSONL your agent already writes |
duel |
Deterministic AI agent arena in the terminal — any executable can fight, every match is a replayable file |
mogwai |
A terminal pet whose brain is a git repo; memory decays, forgetting is reversible |
System & environment
| Repository | What it is |
|---|---|
synapse |
Connectivity monitoring, diagnosis and config repair for LLM relay endpoints — pure Python stdlib |
vault-doctor / apt-diff / pi-guard |
nova — Obsidian vault health checks, Debian system-change audits, Raspberry Pi thermal/throttle watchdog |
Measurement
| Repository | What it is |
|---|---|
model-performance-test |
Frontend benchmark for LLMs: one prompt, one 100-point rubric, several models |
agentic-risk-scan |
Dependency-free scanner for agent workflows, MCP servers and instruction-file risks |
Also m2html — Markdown → HTML in Rust, GFM and syntax highlighting, with CLI and Python bindings.
Awards: 🏆 First Prize (Central-South Region), 19th Chinese Collegiate Computing Competition (2026) — yifang-pavilion · National Third Prize, Xiaomi Cup (2025) — cyberdog-motion-lab
Coursework: xv6-riscv — xv6 / RISC-V labs, OS internals in C. Older coursework archives (whu-term3-archive, software-coursework-lab, whu-coad-experiments, cpp-systems-lab) stay public as a learning record.
Python · PyTorch · C / C++ · Rust · JavaScript · Linux · Slurm · LaTeX
Research code lives in private repositories while the work is in progress — happy to walk through any of it on request.





