Repository navigation
fix(SESF-66): skip Codex external-import turns at ingest - #88
Conversation
Codex Desktop imports Claude Code sessions into rollouts. Their turns start with a task_started event whose turn_id begins with "external-import-", and they copy transcripts that are already indexed as claude_code_cli. Measured on the live index, they were 47,355 of 103,185 FTS rows (46%), nearly all duplicates. parse_source tracks the current turn from task_started and skips message items inside an import turn. A native turn after an import in the same rollout is kept, and an imported user message is never paired with a native answer. With this parser, cleanup.py prune-codex (dry run) finds 27,254 indexed import turns whose rollout is still on disk. Import turns of deleted rollouts (26,454 rows in 59 sessions) are left alone: 22 of those sessions have no overlap with the Claude Code index.
|
@coderabbitai review |
|
Warning Review limit reachedYou've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Next included review available in 16 minutes. View limit details
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Up to standards ✅🟢 Issues
|
| Category | Results |
|---|---|
| CodeStyle | 1 minor |
🟢 Metrics 13 complexity · 2 duplication
Metric Results Complexity ✅ 13 (≤ 100 complexity) Duplication ✅ 2 (≤ 5 duplication)
NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.
Part 1 of SESF-66. Part 2, which strips boilerplate tags at
add_turnsand cleans indexed rows, comes later in a separate PR.Why
Codex Desktop imports Claude Code sessions into Codex rollouts. Each imported turn starts with a
task_startedevent whoseturn_idbegins withexternal-import-. Those turns copy transcripts that are already indexed asclaude_code_cli.On the live index they were 47,355 of 103,185 FTS rows (46%), and they lead search results with
[external_agent_tool_call: …]blocks. Content sampling against theclaude_code_clirows:max_turn_charstruncation, which SESF-62 tracks.Change
provider_codex.CodexAdapter.parse_sourcetracks the current turn fromtask_started, because message items carry no turn id. It skips messages inside an import turn.CLAUDE.mdandAGENTS.md.Cleanup plan (after merge and restart)
cleanup.py prune-codexdeletes indexed Codex turns that a full parse of a rollout still on disk no longer produces, so it needs no new code. Dry runs against the live index:mainImport turns of deleted rollouts are left alone: 26,454 rows in 59 sessions, each at least 60% import rows. 22 of those sessions have no overlap with the Claude Code index, so their originals may no longer be indexed. Deleting them could lose the only copy.
Tests
tests/test_provider_codex.py: an import-only rollout yields nothing, native turns after imports are kept, there is no cross-pairing, and a resumed parse still skips imports.ruff check .is clean.pytest: 699 passed, 120 skipped.