Issue: Cancel in-flight bot streaming on new message; enable async multi-thread replies (if feasible)

[anriltine]

Context

We currently stream bot responses to Telegram with strict Telegram constraints (HTML-only, 4096 chars per message, fast webhook ack). See app/docs/bot_async_streaming.md for the current design and constraints.

Problem

• Cancellation UX: When a user sends a new message in the same chat/thread while the bot is still generating/streaming the previous answer, the bot should stop the previous generation and stop editing the previously streamed message as soon as possible (“latest prompt wins”).
• Async multi-threading: We want (if possible) the ability to run multiple independent threads concurrently inside a single Telegram chat, without blocking unrelated threads, while keeping ordering and cancellation predictable.

Current behavior (summary)

From bot_async_streaming.md:

• Webhook serializes updates per chat (chatQueue in app/bot/webhook.ts), so messages for the same chat are handled sequentially.
• app/bot/responder.ts has a per-chat generation counter and shouldAbortSend() checks latest telegram_update_id for a thread to cancel in-flight streams.
• Only the first 4096 chars are streamed live; overflow is sent after completion via continuation messages.

Goals

• G1 — Hard stop previous stream: When a new user message arrives in the same logical thread, immediately stop streaming/generating the previous response and stop sending further edits to Telegram for that response.
• G2 — Multi-thread concurrency: Allow running several threads asynchronously in one chat (e.g. parallel “topic threads” or “tasks”), only if a clear architecture is available.
• G3 — Keep Telegram-safe messaging: Preserve the existing HTML + 4096 chunking safety guarantees.
• G4 — No regressions: Don’t reintroduce “clock/flash/reorder” issues; keep deterministic ordering within a thread.

Non-goals

• Implementing full multi-segment live streaming across multiple Telegram messages (nice-to-have, out of scope unless it becomes necessary).
• Changing model/provider behavior; this is about orchestration/cancellation/concurrency.

Proposed approach (only if architecture is clear)

A) Cancellation: make it “hard”

• Ensure there is a single cancellation authority per thread, not just per chat.
• When a new message arrives for thread T:
  • Mark prior generation for thread T as cancelled (in-memory + persisted signal if needed).
  • Abort OpenAI stream via AbortController / abort signal.
  • Stop edit loop (sendOrEditQueue) for that reply immediately.
  • Optionally finalize with a last “interrupted” edit if we already streamed partial text (current sendInterruptedReply behavior), but never keep editing after cancellation.

B) Concurrency: move from per-chat queue → per-thread queue (or hybrid)

• Current webhook serializes per chat (chatQueue), which blocks concurrency across threads.
• If we can reliably determine a thread key (e.g. DB thread_id, reply-to chain, or explicit thread selection command), change to:
  • Queue per thread, not per chat, so different threads can run in parallel.
  • Keep a lightweight per-chat guard only for Telegram API rate/ordering hazards if needed (hybrid model).

Acceptance criteria

• AC1: Send message A, bot starts streaming. Send message B in same thread. Bot stops generating/editing A quickly (no more edits to A after cancellation is detected).
• AC2: Two different threads in same chat can stream concurrently (if implemented), and cancellation in one thread does not kill the other.
• AC3: HTML safety and 4096 chunking remains intact (no Telegram “can’t parse entities” regressions).
• AC4: Webhook still returns 2xx quickly; no increased retry/delay behavior from Telegram.
• AC5: If multi-thread architecture is not clear/safe, implement only the improved cancellation and document why multi-threading was not enabled.

Implementation notes / pointers

• Docs: app/docs/bot_async_streaming.md
• Webhook queue: app/bot/webhook.ts (chatQueue)
• Bot streaming + cancellation: app/bot/responder.ts
• AI streaming + abort hooks: app/ai/transmitter.ts, app/ai/openai.ts

Test plan

• Manual:
  • In a Telegram chat, trigger a long streaming reply, then immediately send another message. Confirm previous stream stops.
  • (If multi-thread enabled) start two threads and verify both stream without blocking each other; cancel one and verify the other continues.
• Logging:
  • Add clear logs for: threadKey, generation id, cancellation decision, abort signal triggered, edits stopped.

Priority

• High for cancellation, but still only if possible to do that handsome and clear (G1)
• Optional/Stretch for multi-thread concurrency (G2), only if a clear architecture exists