feat: AI next-word suggestions (on-device first, default off) - #422
Draft
Lloyd-Jackman-UKPL wants to merge 2 commits into
Draft
feat: AI next-word suggestions (on-device first, default off)#422Lloyd-Jackman-UKPL wants to merge 2 commits into
Lloyd-Jackman-UKPL wants to merge 2 commits into
Conversation
AI next-word prediction source added to the suggestion strip, off by default. - AINextWordDictionary: async candidate source active only in next-word mode, cache-backed, never blocks the suggestion thread; AOSP suggestions stay primary. - AINextWordEngine + flavor factories: offline = on-device GGUF causal completion (reuses proofread ModelHolder, no network); standard = OpenAI-compatible chat; offlinelite = no-op. - Registered in DictionaryFacilitatorImpl gated by pref_ai_next_word; toggle in AI integration / advanced settings; unit tests for prompt + candidate parsing.
Include the AI next-word pref in usesSameSettings so toggling it triggers a dictionary reset; previously the AI source would not (re)build until the IME restarted, so the feature appeared not to activate.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AI next-word suggestions (on-device first, off by default)
Implements Closes #421 — an AI candidate source that adds LLM-driven next-word suggestions to the existing AOSP suggestion strip. Functional across all three flavors, disabled by default (
pref_ai_next_word = false); when off, behaviour is identical to before.What it does
AINextWordDictionary(dictionary/AINextWordDictionary.kt) — aDictionarysubclass that only contributes in next-word mode (composedData.mTypedWordempty, i.e. exactly when the existing AOSP prediction hook runs after each word commit). It builds a prompt from the preceding words (NgramContext.extractPrevWordsContext), serves cached candidates, and kicks an async LLM fetch on its own coroutine scope — never blocks the suggestion thread. AOSP suggestions always stay primary (isInitialized()is false until the engine is ready).AINextWordEngineinterface + per-flavorAINextWordEngineFactory:LlamaHelper/ModelHolderalready used by proofreading), plain-text prompt, no network, single shared loaded model (no double-load).DictionaryFacilitatorImpl.getSuggestions, next-word mode only, gated by the pref at dictionary reset.app/src/test/.../AINextWordDictionaryTest.kt, 8 cases) covering prompt building + candidate parsing + dedupe/cap — pure JVM, no instrumented APIs.On your device
Toggle AI next-word suggestions on, then (for offline: load a GGUF model) — candidates appear after the current word alongside AOSP suggestions.
Known limitations (draft)
postUpdateSuggestions). Usable, but not "instant" — flagged for a follow-up.ModelHolderby design (reuse one loaded model), and needs tuning on a real device (params, top-k choice, timeout).Build + packaging verified locally for all three flavors (offline / standard / offlinelite) plus the unit tests.