Skip to content

feat: model categories by use, modalities and adapters as library metadata - #79

Merged
MrScripty merged 10 commits into
mainfrom
feat/model-categories
Nov 4, 2026
Merged

MrScripty merged 10 commits into
mainfrom
feat/model-categories

Conversation

@MrScripty

Copy link
Copy Markdown
Owner

Implements docs/plans/v080-model-categories/plan.md for 0.8.0.

Models were grouped by one category picked from Hugging Face task tags and weak file and name hints. That category also decided the model's storage folder and ID. On the owner's library this filed Kev and Laya under LLM, LTX-2.3 under audio and embedding, and Granite Speech nar under embedding.

What changes

  • Categories describe what a model is used for.
    • New decision and video categories.
    • A GGUF decision head (<arch>.decision.type, as in Kev and Laya) marks a decision model.
    • Video tasks map to video.
    • ...ForASR architectures map to audio.
    • The source's own task tag outranks derived hints.
    • A hard signal (architecture, config, decision head) is never outvoted by a task tag. When they disagree, confidence drops and the model is flagged for review.
  • The category is metadata. Reclassification and migration keep the model's storage bucket (the first ID segment), so a category change never moves a folder or changes an ID. Only a family change still moves a model.
    • Name tokens no longer count the bucket as evidence. The bucket is only the last fallback.
    • ModelType::ALL is the single list of categories. Strict v2 metadata validation and the RPC derive from it.
  • Inputs and outputs (modalities) are recorded for these tasks, including a new decision modality. When a strong source sets the category against a contradicting tag, the modalities follow the category. The tag stays recorded as evidence.
  • A person's choice wins. set_model_category records a category (source user) that re-detection keeps, or clears it to re-detect. Requests are admitted exactly: a missing category, a task tag or unknown is refused with a reason.
  • Desktop.
    • Each catalog row carries input and output modalities and an adapter flag. LoRA and PEFT packages are detected from their file names, adapter_config.json or the model name.
    • The library is grouped by use in a fixed order under readable headings.
    • Rows show "inputs → outputs" and an Adapter badge.
    • The metadata window has a "Used as" picker.
  • Existing libraries. On open, system rule rows seeded by earlier versions are updated in place. Rules a person edited are left alone.

Verification

  • Owner library (94 models), live refresh with the release build:
    • Kev and Laya → decision (text → decision).
    • LTX-2.3 ×3 → video (image → video).
    • Granite Speech nar → audio (audio → text).
    • The IC-LoRA is flagged as an adapter.
    • 0 reclassify errors, no model ID or folder changes, and a second pass is settled.
    • The first live run found strict-v2 validation rejecting the new categories (fixed in e078233; the tests now use strict v2 metadata and fail without the fix).
  • Rust:
    • pumas-library: 2,363 passed.
    • pumas-rpc: 431 passed.
    • fmt and clippy (all features, -D warnings) are clean.
    • The Windows target check passes.
  • Frontend: 949 tests pass, plus lint and types.
  • Electron: all tests pass, along with lint, desktop-contract check and generator, and conformance in both the Electron and frontend suites.

Known limits

  • The LTX IC-LoRA has no task tag or base-model evidence, so it stays under image generation until someone chooses Video.
  • LTX's tag only states image → video.

🤖 Generated with Claude Code

MrScripty and others added 8 commits November 3, 2026 15:33
…as metadata

WIP for docs/plans/v080-model-categories (M1, M2 core): tests that
assert the old move-on-category-change rule are updated separately.

- ModelType gains `decision` and `video`. Video tasks (text/image/video
  to video) map to video; text-classification and zero-shot
  classification to decision; ForSequenceClassification to decision;
  `...ForASR` architectures to audio. Rule tables accept the new values
  (constraint rebuild) and an idempotent repair updates system rows
  seeded by earlier versions.
- A GGUF declaring `<arch>.decision.type` (Kev, Laya) resolves as a
  decision model ahead of any task tag. File identification reads it.
- The source's pipeline_tag outranks derived hints (search kind,
  download category); a hard signal is never outvoted by a task tag,
  which instead lowers confidence and requests review.
- Task signatures cover image->video, text+image->video, video->video
  and text->decision, so modalities are recorded for these models.
- Name tokens no longer count the storage bucket as evidence; the
  bucket is the last fallback when nothing else describes a model.
- Reclassification and migration keep the storage bucket (first ID
  segment); only a family change moves a model. The category is
  recorded from the resolution, never from the ID.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Reclassification compares the new category with the recorded one, not
with the storage bucket, so a model filed under another bucket settles
after one pass. A category a person chose (source "user") is kept by
reclassify, redetect and migration, which all resolve through one
function; set_model_category records or clears that choice without
moving the model. Tests reproduce the owner-library cases: Kev's
decision head, LTX's image-to-video task, and Granite Speech nar's ASR
architecture against its feature-extraction tag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dapters

M3/M4 of docs/plans/v080-model-categories.

- The desktop catalog row carries input/output modalities (the
  `unknown` placeholder dropped) and an adapter flag. The index projects
  the flag from LoRA/PEFT packages: adapter_config.json or
  adapter_model.*, or `lora` as a whole token in weight files, the
  name or the repository.
- New `set_model_category` RPC: an exact choosable category or an
  explicit null to re-detect. A missing field, a task tag or `unknown`
  is refused with a reason listing the choices. The response is
  decoded against a closed schema whose category is an enum.
- The library groups models by use in a fixed order (language,
  vision-language, decision, embedding, reranking, image generation,
  video generation, audio and speech, vision), with unclassified and
  future categories last, under readable headings. Rows show
  "inputs → outputs" and an Adapter badge.
- The metadata window has a "Used as" picker that records or clears
  the choice and shows refusal reasons.
- Download category hints map video tasks to video and
  text-classification to decision, so new downloads file accordingly.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…gration tests

A category change no longer moves a model or changes its ID; only a
family change moves it, within its kept storage bucket.

- Reclassify tests whose point was classification now assert in place:
  unchanged directory and returned ID, new category in metadata and the
  index row, and the same resolution source. Names that described a move
  are renamed (whisper, florence, rule resolver, file signature).
- A model whose metadata already records the detected category settles
  in its bucket and reports no change.
- Move-mechanics tests (collision, identical-payload refusal, relative
  library root, quant basename, update feed, checkpoint move with the
  metadata write notifier, ID reference remapping, partial-download
  blocking, dry-run findings, lookup reclassification) use fixtures that
  require a family change: a stale path family or a recorded
  architecture family.
- Name tokens ignore the storage bucket, which is only the last
  fallback (model-type-storage-bucket).
- A disagreeing task tag no longer outvotes an architecture rule: the
  rule wins at confidence 0.60 with model-type-hint-conflict.
- Reconciliation tests expect the stale-family model under its llm bucket.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lType list

The live re-scan of the owner library resolved Kev and Laya as decision
and LTX as video, but every write was refused: strict schema-v2
metadata validation kept its own list of categories without decision
and video. ModelType::ALL is now the single list (validation messages
and RPC choosable categories derive from it), and `decision` joins the
canonical modality tokens. Clearing a chosen category records a valid
"re-detection requested" state before re-detecting. The category tests
now use strict v2 metadata and fail with the original refusal without
this change.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d outputs

Granite Speech 4.1 2b-nar became audio from its `...ForASR`
architecture, but its modalities still read text -> embedding from the
repository's contradicting feature-extraction tag. When a category comes
from a strong source (architecture/config rules, a GGUF decision head,
a disambiguation guard or a person's choice) and the task tag implies a
different category, the task and modalities follow the category; the
tag stays recorded as evidence. Weak fallbacks (file signature, name
tokens, storage bucket) never override the tag.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Important

  • Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: ea59a295-68ff-4c69-a02a-148111790696
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@greptile-apps

greptile-apps Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 4/5

[Medium impact] The PR changes model classification and library organization across the shared library and desktop catalog.

Fix the category-clear path before merging; clearing a metadata choice can move a model and leave the desktop using its old ID.

Findings

  1. P1 Clearing categories can move models ▶
  2. P2 Decision outputs stay wrong ▶
  3. P2 Conflicting tags go unflagged ▶

Summary

This PR makes model categories independent of storage folders and adds Decision and Video.

  • Models are classified by their use, including decision and video.
  • Category changes leave each model in its existing folder with the same ID.
  • People can choose a model category and keep it through later scans.
  • The library groups models by use and shows modalities and adapter badges.
  • New audit and planning documents describe repository findings and future work.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Choose or clear category] --> B{Clear choice?}
  B -->|No| C[Save user category]
  B -->|Yes| D[Remove user source]
  D --> E[Reclassify category and family]
  E --> F{Family changed?}
  F -->|Yes| G[Move model within storage bucket]
  F -->|No| H[Keep folder and ID]
  G --> I[Read original model location]
  H --> I
  C --> J[Reindex model]
  J --> K[Project task and modalities]
Loading

Reviews (1) · Last reviewed commit: "docs(plan): record the owner-library acc..." · Reviewed by Greptile

Comment on lines +4375 to +4377
self.reclassify_model(model_id).await?;
let detected = self
.load_metadata(&self.library_root.join(model_id))?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Clearing categories can move models

When a model’s family is stale, clearing its chosen category calls reclassify_model, which can move its folder and change its ID. This code discards the returned ID and reads the old location. The request then fails or returns unknown, while the desktop keeps the old ID. Use an in-place detection path here so clearing a category keeps the promised folder and ID.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 530d0fe: clearing a choice now re-detects the category only, in one guarded write, and never re-detects the family, so the model keeps its folder and ID. Test: clearing_a_choice_keeps_the_model_where_it_is.

Comment on lines +6535 to +6538
let Some(source_tag) = preferred_pipeline_tag(metadata) else {
return false;
};
let category_task = category_task_override(metadata, &source_tag);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Decision outputs stay wrong

A decision GGUF without a task tag never gets decision outputs. apply_task_projection_from_persisted_evidence returns before consulting the detected category, leaving existing text outputs or unknown values unchanged through later scans. The new chosen_category_survives_reclassification_until_cleared fixture already has this shape, but only checks the category. Use strong category evidence when no usable tag exists, and check the task and modalities in that test.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 530d0fe: when there is no task tag and the recorded task contradicts a strongly sourced category, the task (and inputs/outputs) follow the category, so a tagless decision GGUF reports text -> decision. Models with no recorded task are left unchanged. Covered in chosen_category_survives_reclassification_until_cleared.

Comment on lines +43 to +49
if gguf_declares_decision_head(model_dir) {
return Ok(ModelTypeResolution {
model_type: ModelType::Decision,
source: "model-type-gguf-decision-head".to_string(),
confidence: 0.95,
review_reasons: Vec::new(),
});

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Conflicting tags go unflagged

The decision-head branch returns before checking task hints. It always gives confidence 0.95 with no review_reasons, even when the GGUF is tagged text-generation or feature-extraction. Later task projection does not add the missing warning. Keep Decision as the result, but lower confidence and flag conflicting tags as the architecture/config path does.

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 530d0fe: the decision head still decides the category, but a task tag naming another category now adds model-type-hint-conflict and model-type-low-confidence (confidence 0.75). Test: decision_head_with_a_contradicting_tag_is_flagged_for_review.

MrScripty and others added 2 commits November 3, 2026 21:27
Addresses the Greptile review of #79:

- Clearing a chosen category re-detects the category only, in one guarded
  write. It no longer runs a full reclassification, which could re-detect
  the family and move the model before the result was read from the old ID.
- A tagless model whose recorded task contradicts a strongly sourced
  category (a decision-head GGUF recorded as text generation) takes the
  category's task, so it reports text -> decision. Models with no recorded
  task are left unchanged.
- A GGUF decision head still decides the category, but a task tag naming
  another category is recorded as a hint conflict with lowered confidence,
  so a person can confirm it.

Also stops tracking two personal working notes that were committed by
mistake in b1701a9 (the local files are kept).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Download tests waited one second for a spawned worker to reach an event
  the test expects. Those waits now use a 30-second bound; they still
  return as soon as the event arrives. Waits that check something does not
  happen, and best-effort waits, keep their one-second bound.
- The catalog custody test rebuilt an owner right after releasing the
  store lease without the module's after_fork_release retry, so a parallel
  test's fork briefly holding the inherited lock descriptor failed it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MrScripty
MrScripty merged commit 78c52fc into main Nov 4, 2026
27 checks passed
@MrScripty
MrScripty deleted the feat/model-categories branch November 4, 2026 06:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant