Skip to content

feat(skills): add read-side resource for DIAL Core /v2/skills - #138

Open
andrii-novikov wants to merge 2 commits into
developmentfrom
feat/skills-read
Open

feat(skills): add read-side resource for DIAL Core /v2/skills#138
andrii-novikov wants to merge 2 commits into
developmentfrom
feat/skills-read

Conversation

@andrii-novikov

Copy link
Copy Markdown
Contributor

Applicable issues

Description of changes

Adds a skills resource covering the read side of DIAL Core's /v2/skills API, introduced by epam/ai-dial-core#1633 ("DIAL Folder As Resource"). A skill is a folder-shaped resource: a .dial-resource marker pointing at an immutable v/{versionId}/ subtree, containing a mandatory SKILL.md plus arbitrary bundled files.

The write side (PUT/DELETE for whole resources, grouping folders and single files) is deliberately out of scope and tracked in #137 — it carries open questions (multipart part-naming, ETag-only responses, If-Match create-vs-overwrite semantics) that would have held up a complete, useful read capability.

API

Method Endpoint
get_metadata(url, *, limit, token, recursive) GET /v2/metadata/skills/{bucket}/{path}
list_files(url, *, path, limit, token, recursive) GET /v2/metadata/skills/{bucket}/{path}/files[/{sub}]
get_file(url, file_path, etag_if_match) GET /v2/skills/{bucket}/{path}/files/{filePath}
download(url, etag_if_match) GET /v2/skills/{bucket}/{path}application/zip

AsyncSkills adds stream_file / stream_download. The sync client has no stream method, so sync stays non-streaming — the same asymmetry that already exists for Files.

listing = client.skills.get_metadata(client.my_skills_home())

token = None
while True:
    page = client.skills.list_files(skill, recursive=True, limit=1000, token=token)
    ...
    token = page.next_token
    if token is None:
        break

manifest = client.skills.get_file(skill, "SKILL.md")
archive = client.skills.download(skill)   # filename -> "<skill-name>.zip"

Groundwork

The library assumed /v1 throughout, so this is not just one more resource class:

  • API_V2_PREFIX / METADATA_V2_PREFIX constants.
  • V2StorageResourceType = Literal["skills"] joins the parser's resource-type union. The v1 StorageResourceType is left intact so Metadata._get_cast_to's assert_never stays exhaustive.
  • DialStorageResourceMixin gains a per-resource api_prefix.
  • Opt-in allow_bucket_root, so skills/{bucket} parses — Core's children listing accepts an empty {path}. It is opt-in rather than automatic because a two-segment path is ambiguous: files/my-file.txt has the same shape and must stay a missing-bucket error (test_get_api_path_missing_bucket pins this). Only skills.get_metadata enables it, so v1 behaviour is unchanged.
  • api_v2_url property, my_skills_home(), and Skills wired into both _init_resources.

Path validation

skills is the first resource that takes two user-supplied paths. The url argument is fully re-parsed as before, but file_path / path are concatenated onto the already-parsed api path and never go back through the parser — so nothing else would catch a traversal segment. urljoin resolves . and .. while building the request, which shifts the bucket segment:

get_file("skills/mybucket/my-skill", "../../../otherbucket/their-skill/files/SKILL.md")
  -> GET /v2/skills/otherbucket/their-skill/files/SKILL.md

Both are now rejected, and segments are checked as they decode: _percent_encode_relative_url normalizes with unquote before quoting, so %2e%2e would otherwise pass a literal check and still reach urljoin as ... Encoded separators (%2f) are rejected too — they split a segment after validation and leak into the derived filename.

This is skills-specific, not a pre-existing issue: every v1 method takes exactly one path, which is always re-validated, so traversal there already lands on an invalid resource type and is rejected.

Drive-by

_files_error_processor and _prompts_error_processor were byte-identical and skills needed a third, so they are folded into a shared storage_error_processor in helpers/storage_resource.py.

Notes for reviewers

  • Every /v2/skills operation is marked x-preview: true in Core's open_api_core.yaml; the README says so.
  • Core's spec is lossy in places, so behaviour was verified against ai-dial-core@development rather than the OpenAPI doc — notably that the ZIP response carries no Content-Disposition (hence the derived filename) and that neither metadata listing returns an etag on the container (ResourceFolderMetadata extends MetadataBase, which has no such field).
  • The children listing carries no etag and no skill name/description — Core builds it from marker listing metadata without reading the marker body. Documented in the README.
  • make lint fails identically on a clean development (the nox lint venv lacks test deps for pyright); ruff and pyright are clean in the project venv.

Checklist

  • Title of the pull request follows Conventional Commits specification
  • README.md updated with sync + async usage and sample response objects
  • New classes exported from resources/__init__.py
  • Unit tests added (442 pass, up from 383); matrix green on every locally available interpreter

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Adds Skills/AsyncSkills covering the four read operations of Core's
folder-as-resource API (epam/ai-dial-core#1633):

  get_metadata  GET /v2/metadata/skills/{bucket}/{path}
  list_files    GET /v2/metadata/skills/{bucket}/{path}/files[/{sub}]
  get_file      GET /v2/skills/{bucket}/{path}/files/{filePath}
  download      GET /v2/skills/{bucket}/{path}  (application/zip)

AsyncSkills also exposes stream_file/stream_download; the sync client
has no stream method, mirroring Files.

Groundwork, since the library assumed /v1 throughout:
- API_V2_PREFIX and METADATA_V2_PREFIX constants
- V2StorageResourceType joins the parser's resource-type union, and
  DialStorageResourceMixin gains a per-resource api_prefix
- opt-in allow_bucket_root so "skills/{bucket}" parses, which Core's
  children listing accepts as an empty {path}; it stays opt-in because
  a two-segment path is ambiguous with "files/my-file.txt"
- api_v2_url property, my_skills_home(), Skills wired into both clients

file_path and path are validated before being concatenated onto the
already-parsed api path, which never goes back through the url parser:
"." and ".." segments and encoded separators are rejected. Segments are
checked as they decode, since _percent_encode_relative_url normalizes
with unquote before quoting, so "%2e%2e" would otherwise reach urljoin
as ".." and retarget the request at another bucket.

Also folds the byte-identical _files_error_processor and
_prompts_error_processor into a shared storage_error_processor.

Closes #136
Refs #135
Follow-up on review of the /v2/skills read resource.

Bucket-root listing was unroutable: the separator before {path} in Core's
COMPLEX_RESOURCE_METADATA regex is literal, so an empty {path} only matches
with a trailing slash, which api_path (a PurePosixPath) never carries. Emit
one from _prepare_metadata_request; Core strips it back off {path}, so the
deeper paths resolve to the same folder as before.

Drop etag_if_match from the four reads. Neither ComplexResourceController
.get nor .getFile calls ProxyUtil.etag, and neither operation declares an
If-Match parameter or a 412 response, so the header was sent and silently
ignored - advertising a precondition the server does not enforce. The /v1
reads do honour it, which is where the parameter was copied from.

Drop content_length/content_type from SkillFileItem: listFiles builds plain
ResourceItemMetadata entries, which carry neither field.

Document that subfolders in the files listing are reported as "ITEM" too -
listFiles never overrides the node type, so only the trailing "/" of the url
marks a folder. node_type stays a union so an upstream fix does not become a
parsing error here.

Add test_skill_core_routes.py, which matches every built read URL against
Core's own route regexes - the string assertions could not tell a
well-formed url from a routable one.
@andrii-novikov
andrii-novikov marked this pull request as ready for review August 31, 2026 09:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Skills: read side — get_metadata, list_files, get_file, download (/v2/skills)

1 participant