Skip to content

Handle mapping versions for Dart pubspec dependencies #5482 - #5487

Open
Raman-Katiyar wants to merge 1 commit into
aboutcode-org:developfrom
Raman-Katiyar:fix/5482-dart-path-dependency
Open

Raman-Katiyar wants to merge 1 commit into
aboutcode-org:developfrom
Raman-Katiyar:fix/5482-dart-path-dependency

Conversation

@Raman-Katiyar

Copy link
Copy Markdown

Fixes #5482

Problem

Scanning a Dart package whose pubspec.yaml declares a dependency
with a mapping version — for example a path: dependency:

dev_dependencies:
  _json_serial_shared_test:
    path: ../shared_test

crashes the pubspec parser:

File ".../src/packagedcode/pubspec.py", line 216, in build_dep
    if version.replace('.', '').isdigit():
AttributeError: 'dict' object has no attribute 'replace'

The exception aborts the whole parse, so the scan returns no packages
and no dependencies for that package. Reproduced with the real
json_serializable 6.15.2 package from the issue
(https://pub.dev/api/archives/json_serializable-6.15.2.tar.gz).

Root cause

build_dep() in src/packagedcode/pubspec.py assumes the dependency
version is always a string. Dart dependencies can also be mappings
(dependency sources): path:, git:, hosted: (with a nested
version:), and sdk:. Only the {'sdk': ...} mapping form was
handled; every other mapping reached version.replace(...) and
crashed.

Fix

Generalize the mapping handling in build_dep() so any mapping
version is flattened into a requirement string that keeps the source
information, e.g. path: ../shared_test,
git: url: https://github.com/example/git_dep.git, ref: main,
hosted: https://example.com, version: ^1.0.0. The sdk: form keeps
its previous rendering (sdk: flutter), so existing behavior is
unchanged.

Tests

Two new data-driven regression tests in
tests/packagedcode/test_pubspec.py (no test code change needed —
fixtures are picked up automatically):

  • tests/packagedcode/data/pubspec/specs/json-serializable-pubspec.yaml
    (+ expected JSON): the real pubspec.yaml from the issue.
  • tests/packagedcode/data/pubspec/specs/dependency-sources-pubspec.yaml
    (+ expected JSON): a minimal fixture covering path:, git: (string
    and mapping forms), hosted: with a nested version:, and sdk:
    mapping forms.

Fail-before (unmodified code): both new tests fail with
AttributeError: 'dict' object has no attribute 'replace' at
pubspec.py:216, and parsing the real pubspec.yaml raises the same
error.

Pass-after:

  • tests/packagedcode/test_pubspec.py: 14 passed (12 existing +
    2 new), run both with the vendored test dependencies and in a fresh
    venv with pip install -e . (Python 3.12.3).
  • End-to-end with the real ScanCode CLI
    (scancode --package on the extracted json_serializable 6.15.2
    package): 1 package (json_serializable 6.15.2) and 24
    dependencies
    , including
    pkg:pubspec/_json_serial_shared_test with requirement
    path: ../shared_test. Before the fix the same scan returned no
    package at all.
  • ruff check src/packagedcode/pubspec.py: all checks passed.
    (ruff format would reformat the whole file, including untouched
    code, so it was not applied — the new code follows the existing
    single-quote style of the file.)

Also updated CHANGELOG.rst (Next release section).

Tasks

  • Reviewed contribution guidelines
  • PR is descriptively titled and links the original issue above
  • Tests pass — run locally (see validation above)
  • Commits are in uniquely-named feature branch and has no merge conflicts
  • Updated documentation pages (if applicable) — not applicable, no user-facing behavior change beyond the bug fix
  • Updated CHANGELOG.rst

Signed-off-by: Raman Katiyar ramankatiyar273@gmail.com



Dart dependencies can be mappings, not only version strings, e.g. a
path dependency such as:

  dev_dependencies:
    _json_serial_shared_test:
      path: ../shared_test

build_dep() only handled the {'sdk': ...} mapping form and called
version.replace() on every other mapping, crashing with
AttributeError: 'dict' object has no attribute 'replace' and aborting
the whole pubspec parse, so scanning such a package returned no
packages and no dependencies.

Flatten any mapping version into a requirement string that keeps the
source information (e.g. 'path: ../shared_test', 'git: url: ..., ref:
main', 'hosted: ..., version: ^1.0.0'). The 'sdk:' form keeps its
previous rendering ('sdk: flutter').

Add data-driven regression tests with the real json_serializable
6.15.2 pubspec.yaml from the issue and a minimal fixture covering
path, git, hosted and sdk mapping forms.

Signed-off-by: Raman Katiyar <ramankatiyar273@gmail.com>
@Raman-Katiyar

Copy link
Copy Markdown
Author

@AyanSinhaMahapatra @pombredanne — could you please review this when you have a chance? It fixes #5482, where a Dart path: dependency crashed the pubspec parser so the scan returned no packages at all.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Running scan on Dart package does not return any packages or dependencies

1 participant