ngclient: add download_target_bytes()

[ChrisJr404]

Adds Updater.download_target_bytes(), which downloads and verifies a target and hands back the content as bytes instead of writing it to the cache dir, so callers like sigstore-python that just want the verified bytes in memory don't have to round-trip through a file.

The verification path is shared, not duplicated: I pulled the URL-building out of download_target() into a small _target_file_url() helper, and both methods then do the same _fetcher.download_file() + targetinfo.verify_length_and_hashes() against the same downloaded stream. The only difference is what happens after verification succeeds - download_target() copies to disk, download_target_bytes() reads the stream and returns it. So the length/hash checks are byte-for-byte identical between the two.

On the cache-timing concern from #1556 (re #1168): the new method deliberately does not touch the local target cache at all. It doesn't call find_cached_target(), doesn't read an existing cached file, and doesn't persist anything. Since it never consults the cache, there's no cache-hit/miss timing to leak - it always downloads and verifies. Callers who want caching keep using download_target() + find_cached_target().

One thing worth flagging: because a target's hash can't be checked until it's fully downloaded, the bytes variant buffers the whole target in memory. That's fine for the small artifacts this is aimed at, but it's not suitable for very large targets - I noted that in the docstring. The streaming/iterator idea from the issue thread would be a separate, larger API and I left it out here.

Tests in tests/test_updater_fetch_target.py, mirroring the existing download_target tests against the repository simulator:

• test_fetch_target_bytes (dataset-driven, same three target cases): returned bytes match expected content, and nothing gets written to the cache dir (find_cached_target stays None).
• test_invalid_target_download_bytes: hash mismatch and length mismatch both still raise RepositoryError, and nothing is persisted.

tox-equivalent locally: pytest tests/test_updater_*.py green (updater suites 27 passed / 13 subtests), black/isort clean at line-length 80, mypy clean on updater.py, pylint 9.91 (only pre-existing __init__ arg-count warnings, untouched by this change).

Fixes #1556

[jku]

Thanks! On just the description I have some concerns I need to at least think about:

• do we want to provide an API just for small artifacts? I think this is probably acceptable: it does simplify things for us and the user (if you are dealing with a small artifact then dealing with an iterator is just another hoop to jump through)
• Avoiding the cache completely sounds like a bad idea: it promotes bad practices (re-downloading things over and over). The main user experience pain point is that a user that just wants the bytes needs to jump through two hoops: first call tuf API, then read the file. Whether cache is written to is related but not tied to this.
• A related API snag is that reading from cache still requires reading a file... so users can't completely avoid reading files:
  • We should likely have a test or an example that shows the whole API use (get_targetinfo, find_cached_target+ read cache file, download_target_bytes) to make sure we understand how the new API is used
  • to really make the API simpler we would want the new method to cover all of
    • find_cached_target
      • if found, read cache, return bytes
    • download the target bytes
      • write cache
      • return bytes

I did mention cache timing in the bug but that was probably a mistake on my part: We certainly don't want to avoid caching because of fears that this somehow leaks timing info.

Possible paths forward

I can see a few possible potential routes:

1. follow your solution but include writing to cache
   • users in reality still need to deal with reading cached files so it's not as useful as it could be
2. Implement the higher level method idea that combines cache lookup, download and writing to the cache so the user experience is better
   • This was not part of the original API design because we wanted things to work with huge files... but since we're prepared to compromise on that for this bit of API, we could do it
   • I wonder if we can go all the way and have a Updater.get_target_bytes(self, target_path: str) -> bytes instead? This would call get_targetinfo, then find_cached_target (on cache hit read the file and return early), then download target, store it in cache, return bytes
   • this would have most impact, making the API a lot simpler for this small file use case. There might be some unforeseen complications though

I think there might be something to the second option if you're willing to experiment with it and see how it looks like

[jku]

Also I'd appreciate it if you document which parts of the work (code, tests, PR message) are AI generated