Skip to content

ctest: add support for modules - #5459

Draft
dybucc wants to merge 9 commits into
rust-lang:mainfrom
dybucc:feature/ctest-specific-modules
Draft

ctest: add support for modules#5459
dybucc wants to merge 9 commits into
rust-lang:mainfrom
dybucc:feature/ctest-specific-modules

Conversation

@dybucc

@dybucc dybucc commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Description

Add support for (currently) parsing and filtering Rust modules found in
the target crate to test. This is supposed to provide scaffolding for
multiple open feature requests that depend on ctest having support for
modules.

I decided to open this before finishing up work on generating proper
tests on both C and Rust sides, because I am not sure if I should just
let the user deal with including a cohesive set of modules that do not
cause item resolution conflicts, or if I should instead generate
separate tests for each parsed module.

Edit: the above paragraph really goes to say that I am not sure
whether I should generate multiple test templates while recursing
through the modules in TestGenerator::generate_files, or if I should
instead recurse through them in TestTemplate::new (with the passed
FfiItems) and return a collection of test templates for each parsed
module that has not been skipped.

Edit 2: I decided to go for generating multiple test templates for
each parsed module.

Edit 3: I completely forgot about use statements, so most of this
is currently useless.

cc @tgross35 @mbyx

TODO

  • Look into parsing use statements, as those are the one things
    missing from getting modules to work.

  • Update the docs to the public API of TestGenerator to mention
    that the name remapping ought return the C identifiers without
    worrying about potential item resolution conflicts.

  • Look through the populate_roundtrip_tests function, as that one
    I skipped to go first to populate_field_ptr_tests because of the
    order these tests appear in the test template files.

  • Look into properly resolving paths to the types of type aliases,
    constants and statics. These are handled poorly now. An example is
    the check for arrays that is made in
    template::TestTemplate::populate_roundtrip_tests, which does not
    consider whether the type of the alias is itself an alias to be
    resolved recursively (and potentially be an array.)

Update 1

I have changed the way we parse item identifiers, and more specifically,
their absolute paths without crate at the start (e.g.
foo::bar::ctime() instead of crate::foo::bar::ctime().)

This should make filtering continue working as-is right now for items,
but allow filtering on items in nested submodules by using an
appropriate string matching something like the above example.

Update 2

I think the initial set of changes is done. I have gone down the route
of potentially generating multiple templates for each parsed module,
such that more than a single test is generated. I have not yet
implemented the multi-template generation stuff, but that should be
fairly straightforward. What I believe to be done is the logic for
generating test to the right Rust paths while also keeping the single
segment paths on the C side of things.

I am currently working through the tests, to see what is missing and
broken from the prior ctest interface. At this point, I am only trying
to work through the prior tests to see if they continue working and
generating the same set of tests (barring nested modules.)

Checklist

  • Relevant tests in ctest/tests and ctest/src/tests.rs have been
    updated
  • Tested locally (LIBC_BLESS=1 cargo test -p ctest);

@rustbot label +stable-nominated

@rustbot rustbot added ctest Issues relating to the ctest crate stable-nominated This PR should be considered for cherry-pick to libc's stable release branch labels Sep 3, 2026
@dybucc
dybucc force-pushed the feature/ctest-specific-modules branch 9 times, most recently from 060d0cc to 2e69844 Compare September 5, 2026 11:20
Add support for parsing modules in the input Rust crate to `ctest`. This
should allow more easily implementing support for a number of recent
feature additions that have been needed in `libc`.
@dybucc
dybucc force-pushed the feature/ctest-specific-modules branch 2 times, most recently from 30b4a8a to f845200 Compare September 6, 2026 13:35
Add support in `TranslationHelper` to filter out modules based off of a
new type of skip accepted in `TestGenerator`'s public API. Filtering
containing items is done by filtering on stringified paths (e.g.
`foo::bar` to skip function `bar` inside top-level module `crate::foo`.)
Trim comment lines and strings to not surpass an 80-character mark from
the very first character after a newline in source. This has been
applied automatically through relevant unstable options in `rustfmt`.
- Rename `ident` function on all parsed items to `path`. After the
  changes in the last few patches, this function was returning the
  cached string we keep in each item representative of the stringified
  token stream from each item's `path` field. Thus, speaking of an
  identifier here is a bit misleading, and considering it a path to the
  item is more correct.

- Add `ident` function to return the last segment of each parsed item's
  `path` field. This corresponds now with the previous semantics of
  `ident`, now `path`, but breaks the API because it returns a
  fully-owned `String` and not a `&str`. It also incurrs an additional
  allocation on each call, but that can be easily fixed by also
  "caching" this last segment of the item's path in one of the item's
  fields.
- Replace and tweak call sites where there were uses of the `ident`
  function exposed by parsed items into using a combination of both the
  new `ident` (which replicates the former's semantics,) `path` (which
  provides the full path to the item from the crate root,) and a new
  routine; `escape_item_path`. The latter is used to replace the default
  path separator `::` with `_`. Use of the these three is necessary to
  get the tests to both refer to the right Rust items in the parsed
  crate submodules, while keeping the same (single-segment) identifiers
  for C symbols.

- Add `rust_ty` and `rust_val` fields to some of the types gathering
  data for the test templates. This is also necessary for keeping track
  of the actual paths to the types and symbols exposed by the crate, as
  modules introduce the possibility for items to be referred to by a
  path with more than one segment.

- Tweak uses of Rust to C `MapInput` type variants for remapping into
  specifically only returning `CEnum`s in the current module. Before
  modules were supported, checking straight with the list of skips in
  the running `TestGenerator` was enough. This is not the case anymore,
  as those skips are provided by the user with respect to full paths
  (i.e. they apply globally across parsed modules.) The change in this
  patch ensures that when returning a `MapInput` variant that yields a
  type and not an item, the `CEnumType` variant is returned only when
  the current module being parsed contains an alias with the passed
  identifier, and further checks with the `TestGenerator` if the full
  path to that alias (if found) has a `CEnum` remapping set.

- Tweak the Rust test template to reflect the changes made to the
  `template` module. This uses the new fields introduced in this same
  patch, and changes some uses of the `id` field to either one of the
  `rust_ty` or `rust_val` fields, as those keep the full path to the
  item type or item, respectively.
- Tweak one of the tests to adjust to the way module support has been
  implemented. Previously, items in nested modules would be expected to
  surface at the top-level. Now they are meant to be part of the parsed
  nested module within the initial `FfiItems` instance.

- Tweak Rust test template to avoid `unused` lints against some of the
  utility functions. I am still looking through some stuff in the tests,
  but thus far these functions sometimes simply don't get used because
  certain askama loops never iterate when the tests that use them are
  wholly skipped.
@dybucc
dybucc force-pushed the feature/ctest-specific-modules branch from f845200 to 3f32191 Compare September 6, 2026 16:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ctest Issues relating to the ctest crate stable-nominated This PR should be considered for cherry-pick to libc's stable release branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants