Skip to content

DAOS-17948 object: fix bugs for sgl duplicate logic - #16796

Merged
gnailzenh merged 7 commits into
release/2.6from
shilongw/DAOS-17948-2.6
Sep 24, 2025
Merged

gnailzenh merged 7 commits into
release/2.6from
shilongw/DAOS-17948-2.6

Conversation

@wangshilong

@wangshilong wangshilong commented Sep 2, 2025 •

Copy link
Copy Markdown
Contributor
  1. When IO retries occur, a use-after-free scenario may arise during
    scatter-gather list (sgl) merging. This happens because retried I/O's
    sgl references dup_sgls, which could be prematurely freed in obj_reasb_io_fini().

    To fix above issue, dup_sgls must not be freed during retry operations
    and should be called only once per I/O operation.

  2. If both short reads and duplicated SGLs occur, the iov_len of the original
    SGLs must be updated properly to ensure the correct number of read data bytes
    is returned.

  3. A test case has been added to verify that short reads function as expected during SGL merging.

Steps for the author:

  • Commit message follows the guidelines.
  • Appropriate Features or Test-tag pragmas were used.
  • Appropriate Functional Test Stages were run.
  • At least two positive code reviews including at least one code owner from each category referenced in the PR.
  • Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

  • Gatekeeper requested (daos-gatekeeper added as a reviewer).

@wangshilong
wangshilong requested review from a team September 2, 2025 02:24
@github-actions

github-actions Bot commented Sep 2, 2025

Copy link
Copy Markdown

Ticket title is 'Potential use-after-free for ec retry IO'
Status is 'In Review'
https://daosio.atlassian.net/browse/DAOS-17948

@wangshilong wangshilong changed the title DAOS-17948 object: fix potential use-after-free in EC retry path DAOS-17948 object: fix potential use-after-free in io retry path Sep 2, 2025
Comment thread src/object/cli_ec.c Outdated
Comment thread src/object/cli_obj.c Outdated
Comment thread src/object/cli_obj.c Outdated
Comment thread src/object/cli_obj.c
1. When IO retries occur, a use-after-free scenario may arise during
scatter-gather list (sgl) merging. This happens because retried I/O's
sgl references dup_sgls, which could be prematurely freed in obj_reasb_io_fini().

To fix above issue, processed_sgls must not be freed during retry operations
and should be called only once per I/O operation.

2. If both short reads and duplicated SGLs occur, the iov_len of the original
SGLs must be updated properly to ensure the correct number of read data bytes
is returned.

3. Since only fetched data bytes are copied back, it is unnecessary to copy
the original buffer to the merged buffer before fetching.

4. A test case has been added to verify that short reads function as expected
during SGL merging.

Signed-off-by: Wang Shilong <shilong.wang@hpe.com>
@wangshilong
wangshilong force-pushed the shilongw/DAOS-17948-2.6 branch from 864fc2b to 669ca6f Compare September 3, 2025 04:40
@wangshilong wangshilong changed the title DAOS-17948 object: fix potential use-after-free in io retry path DAOS-17948 object: fix bugs for sgl duplicate logic Sep 3, 2025
@daosbuild3

Copy link
Copy Markdown
Collaborator

Test stage Functional on EL 8.8 completed with status UNSTABLE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos//view/change-requests/job/PR-16796/2/testReport/

Signed-off-by: Wang Shilong <shilong.wang@hpe.com>
@daosbuild3

Copy link
Copy Markdown
Collaborator

Test stage Test RPMs on EL 8.6 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-16796/4/display/redirect

@daosbuild3

Copy link
Copy Markdown
Collaborator

Test stage Test RPMs on EL 8.6 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-16796/6/display/redirect

@daosbuild3

Copy link
Copy Markdown
Collaborator

Test stage Functional on EL 8.8 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-16796/6/display/redirect

gnailzenh
gnailzenh previously approved these changes Sep 5, 2025
Comment thread src/object/cli_obj.c
@@ -4907,6 +4907,7 @@ obj_dup_sgls_free(struct obj_auxi_args *obj_auxi)
d_sg_list_t *sg_orig = &ctx->sgls_orig[i];
uint32_t dup_sg_idx = 0;
uint32_t dup_buf_size = 0;

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so the dup_buf_size is only for sanity check after the change, right?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes.

@daosbuild3

Copy link
Copy Markdown
Collaborator

Test stage Functional Hardware Large MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-16796/10/execution/node/925/log

Signed-off-by: Wang Shilong <shilong.wang@hpe.com>
@gnailzenh
gnailzenh merged commit a61eb70 into release/2.6 Sep 24, 2025
43 of 45 checks passed
@gnailzenh
gnailzenh deleted the shilongw/DAOS-17948-2.6 branch September 24, 2025 03:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

5 participants