Repository navigation
Draft: Support arbitrary type in item accessor's subscript operator [with more tests] - #4007
cielavenir wants to merge 11 commits into
Conversation
| py::list l1 = py::list(); | ||
| l1.append(1); | ||
| l1.append(2); | ||
| py::object l2 = l1; |
There was a problem hiding this comment.
This appears to be causing clang-tidy errors and a segfault when using python debug mode.
LIkely because the underlying list holding the elements is deleted and it's using the generic object accessor instead of the specialized list accelerator. This making l[1] still be attached the list object which causes issues.
There was a problem hiding this comment.
No, but I looked a little bit only.
Sorry, I don't have the free bandwidth to help debugging this.
There was a problem hiding this comment.
@rwgk No prob, just wondering if you have seen this behavior before.
Skylion007
left a comment
There was a problem hiding this comment.
Nits: should make clang-tidy happy and are better style more general (allows reference stealing).
|
by the way, L62 is why this one segfaults? |
|
@rwgk @Skylion007 do you have ideas about segfault in debug build? |
Yeah, unfortunately and I am not sure it's an easy fix (at least if it's really the method after (access_list_as_object) that is segfaulting. It's because depending on the type of the object, the accessor either borrows or steals the reference to the underlying object. What's happening here is that the list is deleted which is the only remaining reference to the object since it's not properly borrowed by the item accessor. The GC then frees the item since the ref_count is 0. However, we still have a pointer to that object (returned from the function), but it's now pointing to free memory. Hence, the segfault. Not sure these is an easy solution here as the type of the accessor that needs to be called depends on the object. You may be able to get away with this through virtual inheritance / overloading, but that itself seems bugprone and currently our class inheritance method is non-virtual and pybind11 even supports compilation without RTTI (which this solution may break). We could also try to return strong reference for the accessors by seeing if we can use different CPython APIs, but this may both have performance implications and could introduce ref_count based memory leaks if we are not careful. Not sure what a good solution is here other than ensure that object is properly downcasted or that the argument is a proper Python object. We already pushed a PR that disable implicit conversions of int to handles so that should fix some of the issues that this PR brought up. |
|
@henryiii Do you mind explaining why you closed this pull request? I pushed https://github.com/cielavenir/pybind11/tree/addTests again but the CI does not run anymore. Or is the issue fixed upstream actually? |
|
or do we need to use customized version of pybind11? |
|
I was closing all duplicate PRs, I assumed 4004 was the main one. This said "to run ci", so I thought it was a CI-only duplicate of #4004, which is still open. If it's better to keep this one open instead, let me know and I can switch them. |
|
I opened #6286, which I think solves these cleanly. |
(to run ci)
#4004