perf(python): efficient data convesion between numpy to raw - #40
perf(python): efficient data convesion between numpy to raw#40ktro2828 wants to merge 4 commits into
Conversation
Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>
Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>
This reverts commit db17649.
Signed-off-by: ktro2828 <kotaro.uetake@tier4.jp>
|
cc @zusizusi |
manato
left a comment
There was a problem hiding this comment.
@ktro2828
Thank you very much for improving the performance of python binding! The speed up is quite impressive!
I left a few comments to the places where I noticed. Though they may miss the point, your consideration is appreciated!
| image.format = ImageFormat.RAW | ||
|
|
||
| data_u8 = data.astype(np.uint8) | ||
| data_u8 = np.ascontiguousarray(data, dtype=np.uint8) |
There was a problem hiding this comment.
| data_u8 = np.ascontiguousarray(data, dtype=np.uint8) | |
| data_u8 = np.array(data, dtype=np.uint8, copy=True, order='C') |
According to my search, np.ascontiguousarray behaves differently according if the input data is already contiguous or not:
- if the
datais already contiguous: returnsdataas it is - if the
datais NOT contiguous: returns contiguous copy
If we would prefer to secure that from_numpy method returns a new instance (i.e., independent memory region from input), like which is tested test_common.py::test_image_buffer_input_is_independent_and_numpy_output_can_copy(), using always making copy might be better.
Checikg if this aligns your intention is appreciated 🙏
| np.testing.assert_array_equal(array, np.arange(7, dtype=np.uint8)) | ||
|
|
||
|
|
||
| def test_image_to_numpy(): |
There was a problem hiding this comment.
[ask] For me, this function tests nothing (just filling the object members). Is it intended?
PR Type
Related Links
Description
Imagenow implements Python's read-only buffer protocol and providesImage.to_numpy().to_numpy()returns a zero-copy NumPy view by default, whilecopy=Trueproduces independentmutable storage.
Image.from_numpy()now passes a contiguous NumPy buffer directly to C++, whereone bulk
memcpyreplaces millions of Python-integer conversions. The existingImage.datalist getter and iterable setter remain available for compatibility.
Because a zero-copy view references the C++ vector, callers must not reassign
Image.datawhile aview is alive. The view retains the
Imageobject itself, so deleting the original Python variabledoes not invalidate the view.
Benchmark
Imagenow implements Python's read-only buffer protocol and providesImage.to_numpy().to_numpy()returns a zero-copy NumPy view by default, whilecopy=Trueproduces independentmutable storage.
Image.from_numpy()now passes a contiguous NumPy buffer directly to C++, whereone bulk
memcpyreplaces millions of Python-integer conversions. The existingImage.datalist getter and iterable setter remain available for compatibility.
Because a zero-copy view references the C++ vector, callers must not reassign
Image.datawhile aview is alive. The view retains the
Imageobject itself, so deleting the original Python variabledoes not invalidate the view.
Imagelist -> vector)buffer -> vector)Imageinto NumPyvector -> list)Review Procedure
Remarks
Pre-Review Checklist for the PR Author
PR Author should check the checkboxes below when creating the PR.
Checklist for the PR Reviewer
Reviewers should check the checkboxes below before approval.
Post-Review Checklist for the PR Author
PR Author should check the checkboxes below before merging.
CI Checks