restore zero-filled storage when called (#21917) - #21917
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21917
Note: Links to docs will display an error until the docs builds have been completed. ❌ 4 New FailuresAs of commit 507e40e with merge base 080e4a2 ( NEW FAILURES - The following jobs have failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@JakeStevens has exported this pull request. If you are a Meta employee, you can view the originating Diff in D116459495. |
This PR needs a
|
digantdesai
left a comment
There was a problem hiding this comment.
Review automatically exported from Phabricator review in Meta.
Summary: To reduce GPU memory during compilation, ExecuTorch intentionally keeps shape metadata but frees its storage with resize_(0). The existing workaround restored the zero-filled storage only when called from Inductor’s _unlift_graph. PyTorch 2.13 introduced other cloning paths during AOTI compilation/autotuning. Those paths attempted to clone the shaped tensor while its storage was still zero bytes, producing: ``` required storage: 4194304 bytes actual storage: 0 bytes ``` The fix now detects any intentionally emptied KV-cache tensor before examining the caller and restores zero-filled storage Reviewed By: digantdesai Differential Revision: D116459495
507e40e to
4e68d6d
Compare
Summary:
To reduce GPU memory during compilation, ExecuTorch intentionally keeps shape metadata but frees its storage with resize_(0).
The existing workaround restored the zero-filled storage only when called from Inductor’s _unlift_graph. PyTorch 2.13 introduced other cloning paths during AOTI compilation/autotuning. Those paths attempted to clone the shaped tensor while its storage was still zero bytes, producing:
The fix now detects any intentionally emptied KV-cache tensor before examining the caller and restores zero-filled storage
Reviewed By: digantdesai
Differential Revision: D116459495