Skip to content

Navigation Menu

Appearance settings

View all features
- BY COMPANY SIZE
  Enterprises
  Small and medium teams
  Startups
  Nonprofits
- BY USE CASE
  App Modernization
  DevSecOps
  DevOps
  CI/CD
  View all use cases
- BY INDUSTRY
  Healthcare
  Financial services
  Manufacturing
  Government
  View all industries
View all solutions
View all resources
- ENTERPRISE SOLUTIONS
  Enterprise platformAI-powered developer platform
- AVAILABLE ADD-ONS
  GitHub Advanced SecurityEnterprise-grade security features
  Copilot for BusinessEnterprise-grade AI features
  Premium SupportEnterprise-grade 24/7 support
Pricing

Appearance settings

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

Uh oh!

There was an error while loading. Please reload this page.

pytorch / executorch Public

Notifications You must be signed in to change notification settings
Fork 1.1k
Star 5k

Code
Issues 929
Pull requests 361
Discussions
Actions
Projects
Wiki
Security and quality
Insights

Additional navigation options

Code
Issues
Pull requests
Discussions
Actions
Projects
Wiki
Security and quality
Insights

Keep the CUDA memory pool warm between delegates - #22312

#22312

Merged

shoumikhin merged 6 commits into

pytorch:mainpytorch/executorch:mainfrom

shoumikhin:fix/cuda-mempool-release-thresholdshoumikhin/executorch:fix/cuda-mempool-release-thresholdCopy head branch name to clipboard

Sep 2, 2026

Conversation Commits6 (6)Checks Files changed

Merged

Keep the CUDA memory pool warm between delegates#22312
shoumikhin merged 6 commits into
pytorch:mainfrom
shoumikhin:fix/cuda-mempool-release-threshold

Wait for the device, not a stream, before releasing the pool

109b84e

Select commit

Loading

Failed to load commit list.

Uh oh!

There was an error while loading. Please reload this page.

Sign in for the full log view

Labeler on: pull_request_target 1

label

Docathon Labels Sync on: pull_request_target 1

check-labels

Check Labels on: pull_request_target 1

Check labels

Lint on: pull_request 4

Get changed files / get-changed-files

link-check / lint-urls

link-check / lint-xrefs

link-check / lint-file-size

lintrunner-mypy

lintrunner

Test CUDA Windows Export and E2E on: pull_request 1

Get changed files / get-changed-files

CI run decision / decide

export-model-cuda-windows-artifact

test-model-cuda-windows-e2e

periodic on: pull_request 7

gather-models

test-models-linux (buck2, mv3, portable, linux.2xlarge, 90) / linux-job

test-models-linux (buck2, mv3, xnnpack-quantization-delegation, linux.2xlarge, 90) / linux-job

test-models-linux (cmake, mv3, portable, linux.2xlarge, 90) / linux-job

test-models-linux (cmake, mv3, xnnpack-quantization-delegation, linux.2xlarge, 90) / linux-job

test-models-linux (cmake, vit, portable, linux.2xlarge, 90) / linux-job

test-models-linux (cmake, vit, xnnpack-quantization-delegation, linux.2xlarge, 90) / linux-job

Build documentation on: pull_request 1

build (buck2) / Build doc

upload-gh-pages

Test CUDA Builds on: pull_request 36

Get changed files / get-changed-files

CI run decision / decide

test-executorch-cuda-build-12.6 / linux-job

test-executorch-cuda-build-13.0 / linux-job

test-models-cuda / linux-job

export-cuda-target-smem-cross-arch-a100 / linux-job

unittest-cuda / linux-job

unittest-cuda-runtime / linux-job

test-model-cuda-e2e-${{ matrix.model.name }}-${{ matrix.quant }}

test-muse-glimmer-cuda-e2e-kquant-17gb-solo-text / linux-job

test-muse-glimmer-cuda-e2e-kquant-17gb-dflash-image / linux-job

test-muse-glimmer-cuda-e2e-kquant-dynamic-solo-text / linux-job

test-muse-glimmer-cuda-e2e-kquant-dynamic-dflash-image / linux-job

check-all-cuda-builds

test-cuda-target-smem-cross-arch-a10g / linux-job

test-cuda-pybind

Build Presets on: pull_request 15

apple (macos) / build

apple (ios) / build

apple (ios-simulator) / build

apple (pybind) / build

apple (profiling) / build

apple (llm) / build

zephyr (zephyr) / build

linux (linux, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build

linux (linux, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / build

linux (pybind, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build

linux (pybind, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / build

linux (llm, linux.2xlarge, executorch-ubuntu-22.04-clang12) / build

linux (llm, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11-aarch64) / build

windows (pybind) / build

windows (windows) / build

Test XNNPACK Backend on: pull_request 3

test-xnnpack / test-backend-linux (xnnpack, models) / linux-job

test-xnnpack / test-backend-linux (xnnpack, operators) / linux-job

test-xnnpack / test-backend-macos

test-xnnpack / package-golden-artifacts

Build Aarch64 Linux CUDA Wheels on: pull_request 4

generate-matrix / generate

filter-matrix

pytorch/executorch / build-wheel-py3_10-cuda-aarch6413_0-aarch64

pytorch/executorch / upload / upload-wheel-py3_10-cuda-aarch6413_0-aarch64

Test QNN Windows MSVC build on: pull_request 1

build-qnn-windows-msvc / windows-job

Test ROCm AOTI on: pull_request 1

Get changed files / get-changed-files

CI run decision / decide

AOTI CI run decision

Voxtral CI run decision

unittest-rocm-gfx950-rocm${{ matrix.rocm-version }}

test-voxtral-realtime-rocm-gfx950-rocm${{ matrix.rocm-version }}

test-voxtral-realtime-rocm-gfx1100-rocm${{ matrix.rocm-version }}

Cadence Build & Test on: pull_request

cpu-build

hifi-build

vision-build

cpu-test

hifi-op-test

pull on: pull_request 133

Get changed files / get-changed-files

CI run decision / decide

test-qnn-wheel-packages-linux (3.10) / linux-job

test-qnn-wheel-packages-linux (3.11) / linux-job

test-qnn-wheel-packages-linux (3.12) / linux-job

test-qnn-wheel-packages-linux (3.13) / linux-job

test-setup-linux-gcc / linux-job

test-models-linux-basic (mv3, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job

test-models-linux-basic (mv3, portable, cmake, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11... / linux-job

test-models-linux-basic (mv3, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job

test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job

test-models-linux-basic (mv3, xnnpack-quantization-delegation, cmake, linux.arm64.2xlarge, execut... / linux-job

test-models-linux-basic (mv3, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job

test-models-linux-basic (vit, portable, cmake, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job

test-models-linux-basic (vit, portable, cmake, linux.arm64.2xlarge, executorch-ubuntu-22.04-gcc11... / linux-job

test-models-linux-basic (vit, portable, buck2, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job

test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, linux.2xlarge, executorch-u... / linux-job

test-models-linux-basic (vit, xnnpack-quantization-delegation, cmake, linux.arm64.2xlarge, execut... / linux-job

test-models-linux-basic (vit, xnnpack-quantization-delegation, buck2, linux.2xlarge, executorch-u... / linux-job

test-models-linux (linear, portable, linux.2xlarge) / linux-job

test-models-linux (linear, xnnpack-quantization-delegation, linux.2xlarge) / linux-job

test-models-linux (add, portable, linux.2xlarge) / linux-job

test-models-linux (add, xnnpack-quantization-delegation, linux.2xlarge) / linux-job

test-models-linux (add_mul, portable, linux.2xlarge) / linux-job

test-models-linux (add_mul, xnnpack-quantization-delegation, linux.2xlarge) / linux-job

test-models-linux (ic3, portable, linux.2xlarge) / linux-job

test-models-linux (ic3, xnnpack-quantization-delegation, linux.2xlarge) / linux-job

test-models-linux (mv2, portable, linux.2xlarge) / linux-job

test-models-linux (mv2, xnnpack-quantization-delegation, linux.2xlarge) / linux-job

test-models-linux (resnet18, portable, linux.2xlarge) / linux-job

test-models-linux (resnet18, xnnpack-quantization-delegation, linux.2xlarge) / linux-job

test-models-linux (resnet50, portable, linux.2xlarge) / linux-job

test-models-linux (resnet50, xnnpack-quantization-delegation, linux.2xlarge) / linux-job

test-models-linux (mobilebert, portable, linux.2xlarge) / linux-job

test-models-linux (mobilebert, xnnpack-quantization-delegation, linux.2xlarge) / linux-job

test-models-linux (emformer_transcribe, portable, linux.2xlarge) / linux-job

test-models-linux (emformer_transcribe, xnnpack-quantization-delegation, linux.2xlarge) / linux-job

test-models-linux (ic4, portable, linux.4xlarge.memory) / linux-job

test-models-linux (ic4, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job

test-models-linux (emformer_join, portable, linux.4xlarge.memory) / linux-job

test-models-linux (emformer_join, xnnpack-quantization-delegation, linux.4xlarge.memory) / linux-job

test-models-linux (phi_4_mini, portable, linux.4xlarge.memory) / linux-job

test-models-linux (llama3_2_vision_encoder, portable, linux.4xlarge.memory) / linux-job

test-models-linux (w2l, portable, linux.4xlarge.memory) / linux-job

test-parakeet-xnnpack-linux / linux-job

test-voxtral-realtime-xnnpack-linux / linux-job

test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job

test-llama-runner-linux (fp32, xnnpack+custom+qe, linux.arm64.2xlarge, executorch-ubuntu-22.04-gc... / linux-job

test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04... / linux-job

test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu... / linux-job

test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job

test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu-22.04-... / linux-job

test-llama-runner-linux (bf16, custom, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job

test-llama-runner-linux-android / linux-job

test-custom-ops-linux / linux-job

test-selective-build-linux / linux-job

test-multimodal-linux

test-moshi-linux / linux-job

test-quantized-aot-lib-linux / linux-job

test-binary-size-linux-gcc / linux-job

test-binary-size-linux / linux-job

test-arm-cortex-m-size-test (bare_metal) / linux-job

test-arm-cortex-m-size-test (zephyr-preset) / linux-job

test-mcu-cortex-m-backend / linux-job

android / build-android

android / run-emulator

unittest / Resolve CI docker image / resolve

unittest / macos / macos-job

unittest / windows / windows-job

unittest / linux / linux-job

unittest-editable / Resolve CI docker image / resolve

unittest-editable / macos / macos-job

unittest-editable / windows / windows-job

unittest-editable / linux / linux-job

unittest-buck / Resolve CI docker image / resolve

unittest-buck / macos / macos-job

unittest-buck / windows

unittest-buck / linux / linux-job

test-qnn-buck-build-linux / linux-job

test-arm-backend-no-driver (test_pytest_ops_no_target) / linux-job

test-arm-backend-no-driver (test_pytest_ops_tosa) / linux-job

test-arm-backend-no-driver (test_pytest_models_tosa) / linux-job

test-arm-backend-no-driver (test_run_tosa) / linux-job

test-arm-backend-public-api-backward-compatibility / linux-job

test-llama-runner-qnn-linux (fp32, qnn_16a16w, qnn) / linux-job

test-llama-runner-qnn-linux (fp32, qnn_8a8w, qnn) / linux-job

test-static-llama-qnn-linux (stories_110m) / linux-job

test-static-llama-qnn-linux (stories_260k_bc) / linux-job

test-sqnr-static-llm-qnn-linux (smollm2_135m) / linux-job

test-qnn-models-linux (mv2) / linux-job

test-qnn-models-linux (mv3) / linux-job

test-qnn-models-linux (dl3) / linux-job

test-qnn-direct-build-linux / linux-job

test-qnn-testsuite-linux / test-backend-linux (qnn, models) / linux-job

test-qnn-testsuite-linux / test-backend-linux (qnn, operators) / linux-job

test-qnn-testsuite-linux / test-backend-macos

test-qnn-testsuite-linux / package-golden-artifacts

test-qnn-passes-linux / linux-job

test-qnn-delegate-linux / linux-job

test-phi-3-mini-runner-linux / linux-job

test-qnn-python-imports-linux / linux-job

test-eval_llama-wikitext-linux / linux-job

test-llama_runner_eager-linux / linux-job

test-lora-linux / linux-job

test-lora-multimethod-linux / linux-job

test-mediatek-models-linux / linux-job

test-openvino-linux / linux-job

test-build-wasm-linux / linux-job

unittest-wasm-bindings / linux-job

unittest-wasm-bindings (--enable-etdump) / linux-job

unittest-nxp-neutron / linux-job

test-samsung-quantmodels-linux

test-samsung-models-linux

test-vulkan-models-linux / linux-job

test-vulkan-operators-linux / linux-job

nxp-build-test / linux-job

test-arduino-library / run / arduino-library

test-minimal-wheel-linux

test-coreml-bc-macos (${{ matrix.runner }})

Test Cortex-M Backend on: pull_request

Get changed files / get-changed-files

test-cortex-m

Build Linux CUDA Wheels on: pull_request 6

generate-matrix / generate

filter-matrix

pytorch/executorch / build-manywheel-py3_10-cuda13_0

pytorch/executorch / upload / upload-manywheel-py3_10-cuda13_0

Test ARM Backend on: pull_request 6

test-arm / test-backend-linux (arm_tosa_fp, models) / linux-job

test-arm / test-backend-linux (arm_tosa_fp, operators) / linux-job

test-arm / test-backend-macos

test-arm / package-golden-artifacts

test-arm-vgf / test-backend-linux (arm_vgf_fp, models) / linux-job

test-arm-vgf / test-backend-linux (arm_vgf_fp, operators) / linux-job

test-arm-vgf / test-backend-macos

test-arm-vgf / package-golden-artifacts

Test WebGPU Backend on: pull_request 3

test-webgpu / test-backend-linux (webgpu, models) / linux-job

test-webgpu / test-backend-linux (webgpu, operators) / linux-job

test-webgpu / test-backend-macos

test-webgpu / package-golden-artifacts

Test QNN Backend on: pull_request 3

test-qnn / test-backend-linux (qnn, models) / linux-job

test-qnn / test-backend-linux (qnn, operators) / linux-job

test-qnn / test-backend-macos

test-qnn / package-golden-artifacts

Test NXP Backend on: pull_request

Get changed files / get-changed-files

test-nxp

Test Vulkan Backend on: pull_request 3

test-vulkan / test-backend-linux (vulkan, models) / linux-job

test-vulkan / test-backend-linux (vulkan, operators) / linux-job

test-vulkan / test-backend-macos

test-vulkan / package-golden-artifacts

Test CoreML Backend on: pull_request

Get changed files / get-changed-files

test-coreml

Meta CLA

Meta CLA Check

Meta CodeSync

Meta Import Checks

Meta Internal-Only Changes Check

Annotations

1 warning

Complete job

Node.js 20 is deprecated. The following actions target Node.js 20 but are being forced to run on Node.js 24: actions/labeler@v5. For more information see: https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/

label

succeeded Sep 2, 2026 in 6s

Loading

Re-running jobs...

Footer

© 2026 GitHub, Inc.

Footer navigation

Terms
Privacy
Security
Status
Community
Docs
Contact

You can’t perform that action at this time.