Skip to content

Add manual tip-position chunking for NumPy and PyTorch dI/dV backends - #84

Open
GianmarcoMidena wants to merge 8 commits into
masterfrom
70-manually-scale-numpypytorch-didv-backends-for-high-stress-workloads
Open

Add manual tip-position chunking for NumPy and PyTorch dI/dV backends#84
GianmarcoMidena wants to merge 8 commits into
masterfrom
70-manually-scale-numpypytorch-didv-backends-for-high-stress-workloads

Conversation

@GianmarcoMidena

Copy link
Copy Markdown
Contributor

Description

Adds manual tip-position chunking to the vectorized dI/dV implementations so that high-stress NumPy and PyTorch workloads can trade runtime for lower peak memory use.

Changes

  • Adds n_tip_position_chunks support through:
    • ProbeSTM.dIdV
    • ProbeSTM.STM
    • ProbeSTM.MSTM
    • STMutils.run_stm_scan
    • NumPy and PyTorch vectorized dI/dV backends
  • Refactors vectorized dI/dV evaluation to flatten tip positions, process them in configurable chunks, then reshape back to the scan output shape.
  • Adds backend-specific vertical stacking helpers for NumPy and PyTorch.
  • Expands dI/dV tests to cover default, one-chunk, and two-chunk NumPy/PyTorch paths.
  • Adds advanced.n_tip_position_chunks TOML support with a Si 7x7 NumPy chunking example.
  • Adds runtime logging setup in ppstm_run.py and debug logging around chunk progress.
  • Removes the unused global pytest backend fixture from tests/conftest.py.

Closes #70

@GianmarcoMidena
GianmarcoMidena force-pushed the 70-manually-scale-numpypytorch-didv-backends-for-high-stress-workloads branch from 5068e61 to 391de75 Compare August 26, 2026 13:08
Comment thread examples/Si_7x7/fixed_numpy_chunking.toml
Comment thread pyPPSTM/probe_stm_numpy.py
Comment thread pyPPSTM/probe_stm_vectorized.py
Comment thread tests/didv/test_didv_r_a_sign.py Outdated
Comment thread tests/conftest.py Outdated
@GianmarcoMidena
GianmarcoMidena force-pushed the 70-manually-scale-numpypytorch-didv-backends-for-high-stress-workloads branch from 391de75 to 357070b Compare August 26, 2026 14:35
Comment thread pyPPSTM/ProbeSTM.py
Comment thread pyPPSTM/ProbeSTM.py

@ondrejkrejci ondrejkrejci left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 change, but couple of potential changes on top + few comments

@GianmarcoMidena
GianmarcoMidena force-pushed the 70-manually-scale-numpypytorch-didv-backends-for-high-stress-workloads branch 2 times, most recently from 2626f17 to ceba2ed Compare August 27, 2026 14:05

@ondrejkrejci ondrejkrejci left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While there are few comments, the code itself looks good.

@GianmarcoMidena
GianmarcoMidena force-pushed the 70-manually-scale-numpypytorch-didv-backends-for-high-stress-workloads branch from 2bc66cb to 3770521 Compare August 28, 2026 08:33
Base automatically changed from 78-r_a-sign-affects-scans to master August 28, 2026 13:18
@GianmarcoMidena
GianmarcoMidena force-pushed the 70-manually-scale-numpypytorch-didv-backends-for-high-stress-workloads branch from 3770521 to 1b53a48 Compare August 28, 2026 13:18
@ondrejkrejci

Copy link
Copy Markdown
Contributor

All resolved now and good to go!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Allow manual scaling for NumPy/PyTorch dI/dV backends under high-stress workloads

2 participants