Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .github/workflows/infra-generate-documentation.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,8 @@ jobs:
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Install dependencies
run: |
pip install .
Expand Down
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ This file contains the changelog for the Deeploy project. The changelog is divid
- Add SoCDAML Part III: hands-on lab for adding a new int8 operator [#194](https://github.com/pulp-platform/Deeploy/pull/194)
- Match GAP9 SDK MLPerf Tiny Performance with Specialised Depthwise and Pointwise Kernels [#206](https://github.com/pulp-platform/Deeploy/pull/206)
- Add support for the X-HEEP platform [#200](https://github.com/pulp-platform/Deeploy/pull/200)
- PULP-Open iDMA Support and Tests, and LLVM Update to Clang 20 [#216](https://github.com/pulp-platform/Deeploy/pull/216)

### Added
- tests for Regular and DW Conv2D with 3x3 kernel
Expand All @@ -48,6 +49,10 @@ This file contains the changelog for the Deeploy project. The changelog is divid
- Specialised PULPOpen kernels for 3x3 depthwise (`PULPDWConv3x3.c`), 1x1 pointwise (`PULPPWConv1x1.c`) and a three-channel 3x3 stem (`PULPStemConv3x3.c`), with bindings and tile constraints for PULPOpen and GAP9
- Lowering passes `PULPNCHWtoNHWCPwConvPass` and `PULPNCHWtoNHWCConvPass`, keeping a 1x1 convolution's output and a depthwise-feeding stem channels-first so the transposes at depthwise boundaries cancel
- X-HEEP platform support, including compilation for FPGA targets and simulation with Verilator.
- iDMA support for PULP-Open (`--dma {mchan,idma}`, default `idma`), with a CMake `PULP_DMA` check
- PULP-Open test suite (`pulpopen` and `pulpopen_tiled` markers), including FP32 tests now that the toolchain supports `zfinx`
- `cmake/clang_resource_dir.cmake` to locate compiler-rt without a hardcoded clang version
- picolibc build for `rv32imc_zfinx`

### Changed
- Refactor the topology optimization pass `NeurekaReshapePointwiseConvolutionPass` and Neureka's Tile constraints
Expand All @@ -67,6 +72,12 @@ This file contains the changelog for the Deeploy project. The changelog is divid
- Aligned CLI commands across the project
- Added @runwangdl as a code owner
- Skip emitting duplicate `testInputVector` data for inputs placed in L3 (loaded at runtime from the readfs hex instead), reducing test binary size
- Move the PULP LLVM toolchain to clang 20.1.8 (`albsposito/llvm-project`, `port/20`), dropping the local crash-fix patch
- Update GVSoC (with the `pulp-open` target) and the PULP SDK
- LLVM toolchains link with `-nostartfiles -nodefaultlibs` and explicit `-lc -lclang_rt.builtins-riscv32`, as clang 20 adds `crt0.o` and links compiler-rt by absolute path
- MemPool toolchain picks `-misched-topdown` or `-misched-prera-direction=topdown` depending on the LLVM version, and disables GlobalMerge to avoid Banshee corrupting unaligned ELF segment tails
- PULP-Open compiles for `rv32imc_zfinx_xpulpv2` (hardware FP32 instead of soft-float)
- Build LLVM with `-DLLVM_DISABLE_ASSEMBLY_FILES=ON` so ccache does not break the BLAKE3 assembly sources

### Fixed
- Fix Neureka's output-channels subtile size (in ConvTemplate) and Dense/DW/PW tile constraints
Expand All @@ -86,6 +97,11 @@ This file contains the changelog for the Deeploy project. The changelog is divid
- Fix invalid escape sequence python error in DeeployTypes.py: appearing when using pytest to launch regressions
- Fix GAP9 board tests with `--defaultMemLevel L3` reading garbage inputs: place all gapy `--flash-property` options before the positional subcommand and use `image flash run` so the readfs partition (input hex files) is flashed to the device
- Fix Deeploy 101 tutorial errors: `--profileTiling` usage and the moved intrinsics inventory path
- Snitch `CycleCounter` no longer places its state in a `.l1` section, which lld put on top of the L3 heap
- `ReshapeTemplate` alias assignment now runs on all cores instead of only under `SINGLE_CORE`, fixing undefined behaviour on the other cores
- Relax GVSoC's `markdown-it-py>=4` pin to `>=3.0` in the container so it can be installed alongside `myst-parser` on Python 3.10
- `MchanDma` splits transfers longer than the 17-bit mchan length into several commands instead of asserting
- Pin Python 3.11 in the documentation workflow, as the unpinned `setup-python` picked the runner's Python 3.14, which `numpy<2.0.0` does not support

### Removed
- removed experimental `enable3x3` flag, from Neureka Engine. Now, 3x3 mode is enabled by default.
Expand Down
9 changes: 8 additions & 1 deletion Container/Dockerfile.deeploy
Original file line number Diff line number Diff line change
Expand Up @@ -28,12 +28,16 @@ COPY Makefile ./

# Compile emulators
# WIESEP: We need to already clean up some space, otherwise the GitHub runners run out of disk space
# GVSoC pins markdown-it-py>=4, but no myst-parser for Python 3.10 (docs) supports it; 3.x works for GVSoC
RUN --mount=type=cache,target=/ccache \
ccache -z && \
make pulp-sdk chimera-sdk qemu mempool banshee xtensor && \
make gvsoc && \
cp -r toolchain/gvsoc/core/requirements.txt /app/core-requirements.txt && \
cp -r toolchain/gvsoc/gapy/requirements.txt /app/gapy-requirements.txt && \
cp -r toolchain/gvsoc/gvrun/requirements.txt /app/gvrun-requirements.txt && \
cp -r toolchain/gvsoc/config_tree/requirements.txt /app/config-tree-requirements.txt && \
sed -i 's/^markdown-it-py.*/markdown-it-py>=3.0/' /app/config-tree-requirements.txt && \
rm -rf toolchain/pulp-sdk toolchain/qemu toolchain/mempool toolchain/banshee toolchain/xtensor toolchain/xtl toolchain/xsimd toolchain/gvsoc && \
ccache -s

Expand Down Expand Up @@ -133,8 +137,11 @@ RUN sh -c "$(curl -fsSL https://raw.githubusercontent.com/ohmyzsh/ohmyzsh/master

COPY --from=toolchain /app/core-requirements.txt ./core-requirements.txt
COPY --from=toolchain /app/gapy-requirements.txt ./gapy-requirements.txt
COPY --from=toolchain /app/gvrun-requirements.txt ./gvrun-requirements.txt
COPY --from=toolchain /app/config-tree-requirements.txt ./config-tree-requirements.txt
COPY requirements-dev.txt ./
RUN pip install -r requirements-dev.txt -r core-requirements.txt -r gapy-requirements.txt
RUN pip install -r requirements-dev.txt -r core-requirements.txt -r gapy-requirements.txt \
-r gvrun-requirements.txt -r config-tree-requirements.txt

# Copy pre-built toolchains and emulators
COPY --from=toolchain /app/install /app/install
Expand Down
4 changes: 3 additions & 1 deletion Deeploy/Targets/Generic/Templates/ReshapeTemplate.py
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,9 @@ def alignToContext(self, ctxt: NetworkContext,
return ctxt, operatorRepresentation, []


# Executed by every core: ${data_out} can be a local of the per-core RunNetwork,
# so a SINGLE_CORE assignment would leave it uninitialized on the other cores.
referenceTemplate = _ReshapeTemplate("""
// Reshape (Name: ${nodeName}, Op: ${nodeOp})
SINGLE_CORE ${data_out} = ${data_in};
${data_out} = ${data_in};
""")
Original file line number Diff line number Diff line change
Expand Up @@ -31,10 +31,16 @@ class ProfilingPULPClusterTilingGenerationDB(DoubleBufferingTilingCodeGeneration
class PULPClusterTiling(CodeTransformationPass):

def __init__(self, externalMemory: str, localMemory: str, dma: AsyncDma):
self.SB = PULPClusterTilingGenerationSB(externalMemory, localMemory, dma)
self.profilingSB = ProfilingPULPClusterTilingGenerationSB(externalMemory, localMemory, dma)
self.DB = PULPClusterTilingGenerationDB(externalMemory, localMemory, dma)
self.profilingDB = ProfilingPULPClusterTilingGenerationDB(externalMemory, localMemory, dma)
self.externalMemory = externalMemory
self.localMemory = localMemory
self.setDma(dma)

def setDma(self, dma: AsyncDma) -> None:
self.dma = dma
self.SB = PULPClusterTilingGenerationSB(self.externalMemory, self.localMemory, dma)
self.profilingSB = ProfilingPULPClusterTilingGenerationSB(self.externalMemory, self.localMemory, dma)
self.DB = PULPClusterTilingGenerationDB(self.externalMemory, self.localMemory, dma)
self.profilingDB = ProfilingPULPClusterTilingGenerationDB(self.externalMemory, self.localMemory, dma)

def apply(self,
ctxt: NetworkContext,
Expand Down
2 changes: 1 addition & 1 deletion Deeploy/Targets/PULPOpen/DMA/L3Dma.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ class L3Dma(AsyncDma):
_transferTemplates = {
2:
NodeTemplate(
"pi_cl_ram_copy_2d(get_ram_ptr(), ${ext}, ${loc}, ${transfer_size}, ${stride}, ${length}, ${ext2loc}, &${future});"
"pi_cl_ram_copy_2d(get_ram_ptr(), (uint32_t)${ext}, ${loc}, ${transfer_size}, ${stride}, ${length}, ${ext2loc}, &${future});"
)
}
_waitingStrategy = PerTensorWaitingStrategy(L3DmaFuture)
Expand Down
24 changes: 17 additions & 7 deletions Deeploy/Targets/PULPOpen/DMA/MchanDma.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,9 +27,18 @@ class MchanChannelFuture(Future):

class MchanDma(AsyncDma):

includeList = ["mchan_siracusa.h"]

# The *_split helpers (mchan_v7.h) issue one command per 2^17 - 1 bytes on
# the same channel, so transfers longer than the 17-bit mchan length work
# too. The size can differ per tile, so the split is decided at runtime: a
# transfer that fits is still a single command.
_transferTemplates = {
1: NodeTemplate("mchan_transfer_1d(${cmd}, ${loc}, ${ext});"),
2: NodeTemplate("mchan_transfer_2d_ext_strided(${cmd}, ${loc}, ${ext}, ${size_1d}, ${stride_2d});"),
1:
NodeTemplate("mchan_transfer_1d_split(${flags}, ${size}, ${loc}, ${ext});"),
2:
NodeTemplate(
"mchan_transfer_2d_ext_strided_split(${flags}, ${size}, ${loc}, ${ext}, ${size_1d}, ${stride_2d});"),
}
_waitingStrategy = DirectionWaitingStrategy(MchanChannelFuture, "channel")

Expand Down Expand Up @@ -65,12 +74,13 @@ def transferOpRepr(self, externalBuffer: VariableBuffer, localBuffer: VariableBu
mchanFlags += (1 << 3) # event enable

mchanTransferSize = math.prod(shape)
mchanTransferSizeBits = math.ceil(math.log2(mchanTransferSize))
assert mchanTransferSizeBits <= 17, (
"The transfer size is not representable with 17 bits. "
f"Received transfer size {mchanTransferSize} that requires {mchanTransferSizeBits}")
mchanMaxLen = (1 << 17) - 1
if transferRank == 2:
assert shape[1] <= mchanMaxLen, (
f"A 2D mchan row of {shape[1]} bytes does not fit in one command (max {mchanMaxLen})")

operatorRepresentation["cmd"] = (mchanFlags << 17) + mchanTransferSize
operatorRepresentation["flags"] = mchanFlags
operatorRepresentation["size"] = mchanTransferSize

if transferRank == 2:
operatorRepresentation["size_1d"] = shape[1]
Expand Down
99 changes: 99 additions & 0 deletions Deeploy/Targets/PULPOpen/DMA/iDMA.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
# Copyright (c) 2025 FondazioneChipsIT
# SPDX-License-Identifier: Apache-2.0

from typing import Dict, Optional, Tuple

from Deeploy.DeeployTypes import NetworkContext, NodeTemplate, OperatorRepresentation, VariableBuffer
from Deeploy.TilingExtension.AsyncDma import AsyncDma, DirectionWaitingStrategy, DmaDirection, Future

# HAL pulp_cl_idma_* calls take the (row) length as unsigned short;
# 1D goes through deeploy_idma_transfer_1d (pulp_idma_utils.h), split at runtime
_MAX_LEN = 2**16 - 1

# Direction encoding expected by the deeploy_idma_transfer_* helpers in
# pulp_idma_utils.h (`direction == 1` means L2 -> L1): keep the two in sync
_DIR_EXT_TO_LOC = 1
_DIR_LOC_TO_EXT = 0


class iDMAChannelFuture(Future):

_initTemplate = NodeTemplate("")
_deinitTemplate = NodeTemplate("")
_waitTemplate = NodeTemplate("deeploy_idma_wait_${name}();")
_allocTemplate = NodeTemplate("")


class iDMA(AsyncDma):

includeList = ["pulp_core.h", "pulp_idma_utils.h"]

_transferTemplates = {
1:
NodeTemplate("deeploy_idma_transfer_1d(${direction}, "
"(unsigned int)${ext}, (unsigned int)${loc}, ${length});"),
2:
NodeTemplate("deeploy_idma_transfer_2d(${direction}, (unsigned int)${ext}, "
"(unsigned int)${loc}, ${length}, ${strideExt}, ${strideLoc}, "
"${num_reps});"),
3:
NodeTemplate("deeploy_idma_transfer_3d(${direction}, (unsigned int)${ext}, "
"(unsigned int)${loc}, ${length}, ${strideExt}, ${strideLoc}, "
"${num_reps}, ${strideExt_3d}, ${strideLoc_3d}, ${num_reps_3d});")
}
_waitingStrategy = DirectionWaitingStrategy(iDMAChannelFuture, "channel_id")

def __init__(
self,
transferTemplates: Optional[Dict[int, NodeTemplate]] = None,
) -> None:
if transferTemplates is None:
transferTemplates = dict(type(self)._transferTemplates)
super().__init__(transferTemplates)

def checkTransfer(
self,
ctxt: NetworkContext,
externalBuffer: VariableBuffer,
localBuffer: VariableBuffer,
shape: Tuple[int, ...],
strideExt: Tuple[int, ...],
strideLoc: Tuple[int, ...],
direction: DmaDirection,
) -> None:

super().checkTransfer(ctxt, externalBuffer, localBuffer, shape, strideExt, strideLoc, direction)
# 1D is split at runtime; in 2D/3D the row length is 16-bit (strides and reps are 32-bit)
if len(shape) >= 2:
length = shape[-1]
assert length <= _MAX_LEN, (f"iDMA row length {length} exceeds the HAL's 16-bit limit ({_MAX_LEN})")
assert strideExt[-2] >= length, "External stride must be at least equal to the length"
assert strideLoc[-2] >= length, "Local stride must be at least equal to the length"

def transferOpRepr(
self,
externalBuffer: VariableBuffer,
localBuffer: VariableBuffer,
shape: Tuple[int, ...],
strideExt: Tuple[int, ...],
strideLoc: Tuple[int, ...],
direction: DmaDirection,
future: Future,
) -> OperatorRepresentation:

opRepr = super().transferOpRepr(externalBuffer, localBuffer, shape, strideExt, strideLoc, direction, future)

opRepr["direction"] = _DIR_EXT_TO_LOC if direction == "ExternalToLocal" else _DIR_LOC_TO_EXT
opRepr["length"] = shape[-1]
# 2nd dimension (rows): stride between rows and number of rows
if len(shape) >= 2:
opRepr["strideExt"] = strideExt[-2]
opRepr["strideLoc"] = strideLoc[-2]
opRepr["num_reps"] = shape[-2]
# 3rd dimension (planes): stride between planes and number of planes
if len(shape) == 3:
opRepr["strideExt_3d"] = strideExt[-3]
opRepr["strideLoc_3d"] = strideLoc[-3]
opRepr["num_reps_3d"] = shape[-3]

return opRepr
61 changes: 57 additions & 4 deletions Deeploy/Targets/PULPOpen/Platform.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,17 @@
#
# SPDX-License-Identifier: Apache-2.0

import copy
from enum import Enum
from typing import Type, Union

import numpy as np
import onnx_graphsurgeon as gs

from Deeploy.CommonExtensions.OptimizationPasses.TopologyOptimizationPasses.LoweringOptimizationPasses import \
RemoveEmptyConvBiasPass, RemoveOnlySingletonReduceMeanPass
from Deeploy.DeeployTypes import ConstantBuffer, DeploymentEngine, DeploymentPlatform, NetworkContext, NodeMapper, \
NodeTemplate, StructBuffer, TopologyOptimizer, TransientBuffer, VariableBuffer
NodeTemplate, ONNXLayer, StructBuffer, TopologyOptimizer, TransientBuffer, VariableBuffer
from Deeploy.MemoryLevelExtension.MemoryLevels import MemoryHierarchy, MemoryLevel
from Deeploy.MemoryLevelExtension.NetworkDeployers.MemoryLevelDeployer import MemoryPlatform, MemoryPlatformWrapper
from Deeploy.Targets.Generic.Bindings import BasicGEMMBindings, BasicPad1DBindings, BasicPad2DBindings, \
Expand All @@ -31,6 +35,9 @@
SkipEmptyConcatPass, SkipUnityRequantPass, iGELURequantMergePass, iHardswishRequantMergePass
from Deeploy.Targets.PULPOpen.Bindings import BasicDequantBindings, BasicQuantBindings, PULPDMASliceBindings, \
PULPDWConv1DBinding
from Deeploy.Targets.PULPOpen.CodeTransformationPasses.PULPClusterTiling import PULPClusterTiling
from Deeploy.Targets.PULPOpen.DMA.iDMA import iDMA
from Deeploy.Targets.PULPOpen.DMA.MchanDma import MchanDma
from Deeploy.Targets.PULPOpen.Layers import PULPRQSConvLayer, PULPRQSGEMMLayer
from Deeploy.Targets.PULPOpen.Parsers import PULPConv1DParser, PULPConv2DParser, PULPDWConv1DParser, \
PULPDWConv2DParser, PULPFPConv2DParser, PULPFPDWConv2DParser, PULPGEMMParser, PULPMatrixVecParser, \
Expand All @@ -52,6 +59,7 @@
PULPTransposeTilingReadyBindings, PULPUniformRQSTilingReadyBindings
from Deeploy.Targets.PULPOpen.TopologyOptimizationPasses.Passes import PULPAddRequantMergePass, \
PULPConvRequantMergePass, PULPGEMMRequantMergePass, PULPMatMulRequantMergePass
from Deeploy.TilingExtension.AsyncDma import AsyncDma

RQAddMapper = NodeMapper(RQAddParser(), PULPRQAddTilingReadyBindings)
AddMapper = NodeMapper(AddParser(), PULPAddTilingReadyBindings)
Expand Down Expand Up @@ -252,22 +260,67 @@ class PULPStructBuffer(StructBuffer):
name = "PULPOptimizer")

# SCHEREMO: stdint is included before pulp_nn_kernels.h because it is supposed to be included in there, but isn't...
# DMA-specific headers are appended by PULPClusterEngine.setDma
_includeList = [
"pmsis.h", "stdint.h", "pulp_nn_kernels.h", "DeeployPULPMath.h", "mchan_siracusa.h", "dory_mem.h", "bsp/ram.h",
"perf_utils.h"
"pmsis.h", "stdint.h", "pulp_nn_kernels.h", "DeeployPULPMath.h", "dory_mem.h", "bsp/ram.h", "perf_utils.h"
]


class PULPDma(str, Enum):
"""Cluster DMA engine used for L2<->L1 tiling transfers"""

MCHAN = "mchan"
IDMA = "idma"

# Keep f-strings and CLI args identical across Python versions
def __str__(self) -> str:
return self.value

@property
def dmaClass(self) -> Type[AsyncDma]:
return {PULPDma.MCHAN: MchanDma, PULPDma.IDMA: iDMA}[self]


# Return a copy of `mapping` whose L2<->L1 cluster tiling uses `dma`.
# The DMA is baked into the module-level code transformers (Bindings.py) shared
# by all bindings, so it is fixed at import time. Hence PULPMapping is built
# with MchanDma, and an engine targeting another DMA deep-copies it and swaps
# the DMA in every PULPClusterTiling pass. The copy keeps the shared PULPMapping
# (also used by Siracusa and Neureka) untouched.
def _remapClusterDma(mapping, dma: AsyncDma):
mapping = copy.deepcopy(mapping)
for layer in mapping.values():
if not isinstance(layer, ONNXLayer):
continue
for mapper in layer.maps:
for binding in mapper.bindings:
for _pass in binding.codeTransformer.passes:
if isinstance(_pass, PULPClusterTiling):
_pass.setDma(dma)
return mapping


class PULPClusterEngine(DeploymentEngine):

def __init__(self,
name: str,
Mapping = PULPMapping,
initCode = "",
includeList = _includeList,
n_cores: int = 8) -> None:
n_cores: int = 8,
dma: str = "mchan") -> None:
super().__init__(name, Mapping, initCode, includeList)
self.n_cores = n_cores
self._baseIncludeList = list(includeList)
self.dma = PULPDma.MCHAN # PULPMapping is built with MchanDma
self.setDma(dma)

def setDma(self, dma: Union[PULPDma, str]) -> None:
dma = PULPDma(dma)
if dma != self.dma:
self.Mapping = _remapClusterDma(self.Mapping, dma.dmaClass())
self.dma = dma
self.includeList = self._baseIncludeList + dma.dmaClass.includeList


class PULPPlatform(DeploymentPlatform):
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,5 +20,11 @@ def apply(self,
name: str,
verbose: CodeGenVerbosity = _NoVerbosity) -> Tuple[NetworkContext, ExecutionBlock]:
executionBlock.addLeft(NodeTemplate(f"if (snrt_is_{self.coreType}_core()) {{\n"), {})
executionBlock.addRight(NodeTemplate("}\n"), {})
if self.coreType == "compute":
# The FPU is decoupled from the integer core: wait until all issued
# FP instructions (e.g. a trailing fdiv + store) have completed, so
# the following barrier / DMA transfer sees the final results.
executionBlock.addRight(NodeTemplate("snrt_fpu_fence();\n}\n"), {})
else:
executionBlock.addRight(NodeTemplate("}\n"), {})
return ctxt, executionBlock
Loading
Loading