CONTNM-128: Step by step guide to develop custome features

docs/contributing/create-new-nodes/_category_.json

@@ -0,0 +1,8 @@
+{
+  "label": "Create New Nodes",
+  "position": 2,
+  "link": {
+    "type": "doc",
+    "id": "contributing/create-new-nodes/overview"
+  }
+}

docs/contributing/create-new-nodes/overview.md

@@ -0,0 +1,58 @@
+---
+sidebar_position: 1
+sidebar_label: Overview
+title: Create New Nodes
+slug: /contributing/create-new-nodes
+---
+
+# Create New Nodes
+
+This guide walks you through building a **feature worker** with your own workflow node, from an empty folder to a passing build and a worker that registers its node with the platform. The running example is a **Word Counter** node that counts the words in a text column and appends the count as a new column.
+
+You need to know some Kotlin and Gradle. You don't need to know Temporal, Kafka or Spring internals; the Continuum Gradle plugins and the worker starter handle those.
+
+:::info Versions used in this guide
+Verified against **Continuum platform `0.0.20`** (Spring Boot 4.0.6, Kotlin 2.2, Jackson 3, Gradle 8.14, JDK 21). Check [Maven Central](https://central.sonatype.com/artifact/org.projectcontinuum.core/continuum-commons) for the latest platform version, and use the same version everywhere in your project.
+:::
+
+## How the pieces fit
+
+A feature worker is a small Gradle multi-project:
+
+```
+continuum-feature-demo/
+├── settings.gradle.kts           # plugin versions + module list
+├── gradle.properties             # continuumPlatformVersion, featureVersion
+├── features/
+│   └── continuum-feature-demo/   # library module: your node classes
+└── worker/                       # thin Spring Boot app that hosts the nodes
+```
+
+- **Feature modules** (`org.projectcontinuum.feature` plugin) hold the node classes. They're plain libraries and can be published and reused.
+- **The worker module** (`org.projectcontinuum.worker` plugin) is a Spring Boot app with no node logic of its own. It depends on your feature modules and on `continuum-worker-springboot-starter`. At startup the starter finds every `@ContinuumNode` bean, polls a Temporal task queue for node executions, and publishes one registration message per node to Kafka.
+
+See [Node Model & Feature Registration](../../architecture/node-model.md) for what happens after registration.
+
+## Prerequisites
+
+| Requirement | Notes |
+|-------------|-------|
+| JDK 21 | Eclipse Temurin recommended. The plugins set a Java 21 toolchain. |
+| Gradle wrapper 8.14+ | Older wrappers (for example 8.4) can't load the Spring Boot 4 Gradle plugin. |
+| Docker | Only needed to run the worker against the [local stack](../../getting-started/local-setup.md). |
+
+You **don't** need a GitHub token. All Continuum artifacts and Gradle plugins are published to **Maven Central**.
+
+## The steps
+
+1. [Create the root project](./step-1-root-project.md)
+2. [Create the feature module](./step-2-feature-module.md)
+3. [Write the node](./step-3-write-the-node.md)
+4. [Add node documentation](./step-4-node-documentation.md)
+5. [Write unit tests](./step-5-unit-tests.md)
+6. [Create the worker module](./step-6-worker-module.md)
+7. [Build and test](./step-7-build-and-test.md)
+8. [Run the worker against the local stack](./step-8-run-locally.md)
+9. [Ship it](./step-9-ship-it.md)
+
+If something goes wrong along the way, see [Troubleshooting](./troubleshooting.md).

docs/contributing/create-new-nodes/step-1-root-project.md

@@ -0,0 +1,48 @@
+---
+sidebar_position: 2
+sidebar_label: "1. Root project"
+title: "Step 1: Create the root project"
+---
+
+# Step 1: Create the root project
+
+Create the folder and add a Gradle wrapper. If Gradle is installed, run `gradle wrapper --gradle-version 8.14`. Otherwise, copy `gradlew`, `gradlew.bat` and `gradle/wrapper/` from an existing feature repo such as `continuum-feature-base`.
+
+**`gradle.properties`**
+
+```properties
+repoName=your-org/continuum-feature-demo
+continuumPlatformVersion=0.0.20
+featureVersion=0.0.1
+
+# Spring Boot 4 pulls in a large Kotlin compiler jar; give the Gradle daemon enough heap.
+org.gradle.jvmargs=-Xmx4g -XX:MaxMetaspaceSize=1g
+```
+
+- `continuumPlatformVersion` selects the Gradle plugin version and the `continuum-commons` / `continuum-worker-springboot-starter` versions.
+- `featureVersion` is your own version.
+- `repoName` is used in published POM metadata and in the default container image name.
+
+**`settings.gradle.kts`**
+
+```kotlin
+pluginManagement {
+    val continuumPlatformVersion: String by settings
+    repositories {
+        mavenLocal()
+        mavenCentral()
+        gradlePluginPortal()
+    }
+    plugins {
+        id("org.projectcontinuum.feature") version continuumPlatformVersion
+        id("org.projectcontinuum.worker") version continuumPlatformVersion
+    }
+}
+
+rootProject.name = "continuum-feature-demo"
+
+include(":features:continuum-feature-demo")
+include(":worker")
+```
+
+The plugins are resolved from Maven Central, so `mavenCentral()` must be in `pluginManagement.repositories`, not only in `gradlePluginPortal()`.

docs/contributing/create-new-nodes/step-2-feature-module.md

@@ -0,0 +1,61 @@
+---
+sidebar_position: 3
+sidebar_label: "2. Feature module"
+title: "Step 2: Create the feature module"
+---
+
+# Step 2: Create the feature module
+
+**`features/continuum-feature-demo/build.gradle.kts`**
+
+```kotlin
+plugins {
+    id("org.projectcontinuum.feature")
+}
+
+group = "org.projectcontinuum.feature.demo"
+description = "Continuum Feature Demo — demo nodes"
+version = property("featureVersion").toString()
+
+continuum {
+    continuumVersion.set(property("continuumPlatformVersion").toString())
+}
+```
+
+:::warning Always set `continuumVersion`
+If you leave out the `continuum { }` block, the plugin falls back to a built-in default version that is older than the plugin itself. Read the value from `gradle.properties` so plugins and libraries always match.
+:::
+
+The `org.projectcontinuum.feature` plugin applies Kotlin and Spring dependency management and adds these dependencies for you, so you don't declare them yourself:
+
+- Spring Boot starter, web, actuator and autoconfigure
+- `continuum-commons`
+- `kotlin-reflect` and `tools.jackson.module:jackson-module-kotlin`
+- Test: `spring-boot-starter-test`, `kotlin-test-junit5`, `mockito-kotlin`
+
+It also registers the `validateContinuumNodeAnnotations` task, which runs before `classes`. Add a `dependencies { }` block only for extra libraries your nodes need.
+
+## Register the module with Spring
+
+The worker finds your nodes through Spring Boot auto-configuration, so each feature module needs two small files.
+
+**`src/main/kotlin/org/projectcontinuum/feature/demo/AutoConfigure.kt`**
+
+```kotlin
+package org.projectcontinuum.feature.demo
+
+import org.springframework.context.annotation.ComponentScan
+import org.springframework.context.annotation.Configuration
+
+@Configuration
+@ComponentScan
+class AutoConfigure
+```
+
+**`src/main/resources/META-INF/spring/org.springframework.boot.autoconfigure.AutoConfiguration.imports`**
+
+```text
+org.projectcontinuum.feature.demo.AutoConfigure
+```
+
+`@ComponentScan` with no arguments scans the package of `AutoConfigure` and its subpackages. If the imports file is missing or names the wrong class, the build still passes but the worker starts with none of your nodes.

docs/contributing/create-new-nodes/step-3-write-the-node.md

@@ -0,0 +1,174 @@
+---
+sidebar_position: 4
+sidebar_label: "3. Write the node"
+title: "Step 3: Write the node"
+---
+
+# Step 3: Write the node
+
+Put node classes in a `.node` subpackage. The platform derives your **feature ID** by removing the trailing `.node` segment from the class name. For example, `org.projectcontinuum.feature.demo.node.WordCounterNodeModel` gets the feature ID `org.projectcontinuum.feature.demo`.
+
+**`src/main/kotlin/org/projectcontinuum/feature/demo/node/WordCounterNodeModel.kt`**
+
+```kotlin
+package org.projectcontinuum.feature.demo.node
+
+import org.projectcontinuum.core.commons.annotation.ContinuumNode
+import org.projectcontinuum.core.commons.exception.NodeRuntimeException
+import org.projectcontinuum.core.commons.model.ContinuumWorkflowModel
+import org.projectcontinuum.core.commons.node.ProcessNodeModel
+import org.projectcontinuum.core.commons.protocol.progress.NodeProgressCallback
+import org.projectcontinuum.core.commons.utils.NodeInputReader
+import org.projectcontinuum.core.commons.utils.NodeOutputWriter
+import org.slf4j.LoggerFactory
+import org.springframework.http.MediaType.TEXT_PLAIN_VALUE
+import tools.jackson.core.type.TypeReference
+import tools.jackson.databind.ObjectMapper
+
+@ContinuumNode
+class WordCounterNodeModel(
+  private val objectMapper: ObjectMapper
+) : ProcessNodeModel() {
+  companion object {
+    private val LOGGER = LoggerFactory.getLogger(WordCounterNodeModel::class.java)
+    private val WHITESPACE = Regex("\\s+")
+  }
+
+  // 1. Ports: the map keys ("data") are the port IDs used in execute()
+  final override val inputPorts = mapOf(
+    "data" to ContinuumWorkflowModel.NodePort(name = "input table", contentType = TEXT_PLAIN_VALUE)
+  )
+  final override val outputPorts = mapOf(
+    "data" to ContinuumWorkflowModel.NodePort(name = "table with word counts", contentType = TEXT_PLAIN_VALUE)
+  )
+
+  // 2. Where the node appears in the workbench node explorer
+  override val categories = listOf("Demo")
+
+  // 3. JSON Schema for the node's properties
+  val propertiesSchema: Map<String, Any> = objectMapper.readValue(
+    """
+    {
+      "type": "object",
+      "properties": {
+        "inputCol":      { "type": "string",  "title": "Input Column" },
+        "outputCol":     { "type": "string",  "title": "Output Column" },
+        "minWordLength": { "type": "integer", "title": "Minimum Word Length", "minimum": 1, "default": 1 }
+      },
+      "required": ["inputCol", "outputCol"]
+    }
+    """.trimIndent(),
+    object : TypeReference<Map<String, Any>>() {}
+  )
+
+  // 4. JSON Forms UI schema: layout of the configuration dialog
+  val propertiesUiSchema: Map<String, Any> = objectMapper.readValue(
+    """
+    {
+      "type": "VerticalLayout",
+      "elements": [
+        { "type": "Control", "scope": "#/properties/inputCol" },
+        { "type": "Control", "scope": "#/properties/outputCol" },
+        { "type": "Control", "scope": "#/properties/minWordLength" }
+      ]
+    }
+    """.trimIndent(),
+    object : TypeReference<Map<String, Any>>() {}
+  )
+
+  // 5. Metadata: what the workbench and registry see
+  override val metadata = ContinuumWorkflowModel.NodeData(
+    id = this.javaClass.name,
+    nodeModel = this.javaClass.name,
+    title = "Word Counter",
+    subTitle = "Count words per row",
+    description = "Counts the words in a text column and appends the count as a new column",
+    icon = """<svg xmlns="http://www.w3.org/2000/svg" fill="none" stroke="currentColor" stroke-width="1.5" viewBox="0 0 24 24"><path stroke-linecap="round" stroke-linejoin="round" d="M4 6h16M4 12h10M4 18h6M18 14v6m-3-3h6"/></svg>""",
+    inputs = inputPorts,
+    outputs = outputPorts,
+    properties = mapOf("inputCol" to "text", "outputCol" to "word_count", "minWordLength" to 1),
+    propertiesSchema = propertiesSchema,
+    propertiesUISchema = propertiesUiSchema
+  )
+
+  // 6. The node's logic
+  override fun execute(
+    properties: Map<String, Any>?,
+    inputs: Map<String, NodeInputReader>,
+    nodeOutputWriter: NodeOutputWriter,
+    nodeProgressCallback: NodeProgressCallback
+  ) {
+    val inputCol = properties?.get("inputCol") as String?
+      ?: throw NodeRuntimeException(workflowId = "", nodeId = "", message = "inputCol is not provided")
+    val outputCol = properties["outputCol"] as String?
+      ?: throw NodeRuntimeException(workflowId = "", nodeId = "", message = "outputCol is not provided")
+    val minWordLength = (properties["minWordLength"] as Number?)?.toInt() ?: 1
+
+    val reader = inputs["data"] ?: throw NodeRuntimeException(
+      isRetriable = false, workflowId = "", nodeId = "", message = "Input port 'data' is not connected"
+    )
+    val totalRows = reader.getRowCount()
+
+    nodeOutputWriter.createOutputPortWriter("data").use { writer ->
+      reader.use {
+        var row = reader.read()
+        var rowNumber = 0L
+        while (row != null) {
+          val text = row[inputCol]?.toString() ?: ""
+          val count = text.trim().split(WHITESPACE).count { it.length >= minWordLength }
+
+          writer.write(rowNumber, row.toMutableMap().apply { this[outputCol] = count })
+          rowNumber++
+          if (totalRows > 0) nodeProgressCallback.report((rowNumber * 100 / totalRows).toInt())
+          row = reader.read()
+        }
+        LOGGER.info("Counted words in $rowNumber rows")
+      }
+    }
+  }
+}
+```
+
+## The node contract
+
+| Rule | Why |
+|------|-----|
+| Annotate with `@ContinuumNode`, not `@Component`. | `@ContinuumNode` makes the bean prototype-scoped, so each execution gets a fresh instance. The build fails if a node class is missing it (see [Step 7](./step-7-build-and-test.md)). |
+| Extend `ProcessNodeModel` for nodes with inputs, or `TriggerNodeModel` for source nodes. | `TriggerNodeModel` has only `outputPorts` and `execute(properties, nodeOutputWriter)`. |
+| Set `metadata.id` and `metadata.nodeModel` to `this.javaClass.name`. | The fully qualified class name is the node's identity in the registry and in saved workflows. Renaming or moving the class breaks existing workflows that use it. |
+| Use Jackson 3 imports (`tools.jackson.*`). | Platform `0.0.20` is on Spring Boot 4 / Jackson 3. `com.fasterxml.jackson.databind` isn't on the classpath and won't compile. |
+| Take `ObjectMapper` as a constructor parameter. | Spring injects it. In unit tests, pass `jacksonObjectMapper()`. |
+
+## Properties and validation
+
+- `metadata.properties` holds the default values used when a node is dropped onto the canvas.
+- Before `execute()` is called, `ProcessNodeModel.run()` **validates the configured properties against `propertiesSchema`**. Values that break `required`, `type`, `minimum` and similar rules fail the node before your code runs. You can still check values in `execute()` for clearer error messages.
+- `propertiesUISchema` is a [JSON Forms](https://jsonforms.io/) UI schema, used only by the workbench to draw the configuration dialog. For credential pickers (`options.format = "credential"`), see [Credentials](../../architecture/node-model.md#credentials). You can try schemas in the [JSON Forms playground](https://github.com/projectcontinuum-org/continuum-jsonform-playground) before wiring them into a node.
+
+## Reading and writing rows
+
+- `inputs[portId]` is a `NodeInputReader`. `read()` returns the next row as `Map<String, Any>`, or `null` at the end of the table. `getRowCount()` returns the total number of rows.
+- `nodeOutputWriter.createOutputPortWriter(portId)` returns a writer. `write(rowNumber, row)` takes a `Long` row index starting at `0`.
+- Wrap both reader and writer in `.use { }`. The output Parquet file is only complete once the writer is closed.
+- If an input port isn't connected, its key is missing from `inputs`.
+
+Cell values must be one of these types. Any other type makes `write()` throw `IllegalArgumentException: Unsupported type`.
+
+| Kotlin type | Stored as |
+|-------------|-----------|
+| `String` | `application/vnd.continuum.x-string` |
+| `Int`, `Long` | `application/vnd.continuum.x-int`, `x-long` |
+| `Float`, `Double` | `application/vnd.continuum.x-float`, `x-double` |
+| `Boolean` | `application/vnd.continuum.x-boolean` |
+| `List<*>`, `Map<*, *>` | `application/json` |
+| SVG `org.w3c.dom.Document` | `image/svg+xml` |
+
+Convert anything else first, for example call `toString()` on a `LocalDate` or `BigDecimal`. A `null` value can't be stored either: leave the key out instead.
+
+## Errors and retries
+
+Every node runs as a Temporal activity, so an exception thrown from `execute()` is **retried** by default. Throw `NodeRuntimeException(isRetriable = false, ...)` for errors that a retry won't fix, such as missing configuration or a disconnected port. You can pass empty strings for `workflowId` and `nodeId`. The existing nodes do the same.
+
+## Progress
+
+Call `nodeProgressCallback.report(percent)` with a value from 0 to 100 as you go. The workbench uses it to show live progress on the node.

docs/contributing/create-new-nodes/step-4-node-documentation.md

@@ -0,0 +1,15 @@
+---
+sidebar_position: 5
+sidebar_label: "4. Node documentation"
+title: "Step 4: Add node documentation"
+---
+
+# Step 4: Add node documentation
+
+Each node has a Markdown help page that the workbench shows next to it. It's loaded from the classpath on startup, at a path that matches the node's package and class name exactly:
+
+```text
+src/main/resources/org/projectcontinuum/feature/demo/node/WordCounterNodeModel.doc.md
+```
+
+If the file is missing, the node still works, but its help page reads *"Documentation not found for WordCounterNodeModel"*. A good help page covers what the node does, its ports, a properties table, and a small input/output example. See the `.doc.md` files in [`continuum-feature-base`](https://github.com/projectcontinuum-org/continuum-feature-base) for the house style.