Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions docs/modules/ROOT/pages/binders.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,11 @@

The following binder implementations are available:

* https://cloud.spring.io/spring-cloud-stream-binder-rabbit/[RabbitMQ]
* https://cloud.spring.io/spring-cloud-stream-binder-kafka/[Apache Kafka]
* xref:rabbit/rabbit_overview.adoc[RabbitMQ]
* xref:kafka/kafka_overview.adoc[Apache Kafka]
* https://github.com/spring-cloud/spring-cloud-stream-binder-aws-kinesis[Amazon Kinesis]
* https://github.com/spring-cloud/spring-cloud-gcp/tree/master/spring-cloud-gcp-pubsub-stream-binder[Google PubSub _(partner maintained)_]
* https://github.com/SolaceProducts/solace-spring-cloud/tree/master/solace-spring-cloud-starters/solace-spring-cloud-stream-starter#spring-cloud-stream-binder-for-solace-pubsub[Solace PubSub+ _(partner maintained)_]
* https://aka.ms/spring/docs#spring-cloud-stream-binder-for-azure-event-hubs[Azure Event Hubs _(partner maintained)_]
* https://aka.ms/spring/docs#spring-cloud-stream-binder-for-azure-service-bus[Azure Service Bus Binder _(partner maintained)_]
* https://learn.microsoft.com/en-us/azure/developer/java/spring-framework/configure-spring-cloud-stream-binder-java-app-azure-event-hub[Azure Event Hubs _(partner maintained)_]
* https://learn.microsoft.com/en-us/azure/developer/java/spring-framework/configure-spring-cloud-stream-binder-java-app-with-service-bus[Azure Service Bus Binder _(partner maintained)_]
* https://github.com/alibaba/spring-cloud-alibaba/wiki/RocketMQ-en[Apache RocketMQ _(partner maintained)_]
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

This section contains the configuration options used by the Apache Kafka binder.

For common configuration options and properties pertaining to the binder, see the https://docs.spring.io/spring-cloud-stream/docs/current/reference/html/spring-cloud-stream.html#binding-properties[binding properties] in core documentation.
For common configuration options and properties pertaining to the binder, see the xref:spring-cloud-stream/binding-properties.adoc[binding properties] in core documentation.

[[kafka-binder-properties]]
== Kafka Binder Properties
Expand Down Expand Up @@ -81,7 +81,7 @@ If the partition count of the target topic is smaller than the expected value, t
+
Default: `false`.
spring.cloud.stream.kafka.binder.transaction.transactionIdPrefix::
Enables transactions in the binder. See `transaction.id` in the Kafka documentation and https://docs.spring.io/spring-kafka/reference/html/#transactions[Transactions] in the `spring-kafka` documentation.
Enables transactions in the binder. See `transaction.id` in the Kafka documentation and https://docs.spring.io/spring-kafka/reference/kafka/transactions.html[Transactions] in the `spring-kafka` documentation.
When transactions are enabled, individual `producer` properties are ignored and all producers use the `spring.cloud.stream.kafka.binder.transaction.producer.*` properties.
+
Default `null` (no transactions)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ public class KafkaBinderHealthImplementation implements KafkaBinderHealth {

public KafkaBinderHealthImplementation(final KafkaAdmin admin) {
// More about configuring Kafka
// https://docs.spring.io/spring-kafka/reference/html/#configuring-topics
// https://docs.spring.io/spring-kafka/reference/kafka/configuring-topics.html
this.client = AdminClient.create(admin.getConfigurationProperties());
}

Expand Down
2 changes: 1 addition & 1 deletion docs/modules/ROOT/pages/kafka/kafka-binder/metrics.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -24,4 +24,4 @@ class NoOpBindingMeters {
}
```

More details on how to suppress meters selectively can be found https://micrometer.io/docs/concepts#_meter_filters[here].
More details on how to suppress meters selectively can be found https://docs.micrometer.io/micrometer/reference/concepts/meter-filters.html[here].
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ The health indicator requires the dependency `spring-boot-starter-actuator`. For

Spring Cloud Stream Kafka Streams Binder provides a health indicator to check the state of the underlying streams threads.
Spring Cloud Stream defines a property `management.health.binders.enabled` to enable the health indicator. See the
https://docs.spring.io/spring-cloud-stream/docs/current/reference/htmlsingle/#_health_indicator[Spring Cloud Stream documentation].
xref:spring-cloud-stream/health-indicator.adoc[Spring Cloud Stream documentation].

The health indicator provides the following details for each stream thread's metadata:

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ public ApplicationRunner runner() {
}
```

See https://docs.spring.io/spring-cloud-stream/docs/current/reference/html/spring-cloud-stream.html#binding_visualization_control[this section] from the reference docs for more details on this mechanism.
See xref:spring-cloud-stream/binding_visualization_control.adoc[this section] from the reference docs for more details on this mechanism.

NOTE: When controlling the bindings by disabling `auto-startup` as described in this section, please note that this is only available for consumer bindings.
In other words, if you use the producer binding, `process3-out-0`, that does not have any effect in terms of disabling the auto starting of the processor, although this producer binding uses the same `StreamsBuilderFactoryBean` as the consumer bindings.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ spring.cloud.stream.bindings.kstreamProcess-out-0.destination=foobar
```

Things become a bit more complex if you have the same application as above, but is dealing with two different Kafka clusters, for e.g. the regular `process` is acting upon both Kafka cluster 1 and cluster 2 (receiving data from cluster-1 and sending to cluster-2) and the Kafka Streams processor is acting upon Kafka cluster 2.
Then you have to use the https://cloud.spring.io/spring-cloud-stream/reference/html/spring-cloud-stream.html#multiple-binders[multi binder] facilities provided by Spring Cloud Stream.
Then you have to use the xref:spring-cloud-stream/multiple-binders.adoc[multi binder] facilities provided by Spring Cloud Stream.

Here is how your configuration may change in that scenario.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Spring Cloud Stream includes a binder implementation designed explicitly for htt
With this native integration, a Spring Cloud Stream "processor" application can directly use the
https://kafka.apache.org/documentation/streams/developer-guide[Apache Kafka Streams] APIs in the core business logic.

Kafka Streams binder implementation builds on the foundations provided by the https://docs.spring.io/spring-kafka/reference/html/#kafka-streams[Spring for Apache Kafka] project.
Kafka Streams binder implementation builds on the foundations provided by the https://docs.spring.io/spring-kafka/reference/appendix/change-history.html#kafka-streams[Spring for Apache Kafka] project.

Kafka Streams binder provides binding capabilities for the three major types in Kafka Streams - `KStream`, `KTable` and `GlobalKTable`.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,6 @@

By default, no local state is cleaned up when the binding is stopped.
This is the same behavior effective from Spring Kafka version 2.7.
See https://docs.spring.io/spring-kafka/reference/html/#streams-config[Spring Kafka documentation] for more details.
See https://docs.spring.io/spring-kafka/reference/streams.html#streams-config[Spring Kafka documentation] for more details.
To modify this behavior simply add a single `CleanupConfig` `@Bean` (configured to clean up on start, stop, or neither) to the application context; the bean will be detected and wired into the factory bean.

10 changes: 5 additions & 5 deletions docs/modules/ROOT/pages/kafka/kafka_overview.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ Also, 0.11.x.x does not support the `autoAddPartitions` property.

This section contains the configuration options used by the Apache Kafka binder.

For common configuration options and properties pertaining to the binder, see the https://cloud.spring.io/spring-cloud-static/spring-cloud-stream/current/reference/html/spring-cloud-stream.html#binding-properties[binding properties] in core documentation.
For common configuration options and properties pertaining to the binder, see the xref:spring-cloud-stream/binding-properties.adoc[binding properties] in core documentation.

[[kafka-binder-properties]]
==== Kafka Binder Properties
Expand Down Expand Up @@ -384,7 +384,7 @@ Keep in mind that, when using DLQ on a consumer binding that is in batch mode, a
IMPORTANT: Retry within the binder is not supported when using batch mode, so `maxAttempts` will be overridden to 1.
You can configure a `DefaultErrorHandler` (using a `ListenerContainerCustomizer`) to achieve similar functionality to retry in the binder.
You can also use a manual `AckMode` and call `Ackowledgment.nack(index, sleep)` to commit the offsets for a partial batch and have the remaining records redelivered.
Refer to the https://docs.spring.io/spring-kafka/docs/current/reference/kafka/receiving-messages/message-listener-container.html#committing-offsets[Spring for Apache Kafka documentation] for more information about these techniques.
Refer to the https://docs.spring.io/spring-kafka/reference/kafka/receiving-messages/message-listener-container.html#committing-offsets[Spring for Apache Kafka documentation] for more information about these techniques.

[[kafka-producer-properties]]
==== Kafka Producer Properties
Expand Down Expand Up @@ -736,7 +736,7 @@ IMPORTANT: If you deploy multiple instances of your application, each instance n
=== Error Channels

Starting with version 1.3, the binder unconditionally sends exceptions to an error channel for each consumer destination and can also be configured to send async producer send failures to an error channel.
See https://cloud.spring.io/spring-cloud-static/spring-cloud-stream/current/reference/html/spring-cloud-stream.html#spring-cloud-stream-overview-error-handling[this section on error handling] for more information.
See xref:spring-cloud-stream/overview-error-handling.adoc[this section on error handling] for more information.

The payload of the `ErrorMessage` for a send failure is a `KafkaSendFailureException` with properties:

Expand Down Expand Up @@ -772,7 +772,7 @@ class NoOpBindingMeters {
}
```

More details on how to suppress meters selectively can be found https://micrometer.io/docs/concepts#_meter_filters[here].
More details on how to suppress meters selectively can be found https://docs.micrometer.io/micrometer/reference/concepts/meter-filters.html[here].

[[kafka-tombstones]]
=== Tombstone Records (null record values)
Expand Down Expand Up @@ -1024,7 +1024,7 @@ public class KafkaBinderHealthImplementation implements KafkaBinderHealth {

public KafkaBinderHealthImplementation(final KafkaAdmin admin) {
// More about configuring Kafka
// https://docs.spring.io/spring-kafka/reference/html/#configuring-topics
// https://docs.spring.io/spring-kafka/reference/kafka/configuring-topics.html
this.client = AdminClient.create(admin.getConfigurationProperties());
}

Expand Down
14 changes: 7 additions & 7 deletions docs/modules/ROOT/pages/kafka/kafka_tips.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -199,18 +199,18 @@ That sounds like a lot of complex things, but in reality, it boils down to these
Let us analyze each of them.
The first one is the `ListenerContainerCustomizer` bean that takes a `DefaultErrorHandler`.
The container is now customized with that particular error handler.
You can learn more about container customization https://docs.spring.io/spring-cloud-stream/docs/current/reference/html/spring-cloud-stream.html#_advanced_consumer_configuration[here].
You can learn more about container customization xref:spring-cloud-stream/binding-properties.adoc#advanced-consumer-configuration[here].

The second bean is the `DefaultErrorHandler` that is configured with a publishing to a `DLT`.
See https://docs.spring.io/spring-kafka/docs/current/reference/html/#seek-to-current[here] for more details on `DefaultErrorHandler`.
See https://docs.spring.io/spring-kafka/reference/kafka/annotation-error-handling.html#default-eh[here] for more details on `DefaultErrorHandler`.

The third bean is the `DeadLetterPublishingRecoverer` that is ultimately responsible for sending to the `DLT`.
By default, the `DLT` topic is named as the ORIGINAL_TOPIC_NAME.DLT.
You can change that though.
See the https://docs.spring.io/spring-kafka/docs/current/reference/html/#dead-letters[docs] for more details.
See the https://docs.spring.io/spring-kafka/reference/kafka/annotation-error-handling.html#dead-letters[docs] for more details.


We also need to configure an https://docs.spring.io/spring-kafka/docs/current/reference/html/#error-handling-deserializer[ErrorHandlingDeserializer] through application config.
We also need to configure an https://docs.spring.io/spring-kafka/reference/kafka/serdes.html#error-handling-deserializer[ErrorHandlingDeserializer] through application config.

The `ErrorHandlingDeserializer` delegates to the actual deserializer.
In case of errors, it sets key/value of the record to be null and includes the raw bytes of the message.
Expand Down Expand Up @@ -260,7 +260,7 @@ Can you explain?
[[basic-offset-management-in-kafka-binder-solution]]
=== Solution

We encourage you read the https://docs.spring.io/spring-cloud-stream-binder-kafka/docs/current/reference/html/spring-cloud-stream-binder-kafka.html#reset-offsets[docs] section on this to get a thorough understanding on it.
We encourage you read the xref:kafka/kafka-binder/reset-offsets.adoc[docs] section on this to get a thorough understanding on it.

Here is it in a gist:

Expand Down Expand Up @@ -401,7 +401,7 @@ How do I do that?

By default, Kafka binder delegates to the default commit settings in Spring for Apache Kafka project.
The default `ackMode` in Spring Kafka is `batch`.
See https://docs.spring.io/spring-kafka/docs/current/reference/html/#committing-offsets[here] for more details on that.
See https://docs.spring.io/spring-kafka/reference/kafka/receiving-messages/message-listener-container.html#committing-offsets[here] for more details on that.

There are situations in which you want to disable this default commit behavior and rely on manual commits.
Following steps allow you to do that.
Expand Down Expand Up @@ -706,7 +706,7 @@ public Consumer<Message<String>> consume() {
}
```

If you provide a https://docs.spring.io/spring-cloud-stream-binder-kafka/docs/3.1.3/reference/html/spring-cloud-stream-binder-kafka.html#_kafka_binder_properties[custom header mapper] in the application, then this won't work.
If you provide a xref:kafka/kafka-binder/config-options.adoc#kafka-binder-properties[custom header mapper] in the application, then this won't work.
Let's say you have an empty `KafkaHeaderMapper` in the application.

```
Expand Down
4 changes: 2 additions & 2 deletions docs/modules/ROOT/pages/observability.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
= Observability
:page-section-summary-toc: 1

Spring provides support for Observability via https://micrometer.io/[Micrometer] which defines an https://micrometer.io/docs/observation[Observation concept that enables both Metrics and Traces] in applications.
Spring provides support for Observability via https://micrometer.io/[Micrometer] which defines an https://docs.micrometer.io/micrometer/reference/observation.html[Observation concept that enables both Metrics and Traces] in applications.

Spring cloud Stream integrates such support at the level of https://spring.io/projects/spring-cloud-function[Spring Cloud Function] by providing amongst several abstractions an `ObservationFunctionAroundWrapper`, which wraps function to handle observations out of the box.

Expand Down Expand Up @@ -75,7 +75,7 @@ public class DemoStreamApplication {
}
----

The above example emulates attaching an https://projectreactor.io/docs/core/release/reference/#_observation[Observation] to a single message processing (i.e., imperative function), since in this case the unit of observation begins with `Mono.just(..)` and the last operation attaches the `ObservationRegistry` to the subscriber.
The above example emulates attaching an https://projectreactor.io/docs/core/release/reference/metrics.html#observation[Observation] to a single message processing (i.e., imperative function), since in this case the unit of observation begins with `Mono.just(..)` and the last operation attaches the `ObservationRegistry` to the subscriber.

If there is an observation already attached to the subscriber, it will be used to create a child Observation for the chain/segment upstream of `tap`, however as we already stated, by default, the framework does not attach any Observation to the stream chains you return.

2 changes: 1 addition & 1 deletion docs/modules/ROOT/pages/pulsar/pulsar_binder.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -201,7 +201,7 @@ TIP: Any configured custom schema mappings will be consulted when the `schema-ty
Each message typically has header information that needs to be carried along as the message traverses between Pulsar and Spring Messaging via Spring Cloud Stream input and output bindings.
To support this traversal, the framework handles the necessary message header conversion.

https://docs.spring.io/spring-pulsar/docs/current-SNAPSHOT/reference/html/#_pulsar_headers[Pulsar Headers]
https://docs.spring.io/spring-pulsar/docs/1.0.3/reference/reference/pulsar.html#pulsar-headers[Pulsar Headers]

[[custom-header-mapper]]
==== Custom Header Mapper
Expand Down
2 changes: 1 addition & 1 deletion docs/modules/ROOT/pages/rabbit/rabbit_overview.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -82,5 +82,5 @@ NOTE: Currently, a `multiplex` consumer (a single consumer listening to multiple

This section contains settings specific to the RabbitMQ Binder and bound channels.

For general binding configuration options and properties, see the https://cloud.spring.io/spring-cloud-static/spring-cloud-stream/current/reference/html/spring-cloud-stream.html#_configuration_options[Spring Cloud Stream core documentation].
For general binding configuration options and properties, see the xref:spring-cloud-stream/configuration-options.adoc[Spring Cloud Stream core documentation].

Original file line number Diff line number Diff line change
Expand Up @@ -13,15 +13,15 @@ A comma-separated list of RabbitMQ management plugin URLs.
Only used when `nodes` contains more than one entry.
Each entry in this list must have a corresponding entry in `spring.rabbitmq.addresses`.
Only needed if you use a RabbitMQ cluster and wish to consume from the node that hosts the queue.
See https://docs.spring.io/spring-amqp/reference/html/_reference.html#queue-affinity[Queue Affinity and the LocalizedQueueConnectionFactory] for more information.
See https://docs.spring.io/spring-amqp/reference/amqp/connections.html#queue-affinity[Queue Affinity and the LocalizedQueueConnectionFactory] for more information.
+
Default: empty.
spring.cloud.stream.rabbit.binder.nodes::
A comma-separated list of RabbitMQ node names.
When more than one entry, used to locate the server address where a queue is located.
Each entry in this list must have a corresponding entry in `spring.rabbitmq.addresses`.
Only needed if you use a RabbitMQ cluster and wish to consume from the node that hosts the queue.
See https://docs.spring.io/spring-amqp/reference/html/_reference.html#queue-affinity[Queue Affinity and the LocalizedQueueConnectionFactory] for more information.
See https://docs.spring.io/spring-amqp/reference/amqp/connections.html#queue-affinity[Queue Affinity and the LocalizedQueueConnectionFactory] for more information.
+
Default: empty.
spring.cloud.stream.rabbit.binder.compressionLevel::
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ Default: `false`.
batchingEnabled::
Whether to enable message batching by producers.
Messages are batched into one message according to the following properties (described in the next three entries in this list): 'batchSize', `batchBufferLimit`, and `batchTimeout`.
See https://docs.spring.io/spring-amqp//reference/html/_reference.html#template-batching[Batching] for more information.
See https://docs.spring.io/spring-amqp/reference/amqp/sending-messages.html#template-batching[Batching] for more information.
Also see xref:rabbit/rabbit_overview/receiving-batch.adoc[Receiving Batched Messages].
+
Default: `false`.
Expand Down
Loading
Loading