(Kleppman) Designing Data-Intensive Systems Kleppman defines a stream as “data that is incrementally made available over time.” TCP connections, Unix pipes, and (of course) tools like Apache Kafka are all examples of streaming systems.
Generally, the first step of stream processing is to parse it into discrete records (“messages” or “events”). These messages are generally added to a message queue of one kind or another for downstream consumers.
Streams can be one-to-one (as in TCP or Unix pipes), one-to-many, many-to-one, or many-to-many. One-to-many is either for fan-out (a form of multicast) or load balancing (a form of anycast).
When multiple producers need to publish messages to the same stream, a message broker is typically required. Message brokers can also be useful for any of the other cardinalities in certain situations.