RIDDL supports the definition of complex data streaming models. There are two
basic definitions involved in setting up a pipeline:
pipe, and a
processor. There are several kinds of pipes and processors,
and they can be combined to form entire systems of data processing known as a
plant. These metaphors come from chemical processing concepts which you can
visualize as something like this:
All you have to do is remember the 3 P’s:
- Pipes - Pipes are conduits for reliably transmitting messages of a particular type of data between processors that are connected to the pipe.
- Processors - While pipes reliably transport data from its producers to its consumers, processors are the producers, consumers, and transformers of data. Notably, entities are processors of pipes too.
- Plants - Plants combine pipes and processors together with support for rich semantics so that arbitrarily complex streams can be modeled in RIDDL.