ECHO Pub/Sub Framework
Echo is a messaging framework built in Crystal Language, that applies the Pub/Sub pattern with asynchronous messaging service. Echo provides instant event notifications for distributed applications, especially those that are decoupled into smaller, independent building blocks.
Echo uses Topic based messaging, where messages are published to named topics invoked as
Stream(M) type objects. The
Producer(Messate, Stream) is the one who creates these
Consumer subscribe to those topics to receive messages from whereever they appear.
Types of Streams
Echo has 3 types of
Stream built-in these are:
- Redis Streams - Uses Redis Stream data type introduced in Redis 5.0 which models a log data structure in a more abstract way.
- Websockets Streams - Uses WebSockets to provide a long-lived connection to deliver messages from producer to consumers
- In-Memory Streams - Uses Crystal Channels to deliver messages from Producer to Consumers internally within an application
- Stream A named resource to which messages are sent by publishers.
- Producer A named resource representing the stream of events (messages) from a single, specific topic, to be delivered to the subscribing application (consumers).
- Consumer A named resource representing the application/entity subscribed to a stream to receive events.
- Message The combination of data and (optional) attributes that a producer sends to a Stream(Message) and is eventually delivered to consumers.
Producer Consumer Relationship
A producer application creates and sends events to a stream. Consumer applications create a subscription to a event to receive messages from it. Communication can be one-to-many (fan-out), many-to-one (fan-in), and many-to-many.
Common use cases
- Balancing workloads in network clusters. For example, a large queue of tasks can be efficiently distributed among multiple workers.
- Implementing asynchronous workflows. For example, an order processing application can place an order on a stream, from which it can be processed by one or more workers.
- Distributing event notifications. For example, a service that accepts user signups can send notifications whenever a new user registers, and downstream services can subscribe to receive notifications of the event.
- Refreshing distributed caches. For example, an application can publish invalidation events to update the IDs of objects that have changed.
- Logging to multiple systems. For example, an application can write logs to the monitoring system, to a database for later querying, and so on.
- Data streaming from various processes or devices. For example, a residential sensor can stream data to backend servers hosted in the cloud.
- Reliability improvement. For example, a single-zone Compute service can operate in additional zones by subscribing to a common topic, to recover from failures in a zone or region.
Add the dependency to your
dependencies: echo: github: eliasjpr/echo
Echo::Redis is replaceable with
require "echo" struct World include Echo::Message getter name : String = "" def initialize(@name) end end struct Marco include Echo::Message getter name = "Marco" end class WorldProducer include Echo::Producer(World, Echo::Redis) include Echo::Producer(Marco, Echo::Redis) # subscribe and publish methods are now available end class WorldConsumer include Echo::Consumer(World, Echo::Redis) include Echo::Consumer(Marco, Echo::Redis) getter count : Int32 = 0 def on(event : World | Marco) @count += 1 ...do something... end end
- Fork it (https://github.com/eliasjpr/echo/fork)
- Create your feature branch (
git checkout -b my-new-feature)
- Commit your changes (
git commit -am 'Add some feature')
- Push to the branch (
git push origin my-new-feature)
- Create a new Pull Request
- Elias J. Perez - creator and maintainer
- 2 days ago
- May 13, 2020
Tue, 20 Oct 2020 18:03:48 GMT