As mentioned in our previous post (a year in review and ChronosES), Belvedere Trading is excited to announce the open-source release of ChronosES, our cross-platform implementation of a CQRS-ES event store.
Many may not be familiar with the two core concepts that led to our development of ChronosES: Command Query Responsibility Segregation (CQRS) and Event Sourcing (ES). Both are relatively simple concepts that can have a surprising amount of impact once they're fully realized in a large-scale software architecture.
Event sourcing is a pattern in which all changes to an application's state are stored as a linear series of events. Applications that leverage this concept can reconstruct the application's state at any point in the past by replaying events up to the desired time. This property is extremely useful for historical backtesting of a system, reproduction of bugs found in production, and much more. The drawback of such a system, of course, is that replaying these events is not particularly performant. We need some other way to view application state as well.
Enter CQRS. CQRS is the notion that you can use a different model to update information than the model you use to read information. For ChronosES, this means that even though our application's state is stored as a linear series of events, we don't necessarily need to treat it as such. The event stream is very useful to the developer, but most applications care only about the most recent state of the application; therefore, we can use an eager read derivation in all of our clients that are interested in consuming our application's data.
ChronosES was created at Belvedere Trading due to an internal need. We wanted to explore the possibility of utilizing CQRS-ES within our system architecture, but were unable to find any existing open-source tools that could help us accomplish our goal. Although we explored multiple existing implementations, we found that all of them were either limited to a single programming language or seemed unnecessarily complicated to use from the application perspective.
The purpose of ChronosES is to combine all of these ideas into an infrastructural middleware component that could easily be used from any programming language. Along with event sourcing, we also wanted ChronosES to leverage an asynchronous processing architecture complete with real-time notifications of changes in the application state. We think of ChronosES mainly as a platform for the development of other applications.
In order to simplify implementation and to guarantee consistency and durability, ChronosES was not originally designed to be a distributed system, nor a system capable of "hot path" performance characteristics. For the time being, clients connect to only a single instance of ChronosES in production and high-availability is accomplished via hot/cold failover (this is currently how we run ChronosES in production at Belvedere). Despite this initial decision, ChronosES will likely be updated to provide full HA and sharding in the future due to its extensive integration within our own production systems.
One of the primary goals of ChronosES was to make it simple for applications to span multiple different programming languages. This is very important to us at Belvedere because we run three different languages in production:
- C++: For our latency critical trading applications
- Python: For our infrastructural services (performance is not critical)
- C#: For our front-end applications (GUIs)
From our perspective, ChronosES fits nicely into the "infrastructural services" camp. While performance here is important, it is not critical in the same way that it is for our trading algorithms. We treat ChronosES as both a network resource and a database. If we were to include a ChronosES call in our hot path (regardless of its implementation), we would already be far too slow. This, along with its expressive power and ease of use, led us to choose Python as the ChronosES intermediary language along with Google Protocol Buffers (Protobuf) as the serialization protocol.
Following with Domain Driven Design (DDD), ChronosES functions using two core concepts: the Aggregate and Events. The Aggregate is the primary data model in which an application is interested. Each application can define a single Aggregate (although there can be multiple instances of an Aggregate) and multiple Events that interact with the Aggregate instances.
For example, a banking application may define a BankAccount Aggregate with DepositEvent and WithdrawEvent Events. Each person has a separate bank account, each of which would be its own Aggregate instance. The DepositEvent would increase the balance of a specific BackAccount instance, while the WithdrawEvent would do the opposite. In the next post, we'll expand on this example with some working BankAccount code that you'll be able to use to get started with a "Hello, world" for ChronosES.
Applications using ChronosES provide two pieces of information to the service in order to use its functionality:
- A Protobuf file specifying the data model for:
- The Aggregate that the application will use
- The Events that can be applied to the Aggregate instances
- A Python logic file specifying:
- Metadata about the Aggregate (indexing, lifetime, etc.)
- What effect each Event will have on an Aggregate instance when raised
After sending this information to the service, ChronosES validates the Python logic and Protobuf structure. Assuming the validation was successful, clients are then able to begin raising Events against Aggregate instances. Everything in ChronosES is asynchronous - raising an Event does not imply that the Event will be applied immediately, but rather that the Event will be applied at some point in the future (in the order that it was received by the service).
All connected clients are then able to subscribe to updates for all Aggregate instances (or a subset of the Aggregates instances, if they so prefer). Full Aggregate snapshots are sent to all subscribed clients after each Event is processed. In this way, clients can react to the real-time feed of an application and perform actions whenever necessary.
In addition to the real-time feed, ChronosES also has a query interface that allows applications to request the latest state of Aggregate instances based on different criteria (indices). If your client is only interested in performing actions at specific points in time, you can leverage ChronosES queries rather than subscribing to the real-time feed.
Thanks for sticking with us! In the next post, we’ll provide working BankAccount code so you can get started with a “Hello, world” for ChronosES. Until then, we hope you're as excited to learn about ChronosES as we were to create it!