The elastic.io platform is microservices-based, which means that each integration component is placed inside its very own Docker container.
In this article:
When you want to execute a task consisting of, say, two components with a Data Mapper in between, our platform starts each corresponding container (in our case three in total) one after the other in order to pass data from the first container to the last one, stopping every container after the use and proceeding to the next one.
This technique is sufficient for many use cases where the synchronisation is not time critical. These types of flows are called Ordinary flows. They are sufficiently fast, meaning the whole execution could take anywhere between 1 minute to 3 minutes and are less resource intensive.
So what if faster executions are necessary?
What if the nature of integration requires almost instantaneous synchronisation and execution? Luckily we have a solution for you and it is called Realtime flows.
Realtime Flows on demand run at sub-second speeds. In these flows, containers start and don’t stop, while being "glued" together via our messaging queue. Thanks to this, data passes through all of them basically in no time!
How fast can Realtime flow be?
Our latest latency test shows that in average communication between containers is less than 15 milliseconds! Here is a typical graph representing the latency values. On the left we have milliseconds and bottom we have time in hours (this graph is from 15th of July, 2016):
This would mean when you have 3 containers running in one integration flow then elastic.io platform overhead is just 2 x 15 milliseconds = 30 milliseconds in average. This could result in flow executions anywhere between 100 and 400 milliseconds, depending on the data volume and the outside APIs call response times.
Here is the overview of latency for 15 days in July (left milliseconds and bottom time in days):
How to switch to Realtime flow?
Every integration flow can be switched to a realtime flow. To do so use the setup menu found on the right side of every flow in Dashboard.
Please note if you want one of your active ordinary flows to become a realtime flow you must stop it first then select the same menu, otherwise without stopping it the menu item will be greyed out and not selectable.
Here is all the procedure in one picture: