Data services and APIs are usually described by developers as slow, unreliable, hard to use, and overly complex. If the best design is the simplest one, then how do you expect a front-end developer to provide the best possible experience with such a complex service layer? The answer is to provide your developers with sensible data services that actually contribute to the overall end-user experience. This blog series will focus on topics central to building fast, extensible, and reliable data services that will elevate your customers’ experience to a whole new level. In this first post of the series, I will talk about event-based systems, what they are, why you should care, and how to implement them.
Things move at the speed of life. That phrase sounds cool, but what does it actually mean? In the technology world it means that when a user performs an action, they are expecting certain results within a specific time. This time-based expectation is especially critical for a number of areas, such as e-commerce and loyalty programs. After a user makes a purchase on your app, their loyalty account should be updated almost immediately. Having a system that allows you to reward your customer at the exact right time will have a direct impact on your business.
Event-based systems are conceptually very simple; when something external to the system happens, a data pipeline is triggered which can have one or more outputs. For example, when an editor marks content as ready to publish, the content should then be transformed into the proper format and published to the live website. A more complex example is dynamic pricing on hotel rooms. As demand changes (represented as a combination of purchases and inquiries), a service should be analyzing and updating the live room prices. Both these examples were traditionally done in a batch process once or twice a day, usually at off-peak times (classically around 12:01 AM). Now with an event-based system, the user gets content when it is relevant, and the hotel has a tremendous opportunity to optimize revenue.
Why You Should Use Them
Cloud computing has completely untethered UX by allowing services to be flexible enough to process the right information at the right time. Previously it did not make sense to process data on-demand because that meant purchasing additional servers to handle peak loads. Instead, the smart decision was to do batch processing at night when loads were light and the servers were under-utilized. With cloud computing, you only pay for what you use, so for a given set of information it will cost you the same to process it live, or process it in batches later - the difference is the experience.
How to Get Started
The foundation of an event-based system is automation. Automation provides myriad benefits beyond these real-time systems, such as significantly reduced development and operating costs, but that is a topic for a later post. Services can only benefit from the pay-per-use model if they scale up and down automatically and do not rely on people to manage them.
The first step in moving to an automated, event-based system is clearly identifying the best way for information to move through your enterprise. This step should feel natural to the business. You are simply recognizing the sequence of events that should occur for any given input. For example, when a user creates an account, what should happen next? After a technical pipeline is established, the implementation of each piece may commence. Adapters can be developed that sit on top of legacy services so that a complete overhaul is avoided.
Finally, each component should be placed in a managed service or auto-scaling virtual server environment. Amazon Web Services is a great cloud provider that offers common data pipeline components as a service; they even have a Data Pipeline service that manages the entire orchestration. However, it is possible to implement this architecture in any environment that supports virtual servers and auto-scaling.
After completing this process, you will have a much more clearly defined business process and a completely flexible and agile system. You will be able to delight your users (and developers) with features they did not expect and deliver on features they were willing to overlook.