Balancing between Zippy and Consistent APIs: Caching the right way


What is a cache?

A cache is, in my definition, a store of some data. This data could be the actual copy or a facsimile of some data stored somewhere else and held for a duration

A caching solution implementation hold this information in memory (volatile) or offload to a persistent store. When not holding the true copy of the information, invalidation or refreshing becomes an issue

The internet was built on caches and if you do microservices then you should know about how to use one to save your life! For example, Web Portals implementations die an early death during performance testing because the backend response to requests “took a long time”

Example of two microservices for a Portal channel serving straight from the system of record

Know how to implement caches, especially for responses and when to use what type of cache to build an effective solution and a snappy application!

Cache types and patterns

  • Persistent vs Volatile: Does it remain when the power is turned off?
  • Cache-aside vs Operational data: Is it the source of truth or does it periodically refresh this information from another source
Cache Storage types

Invalidating a cache

Simply the hardest problem in Computer Science. Knowing when to clear your copy of the data is key, especially when you do not master the information

Some of the invalidation strategies are

  • Events from the master: Subscribe to refresh events or full updates from master system
  • Periodic refreshes: Use a timer to refresh your cache (especially if there are known update cycles in the master data system)
  • Explicit invalidation: Use an API to clear your cache
Cache implementation patterns

Know when to cache ’em

  1. Availability: It is 2020 and your users want a response now! Drop-downs need to be snappy, lookups in O(1) and fewer network calls across the pond
  2. Reliability: You like your consumers but not enough to let them smash the heck out of your core business systems. Self-service portals/mobile-apps backed by APIs are more vulnerable to scripted attacks and if your service is lazy and always going to the system-of-record then you are risking an outage
  3. De-Coupling: Separation of concerns – Command vs Query. You want to isolate the process that accepts a request to “creating/updating” vs “reading” data to reduce coupling the two contexts. This is to prevent scenarios where a sudden rush of users trying to read the state of a transaction do not block the creation of new transactions. For example, Order booking can continue even if there is a flood of requests for order queries
Applying Caching on Queries

Know when not to cache

  1. Consistency: The application needs to serve the current true state of the information (How much money to I have now vs my transaction history)
  2. Cache Key ComplexitySearches are hard to cache because they generate large and complex set of search keys for the results. For example,  consider implementing a cache for a type-ahead search where each word typed is a callout to a search API. The result set for each word would require a large memory footprint and is notoriously hard to size for. A better approach is to only cache the individual item (resources) returned in the result set or array and not cache the search result array
  3. Ambiguity: This relates to consistency. If you do know know when to refresh your cache, especially if you are not the master system or if this information changes in real-time then look for other solution options. For example, a website has a system that updates a user’s account balance in real-time (betting and gambling?) and the Account API for the user is looking to scale to hundreds or thousands of user requests per second (Melbourne cup day?) – would you cache the user’s account (money) information or look at some other strategy (streaming, HTTP push)

Summary

One of the API anti-patterns is going straight to the system of record data, especially for retrieving data in public facing web applications. The best way to serve information is to understand where on the spectrum of static to dynamic does it sit and then implement a solution to serve the data with the highest degree of consistency

Static non-changing resources are served by CDNs. The next layer is reliant on your APIs and how effectively you implement caching. Hope you got a taste of the types of cache and strategies in this post. There is certainly a lot more to caching than I have talked about here, the internet is a giant knowledge cache – happing searching!

Published by

techiecook

I apply my degree in computing with my experience in building networked software to create products that people will love. More about me here https://techiecook.com/about/

Leave a Comment

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s