Definition
Buffering is used to temporarily store data while it is being moved from one place to another.
How it can help
Cloud migration today has become the de facto ultimate solution for hosting services. I totally agree that moving to the cloud is truly amazing and has tremendous power with a positive impact on the way we work, since we care less about performance and much more about business logic. However, either the cloud hardware is limited or it's very costly.
Regardless of which architecture you're working on — a monolithic, microservice, or serverless architecture — sooner or later you'll face how overwhelming a database, web server, or third-party service can be, with your service struggling to process requests.
This is where the buffer becomes handy.
By implementing the following, you can prevent or minimize the impact of overwhelming a service:
- Throttling
- Request aggregation
- Rate limiting
In this post I'll discuss the second bullet: request aggregation.
Since I believe I have a powerful yet simple-to-implement-and-use solution, I'm offering the following Java library to help you solve your use case in the best possible way.
The Buffer library provides multiple Buffer implementations that support preventing the scenario described above.
Size Buffer
A buffer that holds requests up to a predefined size. When the size threshold is reached, the action is triggered.
Time Buffer
A buffer that holds requests up to a predefined time (millis). When the time threshold is reached, the action is triggered.
Buffer
The widely used implementation includes both time and size thresholds, and when either the time or size threshold is reached, the action is triggered.
The Buffer is a simple utility that includes only two public methods:
append(T t)— you guessed it, for appending tasks to the buffer.run()— for running the required action on the buffered task list, as well as to flush the buffer when required.
Usage
The following code snippet generates a Buffer that triggers the buffer action when reaching 100 ms or 20 requests, whichever is earliest. Upon each action the timer resets and the action is scheduled for the next 100 ms.
Buffer b = new Buffer<>(100,
20,
_ACTION_,
_DEFAULT_ON_FAILURE_);
Buffer demo
IntStream.rangeClosed(1, 101)
.forEach(i -> b.append(i));
Output
As you can see below, the last action was triggered after 100 ms from the last appended request, since the size threshold was not reached yet.
System time: [1600539095335], at this iteration there were [20] entries in the list
System time: [1600539095335], at this iteration there were [20] entries in the list
System time: [1600539095335], at this iteration there were [20] entries in the list
System time: [1600539095335], at this iteration there were [20] entries in the list
System time: [1600539095335], at this iteration there were [20] entries in the list
System time: [1600539095435], at this iteration there were [1] entries in the list
For the full usage and, soon, the tests, please check the GitHub repository.
GitHub repository: https://github.com/jhashoul/buffer