___ __ _____________
/ _ )/ / / / __/ __/ _ \
/ _ / /_/ / _// _// , _/
/____/\____/_/ /_/ /_/|_|
buffr provides a simple mechanism that allows you to collect values in a buffer. Once a certain time has passed or the buffer's capacity is reached, all values will pe processed with the provided function. As an analogy: don't process each drop, wait till the bucket is full and process that.
pip install buffr- Main Features
- Usage Example
- Installation
- Dependencies
- License
- Documentation
- Development
- Contributing to Buffr
- 🐍 Pure Python
- 🖌 Easily configurable
- 👨🎨 User-friendly
You listen to a queue for messages that you'll process and insert into the database. If we receive a message each second, this means that we have to perform an insert 60 times a second.
In this case we could use Buffr to collect messages for 30 seconds before batch inserting them. This reduces the number of costly database-operations from 60 per minute to just 2.
Creating a Buffr is easy.
from typing import List
from buffr import Buffr
# Define the processor
def processing_fn(items:List):
for item in items:
print(item * 2)
# Create a buffr
buffr = Buffr(max_capacity=100, buffer_ttl=30, flush_func=processing_fn)
# Add some values to the Buffr
for i in range(4):
buffr.add(i)The Buffr defined above will flush in three cases:
- the capacity of 100 items is exceeded
- 30 seconds pass (
buffr_ttl) buffr.flush()is called
pip install buffrThe source code is currently hosted on GitHub at: https://github.com/mike-huls/buffr
Binary installers for the latest released version are available at the Python Package Index (PyPI).
Buffr has no dependencies
🔨 Under construction
Find the changelog and list of upcoming features here.
Contributions are always welcome; feel free to submit bug reports, bug fixes, feature requests, documentation improvements or enhancements!