-
-
Notifications
You must be signed in to change notification settings - Fork 342
Add FastStream example to docs #958
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Changes from 2 commits
65d3513
9d5f905
af133cb
0af3d3c
1776d13
ec91c34
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,42 @@ | ||||||||||
| .. _faststream-example: | ||||||||||
|
|
||||||||||
| FastStream example | ||||||||||
| ============= | ||||||||||
|
Comment on lines
+3
to
+4
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. done
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Did you push the changes?
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I started commenting along with adding changes so I can track what's done and what is not. Now I committed everything so you can check
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. And thanks for the review btw :) |
||||||||||
|
|
||||||||||
| .. meta:: | ||||||||||
| :keywords: Python,Dependency Injection,FastStream,Example | ||||||||||
| :description: This example demonstrates a usage of FastStream with Dependency Injector. | ||||||||||
|
|
||||||||||
|
|
||||||||||
| This example shows how to use ``Dependency Injector`` with `FastStream <https://dummy.faststream.airt.ai/0.5/faststream/>`_. | ||||||||||
|
birthdaysgift marked this conversation as resolved.
Outdated
|
||||||||||
|
|
||||||||||
| The source code is available on the `Github <https://github.com/ets-labs/python-dependency-injector/tree/master/examples/miniapps/faststream>`_. | ||||||||||
|
|
||||||||||
| Despite ``FastStream`` uses ``FastDepends`` library for dependency injection, the integration between | ||||||||||
| ``Dependency injector`` and ``FastStream`` has a small difference from already existing :ref:`fastdepends` example. | ||||||||||
|
|
||||||||||
| Since ``FastStream`` also leverages function signatures to determine input data types you have to use ``Depends()`` function | ||||||||||
| with ``cast=False`` argument to make ``FastStream`` ignore your injected dependency argument in the function signature. | ||||||||||
|
|
||||||||||
| Example below shows how to inject ``Counter`` class into ``FastStream`` redis handler so that it will distinguish between | ||||||||||
| message schema (``User``) and injected dependency (``Counter``) and use them both correctly. | ||||||||||
|
|
||||||||||
| Listing of ``consumer.py``: | ||||||||||
|
|
||||||||||
| .. literalinclude:: ../../examples/miniapps/faststream/src/consumer.py | ||||||||||
| :language: python | ||||||||||
|
|
||||||||||
| Listing of ``producer.py``: | ||||||||||
|
|
||||||||||
| .. literalinclude:: ../../examples/miniapps/faststream/src/producer.py | ||||||||||
| :language: python | ||||||||||
|
|
||||||||||
| Sources | ||||||||||
| ------- | ||||||||||
|
|
||||||||||
| Explore the sources on the `Github <https://github.com/ets-labs/python-dependency-injector/tree/master/examples/miniapps/faststream>`_. | ||||||||||
|
|
||||||||||
| .. include:: ../sponsor.rst | ||||||||||
|
|
||||||||||
| .. disqus:: | ||||||||||
|
|
||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| FROM python:3.13-bookworm | ||
|
|
||
| WORKDIR /app | ||
|
|
||
| COPY requirements.txt ./ | ||
| RUN pip install -r requirements.txt | ||
|
|
||
| COPY ./src ./src | ||
|
|
||
| ENV PYTHONUNBUFFERED=1 | ||
| CMD ["python3", "-m", "src.consumer"] | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. For src layout, you have to do few extra steps in
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this suggestion relates to installable python packages (i.e. libraries). Since my example is just a runnable application I don't see why it should follow this approach. Does it make a sense for runnable apps?
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I generally prefer src layout for clear code separation. In larger projects your root of the repo tends to be polluted with other stuff (e.g. django's For example app I think it would be better to stick to flat layout, for simplicity. I.e. remove |
||
|
ZipFile marked this conversation as resolved.
Outdated
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| FROM python:3.13-bookworm | ||
|
|
||
| WORKDIR /app | ||
|
|
||
| COPY requirements.txt ./ | ||
| RUN pip install -r requirements.txt | ||
|
|
||
| COPY ./src ./src | ||
|
|
||
| ENV PYTHONUNBUFFERED=1 | ||
| CMD ["python3", "-m", "src.producer"] |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| FastStream + Dependency Injector Example | ||
| ===================================== | ||
|
birthdaysgift marked this conversation as resolved.
Outdated
|
||
|
|
||
| This is a `FastStream <https://dummy.faststream.airt.ai/0.5/faststream/>`_ + | ||
| `Dependency Injector <https://python-dependency-injector.ets-labs.org/>`_ example application. | ||
|
|
||
| The example application is a simple consumer that counts messages sent to redis channel by producer. | ||
|
|
||
| Counter is provided to faststream handler as a dependency injected by ``dependency_injector`` library. | ||
|
|
||
| Run | ||
| --- | ||
|
|
||
| Everything can be run via docker compose. | ||
|
|
||
| A convenient ``run.sh`` script runs consumer, producer and redis services, prints logs from consumer | ||
| and shuts down once producer exits. | ||
|
|
||
| Ensure that ``run.sh`` has execution permission: | ||
|
|
||
| .. code-block:: bash | ||
|
|
||
| sudo chmod +x ./run.sh | ||
|
birthdaysgift marked this conversation as resolved.
Outdated
|
||
|
|
||
| Run the sciprt: | ||
|
|
||
| .. code-block:: bash | ||
|
|
||
| ./run.sh | ||
|
|
||
| The output should be something like: | ||
|
|
||
| .. code-block:: | ||
|
|
||
| faststream-example-consumer | Message #1 from John: 'As you can see' | ||
| faststream-example-consumer | Message #2 from John: 'messages are counted correctly' | ||
| faststream-example-consumer | Message #3 from John: 'by the counter that is injected' | ||
| faststream-example-consumer | Message #4 from John: 'into faststream handler' | ||
| faststream-example-consumer | Message #5 from John: 'via awesome dependency_injector library.' | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,22 @@ | ||
| name: faststream-example | ||
|
|
||
| services: | ||
|
|
||
| redis: | ||
| container_name: "${COMPOSE_PROJECT_NAME}-redis" | ||
|
birthdaysgift marked this conversation as resolved.
Outdated
|
||
| image: redis | ||
|
|
||
| consumer: | ||
| container_name: "${COMPOSE_PROJECT_NAME}-consumer" | ||
|
birthdaysgift marked this conversation as resolved.
Outdated
|
||
| build: | ||
| dockerfile: Dockerfile.consumer | ||
| depends_on: | ||
| - redis | ||
|
|
||
| producer: | ||
| container_name: "${COMPOSE_PROJECT_NAME}-producer" | ||
|
birthdaysgift marked this conversation as resolved.
Outdated
|
||
| build: | ||
| dockerfile: Dockerfile.producer | ||
|
birthdaysgift marked this conversation as resolved.
Outdated
|
||
| depends_on: | ||
| - consumer | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| .. _faststream-example: | ||
|
|
||
| FastStream example | ||
| ============= | ||
|
|
||
| .. meta:: | ||
| :keywords: Python,Dependency Injection,FastStream,Example | ||
| :description: This example demonstrates a usage of FastStream with Dependency Injector. | ||
|
|
||
|
|
||
| This example shows how to use ``Dependency Injector`` with `FastStream <https://dummy.faststream.airt.ai/0.5/faststream/>`_. | ||
|
|
||
| The source code is available on the `Github <https://github.com/ets-labs/python-dependency-injector/tree/master/examples/miniapps/faststream>`_. | ||
|
|
||
| Despite ``FastStream`` uses ``FastDepends`` library for dependency injection, the integration between | ||
| ``Dependency injector`` and ``FastStream`` has a small difference from already existing :ref:`fastdepends` example. | ||
|
|
||
| Since ``FastStream`` also leverages function signatures to determine input data types you have to use ``Depends()`` function | ||
| with ``cast=False`` argument to make ``FastStream`` ignore your injected dependency argument in the function signature. | ||
|
|
||
| Example below shows how to inject ``Counter`` class into ``FastStream`` redis handler so that it will distinguish between | ||
| message schema (``User``) and injected dependency (``Counter``) and use them both correctly. | ||
|
|
||
| Listing of ``consumer.py``: | ||
|
|
||
| .. literalinclude:: ../../examples/miniapps/faststream/src/consumer.py | ||
| :language: python | ||
|
|
||
| Listing of ``producer.py``: | ||
|
|
||
| .. literalinclude:: ../../examples/miniapps/faststream/src/producer.py | ||
| :language: python | ||
|
|
||
| Sources | ||
| ------- | ||
|
|
||
| Explore the sources on the `Github <https://github.com/ets-labs/python-dependency-injector/tree/master/examples/miniapps/faststream>`_. | ||
|
|
||
| .. include:: ../sponsor.rst | ||
|
|
||
| .. disqus:: | ||
|
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| dependency_injector | ||
| faststream | ||
| pydantic | ||
| redis |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
|
birthdaysgift marked this conversation as resolved.
|
||
| docker compose up \ | ||
| --no-attach=redis \ | ||
| --abort-on-container-exit \ | ||
| --exit-code-from producer | ||
|
|
||
| docker container rm \ | ||
| faststream-example-producer \ | ||
| faststream-example-consumer \ | ||
| faststream-example-redis | ||
|
|
||
| docker image rm \ | ||
| faststream-example-producer \ | ||
| faststream-example-consumer | ||
|
birthdaysgift marked this conversation as resolved.
Outdated
|
||
| Original file line number | Diff line number | Diff line change | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| @@ -0,0 +1,57 @@ | ||||||||||||||||||
| import asyncio | ||||||||||||||||||
| from typing import Annotated | ||||||||||||||||||
|
|
||||||||||||||||||
| from dependency_injector import containers, providers | ||||||||||||||||||
| from dependency_injector.wiring import Provide, inject | ||||||||||||||||||
| from faststream import Depends, FastStream | ||||||||||||||||||
| from faststream.redis import RedisBroker | ||||||||||||||||||
| from pydantic import BaseModel | ||||||||||||||||||
|
|
||||||||||||||||||
|
|
||||||||||||||||||
| class Counter: | ||||||||||||||||||
| def __init__(self): | ||||||||||||||||||
| self.count = 0 | ||||||||||||||||||
|
|
||||||||||||||||||
| def next(self) -> int: | ||||||||||||||||||
| self.count += 1 | ||||||||||||||||||
| return self.count | ||||||||||||||||||
|
|
||||||||||||||||||
|
|
||||||||||||||||||
| class Container(containers.DeclarativeContainer): | ||||||||||||||||||
| counter = providers.Singleton(Counter) | ||||||||||||||||||
|
|
||||||||||||||||||
|
|
||||||||||||||||||
| broker = RedisBroker("redis://redis", logger=None) | ||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Suggested change
If we're demonstrating integration, it would be better to go all-out.
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I tried to keep this example as small as possible to make key idea (adding But anyways I added full DI for So if you think that it's better to have full DI in this example I'll leave this decision up to you, I'm totally fine with both approaches. |
||||||||||||||||||
|
|
||||||||||||||||||
|
|
||||||||||||||||||
| class Message(BaseModel): | ||||||||||||||||||
| user: str | ||||||||||||||||||
| text: str | ||||||||||||||||||
|
|
||||||||||||||||||
|
|
||||||||||||||||||
| @broker.subscriber("messages") | ||||||||||||||||||
| @inject | ||||||||||||||||||
| async def handle_user_message( | ||||||||||||||||||
| message: Message, | ||||||||||||||||||
| counter: Annotated[ | ||||||||||||||||||
| Counter, | ||||||||||||||||||
| Depends( | ||||||||||||||||||
| Provide[Container.counter], | ||||||||||||||||||
| cast=False, # <-- this is the key part | ||||||||||||||||||
| ), | ||||||||||||||||||
| ], | ||||||||||||||||||
| ) -> None: | ||||||||||||||||||
| count = counter.next() | ||||||||||||||||||
| print(f"Message #{count} from {message.user}: '{message.text}'") | ||||||||||||||||||
|
|
||||||||||||||||||
|
|
||||||||||||||||||
| async def main() -> None: | ||||||||||||||||||
| container = Container() | ||||||||||||||||||
| container.wire(modules=[__name__]) | ||||||||||||||||||
|
|
||||||||||||||||||
| await FastStream(broker, logger=None).run() | ||||||||||||||||||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. see comment above
Suggested change
|
||||||||||||||||||
|
|
||||||||||||||||||
|
|
||||||||||||||||||
| if __name__ == "__main__": | ||||||||||||||||||
| asyncio.run(main()) | ||||||||||||||||||
|
|
||||||||||||||||||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| import json | ||
| import time | ||
|
|
||
| import redis | ||
|
|
||
|
|
||
| def main(): | ||
| client = redis.Redis(host="redis", port=6379) | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No DI container?
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. yes, for the sake of simplicity) since we already have
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm not exactly familiar with how you usually separate publishers and subscribers in your faststream apps, but I have gut feeling that you want to reuse same configuration/container for both. In particular, I think it would be nice to reuse the same broker instead of directly sending messages with raw redis client. For the sake of demonstration. https://faststream.ag2.ai/latest/redis/pubsub/publishing/#basic-redis-channel-publishing
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. anyways, I added DI to |
||
|
|
||
| for text in ( | ||
| "As you can see", | ||
| "messages are counted correctly", | ||
| "by the counter that is injected", | ||
| "into faststream handler", | ||
| "via awesome dependency_injector library.", | ||
| ): | ||
| time.sleep(2) | ||
|
|
||
| message = {"user": "John", "text": text} | ||
| client.publish("messages", json.dumps(message)) | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| main() | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.