Support local development environment variables

[Robbe-Superlinear]

It’s recommended to use a .env file to securely manage sensitive configuration information, such as API keys, database credentials, and other environment-specific variables.

To ensure these configurations are used during development we should consider adding env_file to the devcontainer’s docker-compose configuration, pointing it towards the .env file.

As a result, the .env file should be added to the .gitignore file to prevent accidental inclusion in version control.

[lsorber]

To ensure these configurations are used during development we should consider adding env_file to the devcontainer’s docker-compose configuration, pointing it towards the .env file.

There are actually several different effects related to env vars at play here:

1. ✅ Docker-compose will load the .env file and make its variables available within the docker-compose.yml [1].
2. ✅ The .env file will also be available in the Dev Container, so processes that depend on an .env file can load it as usual. For instance, Pydantic can load settings from the .env file [2].
3. ❌ The env vars in the .env file are currently not made exposed as environment variables in the Dev Container. We could change that if we want. The downside to me is that exposing these variables only happens once when you create the Dev Container. So if you update the .env file, you'd need to restart the Dev Container for the env vars to be updated as well.

For the reasons explained in (3), I think making the developer responsible for loading the env vars him/herself in (2) is probably safer. But tell me if you disagree or see other benefits or downsides that I'm missing!

[1] https://docs.docker.com/compose/how-tos/environment-variables/variable-interpolation/
[2] https://docs.pydantic.dev/latest/concepts/pydantic_settings/#dotenv-env-support

As a result, the .env file should be added to the .gitignore file to prevent accidental inclusion in version control.

This is the case already!

[Robbe-Superlinear]

For me, a key aspect of the cookie-cutter approach is to apply and enforce best practices, ensuring a common and consistent way of working. Currently, leaving the responsibility to developers has resulted in inconsistent strategies, such as:

• Using python-dotenv: While this package is useful for development, it’s not ideal for production deployments on platforms like Azure or AWS. Best practices recommend using tools like Azure Key Vault or AWS Secrets Manager to manage environment variables securely in production.

• Hard-coding variables: Universally discouraged, as it increases the risk of exposing sensitive data and creates maintainability issues.

• Custom scripts (e.g., ./set_env_vars.sh): While such scripts could expose environment variables without reopening the container, they introduce additional complexity and potential for error.

After evaluating various approaches, I’ve concluded that using Docker Compose's env_file directly for dev container setup is the cleanest and most practical option. This method offers a straightforward and consistent way to manage environment variables across local and cloud environments, streamlining the transition to deployment on platforms like Azure or AWS. The only requirement for developers is to restart the Dev Container whenever environment variables are updated, ensuring simplicity and reliability in the process. This approach eliminates ambiguity in managing environment variables and establishes a clear strategy.

[lsorber]

In combination with Docker Compose's env_file, I think another best practice would be to use pydantic-settings, which is a separate Python package that makes it convenient and robust to load environment variables (e.g., from a .env file).

Example usage:

from typing import ClassVar
from pydantic_settings import BaseSettings, SettingsConfigDict


class RAGLiteCLIConfig(BaseSettings):
    """RAGLite CLI config."""

    model_config: ClassVar[SettingsConfigDict] = SettingsConfigDict(
        env_prefix="RAGLITE_",  # The prefix of the environment variables to load.
        env_file=".env",  # Load additional env vars from this file.
        extra="allow"  # Whether to add nondeclared env vars as attributes to this model.
    )

    mcp_server_name: str = "RAGLite"  # = RAGLITE_MCP_SERVER_NAME
    db_url: str = "sqlite:///:memory:"  # = RAGLITE_DB_URL
    llm: str = "gpt-4o"  # = RAGLITE_LLM
    embedder: str = "text-embedding-3-large"  # = RAGLITE_EMBEDDER

config = RAGLiteCLIConfig()  # Loads settings from env vars and the .env file.

I'm not sure we can really enforce this though. Do you think it would make sense to include pydantic-settings when the user scaffolds the project with with_typer_cli=1?