Troubleshooting

Troubleshooting

This page covers common issues encountered when running, developing, or deploying the IBF-SLM application. Each entry follows a Symptom → Cause → Fix structure.

---

Table of Contents

1. Startup Errors
2. Database Connection Issues
3. Authentication Errors
4. Alembic Migration Errors
5. Common Python / Dependency Errors
6. Reading Logs

---

1. Startup Errors

Port already in use

Symptom

ERROR:    [Errno 98] Address already in use

Cause Another process is already bound to the configured port (default 8000).

Fix Find and kill the conflicting process:

# identify the PID
lsof -ti tcp:8000

# kill it
kill -9 $(lsof -ti tcp:8000)

Or start uvicorn on a different port:

uvicorn app.main:app --port 8001

---

Application fails to import on startup

Symptom

ImportError: cannot import name 'X' from 'Y'
ModuleNotFoundError: No module named 'Z'

Cause Dependencies are missing or the virtual environment is not activated.

Fix

# activate the virtual environment first
source .venv/bin/activate   # Linux/macOS
# or
.venv\Scripts\activate      # Windows

# reinstall all dependencies
pip install -r requirements.txt

If the error persists, check that the correct Python version is active (python --version; must be 3.11+).

---

uvicorn exits immediately with no error message

Symptom The process starts and stops with exit code 1 but little output.

Cause Usually a syntax error or import-time exception in the application code, or missing environment variables.

Fix Run with explicit reload and debug flags to surface the traceback:

uvicorn app.main:app --reload --log-level debug

Also verify that all required environment variables are set (database URL, secret key, etc.):

python -c "from app.core.config import settings; print(settings)"

---

2. Database Connection Issues

Connection refused on startup

Symptom

asyncpg.exceptions.ConnectionRefusedError: Connection refused
sqlalchemy.exc.OperationalError: (asyncpg...) could not connect to server

Cause PostgreSQL is not running, or the host/port in DATABASE_URL is wrong.

Fix

1. Verify PostgreSQL is running:

   pg_isready -h localhost -p 5432
   # or
   systemctl status postgresql

2. Start it if needed:

   systemctl start postgresql   # systemd
   brew services start postgresql  # macOS Homebrew

3. Double-check DATABASE_URL in your .env file:

   DATABASE_URL=postgresql+asyncpg://user:password@localhost:5432/ibf_slm
   

---

Authentication failed for database user

Symptom

asyncpg.exceptions.InvalidPasswordError: password authentication failed for user "..."

Cause The credentials in DATABASE_URL do not match the PostgreSQL role.

Fix

# reset the password in psql
psql -U postgres -c "ALTER USER ibf_slm_user WITH PASSWORD 'newpassword';"

Update DATABASE_URL accordingly. If using pg_hba.conf peer authentication in development, ensure the OS user matches the PostgreSQL role name.

---

Too many connections / pool exhaustion

Symptom

asyncpg.exceptions.TooManyConnectionsError
sqlalchemy.exc.TimeoutError: QueuePool limit of size X overflow Y reached

Cause The SQLAlchemy async connection pool has been exhausted, often caused by unclosed sessions or a pool size that is too small for the load.

Fix

1. Ensure every database session is properly closed. Use async with or try/finally:

   async with AsyncSession(engine) as session:
       ...

2. Tune pool settings in the engine configuration:

   engine = create_async_engine(
       DATABASE_URL,
       pool_size=10,
       max_overflow=20,
       pool_timeout=30,
   )

3. In development, confirm there are no leaked sessions by adding echo_pool=True to create_async_engine.

---

Database does not exist

Symptom

asyncpg.exceptions.InvalidCatalogNameError: database "ibf_slm" does not exist

Cause The database has not been created yet, or the name in DATABASE_URL is incorrect.

Fix

createdb -U postgres ibf_slm
# or via psql
psql -U postgres -c "CREATE DATABASE ibf_slm;"

Then run migrations:

alembic upgrade head

---

3. Authentication Errors

401 Unauthorized on all protected routes

Symptom Every request to a protected endpoint returns 401 Unauthorized, even with a valid-looking token.

Cause

• SECRET_KEY environment variable is not set or does not match the key used to sign the token.
• Token has expired.
• The Authorization header format is wrong.

Fix

1. Confirm SECRET_KEY is set and consistent across restarts:

   echo $SECRET_KEY

2. Verify the header format is exactly:

   Authorization: Bearer <token>
   

3. Decode the token locally to inspect its claims and expiry:

   import jose.jwt as jwt
   payload = jwt.decode(token, SECRET_KEY, algorithms=["HS256"])
   print(payload)

   If exp is in the past, the client must re-authenticate.

---

jose.exceptions.JWTError: Signature verification failed

Symptom Token decoding raises JWTError server-side.

Cause The token was signed with a different SECRET_KEY (e.g., a previous key or a key from a different environment).

Fix Rotate the secret key in .env and require all clients to obtain a new token. In production, use a secrets manager and avoid hardcoding keys.

---

Password verification always fails

Symptom Login always returns 401 even with the correct password. The application logs may show a ValueError or AttributeError from passlib.

Cause passlib does not support bcrypt 4.x / 5.x. The passlib library uses an internal bcrypt.__about__.__version__ attribute that was removed in newer versions of bcrypt.

Fix Remove passlib and call bcrypt directly:

import bcrypt

def hash_password(plain: str) -> str:
    return bcrypt.hashpw(plain.encode(), bcrypt.gensalt()).decode()

def verify_password(plain: str, hashed: str) -> bool:
    return bcrypt.checkpw(plain.encode(), hashed.encode())

Update requirements.txt:

# remove passlib
bcrypt>=4.0.0

---

AttributeError: module 'bcrypt' has no attribute '__about__'

Symptom Stacktrace pointing into passlib/handlers/bcrypt.py.

Cause Same as above: passlib introspects bcrypt.__about__ which no longer exists in bcrypt 4+.

Fix See fix in the section above — replace passlib with direct bcrypt calls.

---

4. Alembic Migration Errors

FAILED: Target database is not up to date

Symptom

alembic.util.exc.CommandError: Target database is not up to date.

Cause There are unapplied migrations in the database that must be resolved before generating a new one, or the database was modified outside of Alembic.

Fix

# check current revision
alembic current

# apply pending migrations
alembic upgrade head

# then generate new migration if needed
alembic revision --autogenerate -m "describe change"

---

Multiple heads / merge conflict

Symptom

alembic.util.exc.CommandError: Multiple head revisions are present for given argument 'head'

Cause Two developers created migrations from the same base revision, resulting in a branched history.

Fix Create a merge migration:

alembic merge heads -m "merge branch"
alembic upgrade head

Review the generated merge migration to ensure no conflicting schema changes are present.

---

Can't locate revision identified by '...'

Symptom

alembic.util.exc.CommandError: Can't locate revision identified by 'abc123'

Cause The alembic_version table in the database references a revision file that no longer exists in alembic/versions/.

Fix

1. Check the current version recorded in the DB:

   psql $DATABASE_URL -c "SELECT * FROM alembic_version;"

2. If the file was deleted by mistake, restore it from version control.
3. If intentionally removed, manually update the version table to a valid revision:

   psql $DATABASE_URL -c "UPDATE alembic_version SET version_num = '<valid_revision>';"

   Then run alembic upgrade head.

---

Autogenerate produces an empty migration

Symptom alembic revision --autogenerate creates a migration with no op.* calls.

Cause

• SQLAlchemy models were not imported before target_metadata is read in env.py.
• The models are defined but not referenced by the metadata object.

Fix In alembic/env.py, ensure all models are imported before target_metadata is assigned:

from app.models import *  # noqa: F401, F403
from app.db.base import Base

target_metadata = Base.metadata

---

5. Common Python / Dependency Errors

AttributeError or TypeError from SQLAlchemy on Python 3.14+

Symptom

AttributeError: ...
TypeError: ...

Stacktrace originates inside sqlalchemy/ with no obvious application code cause.

Cause SQLAlchemy versions below 2.0.36 are incompatible with CPython 3.14 due to internal interpreter changes.

Fix

pip install "sqlalchemy>=2.0.36"

Pin in requirements.txt:

SQLAlchemy>=2.0.36

---

greenlet import error with async SQLAlchemy

Symptom

ImportError: greenlet is required for async support

Cause The greenlet package is missing. It is required by SQLAlchemy's async engine even though it is an optional dependency.

Fix

pip install "sqlalchemy[asyncio]"
# which installs greenlet as a dependency

---

asyncpg not found

Symptom

ModuleNotFoundError: No module named 'asyncpg'

Cause asyncpg is not installed.

Fix

pip install asyncpg

Add to requirements.txt:

asyncpg>=0.29.0

---

python-jose vs jose import confusion

Symptom

ModuleNotFoundError: No module named 'jose'

or unexpected behavior from the wrong jose package.

Cause The package is installed as python-jose on PyPI but imported as jose. A plain jose package also exists on PyPI and shadows it.

Fix

pip uninstall jose python-jose
pip install python-jose[cryptography]

Import as:

from jose import jwt, JWTError

---

Dependency version conflicts after pip install

Symptom pip reports resolver conflicts, or the app crashes with unexpected AttributeError/ImportError at runtime after updating a package.

Fix Use a pinned requirements file and a virtual environment:

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt

To regenerate a fully-pinned lockfile:

pip freeze > requirements.lock

In CI, install from the lockfile to guarantee reproducibility.

---

6. Reading Logs

uvicorn access and error logs

By default uvicorn writes to stdout. Increase verbosity with:

uvicorn app.main:app --log-level debug

Log levels in order of decreasing verbosity: debug, info, warning, error, critical.

To write logs to a file:

uvicorn app.main:app --log-level info 2>&1 | tee app.log

---

Structuring log output for diagnosis

The application uses Python's standard logging module. Useful patterns:

# filter for errors only
grep -i "error\|exception\|traceback" app.log

# show lines around a specific exception
grep -A 20 "Traceback" app.log

# tail live
tail -f app.log

---

SQLAlchemy query logging

To see every SQL statement issued (useful for debugging N+1 queries or incorrect joins), enable engine echo:

engine = create_async_engine(DATABASE_URL, echo=True)

Or set the log level for the SQLAlchemy logger:

import logging
logging.getLogger("sqlalchemy.engine").setLevel(logging.INFO)

Note: Do not leave echo=True or INFO-level engine logging enabled in production — it is extremely verbose and may expose sensitive data in logs.

---

Alembic verbose output

alembic -v upgrade head      # verbose migration output
alembic history --verbose    # full revision history with details
alembic current --verbose    # show current applied revision

---

Inspecting the database directly

# connect to the database
psql $DATABASE_URL

# check applied migrations
SELECT * FROM alembic_version;

# list tables
\dt

# describe a table
\d+ users

---

For issues not covered here, check the SQLAlchemy async docs, the Alembic docs, and the FastAPI docs. Open an issue in the project repository with the relevant log output and your environment details.