Needed to create my own docker env because given one did not have packages installed.

[SStoica12]

Hello,
I have tried pulling from docker docker pull russellrobin/bunny:latest and creating the image: docker run --rm -it --gpus '"device=1"' -v /shared/nas2/sstoica2/Bunny:/workspace/Bunny russellrobin/bunny:latest bash but none of the required packages were there:

`pip list` results in
Package                 Version
----------------------- --------
anaconda-anon-usage     0.4.4
archspec                0.2.3
boltons                 23.0.0
Brotli                  1.0.9
certifi                 2024.2.2
cffi                    1.16.0
charset-normalizer      2.0.4
conda                   24.3.0
conda-content-trust     0.2.0
conda-libmamba-solver   24.1.0
conda-package-handling  2.2.0
conda_package_streaming 0.9.0
cryptography            42.0.5
distro                  1.8.0
idna                    3.4
jsonpatch               1.33
jsonpointer             2.1
libmambapy              1.5.8
menuinst                2.0.2
packaging               23.2
pip                     23.3.1
platformdirs            3.10.0
pluggy                  1.0.0
pycosat                 0.6.6
pycparser               2.21
PySocks                 1.7.1
requests                2.31.0
ruamel.yaml             0.17.21
setuptools              68.2.2
tqdm                    4.65.0
truststore              0.8.0
urllib3                 2.1.0
wheel                   0.41.2
zstandard               0.19.0

When I check the conda env that is active in the env by doing whoami, I get root, so there is not a bleading of my base environment into this docker env.

It took a while because the versions for packages were not provided in the pyproject.toml, but I think I may have a working dockerfile:

FROM nvidia/cuda:11.8.0-cudnn8-devel-ubuntu22.04

WORKDIR /workspace

RUN apt-get update && apt-get install -y python3-pip git && \
    python3 -m pip install --upgrade pip

# PyTorch must come first — apex, flash-attn, deepspeed all build against it
RUN python3 -m pip install \
    torch==2.2.2 \
    torchvision==0.17.2 \
    torchaudio==2.2.2 \
    xformers==0.0.25.post1 \
    --index-url https://download.pytorch.org/whl/cu118

RUN python3 -m pip install ninja packaging

# Core training stack — pinned for mutual compatibility
RUN python3 -m pip install \
    transformers==4.38.0 \
    tokenizers==0.15.2 \
    accelerate==0.30.1 \
    peft==0.10.0 \
    deepspeed==0.14.4 \
    bitsandbytes==0.43.1

# Bunny runtime dependencies. Add wandb for logging router
RUN python3 -m pip install \
    wandb \
    datasets \
    einops \
    einops-exts \
    timm==0.9.16 \
    sentencepiece \
    tiktoken \
    shortuuid \
    tabulate \
    markdown2 \
    openpyxl \
    pypandoc \
    requests \
    scikit-learn \
    protobuf \
    psutil \
    fastapi \
    uvicorn \
    httpx \
    gradio \
    gradio_client \
    "pydantic<2" \
    "numpy<2"

# flash-attn after torch, pinned within Bunny's required range >=2.7.1,<=2.8.0
RUN python3 -m pip install flash-attn==2.7.4.post1 --no-build-isolation

# Copy repo — apex/ must be at e13873d (last commit with apex.amp)
COPY . /workspace/Bunny

# apex with CUDA extensions
RUN pip uninstall apex -y 2>/dev/null || true && \
    cd /workspace/Bunny/apex && \
    rm -rf build && \
    python3 setup.py install --cuda_ext --cpp_ext && \
    cd / && python3 -c "from apex import amp; print('apex amp OK')"

# Install Bunny without re-resolving deps (all already pinned above)
RUN cd /workspace/Bunny && python3 -m pip install -e . --no-deps

# Pre-create triton cache dir to suppress "No such file or directory" warning
RUN mkdir -p /root/.triton/autotune