Using Workspace Connections for Docker Authentication in Serverless Builds

[acube3]

Title

How to properly pass workspace connection credentials to Docker builds on AML serverless compute

Description

Problem Statement

I'm trying to set up a Docker build on Azure ML serverless compute that needs to authenticate to a private PyPI feed during the image build phase. The image needs to install Python packages from a private Azure DevOps Artifacts feed.

What I'm Trying to Achieve

• Create a Docker image that installs packages from a private PyPI mirror
• Use Azure DevOps Personal Access Token (PAT) for authentication (I am happy to use a different way if it is more sustainable and better)
• Run this on AML serverless compute (not a dedicated compute cluster)
• Securely pass credentials without hardcoding them

[acube3]

The docker build happens as part of the AML pipeline job which gets dockerfile as reference. The docker build gets instantiated using serverless job as part of the pipeline job. The docker creates the conda environment, then it tries to install (using pip) python packages from private azure devops PyPI (this is where authentication error happens). It also installs a wheel in the same conda environment. I can run the docker build by providing PAT as Azure devops token as build secret while doing docker build locally. I am not sure how that can be achieved on AML when serverless job runs docker image creation.