pipelines: document KFP MLflow plugin#4423
Conversation
|
Skipping CI for Draft Pull Request. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
2bc6283 to
b8645af
Compare
b8645af to
7d4660a
Compare
| - **tls**: TLS configuration for MLflow communication | ||
| - **insecureSkipVerify**: Set to `true` to skip TLS certificate verification (not recommended for production) | ||
| - **caBundlePath**: Path to the CA certificate for your MLflow server | ||
| - **settings**: See the [user guide](/docs/components/pipelines/user-guides/integrations/mlflow-plugin/) for a complete list of MLflow plugin settings |
There was a problem hiding this comment.
Could you verify if the hyperlink is working ?
|
|
||
| This integration bridges the gap between pipeline orchestration and experiment tracking, giving you a unified view of your machine learning workflows. | ||
| All tracked data is viewable in the MLflow UI, organized by experiments. You can specify a custom experiment name for each run, or use the default experiment configured by your administrator. | ||
| This page provides detailed instructions on how to set up the MLflow plugin for Kubeflow Pipelines at the user level, once it has been deployed and configured at the [admin level](/docs/components/pipelines/operator-guides/mlflow-plugin/). |
There was a problem hiding this comment.
Could verify if the hyperlink is working?
There was a problem hiding this comment.
This link works successfully on my local deployment - did it fail for you?
| - Deployed MLflow on your cluster | ||
| - Enabled and configured the KFP MLflow plugin | ||
|
|
||
| See the [operator guide](/docs/components/pipelines/operator-guides/mlflow-plugin/) for setup instructions. |
There was a problem hiding this comment.
Could verify if the hyperlink is working?
There was a problem hiding this comment.
This link works successfully on my local deployment - did it fail for you?
7d4660a to
6eb5f0e
Compare
Signed-off-by: Alyssa Goins <[email protected]>
6eb5f0e to
ee4551c
Compare
|
|
||
| ## User Configuration | ||
|
|
||
| To use the MLflow plugin with your pipelines, configure the plugin in your KFP API `config.json` file: |
There was a problem hiding this comment.
I think this belongs in the operator/admin docs.
We can include the user level configuration on the kfp-launcher ConfigMap here for multiuser deployments, but I'd tend to put that in the operator/admin docs.
|
|
||
| This integration bridges the gap between pipeline orchestration and experiment tracking, giving you a unified view of your machine learning workflows. | ||
| All tracked data is viewable in the MLflow UI, organized by experiments. You can specify a custom experiment name for each run, or use the default experiment configured by your administrator. | ||
| This page provides detailed instructions on how to set up the MLflow plugin for Kubeflow Pipelines at the user level, once it has been deployed and configured at the [admin level](/docs/components/pipelines/operator-guides/mlflow-plugin/). |
There was a problem hiding this comment.
Could you include screenshots showing a run in KFP and a run in MLflow?
|
|
||
| ## Integration Overview | ||
|
|
||
| The KFP MLflow integration enables automatic experiment tracking for your Kubeflow Pipelines. When enabled, each pipeline run is automatically registered in MLflow, allowing you to: |
There was a problem hiding this comment.
Could you mention the environment variable injection feature and point how a user may run MLflow's autolog (a link to MLflow docs for that would be good)?
| weight = 1 | ||
| +++ | ||
|
|
||
| ## MLflow & Experiment Tracking |
There was a problem hiding this comment.
In general, this doc is a bit too admin focused. Could this be more focused on the end user that doesn't know anything about deploying/configuring KFP but wants to know how the MLflow integration works?
We should also link to the operator/admin guide in the prerequisites section.
|
|
||
| You need an MLflow instance running on your Kubernetes cluster. One recommended option is the [OpenDataHub MLflow Operator](https://github.com/opendatahub-io/mlflow-operator#mlflow-operator), which simplifies MLflow deployment and management on Kubernetes. | ||
|
|
||
| To deploy using the MLflow Operator: |
There was a problem hiding this comment.
We should not be documenting the MLflow operator as that is Red Hat specific.
| weight = 4 | ||
| +++ | ||
|
|
||
| ## KFP MLflow Plugin |
There was a problem hiding this comment.
Could you make this admin guide more focused on the generic upstream MLflow installation (not the Red Hat or the soon to be Kubeflow specific deployment)?
I think it's only worth mentioning the mlflow-integration plugin when discussing the auth configuration and the RBAC requirements.
| When using `bearer` or `basic-auth` authentication, this field references a Kubernetes secret containing the credentials. | ||
|
|
||
| #### workspacesEnabled | ||
| Enable MLflow workspaces for multi-tenant organization. |
There was a problem hiding this comment.
| Enable MLflow workspaces for multi-tenant organization. | |
| Enable MLflow workspaces for multi-tenancy in organizations. |
|
|
||
| ## Prerequisites | ||
|
|
||
| - Admin access to a Kubernetes cluster |
There was a problem hiding this comment.
| - Admin access to a Kubernetes cluster | |
| - Administrative access to a Kubernetes cluster |
|
|
||
| - Admin access to a Kubernetes cluster | ||
| - Kubeflow Pipelines installed on your cluster | ||
| - kubectl configured to access your cluster |
There was a problem hiding this comment.
| - kubectl configured to access your cluster | |
| - Kubectl configured to access your cluster |
Description of Changes
Document the KFP MLflow integration implemented by kubeflow/pipelines#13441
Related Issues
Checklist