Is your feature request related to a problem? Please describe.
Currently, Pegasus manages the job/transformations using Containers. However, it can get cumbersome to create and push/manage containers for each transformation in large workflows.
Describe the solution you'd like
Environment management tools like conda/pixi are very helpful, especially for bioinformatic/AI/ML applications, where the workflow developer can define specific software and their channels in the workflow definition, and Pegasus can create those environments and execute them. This also removes the need to manage containers, because environments are created/cached during the job execution.
Describe alternatives you've considered
Conda is a good starting point, but Pixi is also a great Conda alternative.
Example usage:
# Option 1
bwaenv = Conda(specs = ["conda-forge::bwa=0.17.7", "conda-forge::python=3.12", ...])
bwa = Transformation(...).add_pegasus_profile(conda = bwaenv)
# Options 2
bwa = Transformation(...).add_pegasus_profile(conda = ["conda-forge::bwa=0.17.7", "conda-forge::python=3.12", ...])
Is your feature request related to a problem? Please describe.
Currently, Pegasus manages the job/transformations using Containers. However, it can get cumbersome to create and push/manage containers for each transformation in large workflows.
Describe the solution you'd like
Environment management tools like
conda/pixiare very helpful, especially for bioinformatic/AI/ML applications, where the workflow developer can define specific software and their channels in the workflow definition, and Pegasus can create those environments and execute them. This also removes the need to manage containers, because environments are created/cached during the job execution.Describe alternatives you've considered
Conda is a good starting point, but Pixi is also a great Conda alternative.
Example usage: