Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -21,3 +21,4 @@ yarn-error.log*



volcano/
4 changes: 2 additions & 2 deletions blog/Volcano-1.10.0-release.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ Volcano introduced the elastic queue capacity scheduling feature in version v1.9

For detailed design information on elastic queue capacity scheduling, refer to the [Capacity Scheduling Design Document](https://github.com/volcano-sh/volcano/blob/master/docs/design/capacity-scheduling.md).

For a step-by-step guide on using the capacity plugin, see the [Capacity Plugin User Guide](/docs/UserGuide/user_guide_how_to_use_capacity_plugin).
For a step-by-step guide on using the capacity plugin, see the [Capacity Plugin User Guide](/docs/Scheduler/Plugins/capacity).

Configure each dimension deserved resource samples for the queue:

Expand All @@ -88,7 +88,7 @@ For instructions on using the `Device Plugin` to report various GPU models, plea

In version v1.10.0, the `capacity` plugin is the default for queue management. Note that the `capacity` and `proportion` plugins are incompatible, so after upgrading to v1.10.0, you must set the `deserved` field for queues to ensure proper functionality.

For detailed instructions, please refer to the [Capacity Plugin User Guide](/docs/UserGuide/user_guide_how_to_use_capacity_plugin).
For detailed instructions, please refer to the [Capacity Plugin User Guide](/docs/Scheduler/Plugins/capacity).

The `capacity` plugin allocates cluster resources based on the `deserved` value set by the user, while the `proportion` plugin dynamically allocates resources according to queue weight. Users can select either the `capacity` or `proportion` plugin for queue management based on their specific needs.

Expand Down
6 changes: 3 additions & 3 deletions blog/Volcano-1.13.0-release.md
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ Sincerely thanks to community developer: @[Wonki4](https://github.com/Wonki4)

Design documentation: [Ray Framework Plugin Design Doc](https://github.com/volcano-sh/volcano/blob/master/docs/design/distributed-framework-plugins.md).

Usage documentation: [Ray Plugin User Guide](/docs/UserGuide/user_guide_how_to_use_ray_plugin).
Usage documentation: [Ray Plugin User Guide](/docs/Controller/Plugins/ray).

## Introduce HCCL Plugin Support

Expand Down Expand Up @@ -189,7 +189,7 @@ Sincerely thanks to community developers: @[JesseStutler](https://github.com/Jes

NodeGroup design documentation: [NodeGroup Design.](https://github.com/volcano-sh/volcano/blob/master/docs/design/node-group.md)

NodeGroup usage documentation: [NodeGroup User Guide.](/docs/UserGuide/user_guide_how_to_use_nodegroup_plugin)
NodeGroup usage documentation: [NodeGroup User Guide.](/docs/Scheduler/Plugins/nodegroup)

## Introduce ResourceStrategyFit Plugin

Expand Down Expand Up @@ -274,7 +274,7 @@ Sincerely thanks to community developers: @[LY-today](https://github.com/LY-toda

Design documentation: [ResourceStrategyFit Design](https://github.com/volcano-sh/volcano/blob/master/docs/design/resource-strategy-fit-scheduling.md)

Usage documentation: [ResourceStrategyFit User Guide](/docs/UserGuide/user_guide_how_to_use_resource_strategy_fit_plugin)
Usage documentation: [ResourceStrategyFit User Guide](/docs/Scheduler/Plugins/resource_strategy_fit)

## Decouple Colocation from OS

Expand Down
2 changes: 1 addition & 1 deletion blog/Volcano-1.7.0-release-en.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ Other enhanced plugins include those for TensorFlow, MPI, and PyTorch Jobs. They
Volcano also provides an extended development framework for you to tailor Job plugins to your needs.

Design Documentation: [Pytorch-plugin](https://github.com/volcano-sh/volcano/blob/master/docs/design/distributed-framework-plugins.md#pytorch-plugin)<br />
User Guide: [Pytorch Plugin User Guide](/docs/UserGuide/user_guide_how_to_use_pytorch_plugin)<br />
User Guide: [Pytorch Plugin User Guide](/docs/Controller/Plugins/pytorch)<br />
Issue:[#2292](https://github.com/volcano-sh/volcano/issues/2292)<br />


Expand Down
2 changes: 1 addition & 1 deletion blog/Volcano-1.9.0-release.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ spec:
```

For a complete usage example of queue elastic capacity scheduling, please refer to:
[How to use capacity plugin](/docs/UserGuide/user_guide_how_to_use_capacity_plugin).
[How to use capacity plugin](/docs/Scheduler/Plugins/capacity).

For the elastic queue capacity design document, please refer to:
[Capacity scheduling Design](https://github.com/volcano-sh/volcano/blob/master/docs/design/capacity-scheduling.md).
Expand Down
2 changes: 1 addition & 1 deletion docs/Concepts/Queue.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ deserved indicates the expected resource amount for all PodGroups in this queue.

> **Note**:
>
> 1. This field can only be configured when the capacity plugin is enabled, and must be less than or equal to the capability value. The proportion plugin uses weight to automatically calculate the queue's deserved value. For more information on using the capacity plugin, see: [capacity plugin user guide](../UserGuide/user_guide_how_to_use_capacity_plugin.md)
> 1. This field can only be configured when the capacity plugin is enabled, and must be less than or equal to the capability value. The proportion plugin uses weight to automatically calculate the queue's deserved value. For more information on using the capacity plugin, see: [capacity plugin user guide](../Scheduler/Plugins/capacity.md)
> 2. If the allocated resources of a queue exceed its configured deserved value, the queue cannot reclaim resources from other queues

* weight, *optional*
Expand Down
2 changes: 1 addition & 1 deletion docs/Contribution/_category_.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{
"label": "Contribution",
"position": 9
"position": 10
}
7 changes: 7 additions & 0 deletions docs/Controller/Overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
title: Overview
---

## Introduction

The Volcano Job Controller manages the lifecycle of jobs and dynamically configures job environments using plugins such as SSH, SVC, MPI, and PyTorch.
4 changes: 4 additions & 0 deletions docs/Controller/Plugins/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"label": "Plugins",
"key": "controller-plugins"
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Volcano Job Plugin -- Env User Guide"
title: Env

---

Expand Down
59 changes: 59 additions & 0 deletions docs/Controller/Plugins/hcclrank.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
---
title: HCCLRank
---

## Introduction

In distributed AI training, particularly when using Ascend NPUs (Neural Processing Units) or MindSpore frameworks, the compute nodes need a deterministic rank or index to communicate over HCCL (Huawei Collective Communication Library).

The **HCCLRank Plugin** is a Volcano Job plugin that automatically injects a `hccl/rankIndex` annotation into the Pods of a Volcano Job. It calculates a unique rank for each pod based on its task type (`master` or `worker`) and its replica index.

## Mechanism

During the Pod creation phase (`OnPodCreate`), the HCCLRank Plugin intercepts the pod and adds the `hccl/rankIndex` annotation to it.

The calculation is as follows:
- **Master Role**: Rank = Pod Index
- **Worker Role**: Rank = (Total Master Replicas) + Pod Index

If the Pod already has a `RANK` environment variable defined in its container specifications, the plugin will use that value instead and simply map it to the `hccl/rankIndex` annotation.

## Configuration

To enable the HCCLRank plugin, configure it within the Volcano job controller's configuration or add it to the `plugins` field of your `VolcanoJob` spec.

```yaml
apiVersion: batch.volcano.sh/v1alpha1
kind: Job
metadata:
name: ascend-distributed-training
spec:
minAvailable: 3
schedulerName: volcano
plugins:
hcclrank:
- --master=master
- --worker=worker
tasks:
- replicas: 1
name: master
template:
spec:
containers:
- name: master
image: my-ascend-image
- replicas: 2
name: worker
template:
spec:
containers:
- name: worker
image: my-ascend-image
```

### Arguments

The HCCLRank plugin supports overriding the default task names used to identify master and worker roles:

- **`--master`**: The name of the master role task in your Job spec. Default is `master`.
- **`--worker`**: The name of the worker role task in your Job spec. Default is `worker`.
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "MPI Plugin User Guide"
title: MPI

---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Pytorch Plugin User Guide"
title: Pytorch
---


Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Ray Plugin User Guide"
title: Ray
---


Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Volcano Job Plugin -- SSH User Guide"
title: SSH

---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Volcano Job Plugin -- SVC User Guide"
title: SVC
---

## Background
Expand Down
4 changes: 4 additions & 0 deletions docs/Controller/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"label": "Controller",
"position": 6
}
2 changes: 1 addition & 1 deletion docs/Ecosystem/RayOnVolcano.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,4 +203,4 @@ ray job submit --address http://localhost:8265 -- python -c "import ray; ray.ini
### Learn More

- For KubeRay integration details, visit the [KubeRay Volcano Scheduler Documentation](https://docs.ray.io/en/latest/cluster/kubernetes/k8s-ecosystem/volcano.html#kuberay-integration-with-volcano)
- For Volcano Job Ray plugin details, see the [Volcano Ray Plugin Guide](/docs/UserGuide/user_guide_how_to_use_ray_plugin)
- For Volcano Job Ray plugin details, see the [Volcano Ray Plugin Guide](/docs/Controller/Plugins/ray)
2 changes: 1 addition & 1 deletion docs/Ecosystem/_category_.json
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
{
"label": "Ecosystem",
"position": 6
"position": 9
}
2 changes: 1 addition & 1 deletion docs/KeyFeatures/QueueResourceManagement.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ Queue scheduling in Volcano involves the following core actions:
Volcano provides two core queue scheduling plugins:

#### capacity plugin
[capacity plugin](/docs/UserGuide/user_guide_how_to_use_capacity_plugin) supports setting queue's deserved resource amount through explicit configuration, as shown in this example:
[capacity plugin](/docs/Scheduler/Plugins/capacity) supports setting queue's deserved resource amount through explicit configuration, as shown in this example:

```yaml
apiVersion: scheduling.volcano.sh/v1beta1
Expand Down
4 changes: 4 additions & 0 deletions docs/Scheduler/Plugins/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
{
"label": "Plugins",
"key": "scheduler-plugins"
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Capacity Plugin User Guide"
title: Capacity

---

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Cooldown Protection Plugin User Guide"
title: CDP

---

Expand Down
41 changes: 41 additions & 0 deletions docs/Scheduler/Plugins/deviceshare.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
---
title: DeviceShare
---

## Introduction

The **DeviceShare Plugin** is an advanced resource scheduling plugin in Volcano that provides a common framework for sharing specialized hardware devices (like GPUs, NPUs, FPGAs) across multiple pods.

Rather than implementing fragmented logic for each new hardware accelerator, Volcano exposes a unified `Devices` interface. The `deviceshare` plugin leverages this interface to perform robust allocation, node filtering, and resource tracking for shared devices.

## Mechanism

The DeviceShare plugin works in conjunction with device-specific implementations. It exposes standard scheduling operations such as `Predicate` (filtering nodes based on available device capacity) and `Allocate`/`Release` (assigning portions of a device to specific pods).

Currently, the `deviceshare` plugin serves as the underlying engine powering features like:
- **GPU Sharing**: Allowing multiple pods to request fractions of a single physical GPU's memory.
- **vGPU and vNPU**: Virtualizing accelerator slices.
- **GPU Exclusive**: Restricting a pod to exclusively own a GPU to avoid contention.

## Configuration and Usage

The `deviceshare` plugin is typically enabled implicitly when you enable device sharing predicates in the Volcano scheduler config map. However, if you are developing custom device sharing logic or need to explicitly declare it, it can be configured in your `volcano-scheduler-configmap`:

```yaml
actions: "enqueue, allocate, backfill"
tiers:
- plugins:
- name: priority
- name: gang
- name: conformance
- name: deviceshare # Enable the device share framework plugin
- plugins:
- name: overcommit
- name: drf
- name: predicates
- name: proportion
- name: nodeorder
- name: binpack
```

> **Note:** For specific guides on how to configure your workloads to request shared GPUs or NPUs, please refer to the dedicated guides for [GPU Sharing](../../UserGuide/user_guide_how_to_use_gpu_sharing) and [vNPU](../../UserGuide/user_guide_how_to_use_vnpu).
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
---
title: "Nodegroup Plugin User Guide"
title: Nodegroup

---

Expand Down
44 changes: 44 additions & 0 deletions docs/Scheduler/Plugins/overcommit.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
---
title: Overcommit
---

## Introduction

In typical cluster environments, the scheduler calculates available idle resources strictly based on physical node capacity minus allocated resources. However, when cluster resources are nearly fully utilized, many PodGroups are rejected from entering the scheduling pipeline and are left completely un-enqueued, which might not be desirable for scenarios where you want the scheduler to tolerate a larger backlog of `pending` pods.

The **Overcommit Plugin** allows the scheduler to artificially inflate the apparent "idle resources" of the cluster by applying an `overcommit-factor`. This permits more jobs to be enqueued and wait in the scheduling pipeline than the physical resources might typically allow.

## Mechanism

The Overcommit plugin evaluates whether a job can be enqueued based on the requested `MinResources` of the PodGroup and the expanded idle resources.

Expanded idle resource is calculated as:
`Idle Resource = (Total Resource * overcommit-factor) - Used Resource`

If the job's minimal requested resources can fit into this expanded idle resource pool, the job is permitted to be enqueued.

## Configuration

To use the Overcommit Plugin, add it to your `volcano-scheduler-configmap` under the `enqueue` tier, and provide an `overcommit-factor`.

```yaml
actions: "enqueue, allocate, backfill"
tiers:
- plugins:
- name: overcommit # Enable the overcommit plugin
arguments:
overcommit-factor: 1.2 # The overcommit factor. Default is 1.2
- name: priority
- name: gang
- name: conformance
- plugins:
- name: drf
- name: predicates
- name: proportion
- name: nodeorder
- name: binpack
```

### Arguments

- **`overcommit-factor`**: A float value greater than or equal to `1.0`. For example, `1.2` means the scheduler will pretend the cluster has 20% more total resources when deciding whether to enqueue jobs into the pipeline. If a value less than `1.0` is provided, the plugin will automatically fallback to the default value of `1.2`.
43 changes: 43 additions & 0 deletions docs/Scheduler/Plugins/pdb.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
---
title: PDB
---

## Introduction

When users deploy highly available jobs or applications on Volcano, they often need to limit the number of pod replicas that can be evicted or destroyed simultaneously to avoid downtime. This constraint is managed via Kubernetes **PodDisruptionBudget (PDB)** resources.

The **PDB Plugin** ensures that Volcano respects user-defined PDB constraints during the scheduling process, specifically during eviction actions like `reclaim`, `preempt`, and `shuffle`.

## Prerequisites

- Your Kubernetes version must be 1.21 or later.
- You must have created valid `PodDisruptionBudget` resources for your workloads.

## Mechanism

The PDB Plugin registers several functions (`ReclaimableFn`, `PreemptableFn`, and `VictimTasksFn`) under the `reclaim`, `preempt`, and `shuffle` actions. It maintains a cache of PDBs using `v1.PodDisruptionBudgetLister`.

During eviction scenarios, the plugin filters out tasks whose eviction would violate the configured PDB constraints. It calculates the `DisruptedPods` (pods whose eviction was processed but not yet observed by the PDB controller) and ensures the remaining available replicas satisfy the budget.

## Configuration

To enable the PDB Plugin, update the `volcano-scheduler-configmap` to include the `pdb` plugin in your configuration tiers.

```yaml
actions: "reclaim, preempt, shuffle"
tiers:
- plugins:
- name: pdb # Enable the PDB plugin
- name: priority
- name: gang
- name: conformance
- plugins:
- name: overcommit
- name: drf
- name: predicates
- name: proportion
- name: nodeorder
- name: binpack
```

*Note: The PDB plugin will be actively invoked when actions like `reclaim`, `preempt`, or `shuffle` are executed in the scheduler workflow.*
Loading