diff --git a/content/en/docs/user_guide_how_to_use_binpack_plugin.md b/content/en/docs/user_guide_how_to_use_binpack_plugin.md new file mode 100644 index 00000000..85a994c7 --- /dev/null +++ b/content/en/docs/user_guide_how_to_use_binpack_plugin.md @@ -0,0 +1,86 @@ ++++ +title = "Binpack Plugin User Guide" +date = 2024-05-10 +type = "docs" +weight = 50 +url = "/en/docs/user-guide/how_to_use_binpack_plugin/" +[menu.docs] + parent = "user-guide" ++++ + +## Introduction + +The Binpack plugin attempts to pack as many pods as possible onto a single node before moving on to the next. This minimizes the fragmentation of cluster resources and leaves larger, empty nodes available for future workloads that may require massive amounts of contiguous resources. By heavily utilizing a subset of nodes, it is also beneficial for cluster autoscalers to downscale completely empty nodes. + +## Environment setup + +### Update scheduler configmap + +Ensure the `binpack` plugin is enabled and configured with weights for different resources in the `volcano-scheduler-configmap`. + +```yaml +kind: ConfigMap +apiVersion: v1 +metadata: + name: volcano-scheduler-configmap + namespace: volcano-system +data: + volcano-scheduler.conf: | + actions: "enqueue, allocate, backfill" + tiers: + - plugins: + - name: priority + - name: binpack + arguments: + binpack.weight: 10 + binpack.cpu: 5 + binpack.memory: 5 +``` + +The arguments dictate the relative importance of binpacking by CPU vs Memory. + +## How to use Binpack Plugin + +When you submit a workload to Volcano, the Binpack plugin automatically influences the node scoring phase. + +### Example YAML + +Create a file named `binpack-job.yaml`: + +```yaml +apiVersion: batch.volcano.sh/v1alpha1 +kind: Job +metadata: + name: binpack-job +spec: + minAvailable: 1 + schedulerName: volcano + queue: default + tasks: + - replicas: 10 + name: worker + template: + spec: + containers: + - image: nginx + name: nginx + resources: + requests: + cpu: "500m" + memory: "500Mi" + restartPolicy: OnFailure +``` + +## Verification + +1. Apply the job to your cluster: + ```bash + kubectl apply -f binpack-job.yaml + ``` + +2. Check the node assignment of the pods: + ```bash + kubectl get pods -o wide | grep binpack-job + ``` + +3. **Expected Result**: Without Binpack (using standard Kubernetes spreading), the scheduler usually distributes pods evenly across all available nodes. With the Binpack plugin, you will observe that Volcano schedules pods densely onto `Node A` until it is completely full, and only then starts placing pods on `Node B`. diff --git a/content/en/docs/user_guide_how_to_use_drf_plugin.md b/content/en/docs/user_guide_how_to_use_drf_plugin.md new file mode 100644 index 00000000..aead54da --- /dev/null +++ b/content/en/docs/user_guide_how_to_use_drf_plugin.md @@ -0,0 +1,113 @@ ++++ +title = "DRF Plugin User Guide" +date = 2024-05-10 +type = "docs" +weight = 50 +url = "/en/docs/user-guide/how_to_use_drf_plugin/" +[menu.docs] + parent = "user-guide" ++++ + +## Introduction + +The DRF (Dominant Resource Fairness) scheduling plugin allocates resources based on the dominant resource requested by a job. The dominant resource is the resource type (e.g., CPU or Memory) that takes up the largest percentage of the total cluster capacity. The DRF algorithm calculates the dominant resource share for each job and prioritizes jobs with the lowest share, ensuring fairness across multiple resource types. + +This is highly effective in mixed workloads where some jobs are CPU-intensive and others are memory-intensive. + +## Environment setup + +### Update scheduler configmap + +Ensure the `drf` plugin is enabled in the `volcano-scheduler-configmap` under the `tiers` section. + +```yaml +kind: ConfigMap +apiVersion: v1 +metadata: + name: volcano-scheduler-configmap + namespace: volcano-system +data: + volcano-scheduler.conf: | + actions: "enqueue, allocate, backfill" + tiers: + - plugins: + - name: priority + - name: gang + - name: drf +``` + +## How to use DRF Plugin + +DRF works automatically out of the box when enabled. To see DRF in action, we can create two jobs in the same queue: one that is CPU-intensive and another that is Memory-intensive. + +### Example YAML + +Create a file named `drf-jobs.yaml`: + +```yaml +apiVersion: batch.volcano.sh/v1alpha1 +kind: Job +metadata: + name: cpu-heavy-job +spec: + minAvailable: 1 + schedulerName: volcano + queue: default + tasks: + - replicas: 5 + name: worker + template: + spec: + containers: + - image: nginx + name: nginx + resources: + requests: + cpu: "2" + memory: "500Mi" + restartPolicy: OnFailure +--- +apiVersion: batch.volcano.sh/v1alpha1 +kind: Job +metadata: + name: mem-heavy-job +spec: + minAvailable: 1 + schedulerName: volcano + queue: default + tasks: + - replicas: 5 + name: worker + template: + spec: + containers: + - image: nginx + name: nginx + resources: + requests: + cpu: "500m" + memory: "4Gi" + restartPolicy: OnFailure +``` + +### How DRF Balances Resources + +If the total cluster capacity is 10 CPUs and 20Gi Memory: +- `cpu-heavy-job` requests 2 CPUs (20% of cluster CPU) and 500Mi Memory (2.5% of cluster Memory) per pod. Its dominant resource is CPU. +- `mem-heavy-job` requests 500m CPUs (5% of cluster CPU) and 4Gi Memory (20% of cluster Memory) per pod. Its dominant resource is Memory. + +As Volcano schedules pods, the DRF plugin constantly recalculates the dominant resource share for both jobs. If `cpu-heavy-job` has 2 pods running (40% CPU share) and `mem-heavy-job` has 1 pod running (20% Memory share), DRF will prioritize the next pod from `mem-heavy-job` because its dominant resource share is currently lower. + +## Verification + +1. Submit both jobs simultaneously: + ```bash + kubectl apply -f drf-jobs.yaml + ``` + +2. Monitor the scheduling order: + ```bash + kubectl get pods -w + ``` + +3. You will observe that Volcano interleaves the scheduling of pods from both jobs to keep their dominant resource shares roughly equal, rather than starving one job entirely while the other finishes. diff --git a/content/en/docs/user_guide_how_to_use_gang_plugin.md b/content/en/docs/user_guide_how_to_use_gang_plugin.md new file mode 100644 index 00000000..1db66195 --- /dev/null +++ b/content/en/docs/user_guide_how_to_use_gang_plugin.md @@ -0,0 +1,133 @@ ++++ +title = "Gang Plugin User Guide" +date = 2024-05-10 +type = "docs" +weight = 50 +url = "/en/docs/user-guide/how_to_use_gang_plugin/" +[menu.docs] + parent = "user-guide" ++++ + +## Introduction + +The Gang scheduling algorithm is one of the core scheduling plugins in Volcano. It meets the "All or nothing" scheduling requirements. This means that a group of pods (usually within a Job) will only be scheduled if the minimum required number of pods can be scheduled together. If the cluster does not have enough resources to satisfy the minimum number of running pods, none of them will be scheduled. This prevents resource deadlock where multiple jobs hold partial resources but none can complete. + +## Environment setup + +### Update scheduler configmap + +To use the Gang plugin, ensure that the `gang` plugin is enabled in your Volcano Scheduler configuration (`volcano-scheduler-configmap`). + +```yaml +kind: ConfigMap +apiVersion: v1 +metadata: + name: volcano-scheduler-configmap + namespace: volcano-system +data: + volcano-scheduler.conf: | + actions: "enqueue, allocate, backfill" + tiers: + - plugins: + - name: priority + - name: gang + enablePreemptable: false + - name: conformance +``` + +## How to use Gang Plugin + +When you submit a `vcjob` (Volcano Job) or use a `PodGroup` with regular Pods, you can define the `minAvailable` field. The Gang plugin will check if the cluster has enough resources to satisfy this number of pods. + +### Example 1: Using Volcano Job (vcjob) + +Create a file named `gang-job.yaml` with the following content: + +```yaml +apiVersion: batch.volcano.sh/v1alpha1 +kind: Job +metadata: + name: gang-job +spec: + minAvailable: 3 + schedulerName: volcano + policies: + - event: PodEvicted + action: RestartJob + tasks: + - replicas: 3 + name: worker + template: + spec: + containers: + - image: nginx + name: nginx + resources: + requests: + cpu: "1" + memory: "1Gi" + restartPolicy: OnFailure +``` + +In this example, `minAvailable` is set to 3. The Gang plugin ensures that either all 3 workers are scheduled simultaneously, or none are. + +### Example 2: Using PodGroup for generic Pods + +You can also use the Gang plugin with native Kubernetes Pods by creating a `PodGroup`. + +Create a file named `gang-podgroup.yaml`: + +```yaml +apiVersion: scheduling.volcano.sh/v1beta1 +kind: PodGroup +metadata: + name: gang-pg +spec: + minMember: 3 + queue: default +--- +apiVersion: v1 +kind: Pod +metadata: + name: gang-pod-1 + labels: + scheduling.volcano.sh/pod-group-name: gang-pg +spec: + schedulerName: volcano + containers: + - name: nginx + image: nginx + resources: + requests: + cpu: "1" +--- +apiVersion: v1 +kind: Pod +metadata: + name: gang-pod-2 + labels: + scheduling.volcano.sh/pod-group-name: gang-pg +spec: + schedulerName: volcano + containers: + - name: nginx + image: nginx + resources: + requests: + cpu: "1" +``` + +In this example, only 2 pods are submitted for a `PodGroup` that requires `minMember: 3`. The pods will remain in the `Pending` state until a 3rd pod belonging to `gang-pg` is created and enough resources are available. + +## Verification + +1. Apply the job to your cluster: + ```bash + kubectl apply -f gang-job.yaml + ``` + +2. Check the status of the pods: + ```bash + kubectl get pods + ``` + If the cluster has at least 3 CPUs and 3Gi of memory available, all 3 pods will transition to `Running`. If not, they will all remain in `Pending` due to the Gang scheduling constraint. diff --git a/content/en/docs/user_guide_how_to_use_nodeorder_plugin.md b/content/en/docs/user_guide_how_to_use_nodeorder_plugin.md new file mode 100644 index 00000000..bbe28c25 --- /dev/null +++ b/content/en/docs/user_guide_how_to_use_nodeorder_plugin.md @@ -0,0 +1,96 @@ ++++ +title = "NodeOrder Plugin User Guide" +date = 2024-05-10 +type = "docs" +weight = 50 +url = "/en/docs/user-guide/how_to_use_nodeorder_plugin/" +[menu.docs] + parent = "user-guide" ++++ + +## Introduction + +The NodeOrder plugin is responsible for scoring nodes during the scheduling process to find the absolute best node for a pod. Rather than just finding *any* node that fits the pod's requirements, NodeOrder ranks the nodes based on configurable scoring dimensions, such as affinity rules or resource availability. + +## Environment setup + +### Update scheduler configmap + +The `nodeorder` plugin must be enabled in the scheduler configuration. + +```yaml +kind: ConfigMap +apiVersion: v1 +metadata: + name: volcano-scheduler-configmap + namespace: volcano-system +data: + volcano-scheduler.conf: | + actions: "enqueue, allocate, backfill" + tiers: + - plugins: + - name: priority + - name: nodeorder + - name: gang +``` + +## How to use NodeOrder Plugin + +The NodeOrder plugin scores nodes automatically based on built-in Kubernetes concepts like `NodeAffinity`, `PodAffinity`, and `PodAntiAffinity`. + +### Example YAML + +Create a file named `nodeorder-job.yaml`: + +```yaml +apiVersion: batch.volcano.sh/v1alpha1 +kind: Job +metadata: + name: nodeorder-job +spec: + minAvailable: 1 + schedulerName: volcano + tasks: + - replicas: 3 + name: worker + template: + spec: + affinity: + nodeAffinity: + preferredDuringSchedulingIgnoredDuringExecution: + - weight: 100 + preference: + matchExpressions: + - key: disktype + operator: In + values: + - ssd + containers: + - image: nginx + name: nginx + resources: + requests: + cpu: "1" + restartPolicy: OnFailure +``` + +In this example, the `nodeAffinity` specifies a `preferredDuringSchedulingIgnoredDuringExecution` rule. The NodeOrder plugin takes this preference into account. It will give a higher score (boosted by the weight of 100) to any node that has the label `disktype=ssd`. + +## Verification + +1. Label one of your nodes with `disktype=ssd`: + ```bash + kubectl label nodes disktype=ssd + ``` + +2. Apply the job: + ```bash + kubectl apply -f nodeorder-job.yaml + ``` + +3. Check where the pods were scheduled: + ```bash + kubectl get pods -o wide | grep nodeorder-job + ``` + +4. You will observe that the Volcano scheduler strongly prefers to place the pods on the node labeled with `disktype=ssd` due to the high score calculated by the NodeOrder plugin. diff --git a/content/en/docs/user_guide_how_to_use_predicates_plugin.md b/content/en/docs/user_guide_how_to_use_predicates_plugin.md new file mode 100644 index 00000000..5234eb11 --- /dev/null +++ b/content/en/docs/user_guide_how_to_use_predicates_plugin.md @@ -0,0 +1,90 @@ ++++ +title = "Predicates Plugin User Guide" +date = 2024-05-10 +type = "docs" +weight = 50 +url = "/en/docs/user-guide/how_to_use_predicates_plugin/" +[menu.docs] + parent = "user-guide" ++++ + +## Introduction + +The Predicates plugin is the primary filtering engine for Volcano. It evaluates each node against the requirements of a pod to filter out nodes that are fundamentally incapable of running the pod. This includes checking if a node has enough CPU/Memory available, evaluating node selectors, matching node affinities, and checking if the node has the requested GPUs. + +## Environment setup + +### Update scheduler configmap + +The `predicates` plugin is a fundamental plugin and should almost always be enabled in your `volcano-scheduler-configmap`. + +```yaml +kind: ConfigMap +apiVersion: v1 +metadata: + name: volcano-scheduler-configmap + namespace: volcano-system +data: + volcano-scheduler.conf: | + actions: "enqueue, allocate, backfill" + tiers: + - plugins: + - name: priority + - name: predicates + - name: gang +``` + +## How to use Predicates Plugin + +You do not need to do anything special to "trigger" the Predicates plugin. It automatically evaluates standard Kubernetes pod constraints. + +### Example YAML (Filtering by NodeSelector) + +Create a file named `predicates-job.yaml`: + +```yaml +apiVersion: batch.volcano.sh/v1alpha1 +kind: Job +metadata: + name: predicates-job +spec: + minAvailable: 1 + schedulerName: volcano + tasks: + - replicas: 1 + name: worker + template: + spec: + nodeSelector: + accelerator: nvidia-tesla-v100 + containers: + - image: nginx + name: nginx + resources: + requests: + cpu: "1" + restartPolicy: OnFailure +``` + +In this example, the `nodeSelector` requires a node with the label `accelerator=nvidia-tesla-v100`. The Predicates plugin will filter out all nodes in the cluster that do not have this exact label. + +## Verification + +1. Apply the job: + ```bash + kubectl apply -f predicates-job.yaml + ``` + +2. If you do not have any nodes labeled with `accelerator=nvidia-tesla-v100`, the pod will remain in the `Pending` state. + +3. Describe the pod to see the Predicates plugin at work: + ```bash + kubectl describe pod -l volcano.sh/job-name=predicates-job + ``` + You will see an event from the Volcano scheduler stating that 0 nodes were available because none matched the node selector. + +4. Label a node to satisfy the predicate: + ```bash + kubectl label nodes accelerator=nvidia-tesla-v100 + ``` + The Predicates plugin will now allow this node to pass the filter, and the pod will be scheduled. diff --git a/content/en/docs/user_guide_how_to_use_priority_plugin.md b/content/en/docs/user_guide_how_to_use_priority_plugin.md new file mode 100644 index 00000000..4eca32db --- /dev/null +++ b/content/en/docs/user_guide_how_to_use_priority_plugin.md @@ -0,0 +1,129 @@ ++++ +title = "Priority Plugin User Guide" +date = 2024-05-10 +type = "docs" +weight = 50 +url = "/en/docs/user-guide/how_to_use_priority_plugin/" +[menu.docs] + parent = "user-guide" ++++ + +## Introduction + +The Priority plugin allows Volcano to schedule Pods and Jobs based on their assigned priority. It introduces preemptive capabilities: if a high-priority Job is submitted and the cluster lacks sufficient resources, Volcano will preempt (evict) lower-priority Jobs to free up resources for the high-priority Job. This is essential for ensuring that critical workloads always run. + +## Environment setup + +### Update scheduler configmap + +1. Ensure the `priority` plugin is enabled in your `volcano-scheduler-configmap`. +2. Ensure the `preempt` and `reclaim` actions are enabled if you want high-priority jobs to evict lower-priority ones. + +```yaml +kind: ConfigMap +apiVersion: v1 +metadata: + name: volcano-scheduler-configmap + namespace: volcano-system +data: + volcano-scheduler.conf: | + actions: "enqueue, allocate, preempt, reclaim, backfill" + tiers: + - plugins: + - name: priority + - name: gang +``` + +## How to use Priority Plugin + +To use the Priority plugin, you must first create a `PriorityClass` object in Kubernetes, and then assign that `PriorityClass` to your `vcjob` or Pods. + +### Step 1: Create Priority Classes + +Create a file named `priority-classes.yaml`: + +```yaml +apiVersion: scheduling.k8s.io/v1 +kind: PriorityClass +metadata: + name: high-priority +value: 1000000 +globalDefault: false +description: "This priority class should be used for critical service pods only." +--- +apiVersion: scheduling.k8s.io/v1 +kind: PriorityClass +metadata: + name: low-priority +value: 1000 +globalDefault: false +description: "This priority class should be used for background data processing." +``` + +### Step 2: Create Jobs with Priorities + +Create a file named `priority-jobs.yaml`: + +```yaml +apiVersion: batch.volcano.sh/v1alpha1 +kind: Job +metadata: + name: low-priority-job +spec: + minAvailable: 1 + schedulerName: volcano + priorityClassName: low-priority + tasks: + - replicas: 5 + name: worker + template: + spec: + containers: + - image: nginx + name: nginx + resources: + requests: + cpu: "2" + restartPolicy: OnFailure +--- +apiVersion: batch.volcano.sh/v1alpha1 +kind: Job +metadata: + name: high-priority-job +spec: + minAvailable: 1 + schedulerName: volcano + priorityClassName: high-priority + tasks: + - replicas: 2 + name: worker + template: + spec: + containers: + - image: nginx + name: nginx + resources: + requests: + cpu: "2" + restartPolicy: OnFailure +``` + +## Verification + +1. Assume your cluster has exactly **10 CPUs** available. +2. Apply the PriorityClasses: + ```bash + kubectl apply -f priority-classes.yaml + ``` +3. Apply *only* the low priority job first: + ```bash + # Extract and apply the low-priority-job from priority-jobs.yaml + kubectl apply -f priority-jobs.yaml + ``` +4. Wait for the `low-priority-job` pods to start running. They will consume all 10 CPUs (5 replicas * 2 CPUs). +5. Now, monitor the pods: + ```bash + kubectl get pods -w + ``` +6. The `high-priority-job` is applied. Because the cluster is full, Volcano will notice the `high-priority-job` is waiting and will trigger the `preempt` action. +7. You will observe that two `low-priority-job` pods are evicted (terminated) to free up 4 CPUs. Immediately after, the two `high-priority-job` pods will be scheduled and transition to `Running`. diff --git a/content/en/docs/user_guide_how_to_use_proportion_plugin.md b/content/en/docs/user_guide_how_to_use_proportion_plugin.md new file mode 100644 index 00000000..78cddc51 --- /dev/null +++ b/content/en/docs/user_guide_how_to_use_proportion_plugin.md @@ -0,0 +1,131 @@ ++++ +title = "Proportion Plugin User Guide" +date = 2024-05-10 +type = "docs" +weight = 50 +url = "/en/docs/user-guide/how_to_use_proportion_plugin/" +[menu.docs] + parent = "user-guide" ++++ + +## Introduction + +The Proportion scheduling plugin ensures fair resource sharing across different queues based on their configured weights. It dynamically controls the maximum resources a queue can consume, preventing a single team or workload from starving others in a multi-tenant cluster. When the cluster is under heavy load, the plugin allocates resources proportionally to each queue's weight. + +## Environment setup + +### Update scheduler configmap + +1. Ensure the `proportion` plugin is enabled in the `volcano-scheduler-configmap`. +2. Define multiple `Queue` resources with different weights. + +```yaml +kind: ConfigMap +apiVersion: v1 +metadata: + name: volcano-scheduler-configmap + namespace: volcano-system +data: + volcano-scheduler.conf: | + actions: "enqueue, allocate, backfill" + tiers: + - plugins: + - name: priority + - name: proportion + - name: gang +``` + +## How to use Proportion Plugin + +First, create the Queues with specific weights. The weight represents the relative share of the cluster's resources that the queue is entitled to. + +### Step 1: Create Queues + +Create a file named `queues.yaml`: + +```yaml +apiVersion: scheduling.volcano.sh/v1beta1 +kind: Queue +metadata: + name: team-a-queue +spec: + weight: 2 + capability: + cpu: "10" + memory: "20Gi" +--- +apiVersion: scheduling.volcano.sh/v1beta1 +kind: Queue +metadata: + name: team-b-queue +spec: + weight: 1 + capability: + cpu: "10" + memory: "20Gi" +``` +In this scenario, `team-a-queue` is entitled to twice as many resources as `team-b-queue` when there is resource contention. + +### Step 2: Submit Jobs to the Queues + +Create a file named `proportion-jobs.yaml`: + +```yaml +apiVersion: batch.volcano.sh/v1alpha1 +kind: Job +metadata: + name: job-team-a +spec: + queue: team-a-queue + schedulerName: volcano + tasks: + - replicas: 10 + name: worker + template: + spec: + containers: + - image: nginx + name: nginx + resources: + requests: + cpu: "1" + restartPolicy: OnFailure +--- +apiVersion: batch.volcano.sh/v1alpha1 +kind: Job +metadata: + name: job-team-b +spec: + queue: team-b-queue + schedulerName: volcano + tasks: + - replicas: 10 + name: worker + template: + spec: + containers: + - image: nginx + name: nginx + resources: + requests: + cpu: "1" + restartPolicy: OnFailure +``` + +## Verification + +1. Apply the Queues and the Jobs: + ```bash + kubectl apply -f queues.yaml + kubectl apply -f proportion-jobs.yaml + ``` + +2. If your cluster has exactly 9 available CPUs, Volcano will schedule pods proportionally according to the queue weights (2:1). + - `team-a-queue` (Weight 2) will get roughly 6 pods running. + - `team-b-queue` (Weight 1) will get roughly 3 pods running. + +3. You can verify this by checking the number of running pods for each job: + ```bash + kubectl get pods -l volcano.sh/job-name=job-team-a + kubectl get pods -l volcano.sh/job-name=job-team-b + ``` diff --git a/content/en/docs/user_guide_how_to_use_sla_plugin.md b/content/en/docs/user_guide_how_to_use_sla_plugin.md new file mode 100644 index 00000000..f83a3045 --- /dev/null +++ b/content/en/docs/user_guide_how_to_use_sla_plugin.md @@ -0,0 +1,81 @@ ++++ +title = "SLA Plugin User Guide" +date = 2024-05-10 +type = "docs" +weight = 50 +url = "/en/docs/user-guide/how_to_use_sla_plugin/" +[menu.docs] + parent = "user-guide" ++++ + +## Introduction + +The SLA (Service Level Agreement) plugin is designed to prevent job starvation. Users can specify the maximum amount of time a job is allowed to wait in the `Pending` state. Once a job reaches its SLA timeout, the SLA plugin will boost its priority to ensure it gets scheduled as soon as possible, potentially preempting other jobs if necessary. + +## Environment setup + +### Update scheduler configmap + +Enable the `sla` plugin in the scheduler configuration. Ensure `preempt` is in the action list so the SLA plugin can forcibly make room for starved jobs. + +```yaml +kind: ConfigMap +apiVersion: v1 +metadata: + name: volcano-scheduler-configmap + namespace: volcano-system +data: + volcano-scheduler.conf: | + actions: "enqueue, allocate, preempt, backfill" + tiers: + - plugins: + - name: priority + - name: sla + - name: gang +``` + +## How to use SLA Plugin + +You can define the SLA requirements directly on the Volcano `Job` using annotations. + +### Example YAML + +Create a file named `sla-job.yaml`: + +```yaml +apiVersion: batch.volcano.sh/v1alpha1 +kind: Job +metadata: + name: sla-job + annotations: + scheduling.volcano.sh/jobWaitingTime: "5m" +spec: + minAvailable: 1 + schedulerName: volcano + tasks: + - replicas: 1 + name: worker + template: + spec: + containers: + - image: nginx + name: nginx + resources: + requests: + cpu: "2" + restartPolicy: OnFailure +``` + +In this configuration, the annotation `scheduling.volcano.sh/jobWaitingTime: "5m"` tells the SLA plugin that this job should not wait in the queue for more than 5 minutes. + +## Verification + +1. Fill up your cluster with low-priority, long-running jobs so there are no free resources. +2. Submit the `sla-job.yaml`. +3. Monitor the job: + ```bash + kubectl get vcjob sla-job + ``` +4. Initially, the job will be in the `Pending` state because the cluster is full. +5. Wait for 5 minutes. +6. Once the 5-minute `jobWaitingTime` is breached, the SLA plugin kicks in. It will boost the `sla-job`'s priority. The scheduler will then preempt (evict) a running pod from another job to free up resources, allowing `sla-job` to finally transition to `Running`. diff --git a/content/en/docs/user_guide_how_to_use_tdm_plugin.md b/content/en/docs/user_guide_how_to_use_tdm_plugin.md new file mode 100644 index 00000000..bdb22d69 --- /dev/null +++ b/content/en/docs/user_guide_how_to_use_tdm_plugin.md @@ -0,0 +1,89 @@ ++++ +title = "TDM Plugin User Guide" +date = 2024-05-10 +type = "docs" +weight = 50 +url = "/en/docs/user-guide/how_to_use_tdm_plugin/" +[menu.docs] + parent = "user-guide" ++++ + +## Introduction + +TDM (Time Division Multiplexing) is a plugin designed for co-located environments where a single set of nodes is shared across different orchestration frameworks (like Kubernetes and Hadoop YARN). The TDM plugin enables time-sharing by marking certain nodes as "revocable". Preemptable workloads can be scheduled on these nodes during specific time windows. When the time window ends, Volcano evicts these workloads to return the resources to the primary framework. + +## Environment setup + +### Update scheduler configmap + +1. Enable the `tdm` plugin in the `volcano-scheduler-configmap`. +2. Ensure you have nodes that you intend to share. + +```yaml +kind: ConfigMap +apiVersion: v1 +metadata: + name: volcano-scheduler-configmap + namespace: volcano-system +data: + volcano-scheduler.conf: | + actions: "enqueue, allocate, backfill" + tiers: + - plugins: + - name: priority + - name: tdm + - name: gang +``` + +## How to use TDM Plugin + +You must configure the nodes and the workloads to interact with the TDM plugin. + +### Step 1: Label the Nodes + +Label the nodes that you want to share with the TDM revocable label: +```bash +kubectl label nodes volcano.sh/revocable-zone=true +``` + +You must also annotate the node with the specific time window during which Volcano is allowed to use it. For example, to allow Volcano to use the node between 00:00 (midnight) and 08:00 AM: +```bash +kubectl annotate nodes volcano.sh/revocable-zone.time-window="00:00-08:00" +``` + +### Step 2: Create a Preemptable Job + +Create a file named `tdm-job.yaml`: + +```yaml +apiVersion: batch.volcano.sh/v1alpha1 +kind: Job +metadata: + name: tdm-batch-job + annotations: + volcano.sh/preemptable: "true" +spec: + minAvailable: 1 + schedulerName: volcano + tasks: + - replicas: 2 + name: worker + template: + spec: + containers: + - image: nginx + name: nginx + restartPolicy: OnFailure +``` + +The annotation `volcano.sh/preemptable: "true"` signals to the TDM plugin that this workload is non-critical and can be scheduled on revocable nodes. + +## Verification + +1. Ensure the current system time is *within* the `time-window` you annotated on the node. +2. Apply the job: + ```bash + kubectl apply -f tdm-job.yaml + ``` +3. The pods will be successfully scheduled onto the revocable node. +4. **The Eviction Phase**: When the system clock passes the end of the time window (e.g., it turns 08:01 AM), the TDM plugin will actively evict the pods running on that node, returning the resources to the cluster so the other framework (e.g., YARN) can use them.