Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions docs/UserGuide/user_guide_how_to_use_nodegroup_plugin.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,47 @@ metadata:
- <groupname>
```

### Configure queue resource limits for each nodegroup

Use the `volcano.sh/nodegroup-resource-limits` annotation to limit the amount of resources a queue can allocate from each nodegroup.

```yaml
apiVersion: scheduling.volcano.sh/v1beta1
kind: Queue
metadata:
name: default
annotations:
volcano.sh/nodegroup-resource-limits: |
{
"groupname1": {"cpu": "100", "memory": "200Gi"},
"groupname2": {"cpu": "50"}
}
spec:
reclaimable: true
weight: 1
affinity:
nodeGroupAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- groupname1
- groupname2
preferredDuringSchedulingIgnoredDuringExecution:
- groupname1
```

For a task from queue `Q` scheduled to a node in nodegroup `G`, the plugin checks:

```text
allocated(Q, G) + task request <= limit(Q, G)
```

If the resulting resource usage exceeds the configured limit, the node is rejected for that task. Other allowed nodegroups remain schedulable, so the queue can prefer `groupname1` and overflow to `groupname2` when the logical limit for `groupname1` is reached.

Only resource dimensions explicitly listed for a nodegroup are limited. For example, `{"groupname2": {"cpu": "50"}}` limits CPU usage in `groupname2` and does not add a memory limit for that nodegroup.

If this annotation is absent, existing nodegroup behavior is unchanged. If a nodegroup is not listed in the annotation, that nodegroup has no extra limit from this feature.

The annotation is not inherited by child queues when hierarchical nodegroup affinity is enabled. Each queue that needs per-nodegroup limits must define its own annotation.

### submit a vcjob

submit vcjob job-1 to default queue.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,47 @@ metadata:
- <groupname>
```

### 为每个 NodeGroup 配置 Queue 资源上限

使用 `volcano.sh/nodegroup-resource-limits` 注解限制 Queue 最多可从各个 NodeGroup 分配的资源量。

```yaml
apiVersion: scheduling.volcano.sh/v1beta1
kind: Queue
metadata:
name: default
annotations:
volcano.sh/nodegroup-resource-limits: |
{
"groupname1": {"cpu": "100", "memory": "200Gi"},
"groupname2": {"cpu": "50"}
}
spec:
reclaimable: true
weight: 1
affinity:
nodeGroupAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- groupname1
- groupname2
preferredDuringSchedulingIgnoredDuringExecution:
- groupname1
```

当 Queue `Q` 中的任务要调度到 NodeGroup `G` 中的节点时,插件会进行以下检查:

```text
allocated(Q, G) + task request <= limit(Q, G)
```

如果调度后的累计资源用量超过配置的上限,插件会拒绝将该任务调度到此节点。其他允许使用的 NodeGroup 仍可参与调度,因此 Queue 可以优先使用 `groupname1`,并在达到 `groupname1` 的逻辑资源上限后将任务调度到 `groupname2`。

只有为 NodeGroup 显式列出的资源维度会受到限制。例如,`{"groupname2": {"cpu": "50"}}` 仅限制 `groupname2` 中的 CPU 用量,不会为该 NodeGroup 添加内存上限。

如果未配置此注解,NodeGroup 的现有行为保持不变。如果注解中未列出某个 NodeGroup,此功能不会对该 NodeGroup 添加额外限制。

启用层级 NodeGroup 亲和性时,子 Queue 不会继承此注解。每个需要 NodeGroup 资源上限的 Queue 都必须单独定义该注解。

### 提交 Volcano Job

将名为 job-1 的 Volcano Job 提交到 default Queue。
Expand Down