You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
description: Learn about Chef orchestration in Azure CycleCloud. Chef is a configuration management tool that automates the deployment and management of applications and infrastructure.
4
+
author: dougclayton
5
+
ms.author: doclayto
6
+
ms.date: 09/23/2025
7
+
ms.custom: compute-evergreen
8
+
---
9
+
10
+
# Chef orchestration
11
+
12
+
[Chef](https://www.chef.io) is a configuration management tool that automates the deployment and management of applications and infrastructure. It uses a domain-specific language (DSL) for writing system configuration "recipes." These recipes are stored in "cookbooks" that can be shared and reused across different systems.
13
+
14
+
Each cluster-init spec can reference one or more [Chef roles](https://docs.chef.io/roles.html) and/or [cookbook recipes](https://docs.chef.io/recipes.html) that need to be executed on the booting VM.
15
+
16
+
> [!NOTE]
17
+
> CycleCloud uses Chef in a stand-alone mode that doesn't rely on a centralized Chef server. The set of Chef cookbooks needed to prepare each VM are downloaded from the Azure Storage Account along with the rest of the files in the project.
18
+
19
+
## Project Structure
20
+
21
+
The **specs** directory in a project can also contain a **chef** subdirectory.
22
+
This directory contains three directories: **site-cookbooks** (for cookbook definitions), **data_bags** (data bag definitions), and **roles** (Chef role definition files):
23
+
24
+
```directories
25
+
myproject
26
+
├── specs
27
+
│ ├── default
28
+
│ └── chef
29
+
│ ├── site-cookbooks
30
+
│ ├── data_bags
31
+
│ └── roles
32
+
```
33
+
34
+
## Chef Runlists
35
+
36
+
You can specify a run_list at the project or spec level within your _project.ini_:
37
+
38
+
```ini
39
+
[spec scheduler]
40
+
run_list = role[a], recipe[b]
41
+
```
42
+
43
+
When a node includes the `scheduler` spec, the run_list you define automatically appends to any previously defined run_list. For example, if the run_list under `[configuration]` is `run_list = recipe[test]`, the final run_list is `run_list = recipe[cyclecloud], recipe[test], role[a], recipe[b], recipe[cluster_init]`.
44
+
45
+
You can also overwrite `run_list` at the spec level on a node. This replacement `run_list` removes any `run_list` included in **_project.ini_**. For example, you can change the node definition to the following definition:
46
+
47
+
```ini
48
+
[cluster-init test-project:scheduler:1.0.0]
49
+
run_list = recipe[different-test]
50
+
```
51
+
52
+
Using this run_list causes the run_list defined in the project to be ignored. The final run_list on the node becomes `run_list = recipe[cyclecloud], recipe[test], recipe[different-test], recipe[cluster_init]`.
53
+
54
+
## Custom Chef and composable specs
55
+
56
+
Each spec can contain a **chef** directory. Before you converge a node, the process untars and extracts each spec into the local chef-repo, replacing any existing cookbooks, roles, and data bags with matching names. The process follows the order in which you define the specs, so the last defined spec wins if there's a naming conflict.
57
+
58
+
## File locations
59
+
60
+
When you upload your project contents, CycleCloud zips the three custom Chef directories and syncs them to your target locker, at `(locker)/projects/(project)/(version)/(spec_name)/chef`.
61
+
62
+
63
+
The node downloads the zipped Chef files during the bootstrapping phase of node startup. The node downloads the files to `$JETPACK_HOME/system/chef/tarballs*`, unzips them to `$JETPACK_HOME/system/chef/chef-repo/`, and uses them to converge the node.
64
+
65
+
> [!NOTE]
66
+
> To run custom cookbooks, you must add them to the run_list for the node.
67
+
68
+
69
+
## Jetpack download resource provider
70
+
71
+
The following example shows how to create a `jetpack_download` lightweight resource provider for Chef users:
72
+
73
+
```ini
74
+
jetpack_download "big-file1.tgz" do
75
+
project "my-project"
76
+
end
77
+
```
78
+
79
+
In Chef, the default download location is `#{node[:jetpack][:downloads]}`. To change the file destination, use the following code:
80
+
81
+
```ini
82
+
jetpack_download "foo.tgz" do
83
+
project "my-project"
84
+
dest "/tmp/download.tgz"
85
+
end
86
+
```
87
+
88
+
When you use the command within Chef, you must specify the project.
description: See a reference for cluster-init objects to be used with Azure CycleCloud. A cluster-init object defines the CycleCloud project specifications to run on a node.
4
-
author: adriankjohnson
5
-
ms.date: 06/29/2025
4
+
author: dougclayton
5
+
ms.author: doclayto
6
+
ms.date: 09/23/2025
6
7
ms.update-cycle: 3650-days
7
-
ms.author: adjohnso
8
8
ms.custom: compute-evergreen
9
9
---
10
10
11
11
# Cluster-Init
12
12
13
-
Cluster-init objects are subordinate to `node` and `nodearray` objects. The cluster-init object defines the [CycleCloud project](~/articles/cyclecloud/how-to/projects.md) specs to run on a node.
13
+
Cluster-init sections are subordinate to `node` and `nodearray` sections.
14
+
The `[[[cluster-init]]]` section defines the [CycleCloud project](~/articles/cyclecloud/how-to/projects.md) specs to run on a node.
15
+
The section uses a shorthand notation to reference the fully qualified spec:
14
16
15
-
When you add a `[[[cluster-init]]]` section to a node, you include a project spec. You can also use shorthand notation to define cluster-init:
17
+
```ini
18
+
[[[cluster-init PROJECT:SPEC:VERSION]]]
19
+
```
20
+
21
+
By default, projects are assumed to be stored in the [locker](../how-to/projects.md#lockers) already. However, if you're using a project defined in GitHub, you can indicate that with the `cyclecloud/` prefix:
This section tells CycleCloud to download the project files from GitHub and upload them to your locker in a special cache area. Without `cyclecloud/` in the cluster-init reference, CycleCloud expects you to upload the project manually.
28
+
29
+
As an example, this cluster template defines one node that uses three specs:
16
30
17
31
```ini
18
32
[cluster my-cluster]
@@ -26,19 +40,27 @@ When you add a `[[[cluster-init]]]` section to a node, you include a project spe
Attribute values that start with `$` reference parameters.
40
49
41
-
The CycleCloud project specs run in the order you list them in the Cluster Template File. In this example, `my-proj:default` runs first because it comes from the node defaults. Next, `myproject:x.y.x` runs, and finally, `my-proj:my-spec` runs.
50
+
The CycleCloud project specs run in the order you list them in the Cluster Template File. In this example, `my-proj:default` runs first because it comes from the node defaults. Next, `myproject:my-spec` runs, which comes from the locker named `test-locker`. Finally, `my-proj:my-spec` runs.
51
+
52
+
The `[[[cluster-init PROJECT:SPEC:VERSION]]]` form is a shorthand for the following section:
53
+
```ini
54
+
[[[cluster-init]]]
55
+
Project = PROJECT
56
+
Version = VERSION
57
+
Spec = SPEC
58
+
```
59
+
60
+
You can also use `[[[cluster-init SOURCE_LOCKER/PROJECT:SPEC:VERSION]]]` to specify a `SourceLocker` for the cluster-init spec. The source locker is optional. Without it, CycleCloud assumes the locker already has the files staged. If set to the special name `cyclecloud`, it uses a built-in project defined in CycleCloud whose contents are stored in GitHub. Otherwise, if set to a different locker, it stages the files from that locker to the target locker before starting the node. This feature is useful for custom cluster-init projects and multi-region deployments. You manually stage the files to a single locker, and CycleCloud uses that locker as a source locker for nodes in other regions.
61
+
62
+
> [!NOTE]
63
+
> Projects that are staged automatically are put in a special cache directory of the target locker so that they don't conflict with projects you stage manually.
42
64
43
65
## Attribute reference
44
66
@@ -48,5 +70,6 @@ Project | String | Name of the CycleCloud project.
48
70
Version | String | Version of the CycleCloud project specification.
49
71
Spec | String | Name of the CycleCloud project specification.
50
72
Locker | String | Name of the locker to download the project specification from.
73
+
SourceLocker | String | Optional. Name of another locker that should be used to stage files from. If set to the special name _cyclecloud_, it uses a built-in project defined in CycleCloud whose contents are stored in GitHub.
74
+
Order | Integer | Optional integer that you can use to override the order of the specs. The default starts at 1000 and goes up by one for each spec.
51
75
52
-
For projects in the CycleCloud project, set the `Locker` attribute to `cyclecloud`.
In the configuration section of each node, you define *cluster-init specs*. Booting VMs use these specifications to prepare for a specific role in the cluster. CycleCloud uses [Chef](https://www.chef.io) as the infrastructure automation platform for preparing and configuring each node. Each *cluster-init spec* maps to one or more [Chef Roles](https://docs.chef.io/roles.html) and/or [Cookbook Recipes](https://docs.chef.io/recipes.html) that need to be executed on the booting VM.
41
+
You can control how nodes are customized on boot by creating a custom *cluster-init project*. A project contains the scripts and other files needed to customize a node, separated into *specs*for the different kinds of roles in a cluster. For example, a project for a batch scheduler such as Slurm comprises a minimum of three specs: one for the scheduler head nodes, one for the compute nodes, and another for the login nodes. [Read more about the CycleCloud projects](~/articles/cyclecloud/how-to/projects.md).
42
42
43
-
CycleCloud uses Chef in a stand-alone mode that doesn't rely on a centralized Chef server. Instead, the entire set of Chef Cookbooks needed to prepare each VM are downloaded from an Azure Storage Account belonging to the user during the VM bootup phase. This set of Cookbooks are cached from the CycleCloud application server into the Storage Account during the cluster creation phase.
43
+
In the node definition, you reference the specs that should run on that node. Jetpack uses these specs on boot to prepare a node for its role in the cluster. The spec files come from the user's Blob Storage Account, and are staged from the CycleCloud application server into the storage account before nodes are started.
44
44
45
-
After you download these cookbooks, Chef processes the list of recipes defined in the node's *cluster-init specs*. It triggers a preparation and configuration phase that converts the VM into a working HPC node.
45
+
> [!NOTE]
46
+
> The specs for built-in templates (such as the Slurm cluster type) are stored in GitHub. CycleCloud automatically downloads them to the user's storage account when the node starts.
46
47
47
-
You author specs as logical collections called *projects*. For example, a project for a batch scheduler such as Slurm comprises a minimum of two specs: one for the scheduler head nodes and the other for the compute nodes. [Read more about the CycleCloud projects](~/articles/cyclecloud/how-to/projects.md).
48
+
When a node boots, Jetpack downloads the specs defined on the node with the `[[[cluster-init]]]` section and processes them in order to *converge*the node to a working state (for instance, to be a compute node).
48
49
49
50
## Node orchestration
50
51
51
52
Depending on the scheduler and services used in a cluster, CycleCloud sometimes needs to orchestrate the preparation phase of nodes in a cluster by coordinating different nodes. For example, some schedulers require that each compute node registers itself with the scheduler daemon. This requirement means that the compute nodes must be aware of the address of the head node. The compute nodes must also recognize that the head node is fully prepared and wait if it's not.
52
53
53
54
CycleCloud uses this element of [Service Discovery](https://en.wikipedia.org/wiki/Service_discovery) for file system server-client relationships.
54
55
56
+
55
57
## More information
56
58
57
-
* Create a [Cluster Template](~/articles/cyclecloud/how-to/cluster-templates.md)
58
-
*[Start a Cluster](~/articles/cyclecloud/how-to/start-cluster.md)
Copy file name to clipboardExpand all lines: articles/cyclecloud/how-to/cluster-templates.md
+3-27Lines changed: 3 additions & 27 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -1,10 +1,10 @@
1
1
---
2
2
title: Cluster Templates
3
3
description: Use or build cluster templates within Azure CycleCloud. See configuration notation, cluster template parameters, machine types, spot virtual machines, and more.
4
-
author: adriankjohnson
5
-
ms.date: 06/30/2025
4
+
author: dougclayton
5
+
ms.author: doclayto
6
+
ms.date: 09/23/2025
6
7
ms.update-cycle: 3650-days
7
-
ms.author: adjohnso
8
8
ms.topic: how-to
9
9
ms.service: azure-cyclecloud
10
10
ms.custom: compute-evergreen
@@ -261,30 +261,6 @@ The GUI includes the **Label** and **Description** attributes, which appear in t
261
261
| Cloud.Credentials | Displays a dropdown containing all of the available credentials. |
262
262
| Cloud.Region | Displays a dropdown containing all available regions. |
0 commit comments