Skip to content

Commit ce42e81

Browse files
author
Doug Clayton
committed
Update cluster-init docs and separate out Chef documentation
1 parent 0fbc3c9 commit ce42e81

8 files changed

Lines changed: 154 additions & 180 deletions

File tree

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,87 @@
1+
---
2+
title: Chef Orchestration in Azure CycleCloud
3+
description: Learn about Chef orchestration in Azure CycleCloud. Chef is a configuration management tool that automates the deployment and management of applications and infrastructure.
4+
author: doclayto
5+
ms.date: 09/23/2025
6+
ms.author: dougclayton
7+
---
8+
9+
# Chef Orchestration
10+
11+
[Chef](https://www.chef.io) is a configuration management tool that automates the deployment and management of applications and infrastructure. It uses a domain-specific language (DSL) for writing system configuration "recipes". These recipes are stored in "cookbooks" that can be shared and reused across different systems.
12+
13+
Each cluster-init spec can reference one or more [Chef roles](https://docs.chef.io/roles.html) and/or [cookbook recipes](https://docs.chef.io/recipes.html) that need to be executed on the booting VM.
14+
15+
[!NOTE]
16+
> CycleCloud uses Chef in a stand-alone mode that doesn't rely on a centralized Chef server. The set of Chef cookbooks needed to prepare each VM are downloaded from the Azure Storage Account along with the rest of the files in the project.
17+
18+
## Project Structure
19+
20+
The **specs** directory in a project can also contain a **chef** subdirectory.
21+
This directory contains three directories: **site-cookbooks** (for cookbook definitions), **data_bags** (data bag definitions), and **roles** (Chef role definition files):
22+
23+
``` directories
24+
myproject
25+
├── specs
26+
│ ├── default
27+
│ └── chef
28+
│ ├── site-cookbooks
29+
│ ├── data_bags
30+
│ └── roles
31+
```
32+
33+
## Chef Runlists
34+
35+
You can specify a run_list at the project or spec level within your _project.ini_:
36+
37+
``` ini
38+
[spec scheduler]
39+
run_list = role[a], recipe[b]
40+
```
41+
42+
When a node includes the `scheduler` spec, the run_list you define automatically appends to any previously defined run_list. For example, if the run_list under `[configuration]` is `run_list = recipe[test]`, the final run_list is `run_list = recipe[cyclecloud], recipe[test], role[a], recipe[b], recipe[cluster_init]`.
43+
44+
You can also overwrite `run_list` at the spec level on a node. This replacement `run_list` removes any `run_list` included in **_project.ini_**. For example, you can change the node definition to the following definition:
45+
46+
``` ini
47+
[cluster-init test-project:scheduler:1.0.0]
48+
run_list = recipe[different-test]
49+
```
50+
51+
Using this run_list causes the run_list defined in the project to be ignored. The final run_list on the node becomes `run_list = recipe[cyclecloud], recipe[test], recipe[different-test], recipe[cluster_init]`.
52+
53+
## Custom Chef and composable specs
54+
55+
Each spec can contain a **chef** directory. Before you converge a node, the process untars and extracts each spec into the local chef-repo, replacing any existing cookbooks, roles, and data bags with matching names. The process follows the order in which you define the specs, so the last defined spec wins if there's a naming conflict.
56+
57+
## File locations
58+
59+
When you upload your project contents, CycleCloud zips the three custom Chef directories and syncs them to your target locker, at `(locker)/projects/(project)/(version)/(spec_name)/chef`.
60+
61+
62+
The node downloads the zipped Chef files during the bootstrapping phase of node startup. The node downloads the files to `$JETPACK_HOME/system/chef/tarballs*`, unzips them to `$JETPACK_HOME/system/chef/chef-repo/`, and uses them to converge the node.
63+
64+
> [!NOTE]
65+
> To run custom cookbooks, you must add them to the run_list for the node.
66+
67+
68+
## jetpack download
69+
70+
The following example shows how to create a `jetpack_download` lightweight resource provider for Chef users:
71+
72+
``` ini
73+
jetpack_download "big-file1.tgz" do
74+
project "my-project"
75+
end
76+
```
77+
78+
In Chef, the default download location is `#{node[:jetpack][:downloads]}`. To change the file destination, use the following code:
79+
80+
``` ini
81+
jetpack_download "foo.tgz" do
82+
project "my-project"
83+
dest "/tmp/download.tgz"
84+
end
85+
```
86+
87+
When you use the command within Chef, you must specify the project.

articles/cyclecloud/cluster-references/cluster-init-reference.md

Lines changed: 29 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -10,9 +10,24 @@ ms.custom: compute-evergreen
1010

1111
# Cluster-Init
1212

13-
Cluster-init objects are subordinate to `node` and `nodearray` objects. The cluster-init object defines the [CycleCloud project](~/articles/cyclecloud/how-to/projects.md) specs to run on a node.
13+
Cluster-init sections are subordinate to `node` and `nodearray` sections.
14+
The `[[[cluster-init]]]` section defines the [CycleCloud project](~/articles/cyclecloud/how-to/projects.md) specs to run on a node.
15+
The section uses a shorthand notation to reference the fully qualified spec:
1416

15-
When you add a `[[[cluster-init]]]` section to a node, you include a project spec. You can also use shorthand notation to define cluster-init:
17+
```ini
18+
[[[cluster-init PROJECT:SPEC:VERSION]]]
19+
```
20+
21+
By default, projects are stored in the [locker](../how-to/projects.md#lockers) defined on the node. To specify a different locker, you can include a locker reference in the `[[[cluster-init]]]` section:
22+
23+
```ini
24+
[[[cluster-init LOCKER/PROJECT:SPEC:VERSION]]]
25+
```
26+
27+
[!NOTE]
28+
> For built-in projects that are downloaded from GitHub, use `cyclecloud` as the locker. This will tell CycleCloud to download the project from GitHub and upload them to your locker in a special cache area. Without `cyclecloud/` in the cluster-init reference, CycleCloud will expect the project to be uploaded by you.
29+
30+
As an example, this cluster template defines one node that uses three specs:
1631

1732
``` ini
1833
[cluster my-cluster]
@@ -26,19 +41,23 @@ When you add a `[[[cluster-init]]]` section to a node, you include a project spe
2641
MachineType = $MachineType
2742
ImageName = $ImageName
2843

29-
[[[cluster-init myspec]]]
30-
Project = myproject
31-
Version = x.y.z
32-
Spec = my-spec
33-
Locker = test-locker
44+
[[[cluster-init test-locker/myproject:my-spec:x.y.z]]]
3445

3546
[[[cluster-init my-proj:my-spec:versionA]]]
36-
3747
```
3848

3949
Attribute values that start with `$` reference parameters.
4050

41-
The CycleCloud project specs run in the order you list them in the Cluster Template File. In this example, `my-proj:default` runs first because it comes from the node defaults. Next, `myproject:x.y.x` runs, and finally, `my-proj:my-spec` runs.
51+
The CycleCloud project specs run in the order you list them in the Cluster Template File. In this example, `my-proj:default` runs first because it comes from the node defaults. Next, `myproject:my-spec` runs, which comes from the locker named `test-locker`. Finally, `my-proj:my-spec` runs.
52+
53+
The `[[[cluster-init LOCKER/PROJECT:SPEC:VERSION]]]` form is a shorthand for the following:
54+
```ini
55+
[[[cluster-init]]]
56+
Project = PROJECT
57+
Version = VERSION
58+
Spec = SPEC
59+
Locker = LOCKER
60+
```
4261

4362
## Attribute reference
4463

@@ -48,5 +67,5 @@ Project | String | Name of the CycleCloud project.
4867
Version | String | Version of the CycleCloud project specification.
4968
Spec | String | Name of the CycleCloud project specification.
5069
Locker | String | Name of the locker to download the project specification from.
70+
Order | Integer | Optional integer that you can use to override the order of the specs. The default starts at 1000 and goes up by one for each spec.
5171

52-
For projects in the CycleCloud project, set the `Locker` attribute to `cyclecloud`.

articles/cyclecloud/concepts/clusters.md

Lines changed: 11 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -38,27 +38,29 @@ CycleCloud provisions VMs from base VM images defined in the cluster template. T
3838

3939
![Node Preparation Diagram](../images/concept-node-prep-diagram.png)
4040

41-
In the configuration section of each node, you define *cluster-init specs*. Booting VMs use these specifications to prepare for a specific role in the cluster. CycleCloud uses [Chef](https://www.chef.io) as the infrastructure automation platform for preparing and configuring each node. Each *cluster-init spec* maps to one or more [Chef Roles](https://docs.chef.io/roles.html) and/or [Cookbook Recipes](https://docs.chef.io/recipes.html) that need to be executed on the booting VM.
41+
You can control how nodes are customized on boot by creating a custom *cluster-init project*. A project contains the scripts and other files needed to customize a node, separated into *specs* for the different kinds of roles in a cluster. For example, a project for a batch scheduler such as Slurm comprises a minimum of three specs: one for the scheduler head nodes, one for the the compute nodes, and another for the login nodes. [Read more about the CycleCloud projects](~/articles/cyclecloud/how-to/projects.md).
4242

43-
CycleCloud uses Chef in a stand-alone mode that doesn't rely on a centralized Chef server. Instead, the entire set of Chef Cookbooks needed to prepare each VM are downloaded from an Azure Storage Account belonging to the user during the VM bootup phase. This set of Cookbooks are cached from the CycleCloud application server into the Storage Account during the cluster creation phase.
43+
In the node definition, you reference the specs that should run on that node. Jetpack uses these specs on boot to prepare a node for its role in the cluster. The spec files come from the user's Blob Storage Account, and are staged from the CycleCloud application server into the storage account before nodes are started.
4444

45-
After you download these cookbooks, Chef processes the list of recipes defined in the node's *cluster-init specs*. It triggers a preparation and configuration phase that converts the VM into a working HPC node.
45+
> [!NOTE]
46+
> The specs for built-in templates (such as the Slurm cluster type) are stored in GitHub, and are automatically downoaded to the user's storage account when the node starts.
4647
47-
You author specs as logical collections called *projects*. For example, a project for a batch scheduler such as Slurm comprises a minimum of two specs: one for the scheduler head nodes and the other for the compute nodes. [Read more about the CycleCloud projects](~/articles/cyclecloud/how-to/projects.md).
48+
When a node boots, Jetpack downloads the specs defined on the node with the `[[[cluster-init]]]` section and processes them in order to *converge* the node to a working state (for instance, to be a compute node).
4849

4950
## Node orchestration
5051

5152
Depending on the scheduler and services used in a cluster, CycleCloud sometimes needs to orchestrate the preparation phase of nodes in a cluster by coordinating different nodes. For example, some schedulers require that each compute node registers itself with the scheduler daemon. This requirement means that the compute nodes must be aware of the address of the head node. The compute nodes must also recognize that the head node is fully prepared and wait if it's not.
5253

5354
CycleCloud uses this element of [Service Discovery](https://en.wikipedia.org/wiki/Service_discovery) for file system server-client relationships.
5455

56+
5557
## More information
5658

57-
* Create a [Cluster Template](~/articles/cyclecloud/how-to/cluster-templates.md)
58-
* [Start a Cluster](~/articles/cyclecloud/how-to/start-cluster.md)
59-
* [Auto Scaling](~/articles/cyclecloud/how-to/configure-autoscaling.md)
60-
* [Terminate a Cluster](~/articles/cyclecloud/how-to/terminate-cluster.md)
61-
* [Node Configuration Reference](~/articles/cyclecloud/cluster-references/configuration-reference.md)
59+
* Create a [Cluster Template](../how-to/cluster-templates.md)
60+
* [Start a Cluster](../how-to/start-cluster.md)
61+
* [Auto Scaling](../how-to/configure-autoscaling.md)
62+
* [Terminate a Cluster](../how-to/terminate-cluster.md)
63+
* [Node Configuration Reference](../cluster-references/configuration-reference.md)
6264

6365
> [!div class="nextstepaction"]
6466
> [Continue to Scheduling Concepts](./scheduling.md)

articles/cyclecloud/how-to/cluster-templates.md

Lines changed: 0 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -261,30 +261,6 @@ The GUI includes the **Label** and **Description** attributes, which appear in t
261261
| Cloud.Credentials | Displays a dropdown containing all of the available credentials. |
262262
| Cloud.Region | Displays a dropdown containing all available regions. |
263263

264-
## Chef Server support
265-
266-
Azure CycleCloud supports [ChefServer](https://docs.chef.io/server_components.html).
267-
268-
Create the file `chefserver.json` and add your credentials. `ValidationKey`
269-
corresponds to the `validation.pem` file for your chef server. You also must provide the
270-
`validation_client_name` if you change it from the default value of `chef-validator`:
271-
272-
``` JSON
273-
{
274-
"AdType" : "Cloud.Locker",
275-
"ValidationKey" : "YOURVALIDATION.PEMHERE",
276-
"ValidationClientName" : "chef-validator",
277-
"Credentials" : "default",
278-
"Location" : "https://mychefserver",
279-
"ChefRepoType" : "chefserver",
280-
"LockerType" : "chefrepo",
281-
"Name" : "chefrepo",
282-
"AccountId" : "default",
283-
"Shared" : false
284-
}
285-
```
286-
287-
Next, place the file in the directory `/opt/cycle_server/config/data`. The server automatically imports the file.
288264

289265
## Custom user images in templates
290266

0 commit comments

Comments
 (0)