Shell provisioner with cloud-init fails on Ubuntu 24.04

[psolymos]

The first shell provisioner fails with cloud-init status --wait on ubuntu-24-04-x64 image, but runs fine with the ubuntu-22-04-x64 image.

==> digitalocean: Creating temporary RSA SSH key for instance...
==> digitalocean: Importing SSH public key...
==> digitalocean: Creating droplet...
==> digitalocean: Waiting for droplet to become active...
==> digitalocean: Using SSH communicator to connect: 147.182.150.138
==> digitalocean: Waiting for SSH to become available...
==> digitalocean: Connected to SSH!
==> digitalocean: Provisioning with shell script: /var/folders/wm/3rsg6gxd2r16r5thmb03nt7w0000gn/T/packer-shell2558422215
    digitalocean: ....................................................................................................................
    digitalocean: status: done
==> digitalocean: Provisioning step had errors: Running the cleanup provisioner, if present...
==> digitalocean: Destroying droplet...
==> digitalocean: Deleting temporary ssh key...
Build 'digitalocean' errored after 1 minute 9 seconds: Script exited with non-zero exit status: 2. Allowed exit codes are: [0]

==> Wait completed after 1 minute 9 seconds

==> Some builds didn't complete successfully and had errors:
--> digitalocean: Script exited with non-zero exit status: 2. Allowed exit codes are: [0]

==> Builds finished but no artifacts were created.

[saschaheim]

Hello,

I had the same problem and spent yesterday the whole day trying to figure out why it just wouldn't work.

In the cloud-init issue tracker on GitHub, I found something interesting from a Canonical employee. They mentioned that the return value of sh has been extended, meaning that not only 0 is considered a success, but now also 2. You can read about it in the commit description here.

Fortunately, the return value is not fixed in Packer, and you can set your own value(s):

{
  "type": "shell",
  "inline": [
    "cloud-init status --wait"
  ],
  "valid_exit_codes": [0, 2]
}

With this small fix now everything works again.

I plan to update and convert the whole droplet-1-clicks repository to the new .pkr.hcl format. The benefits of converting from JSON to PKR.HCL (HashiCorp Configuration Language) include several key advantages related to readability, maintainability, and functionality in the context of using HashiCorp Packer:

Readability

• HCL Syntax: HCL is designed to be human-readable and human-writable. It is more declarative and less verbose than JSON, which makes it easier for developers and operators to read and understand.
• Comments: HCL supports comments, which allow developers to annotate their configuration files with explanations, notes, or instructions, enhancing the readability and maintainability of the code.

Maintainability

• Flexibility: HCL allows for more complex structures and is more forgiving with trailing commas, unlike JSON. This reduces the likelihood of syntax errors.
• Modularity: HCL supports the use of variables, functions, and modules, which can help in breaking down configurations into reusable components, making the configuration more modular and easier to maintain.

Functionality

• Interoperability with Packer: HCL is natively supported by Packer and is the recommended format for defining Packer templates. It integrates better with Packer’s functionalities and the broader HashiCorp ecosystem.
• Advanced Features: HCL allows for advanced templating features, such as conditionals and loops, which are not directly supported in JSON. This allows for more dynamic and flexible configurations.

Versioning and Changes

• Version Control: When using version control systems, HCL's readability and support for comments make it easier to track changes and understand the context behind them.
• Change Management: The structure of HCL allows for clearer diffs when configurations change, making it easier to review and manage changes.

This is a game changer and i hope somone from digitalocean will accept my pull request.

Best regards
Sascha

[lts-rad]

in case its not clear, canonical/cloud-init@a061e81 is a workaround, but the reason the error code is 2 is due to strict validation errors somewhere with noble's cloud-init. so likely also a bug on their end. the output of cloud-init schema --system shows which validations errors are the root cause

[saschaheim]

Yes but the published images from DO itself have the same error:
root@lemponubuntu2204-s-1vcpu-1gb-amd-nyc1-01:~# cloud-init schema --system
Found cloud-config data types: user-data, vendor-data, network-config

1. user-data at /var/lib/cloud/instances/428845611/cloud-config.txt:
   Empty 'cloud-config' found at /var/lib/cloud/instances/428845611/cloud-config.txt. Nothing to validate.

2. vendor-data at /var/lib/cloud/instances/428845611/vendor-cloud-config.txt:
   Invalid vendor-data /var/lib/cloud/instances/428845611/vendor-cloud-config.txt
   Error: Cloud config schema errors: system_info: Additional properties are not allowed ('system_info' was unexpected)

3. network-config at /var/lib/cloud/instances/428845611/network-config.json:
   Invalid network-config /var/lib/cloud/instances/428845611/network-config.json
   Error: Cloud config schema errors: config.0.subnets.0: Additional properties are not allowed ('ipv4' was unexpected), config.0.subnets.1: Additional properties are not allowed ('ipv4' was unexpected), config.1.subnets.0: Additional properties are not allowed ('ipv4' was unexpected)

Error: Invalid schema: vendor-data, network-config

And this happens on ubuntu 22.04 so i think this is not the problem

[josegonzalez]

I'm also seeing the same thing as @ThinkBIG-Company in the release pipeline for Dokku. Link here.

Is this a bug in the generated cloud config, or a bug in cloud-init?