Skip to content

Commit 466ec24

Browse files
ktechmidasclaudevivekgsharma
authored
feat(ci): add Create and Destroy Devnet workflows (#730)
* feat(ci): add Create and Destroy Devnet GitHub Actions workflows Add two new workflow_dispatch workflows for full devnet lifecycle management: - create-devnet.yml: Generates configs, provisions AWS infrastructure via Terraform, deploys all services via Ansible, and persists configs to the dash-network-configs repo. Defaults to 11 ARM HP masternodes, 0 regular masternodes, 1 seed node. Advanced options (node counts, disk size, core version) have sane defaults but can be overridden. - destroy-devnet.yml: Tears down devnet infrastructure (all/platform/network targets), cleans up configs from dash-network-configs repo when fully destroyed. Required secrets: AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_REGION, TERRAFORM_S3_BUCKET, TERRAFORM_S3_KEY, TERRAFORM_DYNAMODB_TABLE (plus existing DEPLOY_SERVER_KEY and EVO_APP_DEPLOY_KEY). Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(ci): add rs_dapi_image version to platform deployments group_vars/all defines rs_dapi_image without a version tag, which causes the dashmate template to use it directly (skipping the regex_replace fallback from dapi_image). This meant rs_dapi would get :latest instead of the specified platform version. Fix by explicitly setting rs_dapi_image in both create-devnet and platform-deploy workflows. Co-Authored-By: Claude Opus 4.6 <[email protected]> * fix(ci): harden devnet workflows against injection and improve reliability - Map all github.event.inputs and secrets to env vars instead of inline ${{ }} expansion in run blocks to prevent shell injection - Write SSH keys with printf instead of echo for safe handling - Use python3 -m pip instead of ambiguous python binary - Add numeric validation for node count inputs before passing to bin/generate - Escape sed-special characters (& \ /) in version strings - Change verify step from if: always() to if: success() and remove || true so Ansible errors are surfaced - Add concurrency groups to prevent parallel runs racing on Terraform state or git push for the same devnet - Validate all three config files (yml, tfvars, inventory) exist in destroy workflow, not just yml - Compare disk size against actual file value instead of hardcoded default - Reference DESTROY_TARGET from job-level env instead of inline expansion in shell blocks Co-Authored-By: Claude Opus 4.6 <[email protected]> * Fix create-devnet workflow deploy step for GitHub Actions detached HEAD --------- Co-authored-by: Claude Opus 4.6 <[email protected]> Co-authored-by: vivekgsharma <[email protected]>
1 parent aedd86e commit 466ec24

3 files changed

Lines changed: 539 additions & 2 deletions

File tree

‎.github/workflows/create-devnet.yml‎

Lines changed: 307 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,307 @@
1+
name: Create Devnet
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
devnet_name:
7+
description: "Devnet name (without 'devnet-' prefix, e.g. 'mytest' creates 'devnet-mytest')"
8+
required: true
9+
type: string
10+
platform_version:
11+
description: "Platform/dashmate version to deploy (e.g. 2.0.0-rc.16)"
12+
required: true
13+
type: string
14+
default: "2.0.0-rc.16"
15+
# Advanced options - sane defaults, only change if you know what you're doing
16+
hp_masternodes_arm_count:
17+
description: "Advanced: Number of ARM HP masternodes"
18+
required: false
19+
type: string
20+
default: "11"
21+
hp_masternodes_amd_count:
22+
description: "Advanced: Number of AMD HP masternodes"
23+
required: false
24+
type: string
25+
default: "0"
26+
masternodes_arm_count:
27+
description: "Advanced: Number of ARM regular masternodes"
28+
required: false
29+
type: string
30+
default: "0"
31+
masternodes_amd_count:
32+
description: "Advanced: Number of AMD regular masternodes"
33+
required: false
34+
type: string
35+
default: "0"
36+
seed_count:
37+
description: "Advanced: Number of seed nodes"
38+
required: false
39+
type: string
40+
default: "1"
41+
core_version:
42+
description: "Advanced: Core (dashd) image version (leave empty for default)"
43+
required: false
44+
type: string
45+
default: ""
46+
hpmn_disk_size:
47+
description: "Advanced: HP masternode disk size in GB"
48+
required: false
49+
type: string
50+
default: "30"
51+
52+
jobs:
53+
create:
54+
name: Create Devnet
55+
runs-on: ubuntu-latest
56+
timeout-minutes: 120
57+
concurrency:
58+
group: "devnet-${{ github.event.inputs.devnet_name }}"
59+
cancel-in-progress: false
60+
61+
env:
62+
NETWORK_NAME: "devnet-${{ github.event.inputs.devnet_name }}"
63+
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
64+
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
65+
AWS_REGION: ${{ secrets.AWS_REGION }}
66+
TERRAFORM_S3_BUCKET: ${{ secrets.TERRAFORM_S3_BUCKET }}
67+
TERRAFORM_S3_KEY: ${{ secrets.TERRAFORM_S3_KEY }}
68+
TERRAFORM_DYNAMODB_TABLE: ${{ secrets.TERRAFORM_DYNAMODB_TABLE }}
69+
ANSIBLE_HOST_KEY_CHECKING: "false"
70+
71+
steps:
72+
- name: Validate devnet name
73+
env:
74+
NAME: ${{ github.event.inputs.devnet_name }}
75+
run: |
76+
if [[ -z "$NAME" ]]; then
77+
echo "Error: devnet_name is required"
78+
exit 1
79+
fi
80+
if [[ "$NAME" =~ ^devnet- ]]; then
81+
echo "Error: Do not include 'devnet-' prefix. Just provide the name (e.g. 'mytest')"
82+
exit 1
83+
fi
84+
if [[ ! "$NAME" =~ ^[a-z0-9][a-z0-9-]*$ ]]; then
85+
echo "Error: devnet_name must be lowercase alphanumeric with optional hyphens"
86+
exit 1
87+
fi
88+
echo "Will create: devnet-$NAME"
89+
90+
- name: Checkout dash-network-deploy
91+
uses: actions/checkout@v4
92+
93+
- name: Set up Node.js
94+
uses: actions/setup-node@v4
95+
with:
96+
node-version: '20'
97+
98+
- name: Install Node.js dependencies
99+
run: npm ci
100+
101+
- name: Set up Terraform
102+
uses: hashicorp/setup-terraform@v3
103+
with:
104+
terraform_version: "1.12.1"
105+
terraform_wrapper: false
106+
107+
- name: Install system dependencies
108+
run: |
109+
sudo apt-get update
110+
sudo apt-get install -y python3-pip python3-netaddr sshpass jq
111+
112+
- name: Install Ansible
113+
run: |
114+
python3 -m pip install --upgrade pip
115+
python3 -m pip install ansible
116+
117+
- name: Install Ansible roles
118+
run: |
119+
ansible-galaxy install -r ansible/requirements.yml
120+
mkdir -p ~/.ansible/roles
121+
cp -r ansible/roles/* ~/.ansible/roles/
122+
123+
- name: Set up SSH keys
124+
env:
125+
DEPLOY_SERVER_KEY: ${{ secrets.DEPLOY_SERVER_KEY }}
126+
EVO_APP_DEPLOY_KEY: ${{ secrets.EVO_APP_DEPLOY_KEY }}
127+
run: |
128+
mkdir -p ~/.ssh
129+
130+
# Server SSH key for connecting to nodes
131+
printf '%s\n' "$DEPLOY_SERVER_KEY" > ~/.ssh/id_rsa
132+
chmod 600 ~/.ssh/id_rsa
133+
134+
# Derive public key from private key
135+
ssh-keygen -y -f ~/.ssh/id_rsa > ~/.ssh/id_rsa.pub
136+
chmod 644 ~/.ssh/id_rsa.pub
137+
138+
# GitHub deploy key for cloning configs repo
139+
printf '%s\n' "$EVO_APP_DEPLOY_KEY" > ~/.ssh/id_ed25519
140+
chmod 600 ~/.ssh/id_ed25519
141+
142+
# SSH config
143+
cat > ~/.ssh/config << 'EOL'
144+
Host github.com
145+
IdentityFile ~/.ssh/id_ed25519
146+
StrictHostKeyChecking no
147+
148+
Host *
149+
IdentityFile ~/.ssh/id_rsa
150+
User ubuntu
151+
StrictHostKeyChecking no
152+
UserKnownHostsFile=/dev/null
153+
EOL
154+
155+
chmod 600 ~/.ssh/config
156+
157+
- name: Create networks/.env
158+
run: |
159+
mkdir -p networks
160+
cat > networks/.env << EOF
161+
PRIVATE_KEY_PATH=$HOME/.ssh/id_rsa
162+
PUBLIC_KEY_PATH=$HOME/.ssh/id_rsa.pub
163+
AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
164+
AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
165+
AWS_REGION=$AWS_REGION
166+
TERRAFORM_S3_BUCKET=$TERRAFORM_S3_BUCKET
167+
TERRAFORM_S3_KEY=$TERRAFORM_S3_KEY
168+
TERRAFORM_DYNAMODB_TABLE=$TERRAFORM_DYNAMODB_TABLE
169+
EOF
170+
171+
- name: Generate network configs
172+
env:
173+
MN_AMD: ${{ github.event.inputs.masternodes_amd_count }}
174+
MN_ARM: ${{ github.event.inputs.masternodes_arm_count }}
175+
HP_AMD: ${{ github.event.inputs.hp_masternodes_amd_count }}
176+
HP_ARM: ${{ github.event.inputs.hp_masternodes_arm_count }}
177+
SEED_COUNT: ${{ github.event.inputs.seed_count }}
178+
run: |
179+
# Validate all counts are numeric
180+
for var in MN_AMD MN_ARM HP_AMD HP_ARM SEED_COUNT; do
181+
val="${!var}"
182+
if [[ ! "$val" =~ ^[0-9]+$ ]]; then
183+
echo "Error: $var must be a number, got '$val'"
184+
exit 1
185+
fi
186+
done
187+
188+
echo "Generating configs for $NETWORK_NAME..."
189+
chmod +x ./bin/generate
190+
./bin/generate "$NETWORK_NAME" \
191+
"$MN_AMD" "$MN_ARM" "$HP_AMD" "$HP_ARM" \
192+
-s="$SEED_COUNT"
193+
194+
echo "Generated config files:"
195+
ls -la networks/devnet-*
196+
197+
- name: Update platform version in config
198+
env:
199+
VERSION: ${{ github.event.inputs.platform_version }}
200+
CORE_VERSION: ${{ github.event.inputs.core_version }}
201+
run: |
202+
CONFIG_FILE="networks/$NETWORK_NAME.yml"
203+
204+
# Escape sed-special characters in version strings
205+
SAFE_VERSION=$(printf '%s' "$VERSION" | sed 's/[&\\/]/\\&/g')
206+
SAFE_CORE_VERSION=$(printf '%s' "$CORE_VERSION" | sed 's/[&\\/]/\\&/g')
207+
208+
echo "Setting platform version to $VERSION..."
209+
210+
# Update dashmate version
211+
sed -i "s/dashmate_version: .*/dashmate_version: $SAFE_VERSION/" "$CONFIG_FILE"
212+
213+
# Update platform service images
214+
sed -i "s|drive_image: dashpay/drive:.*|drive_image: dashpay/drive:$SAFE_VERSION|" "$CONFIG_FILE"
215+
sed -i "s|dapi_image: dashpay/dapi:.*|dapi_image: dashpay/dapi:$SAFE_VERSION|" "$CONFIG_FILE"
216+
217+
# Add rs_dapi_image (not in generated config, but group_vars/all defines it
218+
# without a tag, so we must explicitly set it to get the right version)
219+
if ! grep -q "rs_dapi_image:" "$CONFIG_FILE"; then
220+
echo "rs_dapi_image: dashpay/rs-dapi:$VERSION" >> "$CONFIG_FILE"
221+
else
222+
sed -i "s|rs_dapi_image: dashpay/rs-dapi:.*|rs_dapi_image: dashpay/rs-dapi:$SAFE_VERSION|" "$CONFIG_FILE"
223+
fi
224+
225+
# Update core version if specified
226+
if [[ -n "$CORE_VERSION" ]]; then
227+
echo "Setting core version to $CORE_VERSION..."
228+
sed -i "s|dashd_image: dashpay/dashd:.*|dashd_image: dashpay/dashd:$SAFE_CORE_VERSION|" "$CONFIG_FILE"
229+
fi
230+
231+
echo "Updated config:"
232+
grep -E "(dashmate_version|drive_image|dapi_image|rs_dapi_image|dashd_image)" "$CONFIG_FILE"
233+
234+
- name: Update terraform config
235+
env:
236+
DISK_SIZE: ${{ github.event.inputs.hpmn_disk_size }}
237+
run: |
238+
TFVARS_FILE="networks/$NETWORK_NAME.tfvars"
239+
240+
# Read current value from file (empty if not set)
241+
CURRENT_SIZE=$(grep -oP 'hpmn_node_disk_size\s*=\s*\K[0-9]+' "$TFVARS_FILE" 2>/dev/null || echo "")
242+
243+
if [[ -n "$DISK_SIZE" && "$DISK_SIZE" != "$CURRENT_SIZE" ]]; then
244+
if [[ ! "$DISK_SIZE" =~ ^[0-9]+$ ]]; then
245+
echo "Error: hpmn_disk_size must be a number, got '$DISK_SIZE'"
246+
exit 1
247+
fi
248+
echo "Setting HP masternode disk size to ${DISK_SIZE}GB..."
249+
if [[ -n "$CURRENT_SIZE" ]]; then
250+
sed -i "s/hpmn_node_disk_size = .*/hpmn_node_disk_size = $DISK_SIZE/" "$TFVARS_FILE"
251+
else
252+
echo "hpmn_node_disk_size = $DISK_SIZE" >> "$TFVARS_FILE"
253+
fi
254+
fi
255+
256+
echo "Terraform config:"
257+
cat "$TFVARS_FILE"
258+
259+
- name: Deploy devnet (Terraform + Ansible)
260+
run: |
261+
echo "============================================"
262+
echo "Deploying $NETWORK_NAME"
263+
echo "============================================"
264+
265+
chmod +x ./bin/deploy
266+
# GitHub Actions checks out a detached HEAD; bypass branch safety check.
267+
./bin/deploy -f "$NETWORK_NAME"
268+
269+
- name: Push configs to dash-network-configs
270+
run: |
271+
# Clone the configs repo to a temp directory
272+
git clone [email protected]:dashpay/dash-network-configs.git /tmp/dash-network-configs
273+
274+
# Copy generated config files
275+
cp "networks/$NETWORK_NAME.yml" /tmp/dash-network-configs/
276+
cp "networks/$NETWORK_NAME.tfvars" /tmp/dash-network-configs/
277+
cp "networks/$NETWORK_NAME.inventory" /tmp/dash-network-configs/
278+
279+
# Commit and push
280+
cd /tmp/dash-network-configs
281+
git config user.name "GitHub Actions"
282+
git config user.email "[email protected]"
283+
git add .
284+
git commit -m "Add configs for $NETWORK_NAME" || echo "No changes to commit"
285+
git push
286+
287+
echo "Configs pushed to dash-network-configs repo"
288+
289+
- name: Verify platform services
290+
if: success()
291+
run: |
292+
echo "Verifying platform services on HP masternodes..."
293+
ansible hp_masternodes \
294+
-i "networks/$NETWORK_NAME.inventory" \
295+
--private-key="$HOME/.ssh/id_rsa" \
296+
-b -m shell \
297+
-a 'sudo -u dashmate dashmate status services --format=json | jq -r ".[] | select(.service != \"core\") | \"\(.service): \(.status)\""'
298+
299+
- name: Print summary
300+
if: always()
301+
run: |
302+
echo "============================================"
303+
echo "Devnet: $NETWORK_NAME"
304+
echo "============================================"
305+
echo ""
306+
echo "To update this devnet later, use the 'Platform Version Deployment' workflow"
307+
echo "To destroy this devnet, use the 'Destroy Devnet' workflow"

0 commit comments

Comments
 (0)