Fetch docs without git submodules

[TheLazyCat00]

Summary

• remove the zane-lang/docs git submodule and .gitmodules
• shallow-clone the docs before development and production builds
• remove the clone's .git directory before installing it into src/content/docs
• add npm run sync:docs with optional ZANE_DOCS_REF branch/tag support
• ignore the fetched documentation and temporary checkout directories
• document the new workflow

Why

The website should work from a normal checkout without submodule initialization. A shallow clone keeps the fetching behavior straightforward, avoids the GitHub REST API, and leaves no nested Git repository behind.

Validation

• npm run sync:docs
• confirmed src/content/docs/.git is removed after cloning
• confirmed local startup skips fetching when docs already exist
• npm run build
• Astro generated all 5 static routes successfully

[gemini-code-assist]

Code Review

This pull request replaces the git submodule for documentation with a custom Node.js script (scripts/fetch-docs.mjs) that downloads and extracts the documentation tarball from GitHub. It also updates the build scripts, .gitignore, and documentation to support this new workflow. The review feedback recommends optimizing this process for local development by adding a --skip-if-exists flag to avoid redundant downloads, which prevents rate-limiting and enables offline development. Additionally, it suggests using access instead of readFile to verify the existence of the extracted files.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

Running npm run dev or npm run start currently fetches the documentation archive from GitHub on every single startup. This introduces two major issues:

1. Offline Development Blocked: If a developer is offline or has a spotty connection, fetch will fail, crashing the startup process and preventing them from running the local development server.
2. GitHub API Rate Limiting: Since these are unauthenticated requests, repeatedly running npm run dev can easily trigger GitHub's rate limit (60 requests/hour per IP).

To resolve this, we can introduce a --skip-if-exists flag to check if the documentation directory already exists, and skip the download if it does. This keeps local development fast and offline-friendly, while still ensuring that production builds (npm run build) always fetch the latest documentation.

import { spawnSync } from 'node:child_process';
import { access, mkdtemp, mkdir, rename, rm, writeFile } from 'node:fs/promises';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';

const repository = 'zane-lang/docs';
const ref = process.env.ZANE_DOCS_REF || 'main';
const root = join(dirname(fileURLToPath(import.meta.url)), '..');
const destination = join(root, 'src/content/docs');

if (process.argv.includes('--skip-if-exists')) {
	try {
		await access(destination);
		console.log('Docs already exist, skipping download.');
		process.exit(0);
	} catch {}
}

const workDirectory = await mkdtemp(join(root, 'src/content/.docs-'));
const archive = join(workDirectory, 'docs.tar.gz');
const extracted = join(workDirectory, 'extracted');

[gemini-code-assist]

To support offline development and avoid hitting GitHub API rate limits during local development, pass the --skip-if-exists flag to the sync:docs script in predev and prestart. This ensures the documentation is only downloaded if it is missing locally, while still forcing a fresh download during production builds (prebuild).

Suggested change

	"predev": "npm run sync:docs",
	"dev": "astro dev",
	"prestart": "npm run sync:docs",
	"start": "astro dev",
	"prebuild": "npm run sync:docs",
	"predev": "npm run sync:docs -- --skip-if-exists",
	"dev": "astro dev",
	"prestart": "npm run sync:docs -- --skip-if-exists",
	"start": "astro dev",
	"prebuild": "npm run sync:docs",

[gemini-code-assist]

Using access is more efficient and idiomatic than readFile when you only want to verify the existence of a file without reading its entire contents into memory.

	await access(join(extracted, 'README.md'));

[TheLazyCat00]

/gemini review

[gemini-code-assist]

Code Review

This pull request replaces the src/content/docs git submodule with a custom Node.js script (scripts/fetch-docs.mjs) that downloads and extracts the documentation archive from GitHub. The build, start, and dev scripts in package.json have been updated to run this script automatically. Feedback suggests ensuring that the src/content directory is created recursively before calling mkdtemp to avoid errors on fresh clones, and using the public GitHub archive download URL instead of the REST API to prevent rate-limiting issues in shared CI/CD environments.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

On a fresh repository clone, the src/content directory may not exist because Git does not track empty directories (and the src/content/docs submodule has been removed). If src/content is missing, mkdtemp will fail with an ENOENT error.

Ensure that the src/content directory is created recursively before attempting to create a temporary directory inside it.

const contentDir = join(root, 'src/content');
await mkdir(contentDir, { recursive: true });
const workDirectory = await mkdtemp(join(contentDir, '.docs-'));
const archive = join(workDirectory, 'docs.tar.gz');
const extracted = join(workDirectory, 'extracted');

[gemini-code-assist]

Using the GitHub REST API (api.github.com) without authentication is subject to a strict rate limit of 60 requests per hour per IP address. In shared CI/CD environments (such as Vercel or GitHub Actions), this limit is frequently exceeded by other builds sharing the same outbound IP, which will cause your production builds to fail.

To avoid this, use the public repository archive download URL (https://github.com/{owner}/{repo}/archive/{ref}.tar.gz), which is not subject to these API rate limits and does not require API-specific headers.

	const response = await fetch(
		`https://github.com/${repository}/archive/${encodeURIComponent(ref)}.tar.gz`,
		{
			headers: {
				'User-Agent': 'zane-lang-website',
			},
		},
	);

[TheLazyCat00]

/gemini review

[gemini-code-assist]

Code Review

This pull request replaces the git submodule for documentation with a custom Node.js script (scripts/fetch-docs.mjs) that downloads and extracts the documentation tarball from GitHub during build and development steps. Feedback on the changes suggests improving the robustness of the download script by checking for the existence of the destination directory rather than README.md to skip downloads, and removing a redundant check for README.md after extraction.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

Checking for the existence of README.md to determine if the documentation is already downloaded is fragile. If the README.md file is ever renamed, deleted, or has a different casing (e.g., readme.md on case-sensitive filesystems), this check will fail and trigger an unnecessary re-download. It is more robust to check for the existence of the destination directory itself.

		await access(destination);

[gemini-code-assist]

Verifying the extraction by checking for README.md is fragile and redundant. Since spawnSync already checks the exit status of the tar command (lines 50-52), we can safely trust that the extraction succeeded if tar exited with 0. We should remove this check to avoid breaking the build if README.md is renamed or removed from the repository.

[TheLazyCat00]

/gemini review

[gemini-code-assist]

Code Review

This pull request replaces the git submodule for documentation with a custom Node.js script (scripts/fetch-docs.mjs) that shallow-clones the zane-lang/docs repository before running development or build tasks. The review feedback suggests invoking the Node.js script directly in package.json to avoid forcing a specific package manager, and refactoring the synchronous process spawning (spawnSync) in the fetch script to use asynchronous execution (execFile) to prevent blocking the event loop.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

Calling npm run recursively inside package scripts forces the use of npm even if the user is using another package manager like pnpm, yarn, or bun. It also adds unnecessary overhead by spawning another package manager process.

We can invoke the Node.js script directly instead. Note that the same improvement should be applied to prestart (line 9) and prebuild (line 11).

Suggested change

	"predev": "npm run sync:docs -- --skip-if-exists",
	"predev": "node scripts/fetch-docs.mjs --skip-if-exists",

[gemini-code-assist]

Using synchronous process spawning (spawnSync) blocks the Node.js event loop. Since the rest of this script is fully asynchronous and uses async/await for file system operations, we should use asynchronous process spawning to be consistent and avoid blocking the event loop.

We can use promisify from node:util with execFile from node:child_process to run the git command asynchronously.

import { execFile } from 'node:child_process';
import { access, mkdtemp, mkdir, rename, rm } from 'node:fs/promises';
import { dirname, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { promisify } from 'node:util';

const execFileAsync = promisify(execFile);

[gemini-code-assist]

Update the git clone execution to use the asynchronous execFileAsync helper instead of spawnSync.

	await execFileAsync(
		'git',
		['clone', '--depth=1', '--branch', ref, '--single-branch', repository, checkout],
		{ timeout: 30_000 },
	);

	await rm(join(checkout, '.git'), { recursive: true, force: true });
	await rm(destination, { recursive: true, force: true });
	await rename(checkout, destination);
	console.log('Cloned zane-lang/docs@' + ref + ' to src/content/docs');
} catch (error) {
	throw new Error(error.stderr?.trim() || error.message || 'Failed to clone repository');
} finally {
	await rm(workDirectory, { recursive: true, force: true });
}

[TheLazyCat00]

/gemini review

[gemini-code-assist]

Code Review

This pull request replaces the Git submodule for documentation with a custom Node.js script (scripts/fetch-docs.mjs) that shallow-clones the zane-lang/docs repository. This script is integrated into the build and development lifecycle via npm scripts. The review feedback suggests adding the --skip-if-exists flag to the prebuild script to allow offline builds, implementing a backup-and-restore mechanism during directory renaming to prevent data loss on failure, and wrapping the cleanup in the finally block in a try...catch to avoid shadowing primary errors.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

Running npm run build currently forces a redownload of the documentation every time because prebuild does not use the --skip-if-exists flag. This prevents offline builds and slows down local production testing.

Since production environments (like Vercel or CI) always start with a clean workspace where src/content/docs does not exist, they will still fetch the docs automatically even with --skip-if-exists.

Consider adding --skip-if-exists to prebuild to allow offline builds and speed up local builds.

Suggested change

	"prebuild": "node scripts/fetch-docs.mjs",
	"prebuild": "node scripts/fetch-docs.mjs --skip-if-exists",

[gemini-code-assist]

If rename(checkout, destination) fails after rm(destination) has already run, the existing documentation is lost and the directory is left empty.

To make this operation more robust and prevent data loss on failure, you can rename the existing destination to a temporary backup path first, perform the rename, and then clean up the backup.

	const backup = join(contentDirectory, '.docs-old');
	try {
		await rename(destination, backup);
	} catch {}

	try {
		await rename(checkout, destination);
	} catch (error) {
		try {
			await rename(backup, destination);
		} catch {}
		throw error;
	} finally {
		await rm(backup, { recursive: true, force: true });
	}

[gemini-code-assist]

If an error occurs during rm in the finally block (for example, due to file locking issues on Windows), it will throw and suppress the original error from the try or catch block. This makes debugging the actual clone failure very difficult.

Wrapping the cleanup in a try...catch block ensures that any cleanup errors do not shadow the primary error.

} finally {
	try {
		await rm(workDirectory, { recursive: true, force: true });
	} catch {}
}