Skip to content

Commit 0461194

Browse files
jurerotarjurerotar
authored
feat: reworked build process and package structure (#3)
* added a Dockerfile to create sqlite build environment * added a sqlite build script * added a GitHub Actions workflow which runs tests, check types and validates formatting * added a GitHub Actions workflow which builds Docker image, builds sqlite for wasm, extracts wasm file and bindings and creates a PR to main branch with the updated bindings and wasm file * added Vitest with browser tests for sahpool and oo1 and node tests for sqlite3-node * added tsdown for dead-code-elimination and basic build steps * added lefthook for formatting on push
1 parent c0887bb commit 0461194

45 files changed

Lines changed: 65086 additions & 29836 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.editorconfig‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
root = true
2+
3+
[*]
4+
ij_html_space_inside_empty_tag = true
5+
end_of_line = lf
6+
insert_final_newline = true
7+
charset = utf-8
8+
indent_style = space
9+
indent_size = 2
10+
trim_trailing_whitespace = true
11+
12+
# quote style
13+
ij_javascript_force_quote_style = true
14+
ij_typescript_force_quote_style = true
15+
ij_javascript_use_double_quotes = false
16+
ij_typescript_use_double_quotes = false
17+
18+
# bracket spacing
19+
ij_javascript_spaces_within_object_literal_braces = true
20+
ij_typescript_spaces_within_object_literal_braces = true
21+
ij_javascript_spaces_within_object_type_braces = true
22+
ij_typescript_spaces_within_object_type_braces = true
23+
24+
# imports
25+
ij_javascript_spaces_within_imports = true
26+
ij_typescript_spaces_within_imports = true
27+
ij_javascript_import_merge_members = true
28+
ij_typescript_import_merge_members = true
29+
ij_javascript_import_sort_members = true
30+
ij_typescript_import_sort_members = true

‎.github/actions/node-setup/action.yml‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
runs:
2+
using: 'composite'
3+
steps:
4+
- name: Setup Node.js LTS
5+
uses: actions/setup-node@v6
6+
with:
7+
node-version: 'lts/*'
8+
cache: 'npm'
9+
cache-dependency-path: package-lock.json
10+
11+
- name: Install dependencies
12+
shell: bash
13+
run: |
14+
set -euo pipefail
15+
npm ci --prefer-offline --no-audit --no-fund

‎.github/workflows/build-wasm.yml‎

Lines changed: 119 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,119 @@
1+
name: Build SQLite Wasm
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
sqlite_ref:
7+
description: 'SQLite reference (tag, branch, or commit, or "latest")'
8+
required: true
9+
default: 'master'
10+
11+
jobs:
12+
build:
13+
runs-on: ubuntu-latest
14+
steps:
15+
- name: Checkout repository
16+
uses: actions/checkout@v6
17+
with:
18+
persist-credentials: false
19+
20+
- name: Resolve SQLite reference
21+
id: resolve-ref
22+
run: |
23+
SQLITE_REF="${{ github.event.inputs.sqlite_ref }}"
24+
if [ "$SQLITE_REF" = "latest" ]; then
25+
echo "Fetching latest tag..."
26+
LATEST_TAG=$(git ls-remote --tags --sort="v:refname" https://github.com/sqlite/sqlite.git "refs/tags/version-*" | tail -n 1 | cut -f 2 | sed 's/refs\/tags\///')
27+
echo "Latest tag found: $LATEST_TAG"
28+
SQLITE_REF="$LATEST_TAG"
29+
fi
30+
31+
# Get the full commit SHA
32+
SQLITE_SHA=$(git ls-remote https://github.com/sqlite/sqlite.git "$SQLITE_REF" | head -n 1 | cut -f 1)
33+
if [ -z "$SQLITE_SHA" ]; then
34+
# If not found, maybe it's a tag that needs refs/tags/ prefix or it's already a SHA
35+
SQLITE_SHA=$(git ls-remote https://github.com/sqlite/sqlite.git "refs/tags/$SQLITE_REF" | head -n 1 | cut -f 1)
36+
fi
37+
38+
if [ -z "$SQLITE_SHA" ]; then
39+
# Fallback: assume it's a SHA if ls-remote didn't find it as a ref
40+
SQLITE_SHA="$SQLITE_REF"
41+
fi
42+
43+
echo "sqlite_ref=$SQLITE_REF" >> $GITHUB_OUTPUT
44+
echo "sqlite_sha=$SQLITE_SHA" >> $GITHUB_OUTPUT
45+
echo "branch_name=update-sqlite-wasm-${SQLITE_SHA}" >> $GITHUB_OUTPUT
46+
47+
- name: Check if branch exists
48+
id: check-branch
49+
run: |
50+
BRANCH_NAME="${{ steps.resolve-ref.outputs.branch_name }}"
51+
if git ls-remote --exit-code --heads origin "$BRANCH_NAME"; then
52+
echo "Branch $BRANCH_NAME already exists. Skipping build."
53+
echo "skip=true" >> $GITHUB_OUTPUT
54+
else
55+
echo "skip=false" >> $GITHUB_OUTPUT
56+
fi
57+
58+
- name: Set up Docker Buildx
59+
if: steps.check-branch.outputs.skip != 'true'
60+
uses: docker/setup-buildx-action@v3
61+
62+
- name: Build Docker image
63+
if: steps.check-branch.outputs.skip != 'true'
64+
uses: docker/build-push-action@v6
65+
with:
66+
context: .
67+
load: true
68+
tags: sqlite-wasm-builder:env
69+
cache-from: type=gha
70+
cache-to: type=gha,mode=max
71+
72+
- name: Run build
73+
if: steps.check-branch.outputs.skip != 'true'
74+
run: |
75+
mkdir -p out src/bin
76+
docker run --rm \
77+
-e SQLITE_REF="${{ steps.resolve-ref.outputs.sqlite_ref }}" \
78+
-e HOST_UID="$(id -u)" \
79+
-e HOST_GID="$(id -g)" \
80+
-v "$(pwd)/out":/out \
81+
-v "$(pwd)/src/bin":/src/bin \
82+
sqlite-wasm-builder:env build
83+
# Fallback fix for permissions if chown inside docker failed or was incomplete
84+
sudo chown -R $(id -u):$(id -g) out src/bin
85+
86+
- name: Check for changes
87+
if: steps.check-branch.outputs.skip != 'true'
88+
id: git-check
89+
run: |
90+
# Add files that might be newly created or modified
91+
git add src/bin
92+
git status --porcelain
93+
if [ -n "$(git status --porcelain)" ]; then
94+
echo "changes=true" >> $GITHUB_OUTPUT
95+
else
96+
echo "changes=false" >> $GITHUB_OUTPUT
97+
fi
98+
99+
- name: Create Pull Request
100+
if:
101+
steps.check-branch.outputs.skip != 'true' &&
102+
steps.git-check.outputs.changes == 'true'
103+
uses: peter-evans/create-pull-request@v6
104+
with:
105+
token: ${{ secrets.GITHUB_TOKEN }}
106+
commit-message:
107+
'chore: update SQLite Wasm binaries from ${{
108+
steps.resolve-ref.outputs.sqlite_ref }} (${{
109+
steps.resolve-ref.outputs.sqlite_sha }})'
110+
title:
111+
'chore: update SQLite Wasm binaries from ${{
112+
steps.resolve-ref.outputs.sqlite_ref }}'
113+
body: |
114+
This PR updates the SQLite Wasm binaries in `src/bin` by building them from SQLite reference `${{ steps.resolve-ref.outputs.sqlite_ref }}` (commit `${{ steps.resolve-ref.outputs.sqlite_sha }}`).
115+
116+
Triggered by manual workflow dispatch.
117+
branch: ${{ steps.resolve-ref.outputs.branch_name }}
118+
base: main
119+
delete-branch: true

‎.github/workflows/ci.yml‎

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,69 @@
1+
name: CI
2+
3+
permissions:
4+
contents: read
5+
pull-requests: read
6+
7+
on:
8+
push:
9+
branches: [main]
10+
pull_request:
11+
branches: [main]
12+
13+
concurrency:
14+
group: ${{ github.workflow }}-${{ github.ref }}
15+
cancel-in-progress: true
16+
17+
defaults:
18+
run:
19+
shell: bash
20+
21+
jobs:
22+
type-check:
23+
name: Type check
24+
runs-on: ubuntu-latest
25+
steps:
26+
- name: Checkout repo
27+
uses: actions/checkout@v6
28+
with:
29+
fetch-depth: 0
30+
31+
- name: Set up Node.js
32+
uses: ./.github/actions/node-setup
33+
34+
- name: Run type check
35+
run: npm run check-types
36+
37+
format:
38+
name: Format check
39+
runs-on: ubuntu-latest
40+
steps:
41+
- name: Checkout repo
42+
uses: actions/checkout@v6
43+
with:
44+
fetch-depth: 0
45+
46+
- name: Setup environment (node + install)
47+
uses: ./.github/actions/node-setup
48+
49+
- name: Run format check
50+
run: npx prettier . --check
51+
52+
test:
53+
name: Run tests
54+
runs-on: ubuntu-latest
55+
steps:
56+
- name: Checkout repo
57+
uses: actions/checkout@v6
58+
with:
59+
fetch-depth: 0
60+
61+
- name: Setup environment (node + install)
62+
uses: ./.github/actions/node-setup
63+
64+
- name: Install Playwright browsers
65+
run: npx playwright install --with-deps
66+
67+
- name: Run tests
68+
id: run-tests
69+
run: npm run test

‎.prettierignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
/src/bin
2+
/dist
3+
/node_modules

‎Dockerfile‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
# syntax=docker/dockerfile:1
2+
FROM emscripten/emsdk:latest
3+
4+
ARG DEBIAN_FRONTEND=noninteractive
5+
6+
# runtime / build deps
7+
RUN apt-get update \
8+
&& apt-get install -y --no-install-recommends \
9+
ca-certificates \
10+
git \
11+
build-essential \
12+
autoconf \
13+
automake \
14+
libtool \
15+
m4 \
16+
python3 \
17+
nodejs \
18+
npm \
19+
zip \
20+
unzip \
21+
curl \
22+
&& rm -rf /var/lib/apt/lists/*
23+
24+
# create useful dirs
25+
RUN mkdir -p /build /out /src/bin /work
26+
27+
# copy helper script
28+
COPY scripts/build-sqlite3-wasm.sh /usr/local/bin/build-sqlite3-wasm.sh
29+
RUN chmod +x /usr/local/bin/build-sqlite3-wasm.sh
30+
31+
WORKDIR /work
32+
33+
ENTRYPOINT ["/usr/local/bin/build-sqlite3-wasm.sh"]
34+
CMD ["build"]

‎README.md‎

Lines changed: 71 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -223,14 +223,80 @@ for this package.
223223

224224
(These steps can only be executed by maintainers.)
225225

226-
1. Update the version number in `package.json` reflecting the current
226+
1. Manually trigger the
227+
[GitHub Actions workflow](../../actions/workflows/build-wasm.yml). By
228+
default, it uses the latest SQLite tag. This pull request will contain the
229+
latest `sqlite3.wasm` and related bindings.
230+
231+
2. Once the above pull request is validated and merged, update the version
232+
number in `package.json`, reflecting the current
227233
[SQLite version number](https://sqlite.org/download.html) and add a build
228234
identifier suffix like `-build1`. The complete version number should read
229235
something like `3.41.2-build1`.
230-
1. Run `npm run build` to build the ES Module. This downloads the latest SQLite
231-
Wasm binary and builds the ES Module.
232-
1. Run `npm run deploy` to commit the changes, push to GitHub, and publish the
233-
new version to npm.
236+
237+
## Building the SQLite Wasm locally
238+
239+
1. Build the Docker image:
240+
241+
```bash
242+
docker build -t sqlite-wasm-builder:env .
243+
```
244+
245+
2. Run the build:
246+
247+
**Unix (Linux/macOS):**
248+
249+
```bash
250+
docker run --rm \
251+
-e SQLITE_REF="master" \
252+
-v "$(pwd)/out":/out \
253+
-v "$(pwd)/src/bin":/src/bin \
254+
sqlite-wasm-builder:env build
255+
```
256+
257+
**Windows (PowerShell):**
258+
259+
```powershell
260+
docker run --rm `
261+
-e SQLITE_REF="master" `
262+
-v "${PWD}/out:/out" `
263+
-v "${PWD}/src/bin:/src/bin" `
264+
sqlite-wasm-builder:env build
265+
```
266+
267+
**Windows (Command Prompt):**
268+
269+
```cmd
270+
docker run --rm ^
271+
-e SQLITE_REF="master" ^
272+
-v "%cd%/out:/out" ^
273+
-v "%cd%/src/bin:/src/bin" ^
274+
sqlite-wasm-builder:env build
275+
```
276+
277+
## Running tests
278+
279+
The test suite consists of Node.js tests and browser-based tests (using Vitest
280+
Browser Mode). Tests aim to sanity-check the exported scripts. We test for
281+
correct exports and **very** basic functionality.
282+
283+
1. Install dependencies:
284+
285+
```bash
286+
npm install
287+
```
288+
289+
2. Install Playwright browsers (required for browser tests):
290+
291+
```bash
292+
npx playwright install chromium --with-deps --no-shell
293+
```
294+
295+
3. Run all tests:
296+
297+
```bash
298+
npm test
299+
```
234300

235301
## License
236302

0 commit comments

Comments
 (0)