Skip to content

Commit dddca30

Browse files
committed
Add nbdime for diffing and merging notebooks
1 parent 0b36cee commit dddca30

3 files changed

Lines changed: 111 additions & 5 deletions

File tree

README.md

Lines changed: 52 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,27 +1,71 @@
11
# data-analyses
2+
23
Place for sharing quick reports, and works in progress
34

45
This repository is for quick sharing of works in progress and simple analyses.
56
For collaborative short-term tasks, create a new folder and work off a separate branch.
67
For longer-term projects, consider making a new repository!
78

9+
810
## Using this Repo
911

1012
* Use [this link](https://docs.calitp.org/data-infra/analytics_tools/saving_code.html#onboarding-setup) to get started in JupyterHub, set up SSH, and start commiting to the repo!
1113

14+
15+
### JupyterHub Developers
16+
17+
If you are developing in JupyterHub, follow the [JupyterHub setup docs](https://docs.calitp.org/data-infra/analytics_tools/jupyterhub.html).
18+
19+
1220
### Contributing
13-
#### Pre-commit
1421

15-
This repository uses pre-commit hooks to format code, including [Black](https://black.readthedocs.io/en/stable/index.html). This ensures baseline consistency in code formatting.
22+
Follow these steps to start contributing:
23+
24+
1. [Clone](https://docs.github.com/en/repositories/creating-and-managing-repositories/cloning-a-repository) this `data-analyses` repo.
25+
26+
2. From the repo root (`data-analyses/`), run `make install_env` (runs `uv sync` + pre-commit setup)
27+
28+
3. In JupyterHub, select the **"Pyproject Local"** kernel when opening a notebook
29+
30+
> [!NOTE]
31+
> If you run into the error `No such file or directory`, you may need to [install uv](https://docs.astral.sh/uv/getting-started/installation/) running `pip install uv`.
32+
33+
### uv
34+
35+
This repository uses uv for package management. To learn more go to [uv documentation](https://docs.astral.sh/uv/concepts/projects/dependencies/).
1636

17-
> [!IMPORTANT]
18-
> Before contributing to this project, please install dependencies and pre-commit by running `make install_env` in the root of the repo.
37+
Basic commands:
1938

20-
Once installed, pre-commit checks will run before you can make commits locally. If a pre-commit check fails, it will need to be addressed before you can make your commit. Many formatting issues are fixed automatically within the pre-commit actions, so check the changes made by pre-commit on failure -- they may have automatically addressed the issues that caused the failure, in which case you can simply re-add the files, re-attempt the commit, and the checks will then succeed.
39+
* `uv sync` install missing packages, update existing ones, and remove unnecessary ones to ensure the environment matches the lockfile.
40+
* `uv add <package name>` include and install a new package to the main project.
41+
* `uv add <package name> --dev` include and install new packages/dependencies to the `dev` group.
42+
* `uv add <package name> --portfolio` include and install new packages/dependencies to the `portfolio` group.
43+
* `uv add <package name> --test` include and install new packages/dependencies used only for testing under the `test` group.
44+
* `uv remove <package name>` remove and uninstall packages/dependencies from the project.
45+
46+
### nbdime
47+
48+
[`nbdime`](https://github.com/jupyter/nbdime) provides command-line tools for diffing and merging notebooks.
49+
50+
Basic commands:
51+
52+
* `nbdiff` compare notebooks in a terminal-friendly way.
53+
* `nbmerge` three-way merge of notebooks with automatic conflict resolution.
54+
* `nbdiff-web` shows you a rich rendered diff of notebooks.
55+
* `nbmerge-web` gives you a web-based three-way merge tool for notebooks.
56+
* `nbshow` present a single notebook in a terminal-friendly way.
57+
58+
### Pre-commit
59+
60+
This repository uses pre-commit hooks to format code, including [Black](https://black.readthedocs.io/en/stable/index.html). This ensures baseline consistency in code formatting.
61+
62+
Pre-commit checks will run before you can make commits locally. If a pre-commit check fails, it will need to be addressed before you can make your commit.
63+
Many formatting issues are fixed automatically within the pre-commit actions, so check the changes made by pre-commit on failure -- they may have automatically addressed the issues that caused the failure, in which case you can simply re-add the files, re-attempt the commit, and the checks will then succeed.
2164

2265
Installing pre-commit locally saves time dealing with formatting issues on pull requests. There is a [GitHub Action](./.github/workflows/lint.yml)
2366
that runs pre-commit on all files, not just changed ones, as part of our continuous integration.
2467

68+
2569
## Quick Links - Get Started in Data Analysis
2670

2771
#### Data Analytics Documentation - Welcome
@@ -32,14 +76,17 @@ https://docs.calitp.org/data-infra/analytics_welcome/overview.html
3276

3377
https://docs.calitp.org/data-infra/analytics_tools/overview.html
3478

79+
3580
## Publishing Reports
81+
3682
[The sites folder](./portfolio/sites/) contains the YAML files that drive sites
3783
deployed to [https://analysis.calitp.org/](https://analysis.calitp.org/); the existing
3884
sites can be used as examples/templates for deploying additional sites. Also, the
3985
Data Services Documentation has a [specific chapter](https://docs.calitp.org/data-infra/publishing/overview.html)
4086
dedicated to various ways to publish data.
4187

4288
### Caveats (when using the portfolio site)
89+
4390
Jupyter Book/Sphinx do not play nicely with Markdown headers written out in `display()`
4491
calls. Therefore, [portfolio.py](./portfolio.py) uses a custom Papermill
4592
engine to template Markdown cells directly, following Python formatted-string

pyproject.toml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,9 @@
22
name = "cal-itp-data-analyses"
33
version = "0.1.0"
44
requires-python = ">=3.11.0, <3.12.0"
5+
dependencies = [
6+
"nbdime>=4.0.4",
7+
]
58

69
[tool.uv.workspace]
710
members = [

uv.lock

Lines changed: 56 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)