Sync upstream OpenSfM

.clang-format

@@ -1 +1,2 @@
 BasedOnStyle: Google
+InsertBraces: true

.coveragerc

@@ -0,0 +1,18 @@
+[run]
+data_file = coverage_data/.coverage
+branch = True
+relative_files = True
+source =
+    opensfm
+omit =
+    opensfm/test/*
+
+[report]
+omit =
+    opensfm/test/*
+
+[xml]
+output = coverage_data/coverage-python.xml
+
+[html]
+directory = coverage_data/htmlcov

.dir-locals.el

@@ -0,0 +1,7 @@
+;;; Directory Local Variables         -*- no-byte-compile: t; -*-
+;;; For more information see (info "(emacs) Directory Variables")
+
+((c++-mode    . ((c-basic-offset          . 2)
+                 (tab-width               . 2)))
+ (c++-ts-mode . ((c-ts-mode-indent-offset . 2)
+                 (tab-width               . 2))))

.dockerignore

@@ -3,6 +3,7 @@
 
 # Pick what to include
 !bin
+!configs
 !data/berlin/config.yaml
 !data/berlin/gcp_list.txt
 !data/berlin/images
@@ -11,8 +12,10 @@
 !doc
 !LICENSE
 !opensfm
+!pyproject.toml
+!conda.yml
+!conda-*.lock
 !README.md
-!requirements.txt
 !setup.cfg
 !setup.py
 !viewer

.github/ISSUE_TEMPLATE/bug_report.md

@@ -0,0 +1,25 @@
+---
+name: Bug report
+about: Create a bug report so we can improve
+title: ''
+labels: ''
+assignees: YanNoun
+
+---
+
+**Describe the bug**
+A clear and concise description of what the bug is.
+
+**To Reproduce**
+Please give us a reproduction step (command). If there's NO REPRO steps, WE'LL CLOSE this issue immediately.
+
+**Screenshots or Log**
+Please attach some screenshot or log that show what's happening.
+
+**Platform**
+Linux (which distrib' and version) ?
+WIndows ?
+Mac ?
+
+**Additional context**
+Add any other context about the problem here.

.github/copilot-instructions.md

@@ -0,0 +1,80 @@
+# OpenSfM AI Coding Instructions
+
+## 1. Architecture & "Big Picture"
+OpenSfM is a Structure-from-Motion library with a hybrid architecture:
+- **Core Logic (C++)**: Heavy computational tasks (geometry, bundle adjustment, features) are implemented in C++ under `opensfm/src/`. These are exposed to Python as compiled modules (e.g., `pygeometry`, `pymap`, `pfm`) using **PyBind11**.
+- **Pipeline Layout (Python)**: The application logic, pipeline orchestration, and user interface are in Python (`opensfm/` package).
+- **Data Abstraction**: The `DataSet` class (`opensfm/dataset.py`) is the central definition for filesystem interactions. All pipeline stages read/write to a hardcoded directory structure (`images/`, `config.yaml`, `reconstruction.json`) managed by this class.
+- **State Management**: The `Reconstruction` class (`opensfm/types.py`) wraps the C++ `pymap.Map` object. This pattern (Python class holding a C++ handle) is pervasive.
+
+## 2. Key Developer Workflows
+
+### Building
+The project uses `scikit-build-core` to compile C++ extensions.
+- **Full Build**: `pip install -e .[test]` (Builds C++ extensions and tests, and installs the package in editable mode).
+- **Rebuild C++**: Re-running `pip install -e .` is usually required after changing `.cc` files.
+- **Dependencies**: Managed via `pyproject.toml` for managing Python dependencies. Outer envinvironment (C++ toolchain, libraries) is set up through `conda` and the `conda.yml` file. Any change to `conda.yml` requires recreating the conda environment with `conda env create --file conda.yml --yes`, then activating it with `conda activate opensfm`. Note that before running the build commands, the conda environment must be activated once.
+
+### Testing
+- **Framework**: 
+    - Python : `pytest` is used, and tests are in files `test_YYY.py` under `opensfm/test`.
+    - C++ : `gtest` is used. For a given library XXX, tests are put under `opensfm/src/XXX/test/` as `YYY_test.cc`.
+- **Style**: we favor many tests with few asserts and ideally one function to be tested. Only a few (1-2) larger integration tests per file. We also strive to test using toy examples to check for function correctness and not just check return semantics.
+- **Location**: `opensfm/test/` for Python tests. C++ tests are in `./cmake_build`.
+- **Synthetic Data**: Python tests heavily rely on synthetic scenes generated in `opensfm/test/conftest.py`. Look for fixtures like `scene_synthetic` or `scene_synthetic_cube`.
+- **Run Tests**: `pytest opensfm/test/` for Python tests. C++ tests can be run via `ctest` with `cd cmake_build && ctest --output-on-failure && cd ..`. Don't forget to activate the conda environment before running tests, and run `export LD_PRELOAD=$CONDA_PREFIX/lib/libtcmalloc.so` before running.
+
+### Running the Pipeline
+- **Entry Point**: `bin/opensfm` (shell script) -> `bin/opensfm_main.py`.
+- **Commands**: Each CLI command (e.g., `detect_features`, `reconstruct`) is implemented as a module in `opensfm/commands/`.
+- **Example**: `bin/opensfm reconstruct path/to/dataset` invokes the `run()` method in `opensfm/commands/reconstruct.py`.
+
+## 3. Coding Conventions & Patterns
+
+### Python/C++ Interop
+- **Wrappers**: Do not use C++ objects directly if a Python wrapper exists (e.g., use `opensfm.types.Reconstruction` instead of `opensfm.pymap.Map` where possible). 
+- **Extensions**: C++ extensions are imported from the root package (e.g., `from opensfm import pygeometry`).
+
+
+### Code Philosophy
+- **Readability**: Prioritize clear, simple and maintainable code. Avoid complicated constructs : you're an seasoned engineer that knows about real-world trade-offs, not a rookie trying to expose its design patterns knowledge. You'll use frameworks and design pattern when they're strictly justified at the present moment, not for an hypothetical future. Use descriptive variable and function names, and break down complex functions into smaller, focused ones.
+- **Documentation**: Document all public functions and classes with docstrings. For complex algorithms, include comments explaining the rationale and key steps.
+- **Testing**: Write tests for new features and bug fixes. Use the existing test suite as a reference for style and structure.
+- **C++ Style**: Follow the existing C++ style in the codebase. Use `clang-format` for consistent formatting. Key points:
+    * Stick to use STL and Eigen for data structures. Avoid using OpenCV datastructures in C++, except for image resizing.
+    * Use SoA structuring for performance critical data like point clouds.
+    * Strive to minimize heap allocation and fragmentation by using stack-allocated buffer. In case of heap-allocated, re-use buffers, and avoid tiny buffers in long for loops.
+- **Design**: For the data, try to re-use core lib's data structures. In addition, it is of upmost importance to be able to manipulate large reconstructions efficiently. We strive to handle large amounts of data, so minimize copy operations, use Data-Oriented Design principles, and leverage GPU acceleration where possible (e.g., for map rendering).
+
+### Type Hinting
+- **Strictness**: The codebase uses `pyre-strict`. Ensure all new code has complete type annotations. A comment `# pyre-strict` is often present at the top of files.
+
+### Configuration
+- **Definition**: Default parameters are in `opensfm/config.py` (dataclass `OpenSfMConfig`).
+- **Overrides**: Parameters are overridden by `config.yaml` in the dataset directory.
+- **Access**: Access config values via `data.config['param_name']` (where `data` is a `DataSet` instance).
+
+## 4. Essential Files
+- `opensfm/dataset.py`: **READ THIS FIRST** when dealing with file I/O. Defines where every file lives.
+- `opensfm/types.py`: Defines key data structures (`Reconstruction`, `Shot`, `Camera`).
+- `opensfm/config.py`: Documentation for all tunable parameters.
+- `opensfm/src/map/map.cc`: The backing C++ implementation for `Reconstruction`.
+- `opensfm/commands/`: Implementation of individual pipeline steps.
+
+## 5. Common Pitfalls
+- **Direct File Access**: Avoid manual `open()` calls. Use `dataset.load_*` and `dataset.save_*` methods to ensure consistency with the expected directory structure.
+- **Geometry Types**: Be careful with rotation representations (angle-axis vs matrices). `pygeometry` exposes helper functions; check `opensfm/src/geometry/` for implementation details if behavior is unclear.
+- **Skipping Conda**: when running or building the library, don't forget to activate the conda environment at least once.
+
+## 6. General Algorithmic Notes
+- **Multi-Platforms**: The codebase is designed to run on Windows, Linux, and macOS. Avoid platform-specific code unless absolutely necessary, and use cross-platform libraries (e.g., Qt for GUI, STL/Eigen for C++).
+
+Always refer to instructions/memory.instructions.md for project-specific commands, style guides, and recent progress updates.
+
+If and only if the prompt iS MEMUPDATE, summarize the current state of the project in instructions/memory.instructions.md, including recent progress, lessons learned, and known technical debt. This information should be concise and focused on actionable insights for developers.
+
+## 7. For coding agents:
+- **STRICTLY FORBIDDEN**:
+    - Do not perform any Git operations (commits, pushes, pulls, merges, rebases, etc.). This is a shared repository and all Git operations must be performed by human developers to ensure proper code review and collaboration.
+- **BEST PRACTICES**:
+    - You will favor using bash tools such as sed or awk to perform complex code replacements, and avoid doing complex string manipulation in Python with ad-hoc scripts that open and write to files directly. This is to ensure that code changes are made in a clear and auditable way for humans, and to leverage the power of existing command-line tools for code transformation.

.github/scripts/generate_coverage_badge.py

@@ -0,0 +1,154 @@
+#!/usr/bin/env python3
+"""Generate a repository-local SVG badge from Cobertura XML reports."""
+
+import argparse
+import json
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Dict, List, Optional
+from xml.etree import ElementTree
+from xml.sax.saxutils import escape
+
+
+@dataclass(frozen=True)
+class CoverageTotals:
+    """Coverage counters extracted from a Cobertura XML report."""
+
+    name: str
+    lines_covered: int
+    lines_valid: int
+    branches_covered: int
+    branches_valid: int
+
+    def line_rate(self) -> Optional[float]:
+        if self.lines_valid == 0:
+            return None
+        return (self.lines_covered / self.lines_valid) * 100.0
+
+    def branch_rate(self) -> Optional[float]:
+        if self.branches_valid == 0:
+            return None
+        return (self.branches_covered / self.branches_valid) * 100.0
+
+    def to_dict(self) -> Dict[str, object]:
+        return {
+            "name": self.name,
+            "lines_covered": self.lines_covered,
+            "lines_valid": self.lines_valid,
+            "line_rate": self.line_rate(),
+            "branches_covered": self.branches_covered,
+            "branches_valid": self.branches_valid,
+            "branch_rate": self.branch_rate(),
+        }
+
+
+def _parse_count(attributes: Dict[str, str], key: str) -> int:
+    raw_value = attributes.get(key)
+    if raw_value is None:
+        return 0
+    return int(float(raw_value))
+
+
+def _load_report(report_path: Path) -> CoverageTotals:
+    root = ElementTree.parse(report_path).getroot()
+    return CoverageTotals(
+        name=report_path.stem,
+        lines_covered=_parse_count(root.attrib, "lines-covered"),
+        lines_valid=_parse_count(root.attrib, "lines-valid"),
+        branches_covered=_parse_count(root.attrib, "branches-covered"),
+        branches_valid=_parse_count(root.attrib, "branches-valid"),
+    )
+
+
+def _merge_reports(reports: List[CoverageTotals]) -> CoverageTotals:
+    return CoverageTotals(
+        name="total",
+        lines_covered=sum(report.lines_covered for report in reports),
+        lines_valid=sum(report.lines_valid for report in reports),
+        branches_covered=sum(report.branches_covered for report in reports),
+        branches_valid=sum(report.branches_valid for report in reports),
+    )
+
+
+def _format_percentage(value: Optional[float]) -> str:
+    if value is None:
+        return "n/a"
+    formatted = f"{value:.1f}".rstrip("0").rstrip(".")
+    return f"{formatted}%"
+
+
+def _badge_color(rate: Optional[float]) -> str:
+    if rate is None:
+        return "#9f9f9f"
+    if rate < 50.0:
+        return "#e05d44"
+    if rate < 70.0:
+        return "#fe7d37"
+    if rate < 85.0:
+        return "#dfb317"
+    if rate < 95.0:
+        return "#a4a61d"
+    return "#4c1"
+
+
+def _segment_width(text: str) -> int:
+    return max(10, len(text) * 7 + 10)
+
+
+def _build_badge_svg(label: str, value: str, color: str) -> str:
+    label_width = _segment_width(label)
+    value_width = _segment_width(value)
+    total_width = label_width + value_width
+    label_center = label_width * 5
+    value_center = (label_width + (value_width / 2.0)) * 10
+    aria_label = escape(f"{label}: {value}")
+    label_text = escape(label)
+    value_text = escape(value)
+    return f"""<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"{total_width}\" height=\"20\" role=\"img\" aria-label=\"{aria_label}\">\n  <linearGradient id=\"s\" x2=\"0\" y2=\"100%\">\n    <stop offset=\"0\" stop-color=\"#bbb\" stop-opacity=\".1\"/>\n    <stop offset=\"1\" stop-opacity=\".1\"/>\n  </linearGradient>\n  <clipPath id=\"r\">\n    <rect width=\"{total_width}\" height=\"20\" rx=\"3\" fill=\"#fff\"/>\n  </clipPath>\n  <g clip-path=\"url(#r)\">\n    <rect width=\"{label_width}\" height=\"20\" fill=\"#555\"/>\n    <rect x=\"{label_width}\" width=\"{value_width}\" height=\"20\" fill=\"{color}\"/>\n    <rect width=\"{total_width}\" height=\"20\" fill=\"url(#s)\"/>\n  </g>\n  <g fill=\"#fff\" text-anchor=\"middle\" font-family=\"Verdana,Geneva,DejaVu Sans,sans-serif\" text-rendering=\"geometricPrecision\" font-size=\"110\">\n    <text x=\"{label_center}\" y=\"140\" transform=\"scale(.1)\" fill=\"#010101\" fill-opacity=\".3\">{label_text}</text>\n    <text x=\"{label_center}\" y=\"130\" transform=\"scale(.1)\">{label_text}</text>\n    <text x=\"{value_center}\" y=\"140\" transform=\"scale(.1)\" fill=\"#010101\" fill-opacity=\".3\">{value_text}</text>\n    <text x=\"{value_center}\" y=\"130\" transform=\"scale(.1)\">{value_text}</text>\n  </g>\n</svg>\n"""
+
+
+def _build_summary(reports: List[CoverageTotals], total: CoverageTotals) -> Dict[str, object]:
+    return {
+        "reports": [report.to_dict() for report in reports],
+        "total": total.to_dict(),
+        "badge": {
+            "label": "coverage",
+            "message": _format_percentage(total.line_rate()),
+            "color": _badge_color(total.line_rate()),
+        },
+    }
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("reports", nargs="+", type=Path,
+                        help="Cobertura XML coverage reports")
+    parser.add_argument("--output", required=True, type=Path,
+                        help="Path to the SVG badge to write")
+    parser.add_argument(
+        "--summary-json",
+        required=True,
+        type=Path,
+        help="Path to a JSON summary file written alongside the badge",
+    )
+    args = parser.parse_args()
+
+    reports = [_load_report(path) for path in args.reports]
+    total = _merge_reports(reports)
+    badge_value = _format_percentage(total.line_rate())
+    badge_svg = _build_badge_svg(
+        "coverage", badge_value, _badge_color(total.line_rate()))
+
+    args.output.parent.mkdir(parents=True, exist_ok=True)
+    args.summary_json.parent.mkdir(parents=True, exist_ok=True)
+    args.output.write_text(badge_svg, encoding="utf-8")
+    args.summary_json.write_text(
+        json.dumps(_build_summary(reports, total),
+                   indent=2, sort_keys=True) + "\n",
+        encoding="utf-8",
+    )
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

.github/workflows/conda.yml

@@ -0,0 +1,43 @@
+name: Conda
+
+on: [push, pull_request]
+
+jobs:
+
+  build-test:
+    name: Build conda environment and run tests
+
+    strategy:
+      matrix:
+        include:
+          - os: ubuntu-24.04
+            lockfile: conda-linux-64.lock
+          - os: ubuntu-22.04
+            lockfile: conda-linux-64.lock
+          - os: macos-latest
+            lockfile: conda-osx-arm64.lock
+
+    runs-on: ${{ matrix.os }}
+
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          submodules: true
+
+      - name: Setup Miniconda
+        uses: conda-incubator/setup-miniconda@v3
+        with:
+          environment-file: ${{ matrix.lockfile }}
+          activate-environment: opensfm
+
+      - name: Build OpenSfM
+        shell: bash -l {0}
+        run: pip install -e .[test]
+
+      - name: Run C++ tests
+        shell: bash -l {0}
+        run: cd build && ctest
+
+      - name: Run Python tests
+        shell: bash -l {0}
+        run: export LD_PRELOAD=$CONDA_PREFIX/lib/libtcmalloc.so && python -m pytest