Rework geoloc to support yaw steering and numba for optimisation#229
Rework geoloc to support yaw steering and numba for optimisation#229mraspaud wants to merge 5 commits intopytroll:mainfrom
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #229 +/- ##
==========================================
- Coverage 90.48% 88.69% -1.80%
==========================================
Files 19 19
Lines 3048 3679 +631
==========================================
+ Hits 2758 3263 +505
- Misses 290 416 +126
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| # Module-level cached Transformer — avoids re-creating the PROJ context on | ||
| # every get_lonlatalt() call. | ||
| _GEOCENT_TO_LATLONG = None |
There was a problem hiding this comment.
Would it be simpler to do an @cache decorator? Essentially the same thing. Or is there really a measurable performance difference in just recreating it?
| global _GEOCENT_TO_LATLONG | ||
| if _GEOCENT_TO_LATLONG is None: | ||
| _GEOCENT_TO_LATLONG = Transformer.from_crs( | ||
| dict(proj="geocent"), dict(proj="latlong")) |
There was a problem hiding this comment.
Are there EPSG or pyproj constant versions of these? It seems odd to create these CRSes from the PROJ.4-style dictionary/string rather than something more explicit. Especially if a and b radii can be specified or use an existing definition that match the A and B global in this module.
| if _HAS_NUMBA: | ||
| lon, lat, alt = _numba_ecef_to_lonlatalt(pos[0], pos[1], pos[2]) | ||
| else: | ||
| lon, lat, alt = _get_transformer().transform(*(pos * 1000)) |
There was a problem hiding this comment.
So all of this is to say that the numba version is faster than pyproj? Is that because of the parallel computing part of it or something else in the way the algorithm is implemented?
There was a problem hiding this comment.
There is no magic in the implemented algorithm, so just the parallelism + jit made it much faster than pyproj
There was a problem hiding this comment.
Well pyproj and PROJ are both compiled so I'd assume it is purely the parallelism. I'm mostly worried about numba as a dependency when depending on Cython and openmp (using prange) may be "easier" in terms of getting this optimization all the time or at least in more instances/environments since openmp is also often required for pykdtree optimizations. There would be no conditionals of which function is used, it would just be a C version. Or...dask could be an option too since it is still parallel but with an overhead. Obviously you've already implemented this solution so it is hard to not use it and implement something else.
2 participants
This PR reworks the geoloc module, by providing generic instrument definition builders.
Moreover, we implement yaw-steering and add (optional) numba optimisation.
For generating a full swath of lon/lat, the speed-up goes from 5x to 25x depending on the geometry.
I used AI for this work.