Corrupted outputs after update to ODX 3.8.0

[Orvo300]

How did you install ODX (docker, installer, etc.)?

Browser: None
Operating System: Ubuntu 24.04.4 LTS, WebODM installed in docker
Cient browser: Edge 150 on Windows 11

What is the problem?

After a recent update, the local ODX nodes produced corrupted data for datasets that previously worked. The issue appears only on the local ODX 3.8.0 nodes. Logs show Negative GSD estimated and outputs become visibly degraded (unstable geometry, holes, bad GSD).

With the same datasets and processing options, Lightning (lgt 4.3.0) did not show this warning and produced normal results. We then rolled local nodes back to ODX 3.7.1 and the warning disappeared with local processing returning to normal quality, same datasets.

How can we reproduce this? (What steps trigger the problem? What parameters are you using for processing? Include screenshots. If you are having issues processing a dataset, you must include a copy of your dataset uploaded on Dropbox, Google Drive or https://dronedb.app)

All affected jobs were processed with the same preset. Last week we processed about 50 jobs successfully on the 3.7.1 nodes, while this week 52/52 jobs processed on the new 3.8.0 showed corrupted results.

Reproduction steps:

1 - use WebODM with local ODX nodes upgraded to ODX 3.8.0
2 - run any task
3 - observe task run and assess corrupted data / logs
4 - run equivalent jobs with the same preset on Lightning (lgt 4.3.0) and / or rollback local nodes to ODX 3.7.1
5 - observe that the warnings disappears and output quality returns to normal.

parameters used in affected local tasks:

auto-boundary=true, dem-resolution=2, dsm=true, feature-quality=high, matcher-neighbors=10, mesh-size=120000, min-num-features=15000, optimize-disk-space=true, orthophoto-resolution=1, pc-filter=2.5, pc-quality=high, sfm-algorithm=incremental, skip-3dmodel=true, skip-band-alignment=true.

Example dataset that processes perfect on 3.7.1 but not on 3.8.0:
https://drive.google.com/drive/folders/1eXKnoSwuWJqzrVjnq2AdvkcirtDxIM1X?usp=sharing

[pierotofy]

How can we reproduce this? (What steps trigger the problem? What parameters are you using for processing? Include screenshots. If you are having issues processing a dataset, you must include a copy of your dataset uploaded on Dropbox, Google Drive or https://dronedb.app/)

[Orvo300]

How can we reproduce this? (What steps trigger the problem? What parameters are you using for processing? Include screenshots. If you are having issues processing a dataset, you must include a copy of your dataset uploaded on Dropbox, Google Drive or https://dronedb.app/)

Hello @pierotofy , i provided the dataset in the first post. Thank you !

[pierotofy]

Works fine with the default settings:

report.pdf

Always test with the default settings before opening a bug report. I'm going to guess --matcher-neighbors is the culprit.

[Orvo300]

Thanks for checking, and yes, it know it works with the defaults and I appreciate you testing.
But I don't use the defaults, I use the preset mentioned above and it worked with 3.71 but not with 3.8.0, thats the issue i was trying to post.
Anyway, I will update again to 3.8.0 and play a little with "--matcher-neighbors and reply with what I found.

[Orvo300]

@pierotofy

Hi and thanks again for sharing your ODM report and testing the dataset.

I ran a controlled reproduction on my side using OATS to isolate this behavior in ODX 3.8.0.
I used the same provided dataset (193 images) and ran an opensfm-only matrix so I could compare SfM behavior directly.

I tested three cases on 3.8.0:

ODX defaults (--end-with opensfm)
my defaults without --matcher-neighbors
my defaults with --matcher-neighbors 10
All three runs completed successfully and reached Finished opensfm stage / ODX app finished, but in all three task outputs I still got:
"Negative GSD estimated, this might indicate a flipped Z-axis."

So in my environment, on the provided dataset, the warning is not specific to matcher-neighbors.

I also tested with ODX 3.7.1 on my side, and I do not see this issue there.

Could you please check your run logs and confirm whether you had any Negative GSD warnings during opensfm ?

I mean I can work with 3.7.1 but i like to stay up-to-date and try to change my preset as little as possible, as i have good results with it, thats why i press with this.

Thank you !

[pierotofy]

Try updating, this should be fixed/improved with #32

[Orvo300]

Yes, can confirm, GSD error (negative GSD) is gone in 3.8.1