downstream testing workflow dispatch

[jonhealy1]

Related Issue(s):

Description:

• GitHub Actions workflow for downstream integration testing (stac-fastapi-pgstac and stac-fastapi-elasticsearch-opensearch) to catch breaking changes on pull requests.

PR Checklist:

• pre-commit hooks pass locally
• Tests pass (run make test)
• Documentation has been updated to reflect changes, if applicable, and docs build successfully (run make docs)
• Changes are added to the CHANGELOG.

[jonhealy1]

@vincentsarago I think this is useful. I'm not sure if we want it running on every pr though?

[jonhealy1]

The stac-fastapi-pgstac tests fail as expected with the new v1.1.0 sort conformance class:

=========================== short test summary info ============================
FAILED tests/api/test_api.py::test_default_app[0.9.9] - AssertionError: assert 'https://api.stacspec.org/v1.0.0/ogcapi-features#sort' in ['http://www.opengis.net/spec/cql2/1.0/conf/basic-cql2', 'http://www.opengis.net/spec/cql2/1.0/conf/cql2-json', 'http://www.opengis.net/spec/cql2/1.0/conf/cql2-text', 'http://www.opengis.net/spec/ogcapi-common-2/1.0/conf/simple-query', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/core', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/geojson', 'http://www.opengis.net/spec/ogcapi-features-1/1.0/conf/oas30', 'http://www.opengis.net/spec/ogcapi-features-3/1.0/conf/features-filter', 'http://www.opengis.net/spec/ogcapi-features-3/1.0/conf/filter', 'https://api.stacspec.org/v1.0.0-rc.1/collection-search', 'https://api.stacspec.org/v1.0.0-rc.1/collection-search#fields', 'https://api.stacspec.org/v1.0.0-rc.1/collection-search#filter', 'https://api.stacspec.org/v1.0.0-rc.1/collection-search#free-text', 'https://api.stacspec.org/v1.0.0-rc.1/collection-search#query', 'https://api.stacspec.org/v1.0.0-rc.2/item-search#filter', 'https://api.stacspec.org/v1.0.0/collections', 'https://api.stacspec.org/v1.0.0/collections/extensions/transaction', 'https://api.stacspec.org/v1.0.0/core', 'https://api.stacspec.org/v1.0.0/item-search', 'https://api.stacspec.org/v1.0.0/item-search#fields', 'https://api.stacspec.org/v1.0.0/item-search#query', 'https://api.stacspec.org/v1.0.0/ogcapi-features', 'https://api.stacspec.org/v1.0.0/ogcapi-features#fields', 'https://api.stacspec.org/v1.0.0/ogcapi-features#query', 'https://api.stacspec.org/v1.0.0/ogcapi-features/extensions/transaction', 'https://api.stacspec.org/v1.1.0/collection-search#sort', 'https://api.stacspec.org/v1.1.0/item-search#sort', 'https://api.stacspec.org/v1.1.0/ogcapi-features#sort']
====== 1 failed, 1315 passed, 6 xfailed, 29 warnings in 416.72s (0:06:56) ======

[vincentsarago]

🤔 IMO breaking change should be caught by CI not by testing downstream application.

I did a poor job reviewing the PR which lead to publishing the breaking change in a minor release.

[jonhealy1]

@vincentsarago What do you mean by - caught by CI? I should have noticed there was a breaking change too.

It seems to break downstream quite frequently and I don't think it's usually anyone's fault. Sometimes a minor change that seems fine can cause something to go wrong in SFEOS and/or stac-fastapi-pgstac. Even if this only ran on tags we would at least know if something broke right away

It's not a huge deal because we do pin the stac-fastapi libraries in SFEOS, but we shouldn't need to do this, we should be able to have something like <7.0.0 I think.

[vincentsarago]

What do you mean by - caught by CI?

I meant caught by the python tests. They should tell us when something breaks.

Some people will tell you that downstream app should strictly pin upstream libraries ==6.3.1 and then run tests for breaking change (so the opposite thing compare to this PR).

TBH, I do see the appeal of adding this but on the other hand I don't want to add more GitHub Actions. In general I'm trying to reduce the number of necessary actions to limit the consumption of the repos I work on but now with AI this seems useless 😓

What do you think about making this action a workflow_dispatch only? so the maintainers run this just before merging a PR if they want to double check?

[jonhealy1]

@vincentsarago I think making this a workflow_dispatch only makes a lot of sense. I will update it

[jonhealy1]

@vincentsarago Do you mean you have more work to do now because of ai?

[vincentsarago]

I meant that now AI is using a lot of more powers than all the GitHub Actions 😅

[jonhealy1]

Haha it is true. It's like bitcoin mining is almost environmentally friendly now.