DD-2318: split terms of use and access by jo-pol · Pull Request #251 · DANS-KNAW/dataverse

jo-pol · 2026-07-28T13:53:12Z

What this PR does / why we need it:

This PR prepares the Dataverse codebase for file-specific licenses/terms by splitting the legacy “TermsOfUseAndAccess” concept into two separate models (“TermsOfUseOrLicense” and “TermsOfAccess”) and updating persistence, API, UI, and tests accordingly.

Changes:

Introduces new termsofaccess and termsofuseorlicense tables (Flyway) and updates entity mappings and references.
Updates API/commands/JSON export-import paths to read/write the new split terms objects.
Updates UI (xhtml), exporters, and test fixtures to reflect the new model.
Added validation tests to DatasetVersionTest

Which issue(s) this PR closes:

Closes #

Special notes for your reviewer:

...

Suggestions on how to test this:

check out dans-core-system at least at 64a840ff ... on 7/27/26 at 1:25 PM

Build and on the DANS VM dev_dataversenl_v6.10-PATCH-7_2026-07-19

deploy.py -e shared_dataverse_payara_dir=payara7 --dataverse-war external/dataverse/target/dataverse dev_dataversenl

The deploy above runs the flyway script if not done by a previous deploy.
For manual test of the flyway script, see: DD-2317: split table TermsOfUseAndAccess #247
Note that the name and content of the script changed.
Visit the web-UI of the VM and show a dataset
Upload a file, save the changes, restrict its access.
Edit terms of the same dataset, select 'custom terms' as license and fill fields in both groups.
Publish the changes.
Edit terms of the same dataset and publish again.
Create and use a template with both types of terms.
Commands touched by this PR
- UpdateDatasetTermsOfAccess guides
- UpdateDatasetLicenseCommand guides
- UpdateDatasetFieldsCommand guides
  the lastUpdateTime from the exported metadata causes timestamp outdated error
- RestrictFileCommad guides
- PublishDatasetCommand guides
  requires a draft, api/datasets/9/actions/:publish?type= ... major or minor
- FinalizeDatasetPublicationCommand (called via PublishDatasetCommand)
- CreateTemplateCommand guides
  Not possible to specify TermsOfUseOrLisence nor TermsOfAccess
- CuratePublishedDatasetVersionCommand
  requires a draft, api/datasets/9/actions/:publish?type=updatecurrent
Scenarios for validation errors.
- unchecking request access on a dataset with restricted files, does not cause an error message. When saving, the page only scrolls up. This is not introduced by this PR.
indexing (license facets)
tests covering copilot review comments

Does this PR introduce a user interface change? If mockups are available, please link/include them here:

Is there a release notes update needed for this change?:

Yes, a DB table is split into two tables and its references needs updates, so:

Upgrading requires a maintenance window and downtime. Please plan accordingly, create backups of your database, etc.

Additional documentation:

Cherry picked squashed commits of DANS-KNAW#248

This reverts commit 8ec6937.

Copilot

Pull request overview

This PR prepares the Dataverse codebase for file-specific licenses/terms by splitting the legacy “TermsOfUseAndAccess” concept into two separate models (“TermsOfUseOrLicense” and “TermsOfAccess”) and updating persistence, API, UI, and tests accordingly.

Changes:

Introduces new termsofaccess and termsofuseorlicense tables (Flyway) and updates entity mappings and references.
Updates API/commands/JSON export-import paths to read/write the new split terms objects.
Updates UI (xhtml), exporters, and test fixtures to reflect the new model.

Reviewed changes

Copilot reviewed 68 out of 68 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
tests/shell/DANS-v6-10/template-CCBY.json	Adds a shell-test JSON template used for manual validation.
tests/shell/DANS-v6-10/restrict-file.sh	Adds a manual shell workflow for deploy/restrict/publish scenarios.
tests/shell/DANS-v6-10-restrict-file.sh	Adds a shorter manual shell workflow for file restriction testing.
src/test/java/edu/harvard/iq/dataverse/util/template/TemplateBuilder.java	Updates test template builder to create split terms objects.
src/test/java/edu/harvard/iq/dataverse/util/json/JsonPrinterTest.java	Updates JSON printer tests to assert split terms serialization.
src/test/java/edu/harvard/iq/dataverse/util/FileUtilTest.java	Updates FileUtil tests to use split terms.
src/test/java/edu/harvard/iq/dataverse/pidproviders/doi/datacite/XmlMetadataTemplateTest.java	Updates DataCite XML tests to set split terms objects.
src/test/java/edu/harvard/iq/dataverse/export/SchemaDotOrgExporterTest.java	Updates schema.org export test to use split terms.
src/test/java/edu/harvard/iq/dataverse/engine/command/impl/UpdateDatasetTermsOfAccessCommandTest.java	Updates command test scaffolding for split terms.
src/test/java/edu/harvard/iq/dataverse/engine/command/impl/UpdateDatasetLicenseCommandTest.java	Updates license command tests for split terms and validation.
src/test/java/edu/harvard/iq/dataverse/engine/command/impl/UpdateDatasetFieldsCommandTest.java	Updates field-update command tests for split terms initialization.
src/test/java/edu/harvard/iq/dataverse/engine/command/impl/CreateTemplateCommandTest.java	Updates create-template tests to assert split terms setup.
src/test/java/edu/harvard/iq/dataverse/DatasetVersionTest.java	Updates JSON-LD tests to use split terms.
src/test/java/edu/harvard/iq/dataverse/DatasetVersionDifferenceTest.java	Updates differencing tests for split terms comparison.
src/test/java/edu/harvard/iq/dataverse/api/DatasetsEmbargoAPITest.java	Updates embargo API tests to mock split terms objects.
src/main/webapp/template.xhtml	Passes split terms parameters into template UI includes.
src/main/webapp/manage-templates.xhtml	Passes split terms parameters into manage-templates UI include.
src/main/webapp/guestbook-terms-popup-fragment.xhtml	Updates guestbook/terms popup rendering to use split terms objects.
src/main/webapp/file-edit-button-fragment.xhtml	Updates access-popup binding to `termsOfAccess`.
src/main/webapp/editFilesFragment.xhtml	Updates access-popup binding to `termsOfAccess`.
src/main/webapp/datasetLicenseInfoFragment.xhtml	Updates dataset license/terms display to use split terms objects.
src/main/webapp/dataset.xhtml	Updates dataset page include parameters for split terms.
src/main/webapp/dataset-license-terms.xhtml	Refactors license/terms fragment to use `termsOfUseOrLicense` + `termsOfAccess`.
src/main/resources/db/migration/V6.10.1__DD2318-split-termsofuseandaccess.sql	Adds Flyway migration splitting legacy table into two new tables and updates FKs.
src/main/java/propertyFiles/Bundle.properties	Adds/adjusts bundle keys and TODO notes for split terms messaging.
src/main/java/edu/harvard/iq/dataverse/ValidateTermsOfAccess.java	Renames/retargets validation annotation to TermsOfAccess.
src/main/java/edu/harvard/iq/dataverse/util/json/JsonPrinter.java	Updates JSON export to emit split terms and template terms objects.
src/main/java/edu/harvard/iq/dataverse/util/json/JsonParser.java	Updates JSON parsing to produce split terms objects.
src/main/java/edu/harvard/iq/dataverse/util/FileUtil.java	Updates download-popup logic to check split terms.
src/main/java/edu/harvard/iq/dataverse/util/bagit/OREMap.java	Updates OAI-ORE JSON-LD terms/license output to use split terms.
src/main/java/edu/harvard/iq/dataverse/TermsOfUseOrLicense.java	Introduces/renames the entity holding license + terms-of-use fields.
src/main/java/edu/harvard/iq/dataverse/TermsOfAccessValidator.java	Introduces validator for `TermsOfAccess` compliance with restricted files.
src/main/java/edu/harvard/iq/dataverse/TermsOfAccess.java	Adds new entity for terms-of-access fields and access-request flag.
src/main/java/edu/harvard/iq/dataverse/TemplatePage.java	Updates template page initialization/cloning to use split terms.
src/main/java/edu/harvard/iq/dataverse/Template.java	Updates template entity to reference split terms objects.
src/main/java/edu/harvard/iq/dataverse/search/IndexServiceBean.java	Updates Solr license indexing to use split terms.
src/main/java/edu/harvard/iq/dataverse/license/License.java	Updates License ↔ terms mapping to reference `TermsOfUseOrLicense`.
src/main/java/edu/harvard/iq/dataverse/FilePage.java	Updates file restriction logic and validation checks to use split terms.
src/main/java/edu/harvard/iq/dataverse/FileMetadata.java	Adds per-file terms-of-use/license relation (not yet populated).
src/main/java/edu/harvard/iq/dataverse/engine/command/impl/UpdateDatasetTermsOfAccessCommand.java	Updates terms-of-access command to merge into `TermsOfAccess`.
src/main/java/edu/harvard/iq/dataverse/engine/command/impl/UpdateDatasetLicenseCommand.java	Updates license/custom-terms update logic for split terms objects.
src/main/java/edu/harvard/iq/dataverse/engine/command/impl/UpdateDatasetFieldsCommand.java	Ensures split terms back-references are set before validation.
src/main/java/edu/harvard/iq/dataverse/engine/command/impl/RestrictFileCommand.java	Initializes/updates `TermsOfAccess` when restricting files.
src/main/java/edu/harvard/iq/dataverse/engine/command/impl/PublishDatasetCommand.java	Updates publish-time validation to require license or custom terms-of-use on split terms.
src/main/java/edu/harvard/iq/dataverse/engine/command/impl/FinalizeDatasetPublicationCommand.java	Updates dataset `fileAccessRequest` propagation to use `TermsOfAccess`.
src/main/java/edu/harvard/iq/dataverse/engine/command/impl/CuratePublishedDatasetVersionCommand.java	Updates curation merge logic to handle split terms entities.
src/main/java/edu/harvard/iq/dataverse/engine/command/impl/CreateTemplateCommand.java	Initializes split terms objects when creating templates.
src/main/java/edu/harvard/iq/dataverse/engine/command/impl/AbstractDatasetCommand.java	Updates validation error filtering to match `TermsOfAccess`.
src/main/java/edu/harvard/iq/dataverse/EditDatafilesPage.java	Updates file restriction and terms validation logic to use split terms.
src/main/java/edu/harvard/iq/dataverse/DatasetVersionServiceBean.java	Updates default-custom-terms detection to use new named query/entity.
src/main/java/edu/harvard/iq/dataverse/DatasetVersionDifference.java	Splits terms differencing into access vs use/license sections.
src/main/java/edu/harvard/iq/dataverse/DatasetVersion.java	Updates dataset version entity model, cloning, defaults, and validation for split terms.
src/main/java/edu/harvard/iq/dataverse/datasetutility/AddReplaceFileHelper.java	Updates conflict handling to read validation message from `TermsOfAccess`.
src/main/java/edu/harvard/iq/dataverse/DatasetPage.java	Updates UI/controller logic for access popup and terms validation to use `TermsOfAccess`.
src/main/java/edu/harvard/iq/dataverse/DatasetField.java	Adds a null-guard when grouping fields by metadata block.
src/main/java/edu/harvard/iq/dataverse/dataset/DatasetUtil.java	Updates license lookup to read from `TermsOfUseOrLicense`.
src/main/java/edu/harvard/iq/dataverse/Dataset.java	Updates new-version creation to copy split terms objects.
src/main/java/edu/harvard/iq/dataverse/api/Files.java	Updates restrict-file API error path to validate via `TermsOfAccess`.
src/main/java/edu/harvard/iq/dataverse/api/dto/CustomTermsDTO.java	Updates DTO conversion to produce `TermsOfUseOrLicense`.
src/main/java/edu/harvard/iq/dataverse/api/Dataverses.java	Updates dataset creation to set split terms back-references.
src/main/java/edu/harvard/iq/dataverse/api/Datasets.java	Updates dataset APIs (draft update, terms-of-access update, publish validations) for split terms.
src/main/java/edu/harvard/iq/dataverse/api/datadeposit/SwordServiceBean.java	Updates SWORD license/terms handling to set split terms objects.
src/main/java/edu/harvard/iq/dataverse/api/datadeposit/StatementManagerImpl.java	Updates statement generation to set split terms back-references.
src/main/java/edu/harvard/iq/dataverse/api/Access.java	Updates access-request API to toggle `TermsOfAccess.fileAccessRequest`.
doc/sphinx-guides/source/installation/config.rst	Fixes a small grammar issue in license config docs.
doc/sphinx-guides/source/api/native-api.rst	Adjusts JSON example formatting for terms-of-access API docs.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

workaround to fix broken tests no longer needed

Copilot

Pull request overview

Copilot reviewed 65 out of 65 changed files in this pull request and generated 2 comments.

Suppressed comments (7)

src/main/java/edu/harvard/iq/dataverse/TermsOfAccessValidator.java:44

The block comment describes stricter validation than the code actually implements. The logic allows either fileAccessRequest or non-empty termsOfAccess, but the comment currently says terms "must have terms of access filled in".
src/main/java/edu/harvard/iq/dataverse/util/json/JsonPrinter.java:1730
jsonTermsOfAccess unconditionally adds id using termsOfAccess.getId(). Because JsonObjectBuilder.add(String, long) will unbox the Long, calling this with an unsaved TermsOfAccess (id == null) throws a NullPointerException (the new unit test constructs new TermsOfAccess() without setting an id).
src/main/java/edu/harvard/iq/dataverse/TemplatePage.java:160
When creating a new template in the web UI, TermsOfAccess.fileAccessRequest is no longer defaulted to true (it was previously set on the combined terms object). This changes the default behavior for new templates created via TemplatePage versus templates created via CreateTemplateCommand (which still defaults it to true).

            TermsOfAccess toa = new TermsOfAccess();
            toa.setTemplate(template);
            template.setTermsOfAccess(toa);

src/main/java/edu/harvard/iq/dataverse/util/json/JsonParser.java:365

The new method name parseTermsOfUseAndLicesne contains a spelling error ("Licesne") and doesn't match the JSON it parses (termsOfUseOrLicense). Since this is a public method on JsonParser, the typo is easy to propagate and hard to spot in call sites.
src/test/java/edu/harvard/iq/dataverse/engine/command/impl/UpdateDatasetTermsOfAccessCommandTest.java:68
In setUp(), the stub for datasetVersionMock.getTermsOfUseOrLicense() is created from termsOUAL, but termsOUAL is reassigned afterwards. This makes the assertion assertEquals(termsOUAL, datasetVersionMock.getTermsOfUseOrLicense()) effectively meaningless and could mask regressions.
src/main/java/edu/harvard/iq/dataverse/DatasetPage.java:69
Avoid import static java.lang.Integer.*; here; it pulls in many unrelated symbols and makes usages like MAX_VALUE/parseInt less clear in a large class. Import only what you need (or refer to Integer.MAX_VALUE explicitly).

import static edu.harvard.iq.dataverse.util.JsfHelper.JH;
import static edu.harvard.iq.dataverse.util.StringUtil.isEmpty;
import static java.lang.Integer.*;

src/main/java/edu/harvard/iq/dataverse/engine/command/impl/PublishDatasetCommand.java:15

These imports are now unused: the class never references TermsOfUseOrLicense by name (it uses var), and SettingsServiceBean is not used anywhere. Unused imports can fail builds when static analysis is enforced.

import edu.harvard.iq.dataverse.Dataset;
import edu.harvard.iq.dataverse.DatasetLock;
import edu.harvard.iq.dataverse.Dataverse;
import edu.harvard.iq.dataverse.TermsOfUseOrLicense;
import edu.harvard.iq.dataverse.authorization.Permission;
import edu.harvard.iq.dataverse.authorization.users.AuthenticatedUser;
import edu.harvard.iq.dataverse.engine.command.CommandContext;
import edu.harvard.iq.dataverse.engine.command.DataverseRequest;
import edu.harvard.iq.dataverse.engine.command.RequiredPermissions;
import edu.harvard.iq.dataverse.engine.command.exception.CommandException;
import edu.harvard.iq.dataverse.engine.command.exception.IllegalCommandException;
import edu.harvard.iq.dataverse.settings.SettingsServiceBean;
import edu.harvard.iq.dataverse.util.BundleUtil;

Apply suggestions from code review Co-authored-by: Copilot Autofix powered by AI <[email protected]>

jo-pol · 2026-08-25T08:44:00Z

+    ALTER TABLE datasetversion ADD COLUMN IF NOT EXISTS default_termsofuseorlicense_id BIGINT REFERENCES termsofuseorlicense(id);
+    ALTER TABLE template ADD COLUMN IF NOT EXISTS termsofaccess_id BIGINT REFERENCES termsofaccess(id);
+    ALTER TABLE template ADD COLUMN IF NOT EXISTS termsofuseorlicense_id BIGINT REFERENCES termsofuseorlicense(id);
+    ALTER TABLE filemetadata ADD COLUMN IF NOT EXISTS termsofuseorlicense_id BIGINT REFERENCES termsofuseorlicense(id);


@qqmyers
The already-exists errors are related to eclipse-ee4j/eclipselink#1278
So I filtered them out and are left with the following errors

> journalctl -u payara --no-pager' | grep PER01000 | grep -v 'already exists Aug 25 09:15:39 dataversenl asadmin[769]: PER01000: Got SQLException executing statement "ALTER TABLE DATASETVERSION ADD CONSTRAINT FK_DATASETVERSION_termsofaccess_id FOREIGN KEY (termsofaccess_id) REFERENCES termsofaccess (ID)": org.postgresql.util.PSQLException: ERROR: column "termsofaccess_id" referenced in foreign key constraint does not exist|#] Aug 25 09:15:39 dataversenl asadmin[769]: PER01000: Got SQLException executing statement "ALTER TABLE DATASETVERSION ADD CONSTRAINT FK_DATASETVERSION_default_termsofuseorlicense_id FOREIGN KEY (default_termsofuseorlicense_id) REFERENCES termsofuseorlicense (ID)": org.postgresql.util.PSQLException: ERROR: column "default_termsofuseorlicense_id" referenced in foreign key constraint does not exist|#] Aug 25 09:15:40 dataversenl asadmin[769]: PER01000: Got SQLException executing statement "ALTER TABLE FILEMETADATA ADD CONSTRAINT FK_FILEMETADATA_termsofuseorlicense_id FOREIGN KEY (termsofuseorlicense_id) REFERENCES termsofuseorlicense (ID)": org.postgresql.util.PSQLException: ERROR: column "termsofuseorlicense_id" referenced in foreign key constraint does not exist|#] Aug 25 09:15:40 dataversenl asadmin[769]: PER01000: Got SQLException executing statement "ALTER TABLE TEMPLATE ADD CONSTRAINT FK_TEMPLATE_termsofuseorlicense_id FOREIGN KEY (termsofuseorlicense_id) REFERENCES termsofuseorlicense (ID)": org.postgresql.util.PSQLException: ERROR: column "termsofuseorlicense_id" referenced in foreign key constraint does not exist|#] Aug 25 09:15:40 dataversenl asadmin[769]: PER01000: Got SQLException executing statement "ALTER TABLE TEMPLATE ADD CONSTRAINT FK_TEMPLATE_termsofaccess_id FOREIGN KEY (termsofaccess_id) REFERENCES termsofaccess (ID)": org.postgresql.util.PSQLException: ERROR: column "termsofaccess_id" referenced in foreign key constraint does not exist|#] Aug 25 09:15:40 dataversenl asadmin[769]: PER01000: Got SQLException executing statement "INSERT INTO SEQUENCE(SEQ_NAME, SEQ_COUNT) values ('SEQ_GEN', 0)": org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint "sequence_pkey"

I guess the last error is not alarming too. But now I see I created constraints suffixed with key while most are prefixed with fk. I wonder if this is a problem.

Auto-completion added: "I also see that the constraints are not created on the columns termsofaccess_id and termsofuseorlicense_id, which is what the errors are about."
I doubt that makes sense.

I also see a prefix fx in one case, it originates from V5.11.1.4__hdc-3b2-template-instructions.sql. This sript turns a duplicate exception into already-exists, I wonder whether I should do something similar, if yes, what to catch.

> psql dvndb dvndb=# \d template Table "public.template" Column | Type | Collation | Nullable | Default ------------------------+-----------------------------+-----------+----------+-------------------------------------- id | integer | | not null | nextval('template_id_seq'::regclass) createtime | timestamp without time zone | | not null | instructions | text | | | name | character varying(255) | | not null | usagecount | bigint | | | dataverse_id | bigint | | | termsofaccess_id | bigint | | | termsofuseorlicense_id | bigint | | | Indexes: "template_pkey" PRIMARY KEY, btree (id) "index_template_dataverse_id" btree (dataverse_id) Foreign-key constraints: "fk_template_dataverse_id" FOREIGN KEY (dataverse_id) REFERENCES dvobject(id) "template_termsofaccess_id_fkey" FOREIGN KEY (termsofaccess_id) REFERENCES termsofaccess(id) "template_termsofuseorlicense_id_fkey" FOREIGN KEY (termsofuseorlicense_id) REFERENCES termsofuseorlicense(id) Referenced by: TABLE "dataset" CONSTRAINT "fk_dataset_template_id" FOREIGN KEY (template_id) REFERENCES template(id) TABLE "datasetfield" CONSTRAINT "fk_datasetfield_template_id" FOREIGN KEY (template_id) REFERENCES template(id) TABLE "dataverse" CONSTRAINT "fk_dataverse_defaulttemplate_id" FOREIGN KEY (defaulttemplate_id) REFERENCES template(id) TABLE "dataset" CONSTRAINT "fx_dataset_template_id" FOREIGN KEY (template_id) REFERENCES template(id) dvndb=# \d datasetversion Table "public.datasetversion" Column | Type | Collation | Nullable | Default --------------------------------+-----------------------------+-----------+----------+-------------------------------------------- id | integer | | not null | nextval('datasetversion_id_seq'::regclass) unf | character varying(255) | | | archivalcopylocation | text | | | archivetime | timestamp without time zone | | | createtime | timestamp without time zone | | not null | deaccessionlink | character varying(1260) | | | deaccessionnote | character varying(1000) | | | lastupdatetime | timestamp without time zone | | not null | minorversionnumber | bigint | | | releasetime | timestamp without time zone | | | version | bigint | | | versionnote | character varying(1000) | | | versionnumber | bigint | | | versionstate | character varying(255) | | not null | dataset_id | bigint | | | termsofaccess_id | bigint | | | default_termsofuseorlicense_id | bigint | | | Indexes: "datasetversion_pkey" PRIMARY KEY, btree (id) "index_datasetversion_dataset_id" btree (dataset_id) "one_draft_version_per_dataset" UNIQUE, btree (dataset_id) WHERE versionstate::text = 'DRAFT'::text "unq_datasetversion_0" UNIQUE CONSTRAINT, btree (dataset_id, versionnumber, minorversionnumber) Foreign-key constraints: "datasetversion_default_termsofuseorlicense_id_fkey" FOREIGN KEY (default_termsofuseorlicense_id) REFERENCES termsofuseorlicense(id) "datasetversion_termsofaccess_id_fkey" FOREIGN KEY (termsofaccess_id) REFERENCES termsofaccess(id) "fk_datasetversion_dataset_id" FOREIGN KEY (dataset_id) REFERENCES dvobject(id) Referenced by: TABLE "curationstatus" CONSTRAINT "fk_curationstatus_datasetversion_id" FOREIGN KEY (datasetversion_id) REFERENCES datasetversion(id) TABLE "datasetfield" CONSTRAINT "fk_datasetfield_datasetversion_id" FOREIGN KEY (datasetversion_id) REFERENCES datasetversion(id) TABLE "datasetversionuser" CONSTRAINT "fk_datasetversionuser_datasetversion_id" FOREIGN KEY (datasetversion_id) REFERENCES datasetversion(id) TABLE "filemetadata" CONSTRAINT "fk_filemetadata_datasetversion_id" FOREIGN KEY (datasetversion_id) REFERENCES datasetversion(id) TABLE "guestbookresponse" CONSTRAINT "fk_guestbookresponse_datasetversion_id" FOREIGN KEY (datasetversion_id) REFERENCES datasetversion(id) TABLE "workflowcomment" CONSTRAINT "fk_workflowcomment_datasetversion_id" FOREIGN KEY (datasetversion_id) REFERENCES datasetversion(id) dvndb=# \d filemetadata Table "public.filemetadata" Column | Type | Collation | Nullable | Default ------------------------+------------------------+-----------+----------+------------------------------------------ id | integer | | not null | nextval('filemetadata_id_seq'::regclass) description | text | | | directorylabel | character varying(255) | | | label | character varying(255) | | not null | prov_freeform | text | | | restricted | boolean | | | version | bigint | | | datafile_id | bigint | | not null | datasetversion_id | bigint | | not null | termsofuseorlicense_id | bigint | | | Indexes: "filemetadata_pkey" PRIMARY KEY, btree (id) "index_filemetadata_datafile_id" btree (datafile_id) "index_filemetadata_datasetversion_id" btree (datasetversion_id) Foreign-key constraints: "filemetadata_termsofuseorlicense_id_fkey" FOREIGN KEY (termsofuseorlicense_id) REFERENCES termsofuseorlicense(id) "fk_filemetadata_datafile_id" FOREIGN KEY (datafile_id) REFERENCES dvobject(id) "fk_filemetadata_datasetversion_id" FOREIGN KEY (datasetversion_id) REFERENCES datasetversion(id) Referenced by: TABLE "filemetadata_datafilecategory" CONSTRAINT "fk_filemetadata_datafilecategory_filemetadatas_id" FOREIGN KEY (filemetadatas_id) REFERENCES filemetadata(id) TABLE "vargroup" CONSTRAINT "fk_vargroup_filemetadata_id" FOREIGN KEY (filemetadata_id) REFERENCES filemetadata(id) TABLE "variablemetadata" CONSTRAINT "fk_variablemetadata_filemetadata_id" FOREIGN KEY (filemetadata_id) REFERENCES filemetadata(id)

If I understand, it looks like eclipselink is failing (on an existing db) to add constraints to the table it just tried to create (and failed since it already exists) since it doesn't have the new columns. Your flyway script should try to add those constraints as well (using the syntax in 5.11.1.4 to avoid duplicates). The easiest way to get the naming right is to fire up a new instance where eclipse writes everything and then use the same names you see in the new database. I think you'll probably have to remove the constraint for the column you're removing too (if you haven't already). (I'm not sure why some are fk_ and others have _fkey - possibly changes over time, or perhaps details about how the fields are annotated/defined in Java. In any case, you shouldn't have to figure that out, just use what eclipselink does on a new db.)

The final result is basically that, for a new db, eclipselink does everything, whereas for an existing db, eclipselink will try to create the table and fail, try to create constraints and fail, etc. and then your flyway will run to just make the changes needed. (And, why your flyway will only run once (unless someone edits the flyway_schema_history table), eclipselink will run every startup and switch to complaining about things existing once your flyway has run.)

w.r.t. the INSERT INTO SEQUENCE - that's another eclipselink call that fails on existing databases - not related to your PR at all.

If you're adding new indexes, those are also things you'll need in flyway. Again, easiest to run on an empty db and see what's missing in an existing db.

jo-pol added 3 commits July 28, 2026 11:42

DD-2318 split terms for license per file (squashed)

4a26464

Cherry picked squashed commits of DANS-KNAW#248

fixed unit tests

7700901

version number of flyway script

2c493cf

jo-pol changed the title ~~Dd 2318 spilt terms ouaa~~ DD-2318: spilt terms of use and access Jul 28, 2026

fix flyway

00ce6bb

jo-pol mentioned this pull request Jul 30, 2026

DD-2318: code for splitted table #248

Closed

9 tasks

jo-pol added 2 commits July 30, 2026 11:14

column names to lowercase in flyway

d3e2864

trick to enable flyway

8ec6937

jo-pol commented Jul 30, 2026

View reviewed changes

Comment thread src/main/resources/META-INF/persistence.xml Outdated

jo-pol added 3 commits August 3, 2026 15:01

Revert "trick to enable flyway"

c1ea6ea

This reverts commit 8ec6937.

sync filemetadata class with DB

2e613e0

copilot suggestions

2a430ab

qqmyers reviewed Aug 10, 2026

View reviewed changes

Comment thread src/main/java/edu/harvard/iq/dataverse/engine/command/impl/RestrictFileCommand.java Outdated

jo-pol added 9 commits August 11, 2026 14:17

validate only terms of access

b9f8c40

test script

342ecd0

validate only terms of access

a45a3e6

fix WEB-UI

7502c04

fix api

84a6873

typo

1073924

not null broke updateCurrent

1bc83ea

docs typo + semi manual test protocol

1876fc2

fix broken test

3f40b47

janvanmansum requested a lite review from Copilot August 18, 2026 09:27

Copilot started reviewing on behalf of janvanmansum August 18, 2026 09:28 View session

Copilot AI reviewed Aug 18, 2026

View reviewed changes

janvanmansum changed the title ~~DD-2318: spilt terms of use and access~~ DD-2318: split terms of use and access Aug 18, 2026

jo-pol and others added 4 commits August 18, 2026 13:30

Potential fix for pull request finding

3839d3e

Co-authored-by: Copilot Autofix powered by AI <[email protected]>

no hard coded token

1bf4a55

obsolete file

15d28a7

some of the comments by copilot review

cf45fb2

jo-pol added 8 commits August 20, 2026 08:14

duplicate pointed out in review

8c549e8

restored not intended change

82f5c37

workaround to fix broken tests no longer needed

restored logic

281bce7

restored logic

d489aa3

validation

290401b

comments

7472ff6

duplicate

07ee8b7

split test

0091893

jo-pol requested a lite review from Copilot August 20, 2026 13:44

Copilot started reviewing on behalf of jo-pol August 20, 2026 13:45 View session

Copilot AI reviewed Aug 20, 2026

View reviewed changes

Comment thread src/main/resources/db/migration/V6.10.1__DD2318-split-termsofuseandaccess.sql

Comment thread src/main/resources/db/migration/V6.10.1__DD2318-split-termsofuseandaccess.sql

jo-pol and others added 2 commits August 24, 2026 09:10

Bigserial ID-s

e4f5a8b

Apply suggestions from code review Co-authored-by: Copilot Autofix powered by AI <[email protected]>

select desired violation from returned set

530448e

jo-pol commented Aug 25, 2026

View reviewed changes

jo-pol added 2 commits August 25, 2026 14:00

finetuning test scenario

3cd0afa

constraint, indexes, idempotency

56af4f2

jo-pol commented Aug 31, 2026

View reviewed changes

Comment thread src/main/java/edu/harvard/iq/dataverse/pidproviders/doi/XmlMetadataTemplate.java

jo-pol commented Aug 31, 2026

View reviewed changes

Comment thread src/main/java/edu/harvard/iq/dataverse/util/json/JSONLDUtil.java

irrelevant import changes

972a501

Navigation Menu

Uh oh!

DD-2318: split terms of use and access - #251

DD-2318: split terms of use and access#251
jo-pol wants to merge 35 commits into
DANS-KNAW:v6.10-DANS-DataStationfrom
DANS-KNAW-jp:DD-2318-spilt-termsOUAA

jo-pol commented Jul 28, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

jo-pol Aug 25, 2026

Uh oh!

qqmyers Aug 25, 2026

Uh oh!

qqmyers Aug 25, 2026

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

jo-pol commented Jul 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

jo-pol Aug 25, 2026

Choose a reason for hiding this comment

Uh oh!

qqmyers Aug 25, 2026

Choose a reason for hiding this comment

Uh oh!

qqmyers Aug 25, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jo-pol commented Jul 28, 2026 •

edited

Loading