Background
I am currently working on reconstructing s3DXRD datasets such that the resulting microstructure and elastic strain data can subsequently be used for simulation studies. To filter out sporadic reconstruction artifacts, I started using the DREAM3D-NX (proprietary) / SIMPLNX (open-source) ecosystem because it already provides robust filters for handling and processing 3D datasets, and SIMPLNX additionally exposes these filters through Python wrappers.
At the moment, however, there does not seem to exist a standard vendor-agnostic format for general 3DXRD datasets comparable to H5EBSD (https://doi.org/10.1186/2193-9772-3-4) in the 3D-EBSD ecosystem. While H5EBSD works well for EBSD workflows, diffraction-based datasets introduce fundamentally different information and metadata. In particular, elastic strain tensors and diffraction-specific reconstruction outputs do not conceptually fit into H5EBSD. For this reason, I started exploring whether the existing HDF5 reader infrastructure in SIMPLNX could be adapted to facilitate interoperability between general 3DXRD datasets and SIMPLNX.
While discussing possible reader support with the SIMPLNX maintainers (see issue BlueQuartzSoftware/simplnx#1616), an important question came up regarding the stability and conventions of the HDF5 output format used in ImageD11 workflows.
Since ImageD11 is an actively evolving research code, I completely understand that the internal structures and exported formats naturally evolve over time and that full flexibility is intentionally given to the user. At the same time, I was wondering whether there is any longer-term vision regarding a stable and/or versioned file structure for these datasets from an interoperability perspective.
One possible direction we discussed was whether a future schema could perhaps consist of a small set of stable mandatory datasets/groups together with extensible optional datasets. That would preserve interoperability for the core reconstruction data while still allowing workflow-specific metadata or additional datasets to evolve without breaking compatibility.
More broadly, I was also wondering whether, from your perspective as experts working in this area, a more standardized open format for general 3DXRD datasets would even make sense beyond just ImageD11 workflows, or whether the diversity of reconstruction approaches and beamline-specific data makes such standardization impractical. The reason I ask is that we are currently trying to evaluate whether a dedicated SIMPLNX reader for 3DXRD datasets makes sense.
Many thanks in advance for the discussion and inputs.
Best regards,
Nikhil
Background
I am currently working on reconstructing s3DXRD datasets such that the resulting microstructure and elastic strain data can subsequently be used for simulation studies. To filter out sporadic reconstruction artifacts, I started using the DREAM3D-NX (proprietary) / SIMPLNX (open-source) ecosystem because it already provides robust filters for handling and processing 3D datasets, and SIMPLNX additionally exposes these filters through Python wrappers.
At the moment, however, there does not seem to exist a standard vendor-agnostic format for general 3DXRD datasets comparable to H5EBSD (https://doi.org/10.1186/2193-9772-3-4) in the 3D-EBSD ecosystem. While H5EBSD works well for EBSD workflows, diffraction-based datasets introduce fundamentally different information and metadata. In particular, elastic strain tensors and diffraction-specific reconstruction outputs do not conceptually fit into H5EBSD. For this reason, I started exploring whether the existing HDF5 reader infrastructure in SIMPLNX could be adapted to facilitate interoperability between general 3DXRD datasets and SIMPLNX.
While discussing possible reader support with the SIMPLNX maintainers (see issue BlueQuartzSoftware/simplnx#1616), an important question came up regarding the stability and conventions of the HDF5 output format used in ImageD11 workflows.
Since ImageD11 is an actively evolving research code, I completely understand that the internal structures and exported formats naturally evolve over time and that full flexibility is intentionally given to the user. At the same time, I was wondering whether there is any longer-term vision regarding a stable and/or versioned file structure for these datasets from an interoperability perspective.
One possible direction we discussed was whether a future schema could perhaps consist of a small set of stable mandatory datasets/groups together with extensible optional datasets. That would preserve interoperability for the core reconstruction data while still allowing workflow-specific metadata or additional datasets to evolve without breaking compatibility.
More broadly, I was also wondering whether, from your perspective as experts working in this area, a more standardized open format for general 3DXRD datasets would even make sense beyond just ImageD11 workflows, or whether the diversity of reconstruction approaches and beamline-specific data makes such standardization impractical. The reason I ask is that we are currently trying to evaluate whether a dedicated SIMPLNX reader for 3DXRD datasets makes sense.
Many thanks in advance for the discussion and inputs.
Best regards,
Nikhil