-
Notifications
You must be signed in to change notification settings - Fork 121
Data Storage Arch
The locker requires two primary use cases for data storage, historical archiving and live state.
The intent of the historical archive is to ensure that all collected data is available in a base simple format. This allows for reconstruction to any new required format and for old or recovered disks to have future use. The preferred format is JSON, but if this is technically not possible another format may be used, but it is stressed that it should be simple (near text) and visually inspectable in the future. The archive should contain all the changes and actions on data, not just the current state of the data, meaning deletes, updates, resyncs or anything else are individual records in the historical archive.
It's important to note that while the historical archive is a simple format this requirement does not preclude an index from being created into the data. As design grows, other needs for accessing the archive may arise and the use of an independent index of the data is able to enhance this process. A previous example of this was IJOD, but was reverted until there is further need.
Collections do not have a historical archive. They are intended to represent the collected data, and could be quickly reconstituted from the connector data.