The data load/save mechanism and formats are although working but very slow and inefficient. For instance, the saved list of dicts or dicts of dicts are extremely slow and inefficient.
-
Figure out whether the format that pupil saves to is the way to go during run time or simple pandas pickling mechanism is the way to go.
-
What data formats should be preserved and what formats should be changed i.e. pupil, marker positions dictionaries?
-
What about the other test code that I used before which utilizes the hdf5 format.
-
[ performance evaluation of different formats at least the top 2-3 candidates ]
-
[ Implementing the saving first ]
-
[ Implementing the reading in next]
-
[Test the fail cases i.e. mid analysis failure and follow where it was left off]
The data load/save mechanism and formats are although working but very slow and inefficient. For instance, the saved list of dicts or dicts of dicts are extremely slow and inefficient.
Figure out whether the format that pupil saves to is the way to go during run time or simple pandas pickling mechanism is the way to go.
What data formats should be preserved and what formats should be changed i.e. pupil, marker positions dictionaries?
What about the other test code that I used before which utilizes the hdf5 format.
[ performance evaluation of different formats at least the top 2-3 candidates ]
[ Implementing the saving first ]
[ Implementing the reading in next]
[Test the fail cases i.e. mid analysis failure and follow where it was left off]