Optimise ctransform code

[jonwright]

See #556

• see if xyz + gv can be merged
• sort out threading (one thread for small problems)
• put the benchmark code into test somewhere
• merge into point-by-point
• Locate and upload the frame harvesting code

[AxelHenningsson]

I have a single threaded version of the get_voxel_idx() that runs about 30-40 x faster than current implementations. It requires a one time sorting of the pks and then one extra integer column with indices to be stored.

It is implemented in pure python with some numba.jitting.

Let me know if this is interesting, or you alrady have something even faster than that. To me this seems fast enough.

I would be happy to make a PR and some unit-test for my version during the day. Let me know if it is worth while.

Cheers

[jonwright]

Sure - make a pull request and we can try to compare - it sounds like a similar idea?

[AxelHenningsson]

Yes, probably similar ^^. I am splitting the columnfile into custom omega bins, and then sorting each bin by dty. I then store the bin partition indices, as well as the dty sub-paritions (partitions per omega bin). Access is then a sweep over omega bins (these are 0.2 - 2 degrees or similar, so very few) and for each bin I grab the sub-partition of dtys that are relevant using a padding computed from the derivatives of dty with respect to omega and the selected bin size.

The cool thing about this approach is that the omega bin size can be optimized for the dataset. I.e depending on the dataset, there is an optimal omega bins size that minimises the number of data that needs to be touched.

Is this what you are doing as well?

[jonwright]

The cf_2d is produced in frame order. It should already be sorted by frame_id with a single dty+omega for each frame. If the frames are on a regular grid you just use a sinogram <-> frame_id lookup. So see which frames are hit for a particular voxel and then harvest the frames that can be touched.

It should correspond to doing a single Astra style sample space to sinogram space transform, and then getting the list of peaks in each sinogram pixel. Does that make sense?

[AxelHenningsson]

AH, ok. Yes that makes perfect sense. It is like my algorithm but with a fixed omega bin size equal to the stepsize used in the experiment ( belive )

[AxelHenningsson]

There is now a PR with what I propose here: #558

I am sure you can do better Jon, from experience I know it is difficult to compete with you when it comes to fast codes ^^

Cheers
Axel

[jonwright]

Merged - thanks! I am drowning in admin these days, and I don't want to stop everyone else from making progress while I try to speed up that processing...