Skip to content

A big pile of imporovements#10

Merged
tomba merged 44 commits into
masterfrom
test
Apr 27, 2026
Merged

A big pile of imporovements#10
tomba merged 44 commits into
masterfrom
test

Conversation

@tomba

@tomba tomba commented Apr 23, 2026

Copy link
Copy Markdown
Owner

Main things:

  • Full stride support
  • More formats supported
  • A big improvement to tests
  • Fixed RGB & YUV opencv conv issues

tomba added 30 commits April 23, 2026 13:33
libcamera uses its own names which do not always match pixutils' names.
And trying to match based on fourccs is challenging. So let's add a new
field, 'libcamera_name', so that we can explicitly map between libcamera
and pixutils formats.

Signed-off-by: Tomi Valkeinen <[email protected]>
The 'packed' field was a bit too ambiguous.

Signed-off-by: Tomi Valkeinen <[email protected]>
Add a helper to get the bitspp for RAW formats.

Signed-off-by: Tomi Valkeinen <[email protected]>
Disable GC for the run, do a bit more warmup, use perf_counter.

Signed-off-by: Tomi Valkeinen <[email protected]>
Signed-off-by: Tomi Valkeinen <[email protected]>
Start fixing the multi-plane stride issues by fixing the public API.

Signed-off-by: Tomi Valkeinen <[email protected]>
If the caller gives an integer as the stride, and the format is a
multi-plane format, use similar stride extrapolation as libcamera does.

Signed-off-by: Tomi Valkeinen <[email protected]>
Signed-off-by: Tomi Valkeinen <[email protected]>
Signed-off-by: Tomi Valkeinen <[email protected]>
Signed-off-by: Tomi Valkeinen <[email protected]>
Many of them were quite wrong...

Signed-off-by: Tomi Valkeinen <[email protected]>
Convert a random buffer to BGR888 with each available backend (opencv,
numba, numpy) and compare the outputs pairwise. Catches bugs such as
channel swaps, wrong plane offsets, or off-by-one unpacking that locked
per-backend SHAs in the existing test cannot detect.

Comparison uses max-per-channel mean, 99th percentile, and max of the
per-pixel absolute difference. Per-channel (not aggregate) mean is the
key: a channel swap leaves one channel near zero while the other two
diverge by ~85 on random data, which stands out even when the aggregate
mean overlaps with legitimate algorithmic differences (e.g. different
demosaic methods). Tolerances are split by color class (RGB exact, YUV
a few LSB, RAW loose).

Test discovery probes each format at 32x32 with a shared oversized
buffer and emits test methods only for formats where >=2 backends
accept the conversion, keeping the reported skip count at zero.
Allocate the output once and do the channel swap into it directly. The
alpha-drop paths use a slice-copy instead of np.delete, which is
simpler and avoids the strided fancy-indexing. Results are now
naturally C-contiguous, and gives a nice increase in perf.

Signed-off-by: Tomi Valkeinen <[email protected]>
Shift the intermediate 16-bit array in place and then cast, instead of
allocating a shifted copy and then casting. Saves one full-size buffer
allocation in the RAW conversion path.
np.clip() allocates a new array by default; passing out= makes it
modify the dot-product result in place. Avoids one intermediate
float-array allocation per frame.
All three planes are overwritten immediately after allocation, so
np.zeros() needlessly zero-fills the buffer. np.empty() skips the
fill.
tomba added 14 commits April 23, 2026 13:33
Signed-off-by: Tomi Valkeinen <[email protected]>
Camshark calls internal conversion functions with an integer stride,
which fails now. Add a hack to convert the stride-int to a stride-tuple.
Remove later.

Signed-off-by: Tomi Valkeinen <[email protected]>
In most cases this should be no-op, but it makes sure we will handle a
non-contig and shaped arrays correctly too.

Signed-off-by: Tomi Valkeinen <[email protected]>
Signed-off-by: Tomi Valkeinen <[email protected]>
Make numpy/numba converters return None for unsupported formats so that
the top-level conv.py can manage unsupported formats in one place.

Signed-off-by: Tomi Valkeinen <[email protected]>
Catching any exception hides actual issues.

Signed-off-by: Tomi Valkeinen <[email protected]>
The earlier check did the job, but it wasn't quite correct either.

Signed-off-by: Tomi Valkeinen <[email protected]>
@stefanklug

Copy link
Copy Markdown
Collaborator

@tomba I skimmed through the changes and that looks all sane. I think we could take the risk of just merging it :-)

@tomba
tomba merged commit 0decb81 into master Apr 27, 2026
6 checks passed
@tomba

tomba commented Apr 27, 2026

Copy link
Copy Markdown
Owner Author

@tomba I skimmed through the changes and that looks all sane. I think we could take the risk of just merging it :-)

Risk on!

Comment thread pixutils/conv/conv.py
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants