Light mode

[TheMariday]

It would be good to support cameras which don't have exposure control.
To do so effectively, we need to subtract an initial image to compensate for scene brightness.

[tschundler]

This is how my tool works. Then there's no problem with something bright-ish in the frame confusing it and less need to fuss with threshold values. So I grab several "dark" frames, average and do a blur on it.

Thinking about it again now, rather than a normal blur, I'd take the highest value within a radius. It would end up looking like camera bokeh. That would allow for some minor camera movement / minor background movement.

To count as "not dark" I know I experimented with dark vs average brightness across time (assuming off at least 50% of the time) vs lowest 20-percentile + standard deviation of the value or something like that. If you're interested, I could check exactly what the latest code is doing. (but that doesn't mean it is the best)

---

it gets to be a lot of hyper-parameters quickly. Another thing I considered was using some ML type tools to explore the hyperparameters processing the same dataset different ways to find a good combination. (since some things like how I do thresholding could also impact whether boom filter matching gives better results or worse results)

---

I've thought if I ever build a GUI, ideally I'd also let the user manually mask out tricky areas (eg places with reflections) if they want more accuracy.

[simap]

On a mac, I was unable to get any camera with exposure control working, even supported webcams. I used a 3rd party tool called "webcam settings" to drop exposure way way down until the preview mode started looking right. The app is kind of hit or miss, and only 2 star rating so I can't really recommend it.

Got me thinking, would a diff exposure mode be possible, where it could work with auto exposure and mostly bright frames? Like take a picture with no LEDs, light an LED, subtract the first frame from the second.

[TheMariday]

@simap I'm sorry you couldn't get any webcams working on Mac. Sadly I don't own a mac so can't do a huge amount of testing compared to windows or linux.

And yes bright mode is something I'm thinking about at the moment. Can be addressed once #50 is done.

Thank you for the feedback and I'll keep you posted with any updates to bright mode

[JohnnyMarnell]

Also had trouble on a Mac with multiple webcams. Using my iPhone with free version of Camo Studio app worked pretty well

[TheMariday]

Blimey that's some creative thinking!

I'm going to take off the mac support badge for now until I can verify why mac's are having issues with webcams.

I assume the feed appears fine but the exposure controls are missing?

I will look into how opencv handles webcams on mac in the mean time

[simap]

I assume the feed appears fine but the exposure controls are missing?

Yep. Get this in the terminal:

[INFO/MainProcess] Connecting to device 0 ...
2024-12-17 13:55:15.965 python[90911:3780639] WARNING: AVCaptureDeviceTypeExternal is deprecated for Continuity Cameras. Please use AVCaptureDeviceTypeContinuityCamera and add NSCameraUseContinuityCameraDeviceType to your Info.plist.
[DEBUG/MainProcess] Connected to device 0 with capture method 0
[INFO/MainProcess] setting cam to dark mode
[DEBUG/MainProcess] Setting autofocus to mode 0 with focus 0
[ERROR/MainProcess] Failed to set autofocus to 0
[ERROR/MainProcess] Failed to set focus to 0
[DEBUG/MainProcess] Setting exposure to mode 0
[ERROR/MainProcess] Failed to put camera into manual exposure mode 0
[DEBUG/MainProcess] Setting gain to 0
[ERROR/MainProcess] failed to set camera gain to 0
[DEBUG/MainProcess] Setting exposure to -10
[ERROR/MainProcess] Failed to set exposure to -10

FWIW, this is with the logitech, not the continuity camera. I get similar failed to set xxx messages for the built in camera too.

The image appears, but its full brightness and it fails to detect anything unless I use webcam settings to dial it down to pretty much a black background. FWIW, I hadn't yet installed webcam settings when I first tried this, so I don't think its a conflict with that app. I also don't have the logitech camera settings app installed on this machine, but that could probably have worked too.

I'm super pleased with how things came out in the end, and didn't spend too much time fiddling with it to get it to work. In the end it saved time over very meticulously placing LEDs and mapping with math and fine tuning by hand.

The automatic camera and coordinate detection stuff is just awesome.

[TheMariday]

@simap @JohnnyMarnell I've opened a new ticket here #51 but it looks like opencv just doesn't support exposure adjustment on Mac. I'll update the docs to reflect this.

Glad that everything else seems to be working as expected though!

[tschundler]

Trying to use this today with my laptop's built-in webcam that only supports auto-exposure was pretty challenging, even at threshold 250 and a dim room.

---

I dug up my old code:

    @functools.cached_property
    def black_redch(self):
        f = self.black[:,:,0]
        return cv2.dilate(f, np.ones((3, 3), 'uint8'), iterations=2)
    
    @functools.cached_property
    def blown_mask(self):
        # mask is bigger than problematic area
        blown = cv2.dilate(self.black_grey, np.ones((5, 5), 'uint8'), iterations=2)
        #return blown > to_grey(self.white).mean() * 3
        return blown > min(float(histbin(self.white_grey, thresh=0.95) * 1.44), 254)

    
    def redframe(self, n):
        # only red channel of frame n
        img = self.frame(n)[:,:,0]        
        ref = self.black_redch
        clipped = img < ref
        img = img - ref
        img[clipped] = 0 # these would become negative
        img[self.blown_mask] = 0 # bright areas of reference frame are zero'd
        return img

....but thinking about this now, those "blown" areas are already values around 250, so do I really need both the "black" reference and the blown out mask?

[TheMariday]

I think the best approach here is to capture a frame prior to the scan starting and subtract it from subsequent frames as you have done previously.

Just to clarify that marimapper currently doesn't do any dynamic threshold setting yet

[russss]

I think the best approach here is to capture a frame prior to the scan starting and subtract it from subsequent frames as you have done previously.

I just tried hacking this in and initial impressions are pretty promising using my shitty macbook pro webcam (which obviously doesn't support exposure adjustment). It might be possible to lose the exposure controls entirely.

I'll try and sort out a draft PR with this in but my working copy is a bit of a disaster currently