Object detection approaches

[Marcel-Simon]

Another way to contribute is training an actual object detection model, which does not require the hotspot detections. The input is an image, the output is a set of bounding box. I would consider this more difficult then the hotspot classification, but I would be happy to work with people on this task as well. First ideas for action items:

• Cropping large patches (such as Full HD) out of all images and making a nice dataset with a reasonable number of seals in it.
• Converting the annotations in a common object detection dataset format, such as VOTT or Coco
• Training an off-the-shelf CNN object detector. If you have never worked in this area before, you could use the tensorflow detection API (https://github.com/tensorflow/models/tree/master/research/object_detection) as it is very complete and nicely written. However it is also quite a large piece of code. If anybody knows an easy to use framework, please mention it below.
• On the long-term / more advanced: training of a cascade of detector, which allows for scanning huge amounts of images. The first detector is very fast, but has a high false positive rate. It is used to discard images which certainly do not contain seals. The second detector is slower and only scans images, which were not discarded by the last detector, and so on. This is similar to the Haar cascade for face detection.

[Athapan]

I got the tensorflow object detection working at one point. Someone mentioned that Azure release an objection detection that work quite well when they tried it.

https://docs.microsoft.com/en-us/azure/cognitive-services/custom-vision-service/python-tutorial-od

I'll see if I can sometime to experiment with these.

[Marcel-Simon]

I added a detection code based on PyTorch so people can get started quickly: https://github.com/Microsoft/arcticseals/tree/master/src/SealDetectionRCNN please refer to the README.md on how to get everything running. We will soon also upload a nice dataset of all the labels we have.