Surpassing Humans and Computers with JELLYBEAN: Crowd-Vision-Hybrid Counting Algorithms

Abstract

Counting objects is a fundamental primitive operation in image processing, and has many scientific, health, surveillance, security, and military applications. Existing supervised computer vision techniques typically require large quantities of labeled training data, and even with that, fail to perform well in all but the most stylized settings. We present algorithms to count objects in images using humans, by repeatedly subdividing the image into smaller images with fewer objects. Our algorithms have several desirable properties: (i) they are theoretically optimal, in that they ask as few questions as possible to humans (under certain intuitively reasonable assumptions that we justify in our paper experimentally); (ii) they operate under stand- alone or hybrid modes, in that they can either work independent of computer vision algorithms, or work in concert with them, depending on whether the computer vision techniques are available or useful for the given setting; (iii) they perform very well in practice, returning accurate counts on images that no individual worker or Machine Learning algorithm can count correctly, while not incurring a high cost.