Using computer vision and machine learning to predict corresponding product category (e.g. Italian dish) for a product image (i.e. 2D images).


In our daily lives we often interact with grocery products: e.g. During shopping, or when we want to know the ingredients of a product. Today, such queries are usually triggered by scanning a product’s barcode – a process which requires a bright-light environment, visual contact, active high-resolution camera, close proximity between product and smartphone, and an active command from the user or app to scan the barcode. In the future, such interactions become much more natural – e.g. through smart glasses, which automatically detect a product, even without scanning its barcode. In our project “Image2Product”, we use computer vision to automatically detect product features, e.g. its product category, size, or its identifier.

Our product database which is used as training and test data currently contains approx. 30 000 products including each product’s image, size, text description, nutrient and category data, and is constantly being extended. The machine learning script predicts for each of product image (i.e. 2D images), its corresponding product category (e.g. sweetened beverage) and confidence for the respective prediction.

If you are interested in working together with us on a similar application, please feel free to contact us: Klaus Fuchs



Retailers, manufacturers, app developers, data aggregators


Deep learning, computer vision

Working group