Leveraging mixed reality and food product composition data in order to increase saliency of nutritional quality of groceries at the point of sale.
In order to support research on computer vision based product detection, we invite interested researchers to find and use our labelled dataset from the HoloSelecta study. Please find our dataset and other relevant datasets here: > Labelled Image Data Sets
Please cite the paper if you use the HoloSelecta dataset:
- Fuchs, K., Grundmann, T., Fleisch, E., Towards Identification of Packaged Products via Computer Vision, The 9th International Conference on the Internet of Things (IoT 2019)
Diet-related non-communicable diseases have become the leading cause of mortality globally, accounting for more deaths than non-diet-related mortality causes combined. We believe that machine learning and data science can support making health-beneficial decisions in our daily lives. Motivated through the advances of related technology and deep learning frameworks, we contribute to this field by piloting HoloSelecta, a mixed reality headset application in the field of health and nutrition.
HoloSelecta leverages existing retail data, including image and ingredient data, in order to visualize the degree of nutritional quality of groceries at the point of sale before making a purchase decision. In the case of HoloSelecta, the scenario of a vending machine was chosen, as it often is a source of unhealthy food items and also hinders checking ingredient data as products are locked behind the machine. With HoloSelecta, wearers of mixed-reality headsets can still ‘see’ the Nutri-Score of the products within the vending machine. HoloSelecta serves as an example for potentially very effective mixed reality mediated just-in-time adaptive intervention right in the moment when relevant activities such as buying a snack are detected.