All Projects

HoloSelecta

Leveraging mixed reality and food product composition data in order to increase saliency of nutritional quality of groceries at the point of sale.

Image Datasets

In order to support research on computer vision based product detection, we invite interested researchers to find and use our labelled dataset from the HoloSelecta study. Please find our dataset and other relevant datasets here: > Labelled Image Data Sets

Please cite the paper if you use the HoloSelecta dataset:

    • Fuchs, K., Grundmann, T., Fleisch, E., Towards Identification of Packaged Products via Computer Vision, The 9th International Conference on the Internet of Things (IoT 2019)

Motivation

Diet-related non-communicable diseases have become the leading cause of mortality globally, accounting for more deaths than non-diet-related mortality causes combined. We believe that machine learning and data science can support making health-beneficial decisions in our daily lives. Motivated through the advances of related technology and deep learning frameworks, we contribute to this field by piloting HoloSelecta, a mixed reality headset application in the field of health and nutrition.

HoloSelecta leverages existing retail data, including image and ingredient data, in order to visualize the degree of nutritional quality of groceries at the point of sale before making a purchase decision. In the case of HoloSelecta, the scenario of a vending machine was chosen, as it often is a source of unhealthy food items and also hinders checking ingredient data as products are locked behind the machine. With HoloSelecta, wearers of mixed-reality headsets can still ‘see’ the Nutri-Score of the products within the vending machine. HoloSelecta serves as an example for potentially very effective mixed reality mediated just-in-time adaptive intervention right in the moment when relevant activities such as buying a snack are detected.

Video-Clips

Publications

  • Fuchs, K., Grundmann, T., Haldimann, M., Fleisch, E., Impact of Mixed Reality Food Labels on Product Selection: Insights from a User Study using Headset-mediated Food Labels at a Vending Machine, ACMMM 5th International Workshop on Multimedia Assisted Dietary Management (MADiMa2019)
  • Fuchs, K., Grundmann, T., Fleisch, E., Towards Identification of Packaged Products via Computer Vision, The 9th International Conference on the Internet of Things (IoT 2019)
Working Group
Partners
Contact

Klaus Fuchs, Chair of Information Management, ETH Zurich,
fuchsk@ethz.ch, Telefon: 044 632 42 98.