Study on human-in-the-loop sensing in urban environment analysis

Abstract

Due to the widespread of mobile devices such as smartphones, we can get the appropriate information anytime, anywhere according to the surrounding environments and the human conditions, called ``contexts.'' In order to provide context-aware systems in urban spaces, it is indispensable to collect information in a complicated and vast spaces comprehensively, and to recognize the context based on the analysis of collected information. Hence, we define the human-in-the-loop sensing framework which consists of two sensing approaches: the direct urban sensing using sensors embedded on mobile devices, and the indirect urban sensing by utilizing humans as a sensor, as the scope of our study. In this dissertation, we focus on two challenges to realize the human-in-the-loop sensing framework: 1) how to extract various context, and 2) how to operate the proposed framework sustainably. Regarding the first challenge, we have tackled with two different cases. One is a safety assessment system for sidewalks at night based on sensing illuminance of streetlamps. We devise a method to correct data from mobile sensors, which have large differences in characteristics and accuracy, utilizing collective intelligence. Through the experiment, we confirmed the proposed method reduced the estimation error for 90% samples of streetlamps. Another is a psychological context recognition system based on observing the unconscious behavior of tourists. We devise the tourist emotion and satisfaction recognition model by combining multiple modalities collected during sightseeing. With the real-world experiment, we achieved up to 0.50 of average recall score with three-class emotion recognition task. As for the second challenge, we build a mobile participatory sensing platform, which incorporates citizen communities into the ecosystem of human-in-the-loop sensing framework. Especially, we focus on ``civic tech'' which is one of the ways to realize civic cooperation by using ICT, and have placed civic tech communities as the potential user group of our platform. Through 17 case studies with citizens, we have confirmed the usability and availability of the proposed platform.