Towards a Platform for Smart City-Scale Cognitive Assistance Applications.

Rausch, T., Hummer, W., Stippel, C., Vasiljevic, S., Elvezio, C., Dustdar, S., & Krösl, K.

2021 IEEE Conference Virtual Reality and 3D User Interfaces Abstracts and Workshops

In this paper, we present CognitiveAR: a platform that makes it easier for AR engineers to develop scalable cognitive assistance applications. Figure 1 shows the overall architecture. The platform federates distributed edge resources to run our platform services,and autonomously manages application-specific services for, e.g.,offloading AI-based video processing. Moreover, the platform provides transparent access to sensor data published by edge nodes, via a scalable messaging system. We introduce a global object and user tracking system which we call cyber-physical object positioning(CPOP). CPOP enables the system to accurately display the position of physical objects on the user’s device, that are not in their physical field of view. Object coordinates are published as sensor data via the platform’s messaging system. We discuss several cognitive assistance use cases that highlight requirements and challenges. We present early results that demonstrate (1) how developers can access the system via high-level APIs, (2) the limitations of cloud-based solutions, and (3) the feasibility of running CPOP on edge nodes.

Figure 1: Architecture