The new function of TransBots™ enables AI-driven person recognition. ©TOPPAN INC.

AI-driven person recognition added to TransBots™ solution for centralized control of multiple robot types. Pilot test conducted as part of “CHANGE TOMORROW” event.

Tokyo – Toppan, a global leader in communication, security, packaging, décor materials, and electronics solutions, has developed TransBots™, a digital twin solution for linking physical spaces with virtual reality reconstructions in real time and centrally managing and controlling multiple types of service robots.

As a new function, Toppan has now developed an artificial intelligence (AI) system that makes it possible for multiple moving robots to identify people whose images have been uploaded in advance. This person recognition function has been verified via a pilot test in which images sent from robots in various locations within an event space were analyzed and compiled in the cloud using an AI engine. It was possible to identify specific people from camera images sent by various types of robots and to perform cloud-based confirmation. The test also demonstrated that people’s locations could be pinpointed by using the positions of the robots on TransBots™.

Toppan aims to contribute to a fulfilling and sustainable society by employing TransBots™ and other solutions driving the application of digital twins in diverse fields, such as security, customer analysis, guidance, transportation, and cleaning. Going forward, Toppan intends to help alleviate labor shortages and transform work styles by strengthening collaboration with robot manufacturers, enhancing the multifunction console for service robots and TransBots™, and enabling multiple use cases.

Background

An ageing society and declining birthrate in Japan are contributing to labor shortages, which are becoming severe in security, cleaning, and other areas of the facility management industry. Robots and IT are seen as part of the solution, and robots specialized for tasks such as reception work and cleaning are being introduced. Robot management, however, is complex because multiple different types need to be controlled.

Security cameras and other Internet of Things (IoT) devices are employed for the management of buildings such as hospitals and complexes as well as for managing admission to exhibitions and concerts. But limitations on the scope that can be monitored by people and security cameras mean that robots are expected to play a larger role in the future.

Toppan’s TransBots™ is a digital twin solution for centralized management and control of multiple types of robots. Operators can use a multifunction console to configure courses for robots’ movement in a virtual space. Robots in the real space then move in accordance with the courses to arrive safely at their preset destinations. Robots can also pinpoint their own current position, detect obstacles, and speak to ensure the safety of people nearby. The use of the virtual space for setting courses and controlling robots means that the tasks can be performed remotely.

Toppan has now added an AI-driven person recognition function to the TransBots™ console. When people who have been preregistered on the system enter a physical space, multiple moving robots are able to recognize them. This is expected to have various applications, including remote management of hospitals and complexes and management of large-scale events with restricted entry, such as exhibitions and concerts. In addition to telepresence robots,1 TransBots™ can also be applied for a wide range of service robots, such as robot carriers and mobile picking robots.

About the TransBots™ multifunction console

The TransBots™ multifunction console is part of the digital twin solution. It consolidates elements such as robot location, battery capacity, video from cameras, display images, and audio on the robotics HUB in the cloud and enables them to be controlled via various application consoles.
Main functions include a graphical user interface (GUI) for digital twin robot control, robot map drawing, multi-vendor robot connection, automated/manual operation, a robot management web dashboard, multiple telepresence management, live video streaming, equipment communication, remote central control of multiple locations, video storage, and signage functions.

Features of person recognition function

?Location pinpointed using preregistered information
When multiple robots moving within a specified area recognize a person matching preregistered information, the person’s location can be identified from the positions of the robots.

?Images stored on a server
Images taken by multiple robots when a person is recognized can be stored in the cloud and viewed as a list on the dashboard.

?Reduced costs for robots
Because AI-driven person recognition is performed in the cloud using video images sent by robots, there is no need to equip the robots themselves with edge computers2 to provide AI functions, thus helping to reduce costs. Connecting to TransBots™ also makes it possible for robots of multiple types to take advantage of diverse functions.

Overview of pilot test

Dates: October 29 and 30, 2022
Location: Toppan booth3 inside Workshop Collection at CHANGE TOMORROW (showcase of interesting future just ahead)4 (Venue: Tokyo Portcity Takeshiba)
Objective: To test the new AI-driven person recognition function of TransBots™

Details:
?AI-driven person recognition
To verify the person recognition function, data for recognition was first created by registering images of participants when they joined the event. Based on the data, eight robots dispersed inside the venue recorded images of participants as they moved around. The images were then analyzed using AI to identify the people.
?Dashboard function
The dashboard function enables images taken by robots to be checked in real time. The images taken were used to assess the extent to which each robot had been able to recognize people.
The dashboard was used in a game in which participants could view a large monitor installed at the venue to check which robots had seen them and when.
Result: The effectiveness of cloud-based person recognition using an AI engine was confirmed based on the more than 200 participants who took part in the event.

Event inside the Toppan booth

Event name: Halloween Event: Robot Halloween—Challenge from the Robomon
Target: Kindergarten and young elementary school children and their guardians
Details: Participants wore Halloween masks and took part in a puzzle solving game. As they solved puzzles with their guardians, they received hints from three “Robomon” robot monsters in Halloween costumes who also tried to scare them.

1. Telepresence robot
A robot that someone can control from a remote location to simulate their own presence at the robot’s location.

2. Edge computer
A computer that not only compiles and processes data in the cloud via the Internet, but also processes data on devices on the user side, such as robots, IoT devices, and smartphones.

3. Toppan booth: Workshop Collection in Takeshiba—Experience Towns and Work of the Future!
http://wsc.or.jp/workshop/takeshiba2022/  (Japanese)

4. CHANGE TOMORROW (showcase of interesting future just ahead)
An event held on October 29 and 30 at Tokyo Portcity Takeshiba (Minato City, Tokyo)
https://change-tomorrow.tokyo/ (Japanese)