CIMON is back on Earth after 14 months on the ISS

CIMON has returned to Earth: the Crew Interactive Mobile CompanioN (CIMON), the astronaut assistant featuring artificial intelligence, arrived back on its home planet on board a Dragon spacecraft from US company SpaceX. The SpaceX-18 undocked from the International Space Station (ISS) on 27 August 2019 at 16:59 Central European Summer Time (CEST). The capsule then touched down in the Pacific Ocean around 480 kilometres southwest of Los Angeles and was recovered at 22:21h CEST.

“We expect to have the first CIMON back in Germany by the end of October,” says Dr Christian Karrasch, CIMON Project Manager at the DLR Space Administration. “The CIMON technology experiment has lived up to all our expectations,” Karrasch continues, summarising the past few months. “During its first mission in space – a 90-minute session with German ESA astronaut Alexander Gerst on the ISS in November 2018 – CIMON demonstrated its ability to work well in microgravity conditions and to successfully interact with astronauts. We are still incredibly happy with the only artificial intelligence mission to take place on the space station to date, and have been working on a new, enhanced model for several months. The first CIMON has allowed us to lay the foundations for using social assistance systems in space – systems that support astronauts with their tasks and that at some point may even take some work off their hands.”

Just like its predecessor, the second CIMON is being built by Airbus in Friedrichshafen and Bremen on behalf of the DLR Space Administration and is being funded by the German Federal Ministry for Economic Affairs and Energy (Bundesministerium für Wirtschaft und Energie; BMWi). Airbus in Friedrichshafen has assembled and tested the hardware for the new CIMON, while Airbus in Bremen is working to improve the software for flight and attitude control. IBM is implementing new AI features. “All in all there are several upgrades, including better microphones, a more robust computer, improved flight and attitude control, and new software features for conversation – for example in the areas of voice recognition, conversational skills and understanding intentions,” explains Till Eisenberg, CIMON Project Manager at Airbus.

Matthias Biniok, Project Manager at IBM, adds: “CIMON represents a unique application scenario in an extreme working environment. And we’ve seen that by using AI technology – in our case IBM Watson – we can support astronauts in their work. Our primary objective for CIMON’s next stage of development is to improve the comprehension of speech in specific contexts and the analysis of emotion in language.”

Ethical questions around the future use of CIMON are being examined and evaluated by medical experts at Ludwig-Maximilian University (LMU) in Munich. Privacy rights are affected by the interaction between humans and machines, as CIMON records, processes and interprets images and audio of astronauts. There needs to be high technical standards in data security while recognising that trust in such systems is an important factor for teamwork between humans and machines. Basically, this means deciding what CIMON is permitted to do, know and say. “The new CIMON has an in-built switch that allows the flow of data from all cameras and microphones to be interrupted from the ISS. This means that the astronaut is in control of CIMON at all times, which we thought was particularly important,” says LMU scientist Dr Judith Buchheim.

DLR Space Administration, which is responsible for German contributions to the European Space Agency (ESA), is working with ESA to send the new version of CIMON to the ISS in December 2019 so that it can benefit from crew time with astronauts.

The first CIMON arrived on the ISS as a technology experiment on 2 July 2018. On 15 November 2018, the robotic assistant with a smart ‘face’ received global attention when it was deployed for the first time: it successfully ‘worked’ together with German ESA astronaut Alexander Gerst for 90 minutes. CIMON demonstrated its basic functions, such as its ability to fly in microgravity conditions using autonomous navigation, performing several turns and movements in every direction, and was able to search for Gerst’s face, recognise it, make eye contact and speak to him. It then showed the instructions for an experiment on its ‘face’ – a display in the centre of the sphere – and played music. It also used its cameras to record a video and take a photo of Gerst.

In 2018, the CIMON project won the US ‘Popular Science Award’ in the category ‘Best of What’s New in 2018’ in aerospace. Airbus was also awarded the ‘2019 German Innovation Prize’ (Deutsche Innovationspreis) in the large companies category (#dip19).

•CIMON – the idea

Developed and built in Germany, CIMON is a technology experiment to support astronauts and increase the efficiency of their work. CIMON is able to show and explain information and instructions for scientific experiments and repairs. The voice-controlled access to documents and media is an advantage, as the astronauts can keep both hands free. It can also be used as a mobile camera to save astronaut crew time. In particular, CIMON could be used to perform routine tasks, such as documenting experiments, searching for objects and taking inventory. CIMON can also see, hear, understand and speak. CIMON can orientate itself using its ‘eyes’ – a stereo camera and a high-resolution camera that it uses for facial recognition – as well as two other cameras fitted to its sides that it uses for photos and video documentation. Ultrasound sensors measure distances to recognise potential collisions. Its ‘ears’ consist of eight microphones to identify directions, and an additional directional microphone to improve voice recognition. Its ‘mouth’ is a loudspeaker that it can use to speak or play music. At the heart of the AI for language understanding is the IBM Watson AI technology from the IBM Cloud. CIMON has not been equipped with self-learning capabilities and requires active human instruction. The AI used for autonomous navigation was provided by Airbus and is designed for movement planning and object recognition. Twelve internal rotors allow CIMON to move and rotate freely in all directions. This means it can turn towards the astronaut when addressed, nod and shake its head, and follow the astronaut – either autonomously or on command.

The partners:

The interactive astronaut assistant was developed and built by Airbus in Friedrichshafen and Bremen on behalf of the German Aerospace Center (Deutsches Zentrum für Luft- und Raumfahrt; DLR) Space Administration and funded by the German Federal Ministry for Economic Affairs and Energy (Bundesministerium für Wirtschaft und Energie). Watson AI technology from the IBM Cloud provides voice-controlled artificial intelligence. Scientists from the Ludwig-Maximilian University Hospital in Munich (Klinikum der Ludwig-Maximilians-Universität München, LMU) helped develop and oversee the human aspects of the assistance system. An approximately 50-strong project team from DLR, Airbus, IBM and LMU have been working on the implementation of CIMON since August 2016. CIMON has been on board the ISS since 2 July 2018. It is no coincidence that its name is reminiscent of ‘Professor Simon Wright’, the robotic assistant – or the ‘flying brain’ – from the Japanese science fiction series ‘Captain Future’.