Starwit’s AI Cockpit

In today’s world, AI systems are not only ubiquitous but they also make more and more complex decisions. Decisions that can have severe consequences. Good engineering practice demands, that these decisions should be accessible for human operators, both from an observing as well as from a controlling and correcting perspective. European legislation known as AI Act makes this practice a mandatory legal requirement.

To give users and deployers of artificial software systems a technical tool, to implement and to enforce checks of AI decisions, Starwit has build AI Cockpit. It is a generic tool that can be integrated into any decision making AI system. Its core concepts were developed as part of a federal German research project. For more background information visit project page.

Main component of AI Cockpit is a web application, that shows operators AI modules, decisions and actions. Decisions and actions can be influenced by operators. One AI Cockpit instance can control multiple AI systems. Next image shows a demonstration instance, that controls a number of applications.

Core concept of AI Cockpit is a dialogue that shows an extracted decision from an AI system and the actions, that are supposed to be executed.

This example is from a traffic safety scenario but AI Cockpit is agnostic to application domains and as the next image shows – controlling trash detection works just as well.

Architecture

AI Cockpit is a solution that is integrated into an existing AI application landscape. How this works is shown (briefly) in this architecture diagram. AI Cockpit is integrated in between decision and action for a system landscape and can either be operated as manual circuit breaker or as passive logging component.

Operators can fully decide, in which mode AI Cockpit is run by setting autonomy levels. Following dialogue shows control panel for actions and their respective autonomy. This way operators have fine grained control over the autonomy of a system.

Following both the requirements of AI Act and good engineering practice, AI Cockpit allows operators to stop all actions immediately. Next screen shows this emergency break activated.

As a control instance for complex software systems, AI Cockpit has many additional features. If you want to know more technical details, see project’s source code repository: https://github.com/starwit/ai-cockpit-deployment

Key Features

• Can be integrated with any decision making AI system
• Implements human oversight according to article 14
• Provides information guarantee transparency for deployers as demanded in article 13
• Keeping records for made decisions and their justifications / consequences
• Provides information about AI modules and their used algorithms / training data
• Full open source and adaptable
• Documented APIs

Contact & Questions

You want to know more? You have a question? Great, we would like to hear from you!