Högskolan i Skövde

Autonomous vehicles promise to revolutionize society and improve the daily life of many, making them a coveted aim for a vast research community. To enable complex reasoning in autonomous vehicles, researchers are exploring new methods beyond traditional engineering approaches, in particular the idea of drawing inspiration from the only existing being able to drive: the human. The mental processes behind the human ability to drive can inspire new approaches with the potential to bridge the gap between artificial drivers and human drivers. In this review, we categorize and evaluate existing work on autonomous driving influenced by cognitive science, neuroscience, and psychology. We propose a taxonomy of the various sources of inspiration and identify the potential advantages with respect to traditional approaches. Although these human-inspired methods have not yet reached widespread adoption, we believe they are critical to the future of fully autonomous vehicles. 

This thesis examines Cognitively Inspired Design (CID), which is the process of transferring cognitive science frameworks and theories to intelligent systems in an application context. The thesis studies the relation between cognitive science and the traditional approach to developing systems. There are numerous differences and challenges between those two fields, making the transformation from cognitive science to designing a novel cognitive system a challenging process. To examine this process, the Guest and Martin (2021) multi-layer model has been utilized. The model proposes a sequence of six layers in which a researcher follows from a defined cognitive concept or framework to an empirical experiment of a computational model. This multi-layered model is a path function in which each layer is a function that takes the input from the previous layer and passes the output to the following layer.

The thesis takes the application of self-driving cars as the context of study. Self-driving cars are considered one of the most important applications requiring a high level of intelligence and cognitive ability because they encounter real world scenarios and the risk of failure may cost lives. This thesis analyzes the transformation of CID in three main studies.

The first study theoretically analyzes the applicability and compares the different cognitive paradigms and current AI techniques for self-driving cars. The thesis argues for exploring the emergent paradigm as a less explored paradigm in cognitive systems compared to its main opponent paradigm; the cognitivist. The emergent paradigm is claimed to describe the interactive nature of the human cognition. The analysis highlights the opportunities that the field of self-driving cars benefits from when considering the characteristics of the emergent paradigm.

The second study considers the path function for a selected emergent paradigm theory. The study focuses on the aspect of how humans learn from hypothetical scenarios before encountering them in the real world, in particular, learning how to handle rare scenarios that are difficult to learn in the real world. The study addresses the mechanism for automatically generating these scenarios without being designed and created manually by a developer. The study considers curriculum learning as the candidate theory subject of study. The process of transferring this theory is studied using the path function multilayer model. The study conducts an experiment to address the relation between the importance of the theory in human learning and its equivalence in artificial cognitive systems.

The third study focuses on more debatable theories in the emergent paradigm, in particular enactive and embodiment theories. These theories have gained much attention in research because of the high promise they may deliver for advancing the field of artificial cognitive systems. The applicability of the transition of these theories into artificial cognitive systems is examined in relation to the application of self-driving cars, using the path function multi-layer model. The study considers the aspects that support and hinder such transformation.

The thesis concludes by discussing the current state of CID and the aspects the researchers and developers need to consider in this process before, during, and after the transformation. Overall, the thesis attempts to study cognitive theories mainly from an engineering perspective. In short, the thesis focuses on the transformation of CID, not the promise of delivering a novel cognitive system solution.

Den här avhandlingen undersöker Kognitivt Inspirerad Design (eng. Cognitively Inspired Desig) vilket benämner processen att applicera kognitiva ramverk och teorier i en annan kontext. Avhandlingen studerar relationen mellan kognitionsvetenskap och den traditionella systemutvecklingsprocessen. Det finns flera olikheter och utmaningar som gör det svårt att designa ett nytt kognitivt system utifrån kognitionsvetenskap. För att undersöka denna process har flerlagersmodellen från Guest och Martin (2021) använts. Modellen består av en sekvens av sex lager som en forskare följer i steg från ett kognitivt ramverk till ett empiriskt experiment. Den här flerlagersmodellen är en banfunktion (eng. path function) där varje lager tar input från det tidigare lagret och ger output till nästföljande lager. Avhandlingens genomförs i kontexten av självkörande bilar. Självkörande bilar anses behöva en hög nivå av intelligens och kognitiva förmågor i och med att de måste agera i faktiska körsituationer där felbeslut kan kosta människoliv. Avhandlingen analyserar Kognitivt Inspirerad Design inom ramen för tre huvudsakliga studier.

Den första studien gör en jämförande teoretisk analys kring hur kognitiva paradigm och nuvarande AI-tekniker i självkörande bilar förhåller sig till varandra. Avhandlingen argumenterar för att det emergenta paradigmet är outforskat i denna kontext och bör studeras mer. Det emergenta paradigmet fokuserar på den interaktiva karaktären av mänsklig kognition. Analysen presenterar vilka möjligheter som öppnas för utvecklingen av självkörande bilar genom att applicera emergenta paradigmets olika egenskaper.

Den andra studien fokuserar på flerlagersmodellen i kontexten av en specifik teori inom det emergenta paradigmet. Studien fokuserar på hur människor lär sig från olika hypotetiska scenarier innan de faktiskt behöver träffa på dem i den verkliga världen, med särskilt fokus på ovanliga situationer som sällan inträffar. Studien undersöker hur mekanismer automatiskt kan skapa de här hypotetiska scenarierna, det vill säga utan att någon mänsklig utvecklar behöver manuellt designa situationerna. Studien använder sig av ”gradvis inlärning” (eng. curriculum learning) som en möjlig teoribas som med hjälp av flerlagersmodellen överförs till ett experiment som undersöker relationen mellan gradvis inlärning som teori för människor och för artificiella kognitiva system.

Den tredje studien fokuserar på mer omdebatterade teorier inom det emergenta paradigmet, nämligen så kallad enaktiv (eng. enactive) och förkroppslig (eng. embodied) kognition. De här teorierna har väckt uppmärksamhet på grund av deras möjliga förbättringspotential av artificiella kognitiva system. Flerlagersmodellen används i det här fallet för att undersöka hur dessa teorier kan appliceras på självkörande bilar. Studien beaktar hur de stödjer respektive hindrar appliceringen på självkörande bilar.

Avhandlingen avslutas med en diskussion om var Kognitivt Inspirerad Design står idag och vilka aspekter forskare och utvecklare behöver tänka på före, under och efter transformationen från teori till utvecklat system.

Generellt sett har denna avhandling studerat kognitiva teorier främst från ett ingenjörsperspektiv. Avhandlingens fokus har varit på att studera Kognitiv Inspirerad Design utifrån ett transformationsprocessperspektiv; målet har alltså inte varit att faktiskt skapa ett nytt kognitivt system.

Learning from only real-world collected data can be unrealistic and time consuming in many scenario. One alternative is to use synthetic data as learning environments to learn rare situations and replay buffers to speed up the learning. In this work, we examine the hypothesis of how the creation of the environment affects the training of reinforcement learning agent through auto-generated environment mechanisms. We take the autonomous vehicle as an application. We compare the effect of two approaches to generate training data for artificial cognitive agents. We consider the added value of curriculum learning—just as in human learning—as a way to structure novel training data that the agent has not seen before as well as that of using a replay buffer to train further on data the agent has seen before. In other words, the focus of this paper is on characteristics of the training data rather than on learning algorithms. We therefore use two tasks that are commonly trained early on in autonomous vehicle research: lane keeping and pedestrian avoidance. Our main results show that curriculum learning indeed offers an additional benefit over a vanilla reinforcement learning approach (using Deep-Q Learning), but the replay buffer actually has a detrimental effect in most (but not all) combinations of data generation approaches we considered here. The benefit of curriculum learning does depend on the existence of a well-defined difficulty metric with which various training scenarios can be ordered. In the lane-keeping task, we can define it as a function of the curvature of the road, in which the steeper and more occurring curves on the road, the more difficult it gets. Defining such a difficulty metric in other scenarios is not always trivial. In general, the results of this paper emphasize both the importance of considering data characterization, such as curriculum learning, and the importance of defining an appropriate metric for the task.

Nature has been a great source of inspiration for many inventions and theories. One of the major benefits for this inspiration is perceiving the impossible as possible. The inception of the AI field was no exception with cognitively-inspired approaches with a dream of having an intelligent system that thinks as a human. However, this journey of human intelligence into machine intelligence has been rough and more challenging that resulted in the separation of AI from cognitive studies. In this article, we highlight the main challenges and opportunities for cognitive inspiration for AI development. We then break down the source of inspiration into four abstraction levels in which the researcher may place an inspiration from. These levels then contribute into three main stages for modeling the AI system. The two dimensional mapping from cognitive levels into modeling stages and the relation between them aims to assist the process of cognitively-inspired approaches.

Self-driving cars not only solve the problem of navigating safely from location A to location B; they also have to deal with an abundance of (sometimes unpredictable) factors, such as traffic rules, weather conditions, and interactions with humans. Over the last decades, different approaches have been proposed to design intelligent driving systems for self-driving cars that can deal with an uncontrolled environment. Some of them are derived from computationalist paradigms, formulating mathematical models that define the driving agent, while other approaches take inspiration from biological cognition. However, despite the extensive work in the field of self-driving cars, many open questions remain. Here, we discuss the different approaches for implementing driving systems for self-driving cars, as well as the computational paradigms from which they originate. In doing so, we highlight two key messages: First, further progress in the field might depend on adapting new paradigms as opposed to pushing technical innovations in those currently used. Specifically, we discuss how paradigms from cognitive systems research can be a source of inspiration for further development in modeling driving systems, highlighting emergent approaches as a possible starting point. Second, self-driving cars can themselves be considered cognitive systems in a meaningful sense, and are therefore a relevant, yet underutilised resource in the study of cognitive mechanisms. Overall, we argue for a stronger synergy between the fields of cognitive systems and self-driving vehicles.

The controller of an autonomous vehicle needsthe ability to learn how to act in different driving scenariosthat it may face. A significant challenge is that it is difficult,dangerous, or even impossible to experience and explore variousactions in situations that might be encountered in the realworld. Autonomous vehicle control would therefore benefitfrom a mechanism that allows the safe exploration of actionpossibilities and their consequences, as well as the ability tolearn from experience thus gained to improve driving skills.In this paper we demonstrate a methodology that allows alearning agent to create simulations of possible situations. Thesesimulations can be chained together in a sequence that allowsthe progressive improvement of the agent’s performance suchthat the agent is able to appropriately deal with novel situationsat the end of training. This methodology takes inspirationfrom the human ability to imagine hypothetical situations usingepisodic simulation; we therefore refer to this methodology asepisodic imagination.An interesting question in this respect is what effect thestructuring of such a sequence of episodic imaginations hason performance. Here, we compare a random process to astructured one and initial results indicate  that a structuredsequence outperforms a random one.

For an agent to autonomously interact in a real world environment, it needs tolearn how to behave in the different scenarios that it may face. There are differentapproaches of modeling an artificial agent with interactive capabilities. Oneapproach is providing the agent with knowledge beforehand. Another approachis to let the agent learn from data and interaction. A well-known techniques ofthe former approach is supervised learning. In this approach, data is collected,labeled and provided to train the network as pre-defined input and correct outputas a training set. This requires data to be available beforehand. In a realworld environment however, it is difficult to determine all possible interactionsand provide the correct response to each. The agent thus needs to be able tolearn by itself from the environment to figure out the best reaction in each situation.To facilitate this, the agent needs to be able to sense the environment,make decisions and react back to the environment. The agent repeats this tryingdifferent decisions. To learn from these trials, the agent needs to accumulate oldexperiences, learn and adjust its knowledge and develop progressively after eachinteraction. However, in many applications, experiencing various actions in differentscenarios is difficult, dangerous or even impossible. The agent thereforeneeds an experimental environment where it can safely explore the possibilities,learn from experiences and develop new skills.This research aims to develop a methodology to build an interactive learningagent that can improve its learning performance progressively and perform wellin real world environments. The agent follows the Japanese concept Kaizenwhich refers to activities that continuously improve all functions. It meansstriving for continuous improvement and not radically changing processes. Thecontribution of this research is first to model and develop this agent so thatit can acquire new knowledge based on existing knowledge without negativelyaffecting the old knowledge and skills. Secondly, this research aims to developa novel method to systematically generate synthetic scenarios that contributesto its learning performance.This proposal consists of the background of artificial cognitive systems, acomparison of the theories and approaches in artificial cognitive systems fordeveloping a learning interactive system, and a review of the state of the artin reinforcement learning. Imagination-based learning is discussed and the purposesof imagination are defined. Imagination for creation is used as a scenariogenerator for practicing new skills without the necessity to try them all in thereal world. The research proposal results in the research questions and objectivesto be investigated as well as an outline of the methodology.