Högskolan i Skövde

his.sePublications
Change search
Link to record
Permanent link

Direct link
Publications (8 of 8) Show all publications
Plebe, A., Svensson, H., Mahmoud, S. & Da Lio, M. (2024). Human-inspired autonomous driving: A survey. Cognitive Systems Research, 83, Article ID 101169.
Open this publication in new window or tab >>Human-inspired autonomous driving: A survey
2024 (English)In: Cognitive Systems Research, ISSN 2214-4366, E-ISSN 1389-0417, Vol. 83, article id 101169Article, review/survey (Refereed) Published
Abstract [en]

Autonomous vehicles promise to revolutionize society and improve the daily life of many, making them a coveted aim for a vast research community. To enable complex reasoning in autonomous vehicles, researchers are exploring new methods beyond traditional engineering approaches, in particular the idea of drawing inspiration from the only existing being able to drive: the human. The mental processes behind the human ability to drive can inspire new approaches with the potential to bridge the gap between artificial drivers and human drivers. In this review, we categorize and evaluate existing work on autonomous driving influenced by cognitive science, neuroscience, and psychology. We propose a taxonomy of the various sources of inspiration and identify the potential advantages with respect to traditional approaches. Although these human-inspired methods have not yet reached widespread adoption, we believe they are critical to the future of fully autonomous vehicles. 

Place, publisher, year, edition, pages
Elsevier, 2024
Keywords
Autonomous vehicles, Human cognition, Human driving, Imitation learning, Neuromorphic computing, Autonomous driving, Daily lives, Mental process, Research communities, Traditional engineerings, adoption, autonomous vehicle, cognition, human, human experiment, imitation, learning, neuroscience, psychology, short survey, taxonomy
National Category
Human Computer Interaction Ethics Vehicle Engineering Computer Sciences
Research subject
Interaction Lab (ILAB)
Identifiers
urn:nbn:se:his:diva-23259 (URN)10.1016/j.cogsys.2023.101169 (DOI)001080906500001 ()2-s2.0-85171333787 (Scopus ID)
Note

CC BY-NC-ND 4.0 DEED

© 2023

Corresponding author: E-mail address: alice.plebe@unitn.it (A. Plebe)

Available from: 2023-09-28 Created: 2023-09-28 Last updated: 2023-12-06Bibliographically approved
Mahmoud, S. (2023). Cognitively inspired design: Rethink the wheel for self-driving cars. (Doctoral dissertation). Skövde: University of Skövde
Open this publication in new window or tab >>Cognitively inspired design: Rethink the wheel for self-driving cars
2023 (English)Doctoral thesis, monograph (Other academic)
Abstract [en]

This thesis examines Cognitively Inspired Design (CID), which is the process of transferring cognitive science frameworks and theories to intelligent systems in an application context. The thesis studies the relation between cognitive science and the traditional approach to developing systems. There are numerous differences and challenges between those two fields, making the transformation from cognitive science to designing a novel cognitive system a challenging process. To examine this process, the Guest and Martin (2021) multi-layer model has been utilized. The model proposes a sequence of six layers in which a researcher follows from a defined cognitive concept or framework to an empirical experiment of a computational model. This multi-layered model is a path function in which each layer is a function that takes the input from the previous layer and passes the output to the following layer.

The thesis takes the application of self-driving cars as the context of study. Self-driving cars are considered one of the most important applications requiring a high level of intelligence and cognitive ability because they encounter real world scenarios and the risk of failure may cost lives. This thesis analyzes the transformation of CID in three main studies.

The first study theoretically analyzes the applicability and compares the different cognitive paradigms and current AI techniques for self-driving cars. The thesis argues for exploring the emergent paradigm as a less explored paradigm in cognitive systems compared to its main opponent paradigm; the cognitivist. The emergent paradigm is claimed to describe the interactive nature of the human cognition. The analysis highlights the opportunities that the field of self-driving cars benefits from when considering the characteristics of the emergent paradigm.

The second study considers the path function for a selected emergent paradigm theory. The study focuses on the aspect of how humans learn from hypothetical scenarios before encountering them in the real world, in particular, learning how to handle rare scenarios that are difficult to learn in the real world. The study addresses the mechanism for automatically generating these scenarios without being designed and created manually by a developer. The study considers curriculum learning as the candidate theory subject of study. The process of transferring this theory is studied using the path function multilayer model. The study conducts an experiment to address the relation between the importance of the theory in human learning and its equivalence in artificial cognitive systems.

The third study focuses on more debatable theories in the emergent paradigm, in particular enactive and embodiment theories. These theories have gained much attention in research because of the high promise they may deliver for advancing the field of artificial cognitive systems. The applicability of the transition of these theories into artificial cognitive systems is examined in relation to the application of self-driving cars, using the path function multi-layer model. The study considers the aspects that support and hinder such transformation.

The thesis concludes by discussing the current state of CID and the aspects the researchers and developers need to consider in this process before, during, and after the transformation. Overall, the thesis attempts to study cognitive theories mainly from an engineering perspective. In short, the thesis focuses on the transformation of CID, not the promise of delivering a novel cognitive system solution.

Abstract [sv]

Den här avhandlingen undersöker Kognitivt Inspirerad Design (eng. Cognitively Inspired Desig) vilket benämner processen att applicera kognitiva ramverk och teorier i en annan kontext. Avhandlingen studerar relationen mellan kognitionsvetenskap och den traditionella systemutvecklingsprocessen. Det finns flera olikheter och utmaningar som gör det svårt att designa ett nytt kognitivt system utifrån kognitionsvetenskap. För att undersöka denna process har flerlagersmodellen från Guest och Martin (2021) använts. Modellen består av en sekvens av sex lager som en forskare följer i steg från ett kognitivt ramverk till ett empiriskt experiment. Den här flerlagersmodellen är en banfunktion (eng. path function) där varje lager tar input från det tidigare lagret och ger output till nästföljande lager. Avhandlingens genomförs i kontexten av självkörande bilar. Självkörande bilar anses behöva en hög nivå av intelligens och kognitiva förmågor i och med att de måste agera i faktiska körsituationer där felbeslut kan kosta människoliv. Avhandlingen analyserar Kognitivt Inspirerad Design inom ramen för tre huvudsakliga studier.

Den första studien gör en jämförande teoretisk analys kring hur kognitiva paradigm och nuvarande AI-tekniker i självkörande bilar förhåller sig till varandra. Avhandlingen argumenterar för att det emergenta paradigmet är outforskat i denna kontext och bör studeras mer. Det emergenta paradigmet fokuserar på den interaktiva karaktären av mänsklig kognition. Analysen presenterar vilka möjligheter som öppnas för utvecklingen av självkörande bilar genom att applicera emergenta paradigmets olika egenskaper.

Den andra studien fokuserar på flerlagersmodellen i kontexten av en specifik teori inom det emergenta paradigmet. Studien fokuserar på hur människor lär sig från olika hypotetiska scenarier innan de faktiskt behöver träffa på dem i den verkliga världen, med särskilt fokus på ovanliga situationer som sällan inträffar. Studien undersöker hur mekanismer automatiskt kan skapa de här hypotetiska scenarierna, det vill säga utan att någon mänsklig utvecklar behöver manuellt designa situationerna. Studien använder sig av ”gradvis inlärning” (eng. curriculum learning) som en möjlig teoribas som med hjälp av flerlagersmodellen överförs till ett experiment som undersöker relationen mellan gradvis inlärning som teori för människor och för artificiella kognitiva system.

Den tredje studien fokuserar på mer omdebatterade teorier inom det emergenta paradigmet, nämligen så kallad enaktiv (eng. enactive) och förkroppslig (eng. embodied) kognition. De här teorierna har väckt uppmärksamhet på grund av deras möjliga förbättringspotential av artificiella kognitiva system. Flerlagersmodellen används i det här fallet för att undersöka hur dessa teorier kan appliceras på självkörande bilar. Studien beaktar hur de stödjer respektive hindrar appliceringen på självkörande bilar.

Avhandlingen avslutas med en diskussion om var Kognitivt Inspirerad Design står idag och vilka aspekter forskare och utvecklare behöver tänka på före, under och efter transformationen från teori till utvecklat system.

Generellt sett har denna avhandling studerat kognitiva teorier främst från ett ingenjörsperspektiv. Avhandlingens fokus har varit på att studera Kognitiv Inspirerad Design utifrån ett transformationsprocessperspektiv; målet har alltså inte varit att faktiskt skapa ett nytt kognitivt system.

Place, publisher, year, edition, pages
Skövde: University of Skövde, 2023. p. xxii, 218
Series
Dissertation Series ; 57
Keywords
Artificial cognitive systems, AI, self-driving cars, cognitive paradigms, emergent paradigm, curriculum learning, embodiment, enaction
National Category
Information Systems Computer Systems Robotics
Research subject
Interaction Lab (ILAB)
Identifiers
urn:nbn:se:his:diva-23374 (URN)978-91-987907-1-9 (ISBN)
Public defence
2023-12-21, D201, Högskolan i Skövde, Skövde, 13:00 (English)
Opponent
Supervisors
Available from: 2023-11-21 Created: 2023-11-20 Last updated: 2023-11-21Bibliographically approved
Mahmoud, S., Billing, E., Svensson, H. & Thill, S. (2023). How to train a self-driving vehicle: On the added value (or lack thereof) of curriculum learning and replay buffers. Frontiers in Artificial Intelligence, 6, Article ID 1098982.
Open this publication in new window or tab >>How to train a self-driving vehicle: On the added value (or lack thereof) of curriculum learning and replay buffers
2023 (English)In: Frontiers in Artificial Intelligence, E-ISSN 2624-8212, Vol. 6, article id 1098982Article in journal (Refereed) Published
Abstract [en]

Learning from only real-world collected data can be unrealistic and time consuming in many scenario. One alternative is to use synthetic data as learning environments to learn rare situations and replay buffers to speed up the learning. In this work, we examine the hypothesis of how the creation of the environment affects the training of reinforcement learning agent through auto-generated environment mechanisms. We take the autonomous vehicle as an application. We compare the effect of two approaches to generate training data for artificial cognitive agents. We consider the added value of curriculum learning—just as in human learning—as a way to structure novel training data that the agent has not seen before as well as that of using a replay buffer to train further on data the agent has seen before. In other words, the focus of this paper is on characteristics of the training data rather than on learning algorithms. We therefore use two tasks that are commonly trained early on in autonomous vehicle research: lane keeping and pedestrian avoidance. Our main results show that curriculum learning indeed offers an additional benefit over a vanilla reinforcement learning approach (using Deep-Q Learning), but the replay buffer actually has a detrimental effect in most (but not all) combinations of data generation approaches we considered here. The benefit of curriculum learning does depend on the existence of a well-defined difficulty metric with which various training scenarios can be ordered. In the lane-keeping task, we can define it as a function of the curvature of the road, in which the steeper and more occurring curves on the road, the more difficult it gets. Defining such a difficulty metric in other scenarios is not always trivial. In general, the results of this paper emphasize both the importance of considering data characterization, such as curriculum learning, and the importance of defining an appropriate metric for the task.

Place, publisher, year, edition, pages
Frontiers Media S.A., 2023
Keywords
data generation, curriculum learning, cognitive-inspired learning, reinforcement learning, replay buffer, self-driving cars
National Category
Computer Sciences Computer Vision and Robotics (Autonomous Systems) Robotics
Research subject
Interaction Lab (ILAB)
Identifiers
urn:nbn:se:his:diva-22215 (URN)10.3389/frai.2023.1098982 (DOI)000928959000001 ()36762255 (PubMedID)2-s2.0-85147654896 (Scopus ID)
Funder
EU, Horizon 2020, 731593
Note

CC BY 4.0

Received 15 November 2022, Accepted 05 January 2023, Published 25 January 2023

This article was submitted to Machine Learning and Artificial Intelligence, a section of the journal Frontiers in Artificial Intelligence

This article is part of the Research Topic Artificial Intelligence and Autonomous Systems

Correspondence Sara Mahmoud sara.mahmoud@his.se

Part of this work was funded under the Horizon 2020 project DREAMS4CARS, Grant No. 731593.

Available from: 2023-01-31 Created: 2023-01-31 Last updated: 2023-05-04Bibliographically approved
Mahmoud, S. & Plebe, A. (2022). A critical look into cognitively-inspired artificial intelligence. In: Hadi Banaee; Amy Loutfi; Alessandro Saffiotti; Antonio Lieto (Ed.), AIC 2022 Artificial Intelligence and Cognition 2022: Proceedings of the 8th International Workshop on Artificial Intelligence and Cognition, Örebro, Sweden, 15-17 June, 2022. Paper presented at AIC 2022, 8th International Workshop on Artificial Intelligence and Cognition, Örebro, 15-17 June 2022 (pp. 127-133). CEUR-WS
Open this publication in new window or tab >>A critical look into cognitively-inspired artificial intelligence
2022 (English)In: AIC 2022 Artificial Intelligence and Cognition 2022: Proceedings of the 8th International Workshop on Artificial Intelligence and Cognition, Örebro, Sweden, 15-17 June, 2022 / [ed] Hadi Banaee; Amy Loutfi; Alessandro Saffiotti; Antonio Lieto, CEUR-WS , 2022, p. 127-133Conference paper, Published paper (Refereed)
Abstract [en]

Nature has been a great source of inspiration for many inventions and theories. One of the major benefits for this inspiration is perceiving the impossible as possible. The inception of the AI field was no exception with cognitively-inspired approaches with a dream of having an intelligent system that thinks as a human. However, this journey of human intelligence into machine intelligence has been rough and more challenging that resulted in the separation of AI from cognitive studies. In this article, we highlight the main challenges and opportunities for cognitive inspiration for AI development. We then break down the source of inspiration into four abstraction levels in which the researcher may place an inspiration from. These levels then contribute into three main stages for modeling the AI system. The two dimensional mapping from cognitive levels into modeling stages and the relation between them aims to assist the process of cognitively-inspired approaches.

Place, publisher, year, edition, pages
CEUR-WS, 2022
Series
CEUR Workshop Proceedings, ISSN 1613-0073
Keywords
Cognitively-inspired systems, cognitive abstraction levels, Artificial Intelligence
National Category
Robotics Information Systems, Social aspects Computer Sciences
Research subject
Interaction Lab (ILAB)
Identifiers
urn:nbn:se:his:diva-21895 (URN)2-s2.0-85160825977 (Scopus ID)
Conference
AIC 2022, 8th International Workshop on Artificial Intelligence and Cognition, Örebro, 15-17 June 2022
Note

CC BY 4.0

Available from: 2022-10-03 Created: 2022-10-03 Last updated: 2023-06-21Bibliographically approved
Mahmoud, S., Billing, E., Svensson, H. & Thill, S. (2022). Where to from here?: On the future development of autonomous vehicles from a cognitive systems perspective. Cognitive Systems Research, 76, 63-77
Open this publication in new window or tab >>Where to from here?: On the future development of autonomous vehicles from a cognitive systems perspective
2022 (English)In: Cognitive Systems Research, ISSN 2214-4366, E-ISSN 1389-0417, Vol. 76, p. 63-77Article in journal (Refereed) Published
Abstract [en]

Self-driving cars not only solve the problem of navigating safely from location A to location B; they also have to deal with an abundance of (sometimes unpredictable) factors, such as traffic rules, weather conditions, and interactions with humans. Over the last decades, different approaches have been proposed to design intelligent driving systems for self-driving cars that can deal with an uncontrolled environment. Some of them are derived from computationalist paradigms, formulating mathematical models that define the driving agent, while other approaches take inspiration from biological cognition. However, despite the extensive work in the field of self-driving cars, many open questions remain. Here, we discuss the different approaches for implementing driving systems for self-driving cars, as well as the computational paradigms from which they originate. In doing so, we highlight two key messages: First, further progress in the field might depend on adapting new paradigms as opposed to pushing technical innovations in those currently used. Specifically, we discuss how paradigms from cognitive systems research can be a source of inspiration for further development in modeling driving systems, highlighting emergent approaches as a possible starting point. Second, self-driving cars can themselves be considered cognitive systems in a meaningful sense, and are therefore a relevant, yet underutilised resource in the study of cognitive mechanisms. Overall, we argue for a stronger synergy between the fields of cognitive systems and self-driving vehicles.

Place, publisher, year, edition, pages
Elsevier, 2022
National Category
Robotics Computer Systems Other Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Interaction Lab (ILAB)
Identifiers
urn:nbn:se:his:diva-21894 (URN)10.1016/j.cogsys.2022.09.005 (DOI)000883846400001 ()2-s2.0-85140001741 (Scopus ID)
Note

CC BY 4.0

Available online 1 October 2022

Corresponding author: E-mail address: sara.mahmoud@his.se (S. Mahmoud).

Available from: 2022-10-03 Created: 2022-10-03 Last updated: 2023-01-16Bibliographically approved
Mahmoud, S., Svensson, H. & Thill, S. (2019). Cognitively-inspired episodic imagination for self-driving vehicles. In: Towards Cognitive Vehicles: perception, learning and decision making under real-world constraints. Is bio-inspiration helpful?: Proceedings. Paper presented at TCV2019: Towards Cognitive Vehicles: perception, learning and decision making under real-world constraints. Is bio-inspiration helpful? Workshop held as part of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2019). Macau, China, November 8, 2019. (pp. 28-31).
Open this publication in new window or tab >>Cognitively-inspired episodic imagination for self-driving vehicles
2019 (English)In: Towards Cognitive Vehicles: perception, learning and decision making under real-world constraints. Is bio-inspiration helpful?: Proceedings, 2019, p. 28-31Conference paper, Poster (with or without abstract) (Refereed)
Abstract [en]

The controller of an autonomous vehicle needsthe ability to learn how to act in different driving scenariosthat it may face. A significant challenge is that it is difficult,dangerous, or even impossible to experience and explore variousactions in situations that might be encountered in the realworld. Autonomous vehicle control would therefore benefitfrom a mechanism that allows the safe exploration of actionpossibilities and their consequences, as well as the ability tolearn from experience thus gained to improve driving skills.In this paper we demonstrate a methodology that allows alearning agent to create simulations of possible situations. Thesesimulations can be chained together in a sequence that allowsthe progressive improvement of the agent’s performance suchthat the agent is able to appropriately deal with novel situationsat the end of training. This methodology takes inspirationfrom the human ability to imagine hypothetical situations usingepisodic simulation; we therefore refer to this methodology asepisodic imagination.An interesting question in this respect is what effect thestructuring of such a sequence of episodic imaginations hason performance. Here, we compare a random process to astructured one and initial results indicate  that a structuredsequence outperforms a random one.

National Category
Robotics
Research subject
Interaction Lab (ILAB)
Identifiers
urn:nbn:se:his:diva-18175 (URN)
Conference
TCV2019: Towards Cognitive Vehicles: perception, learning and decision making under real-world constraints. Is bio-inspiration helpful? Workshop held as part of the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2019). Macau, China, November 8, 2019.
Funder
EU, Horizon 2020, 41365
Available from: 2020-01-28 Created: 2020-01-28 Last updated: 2022-12-28Bibliographically approved
Mahmoud, S. (2019). The Kaizen Agent: A self-driving car continuously learns by imagination.
Open this publication in new window or tab >>The Kaizen Agent: A self-driving car continuously learns by imagination
2019 (English)Report (Other academic)
Abstract [en]

For an agent to autonomously interact in a real world environment, it needs tolearn how to behave in the different scenarios that it may face. There are differentapproaches of modeling an artificial agent with interactive capabilities. Oneapproach is providing the agent with knowledge beforehand. Another approachis to let the agent learn from data and interaction. A well-known techniques ofthe former approach is supervised learning. In this approach, data is collected,labeled and provided to train the network as pre-defined input and correct outputas a training set. This requires data to be available beforehand. In a realworld environment however, it is difficult to determine all possible interactionsand provide the correct response to each. The agent thus needs to be able tolearn by itself from the environment to figure out the best reaction in each situation.To facilitate this, the agent needs to be able to sense the environment,make decisions and react back to the environment. The agent repeats this tryingdifferent decisions. To learn from these trials, the agent needs to accumulate oldexperiences, learn and adjust its knowledge and develop progressively after eachinteraction. However, in many applications, experiencing various actions in differentscenarios is difficult, dangerous or even impossible. The agent thereforeneeds an experimental environment where it can safely explore the possibilities,learn from experiences and develop new skills.This research aims to develop a methodology to build an interactive learningagent that can improve its learning performance progressively and perform wellin real world environments. The agent follows the Japanese concept Kaizenwhich refers to activities that continuously improve all functions. It meansstriving for continuous improvement and not radically changing processes. Thecontribution of this research is first to model and develop this agent so thatit can acquire new knowledge based on existing knowledge without negativelyaffecting the old knowledge and skills. Secondly, this research aims to developa novel method to systematically generate synthetic scenarios that contributesto its learning performance.This proposal consists of the background of artificial cognitive systems, acomparison of the theories and approaches in artificial cognitive systems fordeveloping a learning interactive system, and a review of the state of the artin reinforcement learning. Imagination-based learning is discussed and the purposesof imagination are defined. Imagination for creation is used as a scenariogenerator for practicing new skills without the necessity to try them all in thereal world. The research proposal results in the research questions and objectivesto be investigated as well as an outline of the methodology.

Publisher
p. 33
National Category
Other Electrical Engineering, Electronic Engineering, Information Engineering
Research subject
Interaction Lab (ILAB)
Identifiers
urn:nbn:se:his:diva-18176 (URN)
Note

Research proposal, PhD programme, University of Skövde

Available from: 2020-01-28 Created: 2020-01-28 Last updated: 2020-02-24Bibliographically approved
Mahmoud, S. & Svensson, H. (2018). Self-driving cars learn by imagination. In: Tom Ziemke; Mattias Arvola; Nils Dahlbäck; Erik Billing (Ed.), Proceedings of the 14th SweCog Conference: . Paper presented at Swecog 2018, the 14th Swecog conference, Linköping, Sweden, October 11-12, 2018 (pp. 12-15). Skövde: University of Skövde
Open this publication in new window or tab >>Self-driving cars learn by imagination
2018 (English)In: Proceedings of the 14th SweCog Conference / [ed] Tom Ziemke; Mattias Arvola; Nils Dahlbäck; Erik Billing, Skövde: University of Skövde , 2018, p. 12-15Conference paper, Oral presentation with published abstract (Refereed)
Place, publisher, year, edition, pages
Skövde: University of Skövde, 2018
Series
SUSI, ISSN 1653-2325 ; 2018:1
National Category
Robotics
Research subject
Interaction Lab (ILAB)
Identifiers
urn:nbn:se:his:diva-17568 (URN)978-91-983667-3-0 (ISBN)
Conference
Swecog 2018, the 14th Swecog conference, Linköping, Sweden, October 11-12, 2018
Projects
Dreams4Cars
Funder
EU, Horizon 2020, 41365
Available from: 2019-08-22 Created: 2019-08-22 Last updated: 2023-01-02Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0003-0093-3655

Search in DiVA

Show all publications