Högskolan i Skövde

his.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • apa-cv
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Using recurrent neural networks with attention for detecting problematic slab shapes in steel rolling
University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre. (Skövde Artificial Intelligence Lab (SAIL))ORCID iD: 0000-0003-2128-7090
University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre. (Skövde Artificial Intelligence Lab (SAIL))ORCID iD: 0000-0001-7106-0025
University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre. (Skövde Artificial Intelligence Lab (SAIL))ORCID iD: 0000-0001-8884-2154
University of Skövde, School of Informatics. University of Skövde, The Informatics Research Centre. (Skövde Artificial Intelligence Lab (SAIL))ORCID iD: 0000-0003-2973-3112
2019 (English)In: Applied Mathematical Modelling, ISSN 0307-904X, E-ISSN 1872-8480, Vol. 70, p. 365-377Article in journal (Refereed) Published
Abstract [en]

The competitiveness in the manufacturing industry raises demands for using recent data analysis algorithms for manufacturing process development. Data-driven analysis enables extraction of novel knowledge from already existing sensors and data, which is necessary for advanced manufacturing process refinement involving aged machinery. Improved data analysis enables factories to stay competitive against newer factories, but without any hefty investment. In large manufacturing operations, the dependencies between data are highly complex and therefore very difficult to analyse manually. This paper applies a deep learning approach, using a recurrent neural network with long short term memory cells together with an attention mechanism to model the dependencies between the measured product shape, as measured before the most critical manufacturing operation, and the final product quality. Our approach predicts the ratio of flawed products already before the critical operation with an AUC-ROC score of 0.85, i.e., we can detect more than 80 % of all flawed products while having less than 25 % false positive predictions (false alarms). In contrast to previous deep learning approaches, our method shows how the recurrent neural network reasons about the input shape, using the attention mechanism to point out which parts of the product shape that have the highest influence on the predictions. Such information is crucial for both process developers, in order to understand and improve the process, and for process operators who can use the information to learn how to better trust the predictions and control the process.

Place, publisher, year, edition, pages
Elsevier, 2019. Vol. 70, p. 365-377
Keywords [en]
Attention mechanism, Recurrent Neural Networks, Interpretable AI, Steel Rolling
National Category
Computer Sciences
Research subject
INF301 Data Science; Skövde Artificial Intelligence Lab (SAIL)
Identifiers
URN: urn:nbn:se:his:diva-16588DOI: 10.1016/j.apm.2019.01.027ISI: 000468714000021Scopus ID: 2-s2.0-85060941525OAI: oai:DiVA.org:his-16588DiVA, id: diva2:1283525
Projects
DataFlow
Funder
Vinnova, 2017-01531Available from: 2019-01-29 Created: 2019-01-29 Last updated: 2021-04-19Bibliographically approved
In thesis
1. Integrating domain knowledge into deep learning: Increasing model performance through human expertise
Open this publication in new window or tab >>Integrating domain knowledge into deep learning: Increasing model performance through human expertise
2021 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

The research in this thesis is focused on how deep learning models can be designed and implemented to better emulate and integrate the heuristic reasoning process of human experts. Several case studies, within the domains of steel making and drug discovery, are conducted in order to evaluate the performance of these models, in comparison to models that do not consider human expert knowledge from the targeted domain. These case studies focus on separate problems in targeted industries and, for example, deals with predictions of the outcome in the melting of steel and the rolling of steel sheets. This thesis also deals with property predictions of molecules and the generation of new drug candidates.

The research in this thesis addresses three main qustions. Firstly, the impact of different data representations and especially representations that are close to how human experts represent the addressed problem is studied. Secondly, focus is put on how the internal structure of data-driven algorithms can be designed to address problems in the same way as human experts. Finally, it is investigated how constraints, which are specifiedby experts, can be integrated into models to get models that generalise better and prevent non-feasible extrapolation. These questions are addressed in empirical case studies, where the model development is steered by knowledge about the problem provided by domain experts. In the presented cases, the developed algorithms performed significantly better, under the performance metric used, than in other commonly used machine learning models. Hence, this thesis shows the need for the involvement of domain experts in the design of models possessing high performance. This requires models that can both be applied to various data representations and be designed to compute and reason in multiple ways. Hence, the models must be flexible, so that they can be designed to reason in different ways and be applied to different data representations that correspond to the mental representations that is used by the human experts for the targeted problem.

The conclusion of this thesis is that such models can be designed using artificial neural networks, and that these models perform better than conventional machine learning models, when applied to new data. Such models can be designed so that they can be applied to complex representations of the data, representing the targeted problem better. More information regarding the problem can also be provided by designing the model in such a way that the reasoning of human experts is emulated. This can either bedone by emulating the expert behaviour in computational steps or by specifying known boundaries of the problem. The main conclusion is that models that are applied to datarepresentations that capture the real world problem in a good way, emulate the steps that human experts take to solve the problem, or know the limitations around the problem, have more information about the problem, and therefore, have an edge over other models, ultimately resulting in better performance.

Place, publisher, year, edition, pages
Skövde: Högskolan i Skövde, 2021
Series
Dissertation Series ; 39
National Category
Computer Sciences
Research subject
INF301 Data Science; Skövde Artificial Intelligence Lab (SAIL)
Identifiers
urn:nbn:se:his:diva-19630 (URN)978-91-984919-3-7 (ISBN)
Public defence
2021-05-10, Insikten, Kaplansgatan 11 (Portalen), Skövde, 13:00 (English)
Opponent
Supervisors
Note

Publications; Low relevance

VIII. Niclas Ståhl, Göran Falkman, Gunnar Mathiason, and Alexander Karlsson (2018c). “A Self-Organizing Ensemble of Deep Neural Networks for the Classification of Data from Complex Processes.” In: International Conference on Information Processing and Management of Uncertainty in Knowledge-Based Systems. Springer, pp. 248–259.

IX. Juhee Bae, Yurong Li, Niclas Ståhl, Gunnar Mathiason, and Niklas Kojola (2020). “Using Machine Learning for Robust Target Prediction in a Basic Oxygen Furnace System.” In: Metallurgical and materials transactions. B, process metallurgy andmaterials processing science.

X. Juhee Bae, Alexander Karlsson, Jonas Mellin, Niclas Ståhl, and Vicenç Torra (2019). “Complex data analysis.” In: Data Science in Practice. Springer, pp. 157–169.

XI. Denio Duarte and Niclas Ståhl (2019). “Machine learning: a concise overview.” In: Data Science in Practice. Springer, pp. 27–58.

Available from: 2021-04-19 Created: 2021-04-16 Last updated: 2021-07-06Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Ståhl, NiclasMathiason, GunnarFalkman, GöranKarlsson, Alexander

Search in DiVA

By author/editor
Ståhl, NiclasMathiason, GunnarFalkman, GöranKarlsson, Alexander
By organisation
School of InformaticsThe Informatics Research Centre
In the same journal
Applied Mathematical Modelling
Computer Sciences

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 483 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • apa-cv
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf