Hostname: page-component-cb9f654ff-c75p9 Total loading time: 0 Render date: 2025-08-23T12:47:40.061Z Has data issue: false hasContentIssue false
  • English
  • Français

Automatic identification of role-specific information in product development: a critical review on large language models

Published online by Cambridge University Press:  16 May 2024

Dominik Ehring ,
Ismail Menekse ,
Janosch Luttmer  and
Arun Nagarajah
Dominik Ehring*
Affiliation:
University of Duisburg-Essen, Germany
Ismail Menekse
Affiliation:
University of Duisburg-Essen, Germany
Janosch Luttmer
Affiliation:
University of Duisburg-Essen, Germany
Arun Nagarajah
Affiliation:
University of Duisburg-Essen, Germany
*
dominik.ehring@uni-due.de

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

In the era of digitization and the growing flood of information, the automatic, role-specific identification of information is crucial. This research paper aims to investigate whether the adaptation of LLM is suitable for classifying information obtained from standards for corresponding role profiles. This research reveals that with systematic fine-tuning, prediction accuracy can be increased by almost 100%. The validation was carried out using a two-digit number of standards for three predefined roles and demonstrates the significant potential of LM for labelling content with regard to roles.

Keywords

artificial intelligence (AI)large language model (LLM)fine-tuningrole-specific informationknowledge management

Information

Type
Artificial Intelligence and Data-Driven Design
Information
Proceedings of the Design Society , Volume 4: DESIGN 2024 , May 2024 , pp. 2009 - 2018
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
The Author(s), 2024.

References

Arezki, R., Poncelet, P., Dray, G., & Pearson, D. W. (2004). "Information retrieval model based on user profile." Artificial Intelligence: Methodology, Systems, and Applications: 11th International Conference, AIMSA 2004, Varna, Bulgaria, September 2-4, 2004. Proceedings 11 (pp. 490-499). Springer Berlin Heidelberg.Google Scholar
Mahringer, C. A., & Gabler, M. (2018), Wie können Wissensmanagementsysteme nutzerorientiert gestaltet werden? Die Rolle organisationaler Routinen. [online] Available at https://c-mahringer.de/Mahringer_2018_Wie_k%C3%B6nnen_Wissensmanagementsysteme_nutzerorientiert_gestaltet_werden_HMD.pdf (accessed 20.09.2023).CrossRefGoogle Scholar
Chirapurath, J. (2019). Knowledge mining. The Next Wave of Artificial Intelligence-Led Transformation. Harward Business Review Analytic Services.Google Scholar
Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2018). Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: 1810.04805.Google Scholar
Standards, Initiative Digitale (2023), Optimierung von Rechercheprozessen im deutschen Normungswesen durch Methoden des Natural Language Processing. [online] Available at: https://www.dke.de/idis/pilotprojekte/optimierung-von-rechercheprozessen-durch-nlp (accessed 06.11.2019).Google Scholar
Ehring, D., Ferraz-Doughty, P., Luttmer, J., & Nagarajah, A. (2023). "A first step towards automatic identification and provision of user-specific knowledge: A verification of the feasibility of automatic text classification using the example of standards." Procedia CIRP, 119, 1103-1108.CrossRefGoogle Scholar
Kestel, P. (2021). "Assistenzsystem für den wissensbasierten Aufbau konstruktionsbegleitender Finite-Elemente-Analysen." FAU University Press.Google Scholar
Briskilal, J.; Subalalitha, C. N. (2022). "An ensemble model for classifying idioms and literal texts using BERT and RoBERTa". Information Processing & Management 1/59, S. 102756.CrossRefGoogle Scholar
Li, Z., Li, X., Liu, Y., Xie, H., Li, J., Wang, F. L., & Zhong, …, X. (2023). "Label Supervised LLaMA Finetuning". arXiv preprint arXiv:2310.01208.Google Scholar
Shaheen, Z., Wohlgenannt, G., & Filtz, E. (2020). "Large scale legal text classification using transformer models". arXiv preprint arXiv:2010.12871.Google Scholar
Scheible, R., Thomczyk, F., Tippmann, P., Jaravine, V., & Boeker, M. (2020). "GottBERT: A pure German language model". arXiv. arXiv preprint arXiv:2012.02110.Google Scholar
DIN (2012), DIN 743-1:2000-10: Calculation of load capacity of shafts and axles - Part 1: General. DIN Deutschs Institut für Normung e.V., Berlin.Google Scholar
Claesen, M., & De Moor, B. (2015). "Hyperparameter search in machine learning". arXiv preprint arXiv:1502.02127.Google Scholar
Mansour, M., Cumak, E. N., Kutlu, M., & Mahmud, S. (2023). "Deep learning based suture training system". Surgery Open Science, 15, 1-11.CrossRefGoogle ScholarPubMed
Sobhanam, H., & Prakash, J. (2023). "Analysis of fine tuning the hyper parameters in RoBERTa model using genetic algorithm for text classification". International Journal of Information Technology, 15(7), 3669-3677.CrossRefGoogle Scholar
Luttmer, J., Prihodko, V., Ehring, D., Nagarajah, A. (2023): "Requirements extraction from engineering standards: systematic evaluation of extraction techniques". The 33rd CIRP Design Conference; 17.-19.06.2023, Sydney, Australia, In: Procedia CIRP Jg. 119 (2023) S. 794-799.Google Scholar