Research on RAG-Based Cognitive Large Language Model Training Method for Power Standard Knowledge
Downloads
Electrical standards encompass complex technical requirements across multiple disciplines, making their management and application a significant challenge that urgently requires efficient solutions. This paper proposes a knowledge graph retrieval-enhanced training method for large language models (LLMs). By leveraging a pre-trained language model (PLM), highly similar subgraphs are retrieved from the electrical standards knowledge graph. These subgraphs are then parsed into triples using entity linking and semantic reasoning. The triples are converted into natural language text by the LLM, which combines them with the input question to perform reasoning and generate accurate answers. The proposed method addresses the complexity of question answering for electrical standards and offers a novel approach for managing and applying these standards in the field of electrical engineering. Experimental results demonstrate that this approach significantly enhances the model's understanding of electrical standards, enabling it to generate more accurate answers.
Downloads
[1] Almughrabi, A., & Hiary, H. (2024). Hand-drawn Electric Circuit Diagrams Recognition using Deep Learning. 2024 28th International Conference on Information Technology, IT 2024, 1–4. doi:10.1109/IT61232.2024.10475731.
[2] Fang, J., Meng, Z., & Macdonald, C. (2024). Reano: Optimising retrieval-augmented reader models through knowledge graph generation. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, Volume 1: Long Papers, 2094-2112.
[3] Sen, P., Mavadia, S., & Saffari, A. (2023). Knowledge Graph-augmented Language Models for Complex Question Answering. Proceedings of the Annual Meeting of the Association for Computational Linguistics, 1–8. doi:10.18653/v1/2023.nlrse-1.1.
[4] Cuconasu, F., Trappolini, G., Siciliano, F., Filice, S., Campagnano, C., Maarek, Y., Tonellotto, N., & Silvestri, F. (2024). The Power of Noise: Redefining Retrieval for RAG Systems. SIGIR 2024 - Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 719–729. doi:10.1145/3626772.3657834.
[5] Prabhong, T., Kertkeidkachorn, N., & Trongratsameethong, A. (2024). KGC-RAG: Knowledge Graph Construction from Large Language Model Using Retrieval-Augmented Generation. CEUR Workshop Proceedings, 3853.
[6] Xu, Z., Cruz, M. J., Guevara, M., Wang, T., Deshpande, M., Wang, X., & Li, Z. (2024). Retrieval-Augmented Generation with Knowledge Graphs for Customer Service Question Answering. SIGIR 2024 - Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2905–2909. doi:10.1145/3626772.3661370.
[7] Wang, J., Wen, Z., Li, X., Guo, Z., Yang, J., & Liu, Z. (2024). Pair Then Relation: Pair-Net for Panoptic Scene Graph Generation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 10452–10465. doi:10.1109/TPAMI.2024.3442301.
[8] Yu, F., Lu, C., Zhou, J., & Yin, L. (2024). Mathematical model and knowledge-based iterated greedy algorithm for distributed assembly hybrid flow shop scheduling problem with dual-resource constraints. Expert Systems with Applications, 239, 122434. doi:10.1016/j.eswa.2023.122434.
[9] Pramanik, S., Alabi, J., Roy, R. S., & Weikum, G. (2024). UNIQORN: Unified question answering over RDF knowledge graphs and natural language text. Journal of Web Semantics, 83, 100833. doi:10.1016/j.websem.2024.100833.
[10] He, G., Lan, Y., Jiang, J., Zhao, W. X., & Wen, J. R. (2021). Improving Multi-hop Knowledge Base Question Answering by Learning Intermediate Supervision Signals. WSDM 2021 - Proceedings of the 14th ACM International Conference on Web Search and Data Mining, 553–561. doi:10.1145/3437963.3441753.
[11] Roh, J., Kim, M., & Bae, K. (2024). Towards a small language model powered chain-of-reasoning for open-domain question answering. ETRI Journal, 46(1), 11–21. doi:10.4218/etrij.2023-0355.
[12] Joshi, P., Gupta, A., Kumar, P., & Sisodia, M. (2024). Robust Multi Model RAG Pipeline for Documents Containing Text, Table & Images. Proceedings of the 3rd International Conference on Applied Artificial Intelligence and Computing, ICAAIC 2024, 993–999. doi:10.1109/ICAAIC60222.2024.10574972.
[13] Diaz-Pace, J. A., Tommasel, A., & Vazquez, H. C. (2024). The JavaScript Package Selection Task: A Comparative Experiment Using an LLM-based Approach. CLEI Eletronic Journal (CLEIeJ), 27(2), 19. doi:10.19153/cleiej.27.2.4.
[14] Shao, Y., Li, H., Gu, X., Yin, H., Li, Y., Miao, X., Zhang, W., Cui, B., & Chen, L. (2024). Distributed Graph Neural Network Training: A Survey. ACM Computing Surveys, 56(8), 1–39. doi:10.1145/3648358.
[15] Kundu, S., & Aakur, S. N. (2023). IS-GGT: Iterative Scene Graph Generation with Generative Transformers. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2023-June, 6292–6301. doi:10.1109/CVPR52729.2023.00609.
[16] Meng, Y., Xiong, C., Bajaj, P., Tiwary, S., Bennett, P., Han, J., & Song, X. (2021). COCO-LM: Correcting and Contrasting Text Sequences for Language Model Pretraining. Advances in Neural Information Processing Systems, 28, 23102–23114.
[17] Alapati, P. R., Lawrance, J. C., Sambath, P., Murugan, R., Rengarajan, M., Raj, I. I., & Bala, B. K. (2024, April). Cross-Lingual Transfer Learning in NLP: Enhancing English Language Learning for Non-Native Speakers. 10th International Conference on Communication and Signal Processing (ICCSP), 1042-1047. doi:10.1109/ICCSP60870.2024.10544031.
[18] Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language models are unsupervised multitask learners. OpenAI Blog, 1-24.
[19] Ribeiro, E., Ribeiro, R., & De Matos, D. M. (2019). Deep dialog act recognition using multiple token, segment, and context information representations. Journal of Artificial Intelligence Research, 66, 861–899. doi:10.1613/jair.1.11594.
[20] Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., ... & Kiela, D. (2020). Retrieval-augmented generation for knowledge-intensive NLP tasks. Advances in neural information processing systems, 33, 9459-9474.
[21] Meng, G., Tariq, M., Jain, S., Elmetwaly, S., & Schlick, T. (2020). RAG-Web: RNA structure prediction/design using RNA-As-Graphs. Bioinformatics, 36(2), 647-648. doi:10.1093/bioinformatics/btz611.
[22] Datta, V. D., Ganesh, S., Haas, R. E., & Talukder, A. K. (2023). GREAT AI in Medical Appropriateness and Value-Based-Care. International Conference on Big Data Analytics, Springer Nature, Cham, Switzerland. doi:10.1007/978-3-031-49601-1_2.
[23] Jain, S., Tao, Y., & Schlick, T. (2020). Inverse folding with RNA-As-Graphs produces a large pool of candidate sequences with target topologies. Journal of Structural Biology, 209(3), 107438. doi:10.1016/j.jsb.2019.107438.
[24] Wadhwa, S., DeYoung, J., Nye, B., Amir, S., & Wallace, B. C. (2023). Jointly Extracting Interventions, Outcomes, and Findings from RCT Reports with LLMs. Proceedings of Machine Learning Research, 219, 754–771.
[25] Ahmed, T., & Devanbu, P. (2022). Few-shot training LLMs for project-specific code-summarization. ACM International Conference Proceeding Series, 1–5. doi:10.1145/3551349.3559555.
[26] Yu, D., & Yang, Y. (2023). Retrieval-Enhanced Generative Model for Large-Scale Knowledge Graph Completion. SIGIR 2023 - Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, 2334–2338. doi:10.1145/3539618.3592052.
- This work (including HTML and PDF Files) is licensed under a Creative Commons Attribution 4.0 International License.
