Agnostic Multi-Source Retrieval-Augmented Generation for Documents and Database Question Answering
DOI:
https://doi.org/10.62710/qw0ytn73Keywords:
Retrieval-Augmented Generation, Question Answering, Multi-Source, FAISS, Knowledge TransferAbstract
Key personnel turnover creates knowledge gaps in document-based service organizations, where information is distributed across technical specifications, operational databases, and team discussions. This study develops a multi-source Retrieval-Augmented Generation (RAG) based Question Answering (QA) system that automatically integrates heterogeneous knowledge sources through a unified source parameter. Using the Adapter Pattern, the system converts PDF/TXT documents and PostgreSQL tables into a common representation, builds a FAISS vector index, retrieves relevant context, and generates grounded answers with Gemini 2.5 Flash. Evaluation employs eight metrics and three composite scores: Knowledge Transfer Effectiveness (KTE), Multi-Source Retrieval Score (MSRS), and Answer Quality Index (AQI). Experiments were conducted on the BOND_SYS dataset using 25 Indonesian questions covering specification documents, an 8-table PostgreSQL database, and 908 developer discussion messages. Results show perfect retrieval performance (Precision@K = 1.000; MRR = 1.000) across all scenarios. The full hybrid configuration achieves the highest Overall score (0.373), while Scenario C records the highest MSRS (0.825). Scenario E obtains ROUGE-L = 0.181 and BLEU-1 = 0.196 using five manually curated reference answers. Two baseline comparisons further support this contribution: a zero-shot LLM without retrieval correctly answered only 8% of questions, while a BM25 keyword-search baseline, competitive on single-source scenarios, was outperformed on cross-referencing tasks, underscoring the added value of dense multi-source retrieval. The findings demonstrate that integrating formal documents, structured databases, and discussion logs enhances knowledge transfer and question answering for organizational support and employee onboarding.
Downloads
References
Chui, M, Manyika J, Bughin J, Dobbs R, Roxburgh C, Sarrazin H, Sands G, Westergreen M (2012) The social economyL unlocking value and productivity through social technologies. McKinsey Global InstituteFriends and neighbors on the web. Soc Networks 25(3):211–230. https://www.mckinsey.com/industries/technology-media-and-telecommunications/our
insights/the-social-economy
Alavi MJ, Leidner DE (2001) Review: knowledge management and knowledge management system. MIS Q 25(1): 107-136. https://doi.org/10.2307/3250961
Borgeaud S, Mensch A, Hoffman J, Cal T, Rutherford E, Millican K, van den Driessche g, Lespiau JB, Damoc B, Clark A, et al. (2022) Improving language models by retrieving from trillions of tokens. In: Proceedings of the 39th International Conference on Machine Learning (ICML). https://doi.org/10.48550/arXiv.2112.04426
Carbonell J, Goldstein J (1998) The use of MMR, diversity-based reranking for reordering documents and
producing summaries. In: Proceedings of the 21st Annual Annual I nternational ACM SIGIR Conference on Research and Development in Information Retrieval, pp 335-336. https://doi.org/10.1145/290941.291025
Es s, James J, Espinosa-Anke L, Schockaert S (2023) RAGAS: automated evaluation of retrieval augmented generation. arXiv:2309.15217. https://doi.org/10.48550/arXiv.2309.15217
Douze M, Guzhva A, Deng C, Johnson J, Szilvasy G, Mazare PE, Lomeli M, Joulin A, Jegou H (2024) The FAISS library. arXiv:2401.08281. https://doi.org/10.48550/arXiv.2401.08281
Fabbri AR, Krycinski W, McCann B, Xiong C, Socher R, Radev D (2021) SummEval: re-evaluating summarization evaluation. Trans Assoc Comput Linguist 9:391-409. https://doi.org/10.1162/tacl_a_00373
Gao Y, Xiong Y, Gao X, Jia K, Pan J, Bi Y, Dai Y, Sun J, Wang M, Wang H (2024) Retrieval-augmented generation for large language models: a survey. arXiv:2312.1097. https://doi.org/10.48550/arXiv.2312.10997
Gamma E, Helm R, Johnson R, Vlissides J (1994) Design patterns: elements of reusable object-oriented software. Addison-Wesley. https://books.google.co.id/books?id=6oHuKQe3TjQC&printsec=frontcover&source=gbs_ge_su
mmary_r&cad=0#v=onepage&q&f=false
Gartner (2020) Gartner says employees spend too much time on low-value tasks: use AI and automation to fix it. Gartner Newsroom. https://www.gartner.com/en/newsroom/press-releases/2020-01-23
gartner-says-employees-spend-too-much-time-on-low-value-tasks
IDC (2023) 90% of data is unstructured and it’s full of untapped value. IDC Blog. https://blogs.idc.com/2023/05/09/90-of-data-is-unstructured-and-its-full-of-untapped-value/
Izacard G, Grave E (2021) Leveraging passage retrieval with generative models for open domain question answering. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics, pp 874-880. https://doi.org/10.18653/v1/2021.eacl-main.74
Ji Z, Lee N, Frieske R, Yu T, Su D, Xu Y, Ishil E, Bang YJ, Mandotto A, Fung P (2023) Survey of hallucination in natural language generation. ACM Comput Surv 55(12):1-38. https://doi.org/10.1145/3571730
Johnson J, Douze M, Jegou H (2019) Billion-scale similarity search with GPUs. IEEE trans Big Data 7 (3):535-547. https://doi.org/10.1109/TBDATA.2019.2921572
Kapurkhin V, Oguz B, Min S, Lewis P, Wu L, Edunov S, Chen D, Yih W (2020) Dense passage retrieval for open-domain question answering. In: Proceedings of EMNLP 2020, pp 6769-6781. https://doi.org/10.18653/v1/2020.emnlp-main.550
Lewis P, Perez E, Piktus A, Petroni F, Karpukhin V, Goyal N, Kiela D (2020) Retrieval-Augmented generation for knowledge-intensive NLP tasks. In: Advances in Neural Information Processing Systems 33, pp 9459-9474. https://doi.org/10.48550/arXiv.2005.11401
Lin CY (2004) ROUGE: a package for automatic evaluation of summaries. In: Proceedings of the ACL Workshop on Text Summarization Branches Out. https://aclanthology.org/W04-1013
Manning CD, Raghavan P, Schutze H (2008) Introduction to information retrieval. Cambridge University Press, Cambridge. Martin RC (2017) Clean architecture: a craftsman’s guide to software structure and design. Prentice Hall, Upper Saddle River
Nonaka I, Takeuchi H (1995) the knowledge-creating company. Oxford University Press, New York. Papineni K, Roukos S, Ward T, Zhu WJ (2002) BLEU: a method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of Association for Computational Linguistics, pp 311-318. https://doi.org/10.3115/1073083.1073135
Robertson S, Zaragoza H (2009) The probabilistic relevance framework: BM25 and beyond. Found Trends Inf Retr 3(4):333-389. https://doi.org/10.1561/1500000019
Reimers N, Gurevych I (2019) Sentence-BERT: sentence embeddings using Siamese BERT-networks. IN: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10.18653/v1/D19-1410
Ren H, Shi H, Zhao W, Zhao J, Zhao Y (2023) Self-RAG: learning to retrieve, generate, and critique through self-reflection. arXiv:2307.11019. https://doi.org/10.48550/arXiv.2307.11019
Shi P, Li X, Han X, Chang B, Sui Z (2023) REPLUG: retrieval-augmented black-box language models. arXic:2301.12652. https://doi.org/10.48550/arXiv.2301.12652 .
Voorhees EM (1999) The TREC-8 question answering track report. In: Proceedings of the 8th Text retrieval Conference (TREC-8), pp 77-82. https://trec.nist.gov/pubs/trec8/papers/qa_report.pdf .
Yasunaga M, Ren H, Bosselut A, Liang P, Leskovec J (2021) QA-GNN: reasoning with language models and knowledge graphs for question answering. In: Proceedings of the 2021 Conference of the North American Chapter of the association for Computational Linguistics, pp 535-545. https://doi.org/10.18653/v1/2021.naacl-main.45
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Krisna Dwi Setya Adi, Ivan Michael Siregar (Author)
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
