Customizable Retrieval-Augmented Generation Framework for Domain-Specific Intelligent Systems

Abstract

The widespread adoption of Large Language Models (LLMs)has enabled significant advances in automated knowledge systems, yet their application to specialized domains remains challenging due to hallucinations, lack of domain-specific context, and limited customization capabilities. This pa per presents a novel customizable Retrieval-Augmented Generation (RAG) framework that enables users to construct domain-specific intelligent systems tailored to their unique requirements. Un like traditional RAG implementations that rely on fixed knowledge bases and retrieval mechanisms, our framework provides a modular architecture allowing users to upload custom datasets, configure embedding models, define retrieval strategies, and establish evaluation metrics. The system com bines user-defined knowledge repositories with state-of-the-art retrieval techniques and language models to generate contextually accurate, domain-specific responses in real-time. We implement advanced data chunking strategies, multilingual BERT-based vectorization, and comprehensive evaluation metrics including faithfulness, answer relevancy, context recall, and context precision. Experimental validation across educational, healthcare, and analytical domains demonstrates significant improvements in response quality, with faithfulness scores of 0.7044, answer relevancy of 0.9838, and context precision of 0.8756 using GPT-4o mini, substantially outperforming generic LLM approaches. Our customizable framework addresses critical limitations of existing systems by providing users with complete control over knowledge integration, retrieval mechanisms, and response generation, paving the way for democratized AI system development across diverse domains.