WEB-BASED PUBLIC RELATIONS CHATBOT USING LARGE LANGUAGE MODELS AND THE RETRIEVAL-AUGMENTED GENERATION

Abstract

The Public Relations and Protocol Working Group (Pokja Humas) of Politeknik Negeri Ujung Pandang (PNUP) faces challenges in providing interactive and responsive information services. The official website functions only as a one-way medium, and the high volume of repeated questions causes delays in response time. This study developed a public relations chatbot based on Large Language Model (LLM) using the Retrieval-Augmented Generation (RAG) method to improve information services. The chatbot data were obtained through web scraping of the PNUP official website and internal PDF documents, which were processed through preprocessing, text splitting, and embedding using Hugging Face and stored in a FAISS vectorstore. The system was built using FastAPI as the backend and web-based interfaces for admin and user interactions. The results show that User Acceptance Test (UAT) involving 35 respondents achieved 91.93% acceptance (very good). The Retrieval-Augmented Generation Assessment (RAGAS) evaluation achieved average scores of 0.89 for Faithfulness, 0.91 for Answer Relevancy, 0.89 for Context Precision, and 0.89 for Context Recall, indicating that the chatbot produced relevant and contextually accurate answers.