Metodologia orientada a agentes de linguagem para assistência automotiva: integrando engenharia de Prompts em Chatbots avançados

Abstract

The increasing presence of digital systems in vehicles has expanded the number of functionalities available to users. However, recurring doubts related to the use of these features, the interpretation of alerts, and the execution of basic operational procedures still require consulting the owner’s manuals to ensure proper vehicle handling. Although the digitization of these documents represents an advancement over physical formats, access to information remains limited, especially in situations that demand quick responses and accessible language. In this context, this work proposes a language agent–oriented approach, grounded in the Retrieval-Augmented Generation (RAG) technique, with the goal of facilitating specialized consultation of technical content in automotive manuals. The methodology encompasses stages such as segmenting texts into coherent fragments, indexing them in a vector database, crafting prompts tailored to a Large Language Model, and conducting a comparative evaluation of six RAG variants (Conventional, with Gradient Descent, Multi-Query, Step-Back, Self-RAG, and Self-RAG with Gradient Descent). Accordingly, an experiment was conducted in which each variant was evaluated using the LLM-as-a-judge strategy, in which an LLM assigned scores for contextual faithfulness, question relevance, informational completeness, and safety verification. The dataset used comprised twenty questions, ten based on the Volkswagen Polo 2025 manual and ten on the Fiat Argo 2023 manual. Additionally, semantic similarity between pairs of responses was measured using BERTScore. The results indicated that Step-Back achieved the highest overall average score and led in completeness and safety, whereas Self-RAG delivered the best performance in faithfulness and exhibited high semantic convergence with its gradient-based variant. These findings suggest that mechanisms of reformulation, decomposition, and self-assessment improve both the quality and consistency of responses, highlighting the potential of adaptive architectures to enhance technical assistance in embedded systems.