Treinamento consciente de quantização vetorial evolutiva: um método adaptativo para compactar modelos de aprendizado de máquina

Abstract

The consolidation of Large Language Models poses significant challenges to their deployment on edge devices, as the reliance on cloud-based infrastructures compromises data privacy, increases latency, and limits critical applications in scenarios with unstable connectivity. Traditional compression techniques, predominantly applied after training, prove inadequate in this context, as they may degrade model performance or rely on opaque execution pipelines, hindering the analysis and verification of system behavior. In response to these challenges, this thesis proposes Evolving Vector Quantization-Aware Training (EVQAT), whose central principle consists of integrating compression directly into the learning process. In this approach, quantization is no longer treated as a post-training step but rather as an adaptive mechanism during training, allowing the transformations imposed on model parameters to be explicitly controlled and analyzed. mThe methodology is grounded in evolutionary vector quantization theory by treating the weights of the language model as continuous data streams, enabling incremental clustering algorithms such as AutoCloud, Mean-Shift, and Affinity Propagation to dynamically adjust the codebooks in synchrony with the optimization of high-dimensional parameters, thereby enabling model compression with explicit control over quantization error. The integration of codebook evolution into the training process ensures that the partitioning of the vector space follows the learning dynamics, preserving relevant statistical properties and reducing the impact of quantization noise. This characteristic makes EVQAT suitable for quantizationsensitive architectures such as Transformers, extending its applicability from multilayer neural networks to generative language models under strict memory and computational constraints. The proposed approach is evaluated across multiple experimental scenarios, including training without quantization, quantization-aware training using evolutionary vector quantization and 8-bit integer (INT8) representations, as well as post-training quantization under the same variants, with deployments on microcontroller-based platforms. In this context, auditability is achieved through transparent access to model parameters, including the vocabulary, the weights of each hidden layer, and the attention mechanisms. As a result, the methodology enables the automatic generation of standard, self-contained C++ code, with real-time inference capabilities and direct execution on diverse hardware architectures, without reliance on third-party libraries or operating systems, ensuring explainability at the levels of training, model, and execution, as well as interoperability across platforms.