Seminars

Home Schools Computational Sciences Seminars

ABSTRACT: To achieve stable training and improved performance in large language models (LLMs), techniques such as layer normalization, retrieval-augmented generation, and various hallucination detection methods have been widely adopted. This presentation critically examines the limitations of these existing approaches and introduces statistical methodologies that further enhance efficiency, stability, and reliability. First, I present Bounded Hyperbolic Tangent (BHyT), a new activation function designed to address shortcomings of conventional layer normalization. Unlike layer normalization, BHyT does not require variance computation, making it more computationally efficient, while Chebyshev’s inequality is leveraged to promote stable training. Second, I introduce a novel text embedding ensemble framework that integrates the strengths of multiple embedding models. By quantifying model uncertainty for given data via Laplace approximation, the proposed method ensembles embeddings in an uncertainty-aware manner, leading to improved retrieval performance. Finally, to address hallucinations that may still arise despite these advances, I present a new conformal inference–based hallucination detection method, providing statistically grounded reliability guarantees.