Despite recent advances, models of Automatic Speech Recognition (ASR) and Text-to-Speech (TTS) still misrecognize words in challenging conditions, limiting their ability. Reinforcement Learning from Human Feedback (RLHF) limits hallucinations in speech models by replacing purely statistical learning with a human-aligned optimization objective that rewards factual, grounded, and faithful outputs while penalizing hallucinated content. Current speech assistants are typically trained on proprietary data and use metrics such as Word Error Rate (WER) to prove their performance. At the same time, RLHF methods have largely focused on text-only models, leaving a gap in tools and datasets for applying preference alignment training to spoken dialogue systems.