Preference alignment techniques have achieved remarkable results in aligning Large Language Models (LLMs) with human values through output comparison. However, these methods critically rely on human-annotated preference data, whose collection remains a major challenge due to scalability and consistency issues. The complexity further increases in the multi-modal domain, where annotators may focus on isolated aspects of a given modality (e.g., speech tone or rhythm) rather than its overall communicative intent. Positioned within this context, the present work specifically addresses these challenges in the speech domain. The primary goal is to collect human preference data on speech-based interactions, ensuring that annotators are properly guided to provide consistent and meaningful feedback.