Large Language Models for Ambiguity Detection and Resolution in Smart Homes

This thesis studies the use of LLMs within smart home systems for ambiguity detection and resolution. Existing literature often overlooks the significance of user-centric disambiguation, as smart home systems typically interpret user intentions based on context and predefined settings without considering the user's specific definitions of ambiguous concepts (e.g., "large," "cozy," "comfortable"). This oversight potentially impacts the accuracy of results from the user's perspective. This thesis explores how users perceive the interaction with a text-based smart-home system and its results when user-oriented disambiguation is set in place. To achieve this, an add-on system was developed to identify concept ambiguities in user intents and present disambiguation options in both text and image formats using LLMs and multiple prompting strategies. Experiments involving 7 participants were conducted within a simulated smart-home environment, with four predefined intents. The results indicate that users view disambiguation positively, as it reduces the probability space of responses and consequently increases the accuracy. However, it was found that the introduction of the disambiguation add-on decreases the precision. Despite this, users perceive the trade-off favorably, valuing increased accuracy even at the expense of precision; responses aligned closely with the user's context and mental flow were rated higher, even when they deviated from the user's initial intent expectation.