This research is about addressing the need to better understand interaction with conversational user interfaces (CUIs) and how human-technology `conversations' can be improved by drawing on the lessons learned from human-human interaction. It focuses on incorporating abstractions of complex human behaviour, specifically gaze, to enhance interactions with speech agents in conversations. Across four empirical studies, a mix of methods is used to look closely at the interaction between the user and the system.

I offer empirical and conceptual contributions for interaction designers and researchers. First, I present a novel speech interface, Tama, which is a gaze-aware speech agent designed to explore the use of gaze in conversational interactions with smart speakers. Second, I present the empirical contributions, that is, the studies that document the interactions with and around speech interfaces, including ongoing, non-system-directed speech. A moment-by-moment analysis of these interactions highlights the opportunities that the gaze offers as a modality to enhance the interaction with the speech agent, as well as the problems and limitations when such a modality is used. The third contribution is a conceptual contribution made by providing perspective on minimal anthropomorphic design. This produces interactions that are not human-like in terms of behaviour but do take advantage of the skills used in human interaction as a key to advancing interactions with speech agents.

Based on my research work and contributions, I reflect upon advancing interactions with speech interfaces, focusing on what different technologies can offer and the possibility of taking the next step in designing CUIs. I then discuss the need to bridge the work of different fields (i.e. conversation analysis (CA), human-computer interaction (HCI), and human-robot interaction (HRI)) to combine models and approaches from all these fields in order to guide designers building speech systems. I see three competing yet complementary interaction paradigms across CUIs. I call these paradigms Direct Speech Interaction, Agent-Mediated Interaction, and Para-Speech Interaction. Each of these paradigms has specific challenges and opportunities for interaction.