EchoTree: Engaged Conversation when Capabilities are Limited

Abstract

Speech and motor-impaired individuals using assistive tech- nologies to communicate face a common problem; the inevitable conversational ’dead space’ which occurs while the user generates written or artificially-spoken sentences can dramatically reduce conversation quality and partner engagement. To address this problem, we introduce EchoTree, a visualization approach employing predictive language mod- eling to generate and display candidate utterances, thus al- lowing partners to make guesses about the impaired user’s intended conversational direction. By including partners in the process of sentence generation, EchoTree mitigates the dead space problem by (1) engaging partners in a collaborative activity, and (2) reducing the task of word-generation to that of confirming a word choice. Our implementation is browser-based, making it suitable for face-to-face or remote communication, as multiple parties can view and interact with the same stream of EchoTrees from desktops, tablets, or smartphones. We describe the visual and system design choices and the experiments with data sources and language models which led to our current implementation.