skip to content

Language and Gesture Generation in Dialogue

Contact: Paul Piwek (P.Piwek 'at' open.ac.uk)

This theme is motivated by the aim to gain an understanding of how people communicate information, especially in dialogue. It aims to bring together empirical and computational work: empirical data (e.g., audio and video recordings) of how natural language and gestures are used in conversation and computational models that account for such data in terms of the underlying information processing. The theme crosses the boundaries of several disciplines, including Natural Language Processing, Artificial Intelligence and Cognitive Science. There is also an applied strand to this theme which looks at leveraging our understanding of communicative processes to build technologies for communicating across linguistic and sensory barriers.

General areas for a project in this theme are:

Dialogue Simulation. To understand one or more phenomena in dialogue, such as cooperation, coordination of gesture and speech, adversity, or emotion, a project in this area would focus on constructing computational models of dialogue agents. In particular, these agents should have the ability to converse with each other, and the resulting behaviour should mirror, in relevant respects, the human behaviours that we find in naturally-occurring dialogues.

Question Generation. One of the most striking abilities of human beings is that of generating questions. Questions help to get information flowing. We ask questions, among other things, to obtain factual information ("Where would you like to travel to?"), request clarification ("Did you say 'London'?") and gauge other people's knowledge (e.g., in a situation where a teacher, who already knows the answer, asks a pupil a question). In this project, you would work on developing the theory and technology for automatically generating questions from machine-readable resources. Machine-readable resources could be anything from raw text to semantic web ontologies.

Applications to Inclusive Communication. In this project, you would work on building and exploiting computational models of information flow in dialogue to support human-human and/or human-computer communication. You will focus on technologies that help with communicating information across barriers, whether they are linguistic or sensory. Questions that this project could address are: How can we build technologies that help people from different language communities talk with each other? How can we make statistical graphs accessible for a visually-impaired person?