About the project
When artificial agents have to communicate by drawing rather than by exchanging words or tokens, what kind of visual language emerges, and when does it become stable, efficient and understandable to humans? This project studies how populations of agents invent shared visual symbol systems through interaction. You will investigate which properties of the task, the agents and the population determine whether and how such systems arise, and develop the experimental tasks and analysis methods needed to characterise them.
Humans communicate through drawings, diagrams and sketches that are grounded in perception and work across linguistic boundaries. Research on communication between artificial agents, by contrast, has mostly relied on discrete tokens or natural language, and there is no systematic account of how visual communication systems emerge and stabilise. Previous work showed that agents can develop sketch-based communication protocols through interaction using differentiable drawing. This project asks the next question: under what conditions do populations of agents converge on stable, efficient and human-interpretable visual communication?
Unlike tokens, visual messages have no predefined vocabulary: any structure or convention must emerge purely from the pressure to communicate, and many existing measures do not carry over. Characterising what emerges is therefore part of the research. The project will combine interactive, cooperative tasks with new measures of compositionality, abstraction, stability and interpretability, to examine questions such as:
- how do task, bandwidth, architecture and drawing mechanism shape the symbols that emerge?
- do heterogeneous populations converge on a shared visual code, or fragment into local ones?
- how readable are emergent symbols to people, and how do they compare with pictograms or early writing?
- how do design and training choices relate to the properties of the resulting communication?
The emphasis can be adapted to your background and interests, whether computational (learning dynamics), analytical (measures and evaluation) or empirical (human studies). The project spans machine learning, cognitive science, linguistics and human–computer interaction, and informs the design of AI systems that communicate transparently with the people who work alongside them.
The School of Electronics and Computer Science is committed to promoting equality, diversity inclusivity as demonstrated by our Athena SWAN award. We welcome all applicants regardless of their gender, ethnicity, disability, sexual orientation or age, and will give full consideration to applicants seeking flexible working patterns and those who have taken a career break. The University has a generous maternity policy, onsite childcare facilities, and offers a range of benefits to help ensure employees’ well-being and work-life balance. The University of Southampton is committed to sustainability and has been awarded the Platinum EcoAward.
