Introducing The Big Reveal (WT)

Tim Hopkins

Project aims

How can uses of AI and adaptive technology be used to create narrative fiction in audio form, responsive to location, in AR experiences for audiences using headphones and mobile devices?

Imagine an audience member at point A, wearing headphones, listening to pre-scripted narration that reveals creative spirits at large – talking to listeners, drawing them into a story.  Other scripted text awaits at locations B, C, D etc.

The user moves unpredictably between them – as they do, AI generates spoken text that bridges the story from one place to another, regardless of the route, with compelling narrative traction.   The thresholds between AI and not-AI may be undetectable, or may announce themselves, like doors to a different realm…

The Big Reveal (WT) is a project researching how to make this possible, bringing together colleagues from different disciplines: Tim Hopkins, Sam Ladkin, Andrew Robertson, Kat Sinclair and David Weir.  We’ve had very welcome support also from Jo Walton and other colleagues, and have arranged future contribution from Victor Shepardson (voice synthesis.)

New developments in generative pre-trained transformer (GPT) technology extend uses of deep learning to produce text. This is a branch of machine learning/AI that has many potential uses and societal impacts.  The project explores this using AI and AR as a creative space for collaboration between engineers and artists.  

We are interested in affording practitioners across these disciplines a space to learn by making something together, perceiving AI as a new space for creative writing in a multidimensional expressive space, and making something that might offer the public an engaging way to experience and reflect on their space and the growing presence and impact of AI.

The project envisages three elements, each with its own development stage.

1) a text-generation system that is adaptive to context in a way that sustains / plays with suspension of disbelief

2) voice synthesis that can translate text into convincing speech / narrative voices

3) a platform combining software which can fuse detection of user activity and location with adaptive delivery on a mobile device

Progress so far

This has focused on 1 (as this will define scope for an application for a large project grant supporting the research phases 2 and 3).

Our method has been to develop a lab simulation combining some key technology, functionality and artistry as a proof-of-concept.

We come from different disciplines.  One of the inherent stimulations for the project is navigating differences between how we conceive and share what we are trying to do.  For example, David and Andrew (Informatics) have provided essential insights and guidance on current work with language models, to help induct Tim, Sam and Kat (MAH) into a complex and huge field.   T, S and K have experience of writing / creating for related spaces (e.g. games, adaptive systems, branching narratives), as well as more traditional contexts, but the concepts and engineering potentially underpinning our project ask for new understandings.  Similarly, discussions of language features at work in creative writing (e.g. complex implications of syntax) may test the functionality and limits of existing automated language models.

A central task has been to look for an approach that can respond significantly to what might be minimal input (prompts) from the user.  In contrast to some game formats, where players’ active choices overtly steer subsequent events, we are interested in an experience where users might not perceive any instructional dialogue with a system at all, but feel as if they are being told or are immersed in a narrative that recognises what experience they are having, and is able to determine what they should hear next.  This needs to happen according to a given narrative world and its bespoke generative language model – adapting to information detected by a mobile device as to location, orientation, direction, speed.

A series of discussions (April-November 2022) each led to tests based on existing text2text approaches, whereby text is put into a language model that infers suitable responses based on a range of data.  Although ultimately in our user’s experience there may be no apparent text prompt from the user themselves, there is nonetheless a need for an underlying element of this kind in order to stimulate a generated response. ‘Text’ in this case may be adaptive written text users actually hear, or an equivalent input determined by their behaviour, generating ‘text’ or prompts that may be hidden from users’ perception.  Our tests involved texts / prompts written by Andrew, Kat, Tim and Sam, fed through a number of text generation processes (on , a prominent platform for AI projects.)

Instead of shorter prompts leading to consequent longer responses, many of these processes were originally designed to achieve different kinds of results – such as inferring summaries of information from input texts.   This tended to result in short outputs that were not quite right for our purposes in a variety of ways. Extending prompts did not flex responses more effectively. Varying the character of prompts, for example imitating strongly-flavoured genres, had some perceivable impacts, but not decisively.  We needed to develop functionality towards richer responses.  This suggested adjusting our approach, involving two current directions.

Next steps

Firstly,  we continue to explore core needs – text2text generation, and training a GPT2-like model. However, we’re focussing on getting a ‘good start’ (DW) to an automated response of the kind we might want – rather than concerns about the length of response (which can be addressed later.)  We are also identifying specific corpora to fine-tune a model. Andrew has been experimenting for example using ‘film reviews’ as inputs (recently using Kat is supplying examples of poetry (including her own) and shortly larger corpora needed to train a classifier – something that can distinguish between kinds of input in the way we need.  Andrew is now working on building a test model based on GPT2 to be fine-tuned with this.

Secondly,  the creation of some kind of ranking machine.  For example

a) we give the same input to create a range of candidate texts (e.g.100), a machine ranks them according to merit, and randomly chooses from the top of pile

b) we have two blocks of text – one visible one not.  We insert candidate third blocks between the visible and the hidden, and rank the insertions according to how well they work between the two. (This discussion also included ‘similarity metrics’ and BERT representation – ‘Bidirectional Encoder Representations from Transformers’).

c) we compare prompts with two corpora of texts – one has features we are looking for (e.g. of genre or form), the other is overtly more neutral (e.g. informational website like BBC news) – the machine ranks highest those closest to the first.

In the new year (2023) we will pick up on these, aiming to make our proof-of-concept model in the Spring.   Towards our grant application, we will also start scoping Phase 2 – on voice synthesis – with input from Victor Shepardson (Iceland, Dartmouth,  delayed this year due to COVID19 impacts.) We will look at challenges and consequences for translating text responses into persuasive speech simulation, and practical issues around processing – since the outcome envisages accompanying users across ‘borders’ in real time, between recorded adaptive narration and AI assisted/generated narration.

Tune in to International Dawn Chorus Day online

By Alice Eldridge

This weekend is International Dawn Chorus day, a worldwide celebration of nature’s great symphony. Not everyone is of the requisite constitution to get up in time to witness the majesty of the spring dawn chorus, but fear not — you can listen in from the comfort of your own bed. As part of ongoing research into the art, science and technology of wild soundscapes we have installed a high-fidelity, DIY, off-grid live audio transmitter at Knepp Wilding Project in West Sussex.

Dawn mist over the hammer pond at Knepp Rewilding project.

Our live feed is part of a global broadcast, linking the dawn chorus of Sussex to a network of open microphones around the world. Over the weekend of Dawn Chorus day each year, project partners, Sound Camp curate a live global dawn chorus transmission, Reveil. By mixing live feeds from around the globe they create one continuous 24 dawn chorus, following the path of the rising sun around the planet as our feathered friends awaken and warm up their virtuosic synrinxes.

Reveil is a 24 hour live broadcast of the dawn chorus as it circumvents the globe.

You are invited to listen to the Knepp soundscapes both above and below water. One ear is up in an oak tree, roosting with the turtle doves, cuckoos, owls and nightingales that have come to breed, evidence of the astonishing success of the rewilding of this arable farm over the last 20 years. The other ear takes you under water into a little stream where you can variously hear the tinkle of a babbling brook, splashing of a duck bathing, pig drinking, or subtle munching of an, as yet unidentified, freshwater invertebrate.

The soundscape from the canopy of an oak tree is transmitted via a microphone, sound card and 3G dongle perched in the tree.

This technical and artistic experiment complements ongoing scientific and ethnographic research into cultural and natural soundscapes, including the potential to use sound to monitor ecological status. We now recognise that we are on the edge of the sixth great extinction. Various national, European and global strategies such as Biodiversity Net Gain, EU Biodiversity strategy 2030 or the UN Decade on Restoration, aim to halt or reverse biodiversity loss. Such schemes require evidence to monitor progress and inform decision making, but traditional survey methods of ecological health assessment are prohibitively time-consuming. Our previous research, alongside that of an increasingly active international community of ecoacousticians, demonstrates that listening in to ecosystems can provide valuable information about ecological status, biodiversity, and even behavioural changes in certain species.

The research cannot progress within a single discipline. Even within Sussex University over the last few years our research into cultural and natural soundscapes has involved collaborations across disciplines including conservation biology, international development, anthropology, AI, complexity science, neuroscience and music, partnering with artists in London, indigenous communities in Ecuador, fishers in Indonesia, parabiologists in Papua New Guinea, tourism operators in Sweden, anthropologists in Finland, ecoacousticians in Italy and geographers in France. Working together across and beyond disciplines enables technical and methodological innovation alongside enthnographic, cultural and ethical insights, that not only stimulate methodological and theoretical advances in conservation technologies, but bring other voices in to the conversation. In this way we aim to contribute to social and ecological sustainability through creating cost-effective monitoring tools and advancing equitable conservation agendas.

If the soundscape acts as a transdisciplinary nexus for research, it also connects across species boundaries. As you listen to the exquisite nightingale trios in the late evening, the sound of ducks paddling or tiny insects feeding, I defy you to maintain a strong sense of human exceptionalism. Intimately witnessing the moment-to-moment details of the lives of these other beings unfold is a strong, sensory reminder of our interdependence — of the fact that human well being and that of all other living organisms are inseparable. And a reminder that we need to act fast to ensure that all our songs continue long into the future.

— — —

Bringing you 24 hours of dawn chorus around the earth, Reveil runs 5am London time (UTC+1) on Saturday 30 April to 6am on Sunday 1 May 2022. Listen live on the Reveil platform

The live stream from Knepp is a long-term experiment in Rewilding Soundscapes – perhaps the ultimate slow radio. It is funded by Sussex Humanities Lab, Experimental Ecologies strand and is a collaboration between Alice Eldridge and arts cooperative Sound Camp.

You can listen to the live stream from Knepp day and night for years to come here.

Scoping out the best site at for a long term soundscape stream with Grant and Dawn of Sound Camp

Coming soon: An exciting announcement which explains the motivation for and development of this long term audio streaming project …

Utopia on the Tabletop

Utopia on the Tabletop is a forthcoming collection of scintillating interventions and essays about tabletop roleplaying games and their relation to utopian theory and practice. It is being edited by Jo Lindsay Walton and supported by the Sussex Humanities Lab’s Open Practice group. Contributors include:

  • Emma French
  • Francis Gene-Rowe
  • Grant Brewer
  • Kelsey Paige Mason
  • Lesley Guy
  • Vivek Santayana
  • Felix Rose Kawitzky
  • Nicholas Stefanski
  • Maurits W. Ertsen
  • Rafael Carneiro Vasques, Vitor Celli Caria, and Matheus Capovilla Romanetto
  • Benjamin Platt
  • Grace A.T. Worm
  • Allan Hughes and Mark Jackson
  • Jess Wind
  • Nicholas J. Mizer
  • Jo Lindsay Walton
  • Rok Kranjc
  • Kellyn Wee

More to be confirmed!

The collection will launch in late 2022 / early 2023. It will be a collaboration with MAH’s new imprints Ping Press and Both Are Worse. In the meanwhile, you can keep an eye on Vector, where you’ll be able to get a sneak peek at some of the chapters throughout the second half of 2022. See also the related Applied Hope Games Jam.


Welcome to the sparkling new Sussex Humanities Lab blog!

What is this field, called the Digital Humanities? Broadly speaking, it’s about a dialogue – occasionally a tussle, or a two-part harmony – between all the things humanities scholars have traditionally done, and the new and emerging practices which digital technology enables. Here at the SHL, we’re organised into four strands: Digital History/Digital Archives, Digital Media/Computational Culture, Digital Technologies/Digital Performance, and Digital Lives/Digital Memories; we also collaborate extensively with the TAG (Text Analysis Group) Lab, and with the University of Sussex Library. Building innovative archival and analytic tools to reappraise literary and cultural heritage is part of what we do; so is thinking through the ethical implications of the changing nature of privacy and personal data; so is investigating fledging or fleeting everyday cultural practices of social media users. The fore-edges of medieval manuscripts are in our wheelhouse; so are the memes of 4Chan.

But even a permissive definition of the Digital Humanities risks falling short of the sheer richness and diversity of activity taking place under its rubric. The influence of the Digital Humanities spreads wide, as encounters with new cultural forms often cast fresh light on the familiar, revealing what was under our noses all along. Some scholars and artists have already started to prefer the term postdigital. Let’s not forget about that strong practice-led thread either: the Digital Humanities is not only critical and curatorial, but also creative.

Perhaps the best way to understand the Digital Humanities is to keep an eye on what Digital Humanities scholars are up to. That’s where this blog comes in. In the coming months we’ll be bringing you glimpses of dozens of exciting projects and initiatives with which the Sussex Humanities Lab is involved. We hope to make this blog a place of interest to researchers of all disciplines, and to the public at large.

For my part, I mostly work on the intersection of speculative fiction and economics. How are creators of speculative fiction imagining the impact of automation and Artificial Intelligence on society? Can speculative fiction inform the design of new institutions and policies, allowing us to meet the ecological challenges of the future? Or … maybe it can’t? Later in the year, I’ll be sharing my research more fully. But right now, I want to flag up two projects I’ve been involved with editorially, both hot off the presses. On the fiction side, there’s Strange Economics, an anthology from David Schultz, featuring original economics-themed short stories. The ebook will be free for a limited time. Then there’s Vector, the UK’s longest-running magazine of SF criticism. Vector #288, co-edited with Polina Levontin, a special issue devoted to the future of economics. The magazine only goes to members of the British Science Fiction Association, but we’ll be featuring plenty of excerpts on the Vector website over the next few months.