Co-creating an animation: the use of Large Language Models in health research

• Project title: Co-developing patient and public facing activities to explain large language models and how they are used in health research
• Faculty: Faculty of Medicine and Dentistry
• School: Wolfson Institute of Population Health
• Formats: PPIE workshops; animation
• External Partners: Social Action for Health; King's College London; Brighton and Sussex Medical School; Really Bright Media
• Key themes: AI and healthcare; Large language models; Health research; public and patient involvement and engagement

Please could you give us an overview of the project?

The applicant team came together over a shared interest in engaging the public in research on large language models (LLMs). LLMs are a type of artificial intelligence (AI) which are being increasingly used in healthcare, for example for communication, health record documentation, and making clinical risk predictions. Existing videos to explain LLMs are very technical and assume the viewer already knows something about AI. This can be a barrier to members of the public being able to meaningfully engage in LLM research, and to build trust in their use in healthcare.  

The project aimed to create accessible information on LLMs and their use in healthcare and research. We worked with a group of 6 PPIE (Patient and Public Involvement and Engagement) contributors with the help of Social Action for Health, a community-based charity in East London that provides services and support to people most affected by health inequalities. Our initial workshops focused on building understanding of how LLMs worked through games and using different openly available LLM tools.  

The PPIE contributors decided that we should create two animated videos that: explain how LLMs work and the role of LLMs in health care and research. Over seven workshops we worked with the PPIE contributors to agree the type of information that was important for people to know, refine a script, and design visuals. We tested these with a wider group of PPIE contributors and with other researchers, to make sure it made sense. 

Our intention from the start of the project was to make a resource that could be shared locally. As Bengali is the second most spoken language in Tower Hamlets, we had the script and text on screen translated professionally into Bengali, then improved it with a group of 4 Bengali speaking PPIE contributors (two of whom were part of our original PPIE group). 

We worked with an animation company, Really Bright Media, to bring these ideas to life, with the PPIE contributors reviewing and improving the animation design, voice-overs and music. 

Please could you tell us about the external partners you worked with on this project?

Social Action for Health (SAfH) have an existing relationship with Queen Mary University of London, and Lizzie Remfry (PhD student in the William Harvey Research Institute) has worked with them for some time as part of the AI-MULTIPLY project. SAfH identified individuals to invite as PPIE contributors to the project, and helped to support communication between contributors and QMUL researchers, to ensure people felt confident to participate. We had regular meetings and check-ins via email or WhatsApp, and SAfH were reimbursed for their role in the project to ensure staff costs and time were covered. This partnership was beneficial for both sides. One of SAfH’s aims is to support people who might not typically participate in health research to get involved, and to gain knowledge, confidence and skills important to them. By working with SAfH and through their existing community networks, this allowed us to invite individuals not typically involved in research and many of whom come from underserved communities.  

Please could you tell us more about any specific events you delivered as part of your project?

Workshops

As our PPIE contributors were based in Tower Hamlets, we held most of the workshops on the Queen Mary campus in Whitechapel  ). The workshops were 2 hours long, and where possible, were held the same day of the week and at the same time, to make it easier for the group to plan their attendance.  

The initial workshops introduced the PPIE contributors to key concepts around LLMs, encouraged them to share any knowledge and experiences they had with each other, and provided an opportunity to use LLMs. The applicant team designed some simple games to help build understanding of how LLMs work on a technical level, and these were completed as a group. Once the PPIE contributors felt comfortable, we moved on to brainstorming what other people might want to know about LLMs, and then on to write a script and ways to visually represent the concepts. Following this we sought to iteratively improve the storyboard and then the draft animations provided by Really Bright Media. 

We had originally planned to hold 4 workshops but extended this to 7 workshops, as it became clear we needed more time to talk through the potential implications of LLMs, and to design the images and script. Fortunately, we were able to draw on other resources to compensate PPIE contributors for their time, and this enabled us to work at a pace that felt comfortable for the group, rather than rushing through the stages of the project. 

Launch event

We held a launch event towards the end of the project, which gave PPIE contributors an opportunity to show the finished animation to friends and family. We also invited other researchers working on LLMs in health, so that members of the public had an opportunity to ask about real research and applications being carried out at QMUL. One aspect that worked particularly well was that Mel produced some colouring-in sheets based on key messages and images from animation; this was appreciated by both adults and children! 

Why did you choose to create an animation? How did you go about producing this? 

Before starting the project, the team researched available resources related to increasing understanding of LLMs. During the early workshops, we asked the PPIE contributors how best to present this information in an accessible way, for example via infographic, animation, video or social media posts such as TikTok. Animations were chosen as the best medium, and the PPIE contributors decided that two short animations would be more engaging than one longer animation. 

One of our team members had previously worked with Really Bright Media.  At the start of the project, Mel and Lizzie met with the Really Bright Media team to explain what we hoped to achieve and how we planned to involve the PPIE contributors. We agreed on a timeline and process, and kept in regular contact over the course of the project. We provided the team with our ideas on how to translate this complex information to the screen, with some outlines that Mel sketched and then created on PowerPoint. The animation team came up with great looking options and were very responsive to PPIE feedback at each stage of development.  

One aspect that took some work was ensuring that the content remained accessible. For example, at some points in the videos, words in bubbles were moving across the screen on a conveyor belt. While it was more visually engaging for the bubbles to be moving quickly, people felt that they needed to read all the words and then found it hard to keep up. We kept text in the bubbles (rather than changing them to scribbles to represent words), but asked the team to slow down the pace and reduce the number of bubbles overall.  

We are thrilled with the final output, and PPIE contributors loved seeing their ideas translated into the final animations. 

Please could you tell us more about the audience you worked with? 

Social Action for Health (SAfH) helped us recruit a diverse PPIE contributor group reflecting different ages, ethnicities, digital familiarity and PPIE experience. SAfH provided ongoing input to support contributors to attend (e.g. booking taxis, sending reminders, meeting people at a designated familiar location and arriving to QMUL together). The SAfH team also attended some of the workshops and were able to share their insights from their work in the wider community to help us improve the language and approach. While initially the project started with the research team sharing our knowledge with the PPIE contributors, we tried to keep this flexible to the questions raised during the workshops, and as interactive as possible – ‘showing not telling’ when sharing potential risks and biases, as well as benefits, of LLMs. We wanted the final output to be shaped by the PPIE contributors as much as possible in terms of content, language, and visuals – at times this meant the research team had to be open to their own preferences not being included. Working together over several shorter workshops (rather than in one or two long days) gave the whole team an opportunity to build good relationships and helped us have challenging conversations without losing trust or interest in the project. We feel that this approach led to animations that are accessible and acceptable to a wide audience, which the researchers would not have been able to develop on their own. Additionally, three PPIE contributors joined the team in presenting our process at a conference in Glasgow, at which the work won ‘People’s Choice’, demonstrating the benefits of this approach to our wider team.

What kinds of feedback did you receive from audiences you engaged with? 

Feedback from the launch event included comments like:

• “Really proud that the videos were simple and easy to understand”
• “Amazing videos. Very engaging. Love the Bangla version too!” 

We also invited the PPIE contributors to a final meeting to discuss the manuscript and reflect on the project. They were positive about the experience of co-producing the video and the learning and development opportunities that the team were able to extend, such as attending the conference. For example: 

• “I don’t come from a very educated background, I was nervous at first but with the support of everyone it made it easier. I didn’t feel left out. I can talk about my experiences, my everyday life, my health and bring this forward to other projects.”
• “It also helped to build up personal experiences and work experience.”
• Reflecting on talking at the conference: “It has opened doors, I’m more confident, it gave me something to talk about in front of a room of professionals.”

Where do you think this project will go next?

We hope that these animations will be helpful to other researchers and healthcare providers looking to engage PPIE contributors with work on LLMs  

The animations are freely available on YouTube in English and Bengali. We would welcome any opportunities to translate them into other languages to support research and health engagement in LLMs. 

We are working to share this work on social media platforms and within the relevant networks, for example: a conference and journal paper; research blogs; and seminars. We have already conducted a successful launch event for the local community. 

Please could you share any top tips or lessons learned?

• When co-producing a resource, be realistic about the timeline for building knowledge, deciding priorities, developing and refining a script, and agreeing visuals – it takes longer than you think! 

• Don’t be afraid to be creative – the animation company will make it look good, but using basic sketches and shapes can streamline the process and ensure buy-in from your stakeholders 

• Working with a local partner is beneficial, particularly if you are aiming to work with PPIE contributors that are not typically involved in research. Local partners already have existing networks and knowledge which you can learn from. 

• Look at lots of different funding options, many universities have a PPIE fund, such as the ones at QMUL Centre for Public Engagement, if you are collaborating with researchers from other universities look out for these! 

• Try to be hands on where possible, rather than using power point presentations or written materials to explain things, you could instead use creative ways to involve people in research. 

Watch the animations on YouTube.

This project was supported by a Centre for Public Engagement Participatory Research Fund Grant. Find out more about the Participatory Research Fund.

SDGs supported: 

• SDG 3: Good health and well-being

Queen Mary University of London is committed to driving action for the Sustainable Development Goals through our research, education, and engagement. Find out more about the SDGs at Queen Mary University of London.