DERI Seminar with Ivan Chelombiev from Graphcore

When: Thursday, June 9, 2022, 11:00 AM - 12:00 PM
Where: zoom

Speaker: Ivan Chelombiev is a member of the Research Team at Graphcore

Our DERI seminar series returns after the spring break with Ivan Chelombiev who is a member of the Research Team at  Graphcore.

Title of talk:  GroupBERT: a case study of hardware-aware algorithm design

Zoom Link: https://qmul-ac-uk.zoom.us/j/81148100921

Abstract: Attention based language models have become a critical component in state-of-the- art natural language processing systems. However, these models have significant computational requirements. By using Graphcore’s  IPU technology we demonstrate a set of modifications to the structure of a Transformer layer that makes a more efficient architecture. First, we rely on grouped transformations to reduce the computational cost of dense feed- forward layers, while preserving the expressivity of the model . Secondly, we add a grouped convolution module to complement the self-attention module, decoupling the learning of local and global interactions. The resulting GroupBERT model makes a case study of a hardware-aware algorithm design for the IPU – a novel AI acceleration chip.

Bio: Ivan Chelombiev is a member of the Research Team at Graphcore. Since joining the company in 2019, he has been focusing on the study of weight sparsity in neural networks. His area of research consisted in applying structured sparsity techniques to transformer models. Before joining Graphcore Ivan obtained a MSc degree from the University of Bristol, where he studied Mathematics and Computer Science, focusing his research on information propagation in artificial neural networks.