Deep learning is a modern and exciting approach to machine learning that is delivering state-of-the-art performance in many real-world applications of data science. Deep learning achieves its flexibility and power by representing the world as a nested hierarchy of concepts based on networks of primitive processing units, much like a neural architecture. This module provides an introduction to the goals and methods of deep learning with particular focus on its application to problems in the processing of speech signals and natural language text. Topics will include the history of deep learning and its relationship to other machine learning approaches, the mathematical and computational infrastructure of deep learning, a review of popular deep learning architectures and strategies, applications of deep learning to word semantics, parsing, machine translation and question answering, applications of deep learning to speech recognition, speech synthesis and paralinguistics, and the relationship between deep learning and general artificial intelligence. The course will be presented as a mixture of seminars, discussions about recent papers, and hands-on laboratory sessions with deep learning tools. The course is designed to be accessible to psychology and linguistics students with no prior programming expertise.

The course will be presented as a series of interactive seminars and laboratory sessions. The seminars will be used to introduce concepts of deep learning and to describe and demonstrate deep learning applications in speech and natural language processing. The seminars will be based around a set of internet materials which students will have been expected to study before the meeting. The laboratory sessions will provide “hands-on” experience of the design, training and operation of deep learning models. This will be achieved using web-based tools such as Google Colaboratory or TensorFlow.js.

Aims

To provide an introduction to machine learning as a means to build applications in speech and natural language in contrast to knowledge-based approaches
	To provide an introduction to the deep learning approach to machine learning in terms of its history, its methods, and its mathematical and computational foundations
	To provide an overview of essential elements of deep learning systems in terms of data representation, node types, network structure, loss functions, and optimization algorithms.
	To demonstrate how deep learning is being used to build applications in speech and natural language processing
	To provide practical experience in building, training and running deep learning systems on speech and text data.
Learning outcomes

Awareness of the motivations for deep learning approaches to machine learning
	Awareness of the essential structures and algorithms of deep learning
	Awareness of contemporary applications of deep learning in speech and natural language processing
	Experience in building deep learning models applied to speech and natural language
	Ability to contrast deep learning with other approaches to building applications in speech and language
	Ability to discuss the relationship between deep learning, the scientific method for studying language and the problems of artificial intelligence generally.
Undergraduate enquiries to d.newton@ucl.ac.uk
Postgraduate enquiries to pals.langsci@ucl.ac.uk