James Heald

I am a Research Fellow working with Maneesh Sahani at the Gatsby Unit, UCL. Previously, I worked as a postdoctoral researcher at the Zuckerman Mind Brain Behavior Institute, Columbia University with Daniel Wolpert.

I received a PhD in Engineering from the University of Cambridge, where I trained in the Computational and Biological Learning Lab under the supervision of Daniel Wolpert (primary supervisor) and Máté Lengyel (secondary advisor). Prior to my PhD, I worked as a medical doctor in the NHS, and before that, I received an MSc in Cognitive and Computational Neuroscience from The University of Sheffield and an MBChB in Medicine from The University of Manchester.

Email  /  Google Scholar  /  GitHub  /  LinkedIn  /  Twitter

I study motor control in machines and brains using reinforcement learning (RL) and probabilistic machine learning. My research interests span model-based RL, representation learning, transfer learning and imitation learning.

I am an active contributor to the open-source platform MyoSuite, participating in biweekly development meetings. Working collaboratively and independently, I have:

• Ported environments from MuJoCo to MuJoCo XLA, enabling massively parallel, GPU-accelerated simulations.
• Desiged new musculoskeletal models and tasks for the wider research community.
• Built an API to allow users to modify MuJoCo models for their own research needs.

Outstanding research achievements are highlighted below.

MyoChallenge 24: Physiological Dexterity and Agility in Bionic Humans
Team Muscle Heads: James Heald, Kai Biegun, Samo Hromadka, Maneesh Sahani
NeurIPS 2024 Competition Track Program

Our team came 🥇 first place 🥇 in the NeurIPS 2024 MyoChallenge Manipulation Track. The challenge involved coordinating a biologically-realistic arm with 63 muscles and a robotic prosthetic arm with 17 DoFs to transfer an object between two pillars. The object properties and pillar locations were randomized on each episode. To control the musculoskeletal arm, I developed a novel algorithm to learn a low-dimensional control manifold in the high-dimensional action space, enabling efficient policy search. This allowed us to efficiently acquire dexterous reaching and grasping skills. To control the robotic prosthetic arm, we combined a learned policy with inverse kinematic control, enabling smooth hand-offs and object relocation. Curriculum learning and reward shaping were used to solve the task in stages (reach, grasp, handover, move object to goal) .

We review a theoretical approach to formalizing context-dependent learning in the face of contextual uncertainty and the core computations it requires. We show how this approach organizes a large body of disparate experimental observations, from multiple levels of brain organization (including cells, circuits, systems and behavior) and multiple brain regions (most prominently the prefrontal cortex, the hippocampus and motor cortices), into a coherent framework. We argue that contextual inference may also be key to understanding continual learning in the brain.

In this issue of Trends in Cognitive Sciences, we review a computational framework of repertoire learning that provides a unifying account of phenomena across numerous domains, including conditioning, episodic memory, economic decision making and motor learning.

We develop a theory of motor learning based on the principle of contextual inference. Our theory reveals that adaptation can arise by both creating and updating memories and changing how existing memories are differentially expressed.

Different parts of tools are often handled in different ways. This study presents a computational model explaining how humans build separate motor memories for different parts of the same object.

website template