Understanding reality through algorithms | MIT News

Although Fernanda De La Torre still has several years left in her higher education, she is already thinking big when it comes to what the future holds.

“I dream of one day opening a school where I can bring this world of understanding cognition and perception to places that would never have contact with it,” she says.

It’s this kind of ambitious thinking that has brought De La Torre, a doctoral student in MIT’s Department of Brain and Cognitive Sciences, to this point. A recent recipient of the prestigious Paul and Daisy Soros Fellowship for New Americans, De La Torre found a creative and supportive research environment at MIT that allowed him to immerse himself in the cutting-edge science of artificial intelligence. But she is still driven by an innate curiosity for the human imagination and a desire to bring that knowledge to the communities in which she grew up.

An unconventional path to neuroscience

De La Torre’s first exposure to neuroscience was not in the classroom, but in his daily life. As a child, she watched her younger sister struggle with epilepsy. At age 12, she entered the United States illegally from Mexico to find her mother, exposing her to a whole new language and culture. Once in the United States, she had to deal with her mother’s changing personality in the midst of an abusive relationship. “All these different things I was seeing around me made me want to understand more about how psychology works,” De La Torre says, “to understand how the mind works, and how come we can all be in the same environment and feel very different things.

But finding an outlet for this intellectual curiosity was a challenge. As an undocumented immigrant, her access to financial assistance was limited. His high school was also underfunded and lacked elective options. However, mentors along the way encouraged the aspiring scientist, and through a program at her school, she was able to take courses at a community college to meet basic educational requirements.

It took an inspiring dedication to her education, but De La Torre came to Kansas State University for her undergraduate studies, where she majored in computer science and math. At Kansas State, she got her first real taste of research. “I was just fascinated by the questions they were asking and all this space that I hadn’t encountered,” De La Torre says of her experience working in a visual cognition lab and discovering the field. computational neuroscience.

Although Kansas State does not have a dedicated neuroscience program, her cognition research background led her to a machine learning lab run by computer science professor William Hsu. There De La Torre fell in love with the possibilities of using computation to model the human brain. Hsu’s support also convinced her that a career in science was a possibility. “He always made me feel capable of tackling big issues,” she says lovingly.

With the trust bestowed upon her at Kansas State, De La Torre arrived at MIT in 2019 as a post-baccalaureate student in the lab of Tomaso Poggio, Eugene McDermott Professor of Brain and Cognitive Sciences and researcher at the McGovern Institute for Brain Research. . Along with Poggio, also director of the Center for Brains, Minds and Machines, De La Torre began working on the theory of deep learning, an area of ​​machine learning focused on how artificial neural networks modeled on the brain can learn to recognize patterns and learn.

“It’s a very interesting question because we’re starting to use them everywhere,” De La Torre says of neural networks, citing examples ranging from self-driving cars to medicine. “But, at the same time, we don’t fully understand how these networks can go from knowing nothing and just being a bunch of numbers to producing things that make sense.”

Her experience as a post-graduate was De La Torre’s first real opportunity to apply the technical computer skills she developed as an undergraduate to neuroscience. It was also the first time she could fully concentrate on research. “It was the first time I had access to health insurance and a stable salary. It was, in itself, kind of life changing,” she says. “But on the research side, it was very intimidating at first. I was anxious and unsure if I belonged here.

Fortunately, De La Torre says she was able to overcome those insecurities, both through a growing unabashed enthusiasm for the field and through support from Poggio and her other colleagues in MIT’s Department of Brain and Cognitive Sciences. When the opportunity arose to apply for the department’s doctoral program, she jumped at the chance. “It was just knowing that these types of mentors were there and cared about their students,” De La Torre says of his decision to stay at MIT for graduate school. “It was really meaningful.”

Expand notions of reality and imagination

During her two years in the graduate program, De La Torre’s work expanded the understanding of neural networks and their applications to the study of the human brain. Working with Guangyu Robert Yang, a research associate at the McGovern Institute and an assistant professor in the departments of Brain and Cognitive Sciences and Electrical and Computer Engineering, she engaged in what she describes as more philosophical questions about the how one develops a sense of self. as an independent being. She is interested in how this self-awareness develops and why it might be useful.

De La Torre’s main adviser, however, is Professor Josh McDermott, who heads the Computational Hearing Laboratory. With McDermott, De La Torre tries to understand how the brain integrates vision and sound. Although combining sensory input may seem like a basic process, many questions remain unanswered about how our brain combines multiple signals into a coherent impression or perception of the world. Many questions are raised by audio-visual illusions in which what we hear changes what we see. For example, if one sees a video of two discs crossing each other, but the clip contains the sound of a collision, the brain will perceive that the discs are bouncing rather than crossing each other. Faced with an ambiguous image, this simple auditory signal is enough to create a different perception of reality.

“There’s something interesting going on where our brain receives two signals telling us different things and yet we have to somehow combine them to make sense of the world,” she says.

De La Torre uses behavioral experiments to probe how the human brain makes sense of multisensory cues to construct a particular perception. To do this, she created various scenes of objects interacting in 3D space to different sounds, asking research participants to describe features of the scene. For example, in one experiment, she combines images of a block moving across a surface at different speeds with various scratching sounds, asking participants to estimate the roughness of the surface. Eventually, she hopes to bring the experience into virtual reality, where participants will physically push blocks in response to how they perceive the surface, rather than just reporting what they’re experiencing.

Once she collects data, she will move on to the modeling phase of the research, assessing whether multisensory neural networks perceive illusions the way humans do. “What we want to do is model exactly what happens,” says De La Torre. “How is it that we receive these two signals, integrate them and, at the same time, use all our prior knowledge and physics inferences to really make sense of the world?”

Although her two strands of research with Yang and McDermott may seem distinct, she sees clear connections between the two. Both projects aim to understand what artificial neural networks are capable of and what they tell us about the brain. On a more fundamental level, she says how the brain perceives the world from different sensory cues could be part of what gives people a sense of self. Sensory perception consists of constructing a coherent and unitary sense of the world from multiple sources of sensory data. Likewise, she argues, “sense of self is really a combination of actions, plans, goals, emotions, all these different things that are components in themselves, but somehow create a unitary being. “.

It’s an appropriate sentiment for De La Torre, who has worked to make sense of and integrate different aspects of his own life. Working in the Computational Hearing Lab, for example, she began experimenting with combining electronic music with folk music from her native Mexico, connecting her “two worlds”, as she puts it. Having the space to undertake these kinds of intellectual explorations, and colleagues who encourage it, is one of MIT’s De La Torre favorite parts.

“Beyond the professors, there are also a lot of students whose way of thinking amazes me,” she says. “I see a lot of goodness and excitement for science and a bit of – it’s not nervousness, but a love for very specific things – and I just like that.”

Sharon D. Cole