Pinpointing the Brain’s Arbitrator

Caltech researchers ID a brain mechanism that weighs decisions

We tend to be creatures of habit. In fact, the human brain has a learning system that is devoted to guiding us through routine, or habitual, behaviors. At the same time, the brain has a separate goal-directed system for the actions we undertake only after careful consideration of the consequences. We switch between the two systems as needed. But how does the brain know which system to give control to at any given moment? Enter The Arbitrator.

Researchers at the California Institute of Technology (Caltech) have, for the first time, pinpointed areas of the brain—the inferior lateral prefrontal cortex and frontopolar cortex—that seem to serve as this "arbitrator" between the two decision-making systems, weighing the reliability of the predictions each makes and then allocating control accordingly. The results appear in the current issue of the journal Neuron.

According to John O'Doherty, the study's principal investigator and director of the Caltech Brain Imaging Center, understanding where the arbitrator is located and how it works could eventually lead to better treatments for brain disorders, such as drug addiction, and psychiatric disorders, such as obsessive-compulsive disorder. These disorders, which involve repetitive behaviors, may be driven in part by malfunctions in the degree to which behavior is controlled by the habitual system versus the goal-directed system.

"Now that we have worked out where the arbitrator is located, if we can find a way of altering activity in this area, we might be able to push an individual back toward goal-directed control and away from habitual control," says O'Doherty, who is also a professor of psychology at Caltech. "We're a long way from developing an actual treatment based on this for disorders that involve over-egging of the habit system, but this finding has opened up a highly promising avenue for further research."

In the study, participants played a decision-making game on a computer while connected to a functional magnetic resonance imaging (fMRI) scanner that monitored their brain activity. Participants were instructed to try to make optimal choices in order to gather coins of a certain color, which were redeemable for money.

During a pre-training period, the subjects familiarized themselves with the game—moving through a series of on-screen rooms, each of which held different numbers of red, yellow, or blue coins. During the actual game, the participants were told which coins would be redeemable each round and given a choice to navigate right or left at two stages, knowing that they would collect only the coins in their final room. Sometimes all of the coins were redeemable, making the task more habitual than goal-directed. By altering the probability of getting from one room to another, the researchers were able to further test the extent of participants' habitual and goal-directed behavior while monitoring corresponding changes in their brain activity.

With the results from those tests in hand, the researchers were able to compare the fMRI data and choices made by the subjects against several computational models they constructed to account for behavior. The model that most accurately matched the experimental data involved the two brain systems making separate predictions about which action to take in a given situation. Receiving signals from those systems, the arbitrator kept track of the reliability of the predictions by measuring the difference between the predicted and actual outcomes for each system. It then used those reliability estimates to determine how much control each system should exert over the individual's behavior. In this model, the arbitrator ensures that the system making the most reliable predictions at any moment exerts the greatest degree of control over behavior.

"What we're showing is the existence of higher-level control in the human brain," says Sang Wan Lee, lead author of the new study and a postdoctoral scholar in neuroscience at Caltech. "The arbitrator is basically making decisions about decisions."

In line with previous findings from the O'Doherty lab and elsewhere, the researchers saw in the brain scans that an area known as the posterior putamen was active at times when the model predicted that the habitual system should be calculating prediction values. Going a step further, they examined the connectivity between the posterior putamen and the arbitrator. What they found might explain how the arbitrator sets the weight for the two learning systems: the connection between the arbitrator area and the posterior putamen changed according to whether the goal-directed or habitual system was deemed to be more reliable. However, no such connection effects were found between the arbitrator and brain regions involved in goal-directed learning.  This suggests that the arbitrator may work mainly by modulating the activity of the habitual system.

"One intriguing possibility arising from these findings, which we will need to test in future work, is that being in a habitual mode of behavior may be the default state," says O'Doherty. "So when the arbitrator determines you need to be more goal-directed in your behavior, it accomplishes this by inhibiting the activity of the habitual system, almost like pressing the breaks on your car when you are in drive."

The paper in Neuron is titled "Neural computations underlying arbitration between model-based and model-free learning." In addition to O'Doherty and Lee, Shinsuke Shimojo, the Gertrude Baltimore Professor of Experimental Psychology at Caltech, is also a coauthor. The work was completed with funding from the National Institutes of Health, the Gordon and Betty Moore Foundation, the Japan Science and Technology Agency, and the Caltech-Tamagawa Global COE Program. 

Writer: 
Kimm Fesenmaier
Writer: 
Exclude from News Hub: 
No

Moneyball Comes to Caltech

The winter 2014 quarter at the California Institute of Technology (Caltech) is premiering a unique course, The Theory and Practice of Moneyball, taught by Caltech professor of political science and dean of undergraduate students Rod Kiewiet, and Fred Claire, former general manager of the Los Angeles Dodgers and current sports consultant and educator.

Moneyball—the title of a 2003 book by Michael Lewis and a 2011 film starring Brad Pitt—has entered the contemporary lexicon as the informal name for sabermetrics, a term coined by baseball analyst Bill James in 1980, who defines it as "the search for objective knowledge about baseball." (James came up with the term sabermetrics as a reference to the Society for American Baseball Research [SABR].)

Of course, managers, coaches, and players have long attempted to bring objective knowledge to bear on their attempts to find more winning ways to play the game. But it was not until recent decades, with the development of new modes of collecting data about baseball (from the angle of a pitcher's arm to the exact spin on a ball) and ever more sophisticated statistical methods and computer technologies, that sabermetrics really took off and became the burgeoning business it is today. Most major ball clubs have a sabermetrician on staff, and baseball fans are increasingly fluent in the theories that sabermetrics have brought to the forefront. Is a batting average a helpful metric in predicting wins? Sabermetrics says no. Is there a strategic advantage associated with bunting or stealing bases? Common wisdom has said yes for generations, but sabermetrics typically says no, and both practices have become less common in the past decade of professional baseball.

To shed light on this last debate, Kiewiet and Claire invited a special guest to their moneyball class on January 21: Maury Wills, former Dodgers shortstop and the National League's Most Valuable Player in 1962, the year he stole 104 bases, breaking Ty Cobb's 1915 record for bases stolen in a single season. Wills, of course, has every reason to doubt the conventional sabermetric wisdom about the deficiencies of the "small ball" game of bunts and stolen bases. He was an expert at both during his years as a major-league player; at age 81, he continues to coach Dodgers players on the art of setting down the perfect bunt.

As Wills, Claire, and Kiewiet agreed, the statistical techniques employed in sabermetrics can be enlightening, but can also be misleading if they are not sophisticated enough. Bunting and stealing bases may not look like winning strategies in the aggregate, but when deployed properly within the game—by the right players at the right time—they can contribute to a team's success. This is, says Wills, because baseball is a "mental game": "You can be tall, short, skinny, fat, fast, slow, handsome . . . and not-so-handsome. Yet you can be a good baseball player. That's why the game is so great."

Freshman seminars were introduced at Caltech in 2011 to give first-year students more direct contact with faculty in a small class setting (12–15 students total) and the opportunity, says Kiewiet, "to have fun, and not just be grinding away." In addition to The Theory and Practice of Moneyball, this year has also offered freshman seminars on earthquakes, gravitational waves, and courses with evocative names like Albatrosses, Beetles, and Cetaceans—a reflection on scaling in nature—and The Origin of Ideas.

"I feel very strongly about the value of freshman seminars," says Kiewiet, "and as dean, I'm anxious to keep these going, so I thought I should put my money where my mouth is and do one myself." Moneyball was a natural choice for Kiewiet. A baseball fan from childhood, he studied sabermetrics in the late 1980s, when Bill James's work came to greater prominence. Kiewiet mentored Caltech undergraduate Ari Kaplan, BS '92, a baseball player, in a SURF fellowship on pitching statistics that eventually led Kaplan to a career as a sports consultant. Kaplan has now worked for over half the major-league baseball teams in America, in addition to his ongoing career in business software; he co-owns a sabermetrics company, AriBall, with Claire.

As for Kiewiet, he has kept up with baseball statistics, though his scholarly efforts are primarily directed toward research into public-school finance, voting in British elections, and other pursuits typical of a professor of political science. Kiewiet did serve as the sabermetrics specialist for the film Moneyball, though he claims this was "only because they couldn't get Kaplan, because he was busy working for the Cubs."

Kiewiet recently got approval from the faculty board to open freshman seminars to upperclass students. Freshmen have the opportunity to register first, but if there are remaining spaces in these classes after freshmen have signed up, more advanced students can take them too. As a result, Kiewiet and Claire's moneyball seminar has attracted five upperclass students along with four freshmen, almost all of whom are student athletes. "Moneyball is a good laboratory for looking at economic theories," says Kiewiet. "It's got all the issues of strategy and information and of maximizing efficiency with budget constraints. But frankly, I just really like baseball."

Writer: 
Cynthia Eller
Contact: 
Writer: 
Exclude from News Hub: 
No
News Type: 
In Our Community
Monday, May 5, 2014
Moore 070

Teaching Statement Workshop - 2-Part Event

Monday, May 12, 2014
Center for Student Services 360 (Workshop Space)

Teaching Statement Workshop - 2-Part Event

Friday, April 4, 2014
Center for Student Services 360 (Workshop Space)

Spring TA Training

Tuesday, April 1, 2014
Center for Student Services 360 (Workshop Space)

Spring Head TA Lunch

Exploration: The Globe and Beyond

A New Lecture Series at Caltech

Caltech has long had a reputation for wide-ranging exploration, and now its Division of Humanities and Social Sciences is celebrating this theme in a lecture series titled Exploration: The Globe and Beyond. The series is intended to bring together a diverse community to discuss the broad theme of exploration, from antiquity to the present day, from new lands on Earth to other planets in our solar system.

"Exploration," says Professor of History Nicolas Wey-Gomez, "is an indeterminate process. It is about abandoning oneself to a search that may or may not lead somewhere other than where one began. However uncertain it may be at times, it is the prerequisite for any real discovery."

The first lecture in the series—"Junípero Serra and the Spanish 'Craze'"—was given on January 6 by historian Richard L. Kagan of Johns Hopkins University. Kagan described how Serra came to stand in as the "founding father of California, the Columbus of the West." The wave that Serra rode to this new status—well after his death—was part of what Kagan described as a "craze" for all things Spanish that arose, ironically, in the midst of the Spanish-American War in 1898. It was not only Californians who flirted with the tropes of brave conquistadors and pious bringers of civilization to indigenous peoples. Spanish culture flourished in popular songs, stage shows, architecture, and numerous public exhibitions across a young nation that was flexing its own muscles as a new world power.

Chet Van Duzer of the Library of Congress will be the next lecturer in the series, with a talk titled "Watching a Renaissance Cartographer at Work: The Construction of Waldseemüller's Carta Marina of 1516" on March 24. In 1516, mapping was accomplished by marshaling data from texts and travelers' reports and converting it into two-dimensional representations of lands and seas; today satellites make this task much easier.

Future lecturers in the series will include Fletcher Jones Professor of Geology John Grotzinger, who will speak about the Mars Exploration Rover Mission, and Professor Deborah Coen of Barnard College, who will talk about her recent book The Earthquake Observers: Disaster Science from Lisbon to Richter.  All lectures are open to the public.

Writer: 
Cynthia Eller
Writer: 
Exclude from News Hub: 
No
News Type: 
In Our Community

Aiding and Abetting a Culture of Corruption

Watson Lecture Preview

Jean Ensminger is studying a corruption network linked to aid money, using interviews and quantitative analytical methods to follow the money disbursed by a large World Bank project in Africa. Ensminger, Caltech's Edie and Lew Wasserman Professor of Social Sciences, will explain the magnitude of the corruption problem and share some forensic economic techniques that could allow development donors to combat corruption by tracking potential fraud in real time. The talk will be at 8:00 p.m. on Wednesday, January 15, 2014, in Caltech's Beckman Auditorium. Admission is free.

 

Q: What do you do?

A: I'm an economic anthropologist. I have spent my career trying to understand how economic systems work from the grass roots up in developing countries. Right now I'm studying the role of corruption in that process. Most corruption studies look at grand scandals on the national level, but I also want to understand the impacts of corruption when it percolates down to ordinary people at the village level. Once corruption reaches this level, it has usually become the social norm and works like a vortex sucking more and more systems and individuals into its grip.

There is no methods book that teaches one how to conduct a study of this sort. But that has been part of the excitement, as I had to invent the process as I went along. Had I not built up a lot of trusted networks from 30 years of research in Kenya this would not have been possible. Trust is everything in a project of this sort. Another thing this process has taught me is that time is your friend. The data get better and better.

 

Q: How did you get into this line of work?

A: I have worked with the Orma people in a remote part of Kenya for about 30 years. In 2004 a small micro project came to the village in which I stay, and it was part of a large World Bank project. I did a research paper on the corruption in that micro project that caught the attention of the World Bank. Given the reaction to my work by those running the project, I suspected that "where there is smoke there is fire." As I broadened my inquiries, I rapidly deduced that the entire project was ripe for further investigation and important enough for me to redirect my primary research orientation.

Research like this is unusual even for an anthropologist. I was in the right place at the right time to do something that, to the best of my knowledge, has never been done quite like this before. I had a sabbatical coming up; I had research money available to me; and, because I had been working in Kenya so long, I had a network that gave me access to the top of civil society—people who are household names in Kenya helped me get started. I have never regretted this decision.

 

Q: Why are you doing this?

A: Although economists have written a great deal about the negative impact of corruption upon economic growth, the level of corruption in aid is not well measured, and I think it's important to know more. People may love foreign aid or hate foreign aid, but one thing most agree upon is that they would like to see it spent effectively, and certainly not to do harm. Corruption has a corrosive effect upon many individuals who are virtually compelled to participate because the system is pervasive. That is part of the human story.

In spite of all this, there is hope. What keeps me particularly fired up is that this project resonates with so many of the wonderful citizens of Kenya, including the scores of people who bravely provided evidence for this project. People have literally pounded the table to encourage me to keep going, to dig deeper, and to get the word out. It is a good feeling to have the passion I hold for my research shared with Kenyans.

 

 

Named for the late Caltech professor Earnest C. Watson, who founded the series in 1922, the Watson Lectures present Caltech and JPL researchers describing their work to the public. Many past Watson Lectures are available online at Caltech's iTunes U site.

Writer: 
Douglas
Frontpage Title: 
Aiding and Abetting a Culture of Corruption
Listing Title: 
Watson Lecture Preview: Aiding and Abetting a Culture of Corruption
Writer: 
Exclude from News Hub: 
No
News Type: 
In Our Community

Assessing Others: Evaluating the Expertise of Humans and Computer Algorithms

How do we come to recognize expertise in another person and integrate new information with our prior assessments of that person's ability? The brain mechanisms underlying these sorts of evaluations—which are relevant to how we make decisions ranging from whom to hire, whom to marry, and whom to elect to Congress—are the subject of a new study by a team of neuroscientists at the California Institute of Technology (Caltech).

In the study, published in the journal Neuron, Antonio Rangel, Bing Professor of Neuroscience, Behavioral Biology, and Economics, and his associates used functional magnetic resonance imaging (fMRI) to monitor the brain activity of volunteers as they moved through a particular task. Specifically, the subjects were asked to observe the shifting value of a hypothetical financial asset and make predictions about whether it would go up or down. Simultaneously, the subjects interacted with an "expert" who was also making predictions.

Half the time, subjects were shown a photo of a person on their computer screen and told that they were observing that person's predictions. The other half of the time, the subjects were told they were observing predictions from a computer algorithm, and instead of a face, an abstract logo appeared on their screen. However, in every case, the subjects were interacting with a computer algorithm—one programmed to make correct predictions 30, 40, 60, or 70 percent of the time.

Subjects' trust in the expertise of agents, whether "human" or not, was measured by the frequency with which the subjects made bets for the agents' predictions, as well as by the changes in those bets over time as the subjects observed more of the agents' predictions and their consequent accuracy.

This trust, the researchers found, turned out to be strongly linked to the accuracy of the subjects' own predictions of the ups and downs of the asset's value.

"We often speculate on what we would do in a similar situation when we are observing others—what would I do if I were in their shoes?" explains Erie D. Boorman, formerly a postdoctoral fellow at Caltech and now a Sir Henry Wellcome Research Fellow at the Centre for FMRI of the Brain at the University of Oxford, and lead author on the study. "A growing literature suggests that we do this automatically, perhaps even unconsciously."

Indeed, the researchers found that subjects increasingly sided with both "human" agents and computer algorithms when the agents' predictions matched their own. Yet this effect was stronger for "human" agents than for algorithms.

This asymmetry—between the value placed by the subjects on (presumably) human agents and on computer algorithms—was present both when the agents were right and when they were wrong, but it depended on whether or not the agents' predictions matched the subjects'. When the agents were correct, subjects were more inclined to trust the human than algorithm in the future when their predictions matched the subjects' predictions. When they were wrong, human experts were easily and often "forgiven" for their blunders when the subject made the same error. But this "benefit of the doubt" vote, as Boorman calls it, did not extend to computer algorithms. In fact, when computer algorithms made inaccurate predictions, the subjects appeared to dismiss the value of the algorithm's future predictions, regardless of whether or not the subject agreed with its predictions.

Since the sequence of predictions offered by "human" and algorithm agents was perfectly matched across different test subjects, this finding shows that the mere suggestion that we are observing a human or a computer leads to key differences in how and what we learn about them.

A major motivation for this study was to tease out the difference between two types of learning: what Rangel calls "reward learning" and "attribute learning." "Computationally," says Boorman, "these kinds of learning can be described in a very similar way: We have a prediction, and when we observe an outcome, we can update that prediction."

Reward learning, in which test subjects are given money or other valued goods in response to their own successful predictions, has been studied extensively. Social learning—specifically about the attributes of others (or so-called attribute learning)—is a newer topic of interest for neuroscientists. In reward learning, the subject learns how much reward they can obtain, whereas in attribute learning, the subject learns about some characteristic of other people.

This self/other distinction shows up in the subjects' brain activity, as measured by fMRI during the task. Reward learning, says Boorman, "has been closely correlated with the firing rate of neurons that release dopamine"—a neurotransmitter involved in reward-motivated behavior—and brain regions to which they project, such as the striatum and ventromedial prefrontal cortex. Boorman and colleagues replicated previous studies in showing that this reward system made and updated predictions about subjects' own financial reward. Yet during attribute learning, another network in the brain—consisting of the medial prefrontal cortex, anterior cingulate gyrus, and temporal parietal junction, which are thought to be a critical part of the mentalizing network that allows us to understand the state of mind of others—also made and updated predictions, but about the expertise of people and algorithms rather than their own profit.

The differences in fMRIs between assessments of human and nonhuman agents were subtler. "The same brain regions were involved in assessing both human and nonhuman agents," says Boorman, "but they were used differently."

"Specifically, two brain regions in the prefrontal cortex—the lateral orbitofrontal cortex and medial prefrontal cortex—were used to update subjects' beliefs about the expertise of both humans and algorithms," Boorman explains. "These regions show what we call a 'belief update signal.'" This update signal was stronger when subjects agreed with the "human" agents than with the algorithm agents and they were correct. It was also stronger when they disagreed with the computer algorithms than when they disagreed with the "human" agents and they were incorrect. This finding shows that these brain regions are active when assigning credit or blame to others.

"The kind of learning strategies people use to judge others based on their performance has important implications when it comes to electing leaders, assessing students, choosing role models, judging defendents, and so on," Boorman notes. Knowing how this process happens in the brain, says Rangel, "may help us understand to what extent individual differences in our ability to assess the competency of others can be traced back to the functioning of specific brain regions."

The study, "The Behavioral and Neural Mechanisms Underlying the Tracking of Expertise," was also coauthored by John P. O'Doherty, professor of psychology and director of the Caltech Brain Imaging Center, and Ralph Adolphs, Bren Professor of Psychology and Neuroscience and professor of biology. The research was supported by the National Science Foundation, the National Institutes of Health, the Betty and Gordon Moore Foundation, the Lipper Foundation, and the Wellcome Trust.

Writer: 
Cynthia Eller
Writer: 
Exclude from News Hub: 
No
News Type: 
Research News

Focusing on Faces

Researchers find neurons in amygdala of autistic individuals have reduced sensitivity to eye region of others' faces

Difficulties in social interaction are considered to be one of the behavioral hallmarks of autism spectrum disorders (ASDs). Previous studies have shown these difficulties to be related to differences in how the brains of autistic individuals process sensory information about faces. Now, a group of researchers led by California Institute of Technology (Caltech) neuroscientist Ralph Adolphs has made the first recordings of the firings of single neurons in the brains of autistic individuals, and has found specific neurons in a region called the amygdala that show reduced processing of the eye region of faces. Furthermore, the study found that these same neurons responded more to mouths than did the neurons seen in the control-group individuals.

"We found that single brain cells in the amygdala of people with autism respond differently to faces in a way that explains many prior behavioral observations," says Adolphs, Bren Professor of Psychology and Neuroscience and professor of biology at Caltech and coauthor of a study in the November 20 issue of Neuron that outlines the team's findings. "We believe this shows that abnormal functioning in the amygdala is a reason that people with autism process faces abnormally."

The amygdala has long been known to be important for the processing of emotional reactions. To make recordings from this part of the brain, Adolphs and lead author Ueli Rutishauser, assistant professor in the departments of neurosurgery and neurology at Cedars-Sinai Medical Center and visiting associate in biology at Caltech, teamed up with Adam Mamelak, professor of neurosurgery and director of functional neurosurgery at Cedars-Sinai, and neurosurgeon Ian Ross at Huntington Memorial Hospital in Pasadena, California, to recruit patients with epilepsy who had electrodes implanted in their medial temporal lobes—the area of the brain where the amygdala is located—to help identify the origin of their seizures. Epileptic seizures are caused by a burst of abnormal electric activity in the brain, which the electrodes are designed to detect. It turns out that epilepsy and ASD sometimes go together, and so the researchers were able to identify two of the epilepsy patients who also had a diagnosis of ASD.

By using the implanted electrodes to record the firings of individual neurons, the researchers were able to observe activity as participants looked at images of different facial regions, and then correlate the neuronal responses with the pictures. In the control group of epilepsy patients without autism, the neurons responded most strongly to the eye region of the face, whereas in the two ASD patients, the neurons responded most strongly to the mouth region. Moreover, the effect was present in only a specific subset of the neurons. In contrast, a different set of neurons showed the same response in both groups when whole faces were shown.

"It was surprising to find such clear abnormalities at the level of single cells," explains Rutishauser. "We, like many others, had thought that the neurological abnormalities that contribute to autism were spread throughout the brain, and that it would be difficult to find highly specific correlates. Not only did we find highly specific abnormalities in single-cell responses, but only a certain subset of cells responded that way, while another set showed typical responses to faces. This specificity of these cell populations was surprising and is, in a way, very good news, because it suggests the existence of specific mechanisms for autism that we can potentially trace back to their genetic and environmental causes, and that one could imagine manipulating for targeted treatment."

"We can now ask how these cells change their responses with treatments, how they correspond to similar cell populations in animal models of autism, and what genes this particular population of cells expresses," adds Adolphs.

To validate their results, the researchers hope to identify and test additional subjects, which is a challenge because it is very hard to find people with autism who also have epilepsy and who have been implanted with electrodes in the amygdala for single-cell recordings, says Adolphs.

"At the same time, we should think about how to change the responses of these neurons, and see if those modifications correlate with behavioral changes," he says.

Funding for the research outlined in the Neuron paper, titled "Single-neuron correlates of abnormal face processing in autism," was provided by the Simons Foundation, the Gordon and Betty Moore Foundation, the Cedars-Sinai Medical Center, Autism Speaks, and the National Institute of Mental Health. Additional coauthors were Caltech postdoctoral scholar Oana Tudusciuc and graduate student Shuo Wang.

Writer: 
Katie Neith
Writer: 
Exclude from News Hub: 
No
News Type: 
Research News

Pages

Subscribe to RSS - HSS