Wednesday, April 8, 2015
Center for Student Services 360 (Workshop Space) – Center for Student Services

Want to know what works in teaching?

Monday, March 2, 2015
Winnett Lounge – Winnett Student Center

The Secret Life of a Snowflake

How the Brain Learns from the Past and Makes Good Decisions for the Future: A Tour of Neural Reinforcement Learning

Watson Lecture Preview

It is often said that people who do not learn from history are doomed to repeat it. Not being one of those people requires a network of different brain regions to work in concert. On Wednesday, February 4 at 8 p.m. in Caltech's Beckman Auditorium, John P. O'Doherty, professor of psychology and director of the Caltech Brain Imaging Center, will discuss our current understanding of how we learn from experience. Admission is free.


Q: What do you do?

A: I study how we learn from experience. Humans and other animals have to make decisions all the time to maximize their benefits and minimize danger. These decisions range from what I should have for dinner or should I cross the road—which could have life-changing consequences if I'm wrong—to the selection of a life partner. I don't claim that "Who should I marry?" is equivalent to "Carrots or Brussels sprouts?" but we do think that many decisions share certain commonalities. So we look at very simple tasks that give us a window into how the brain solves problems to maximize future rewards.

We study brain activity by putting your head in an fMRI scanner. "MRI" stands for magnetic resonance imaging, and you've probably had one if you've had a sports injury. The "f" stands for "functional," and an fMRI scan detects changes in the oxygenation levels in the blood. If a certain part of the brain is active, its oxygen supply increases. We map those increases onto the brain's anatomy in 3-D while our volunteers perform some task that involves learning.

A task might be playing virtual slot machines. You have a choice of three machines, and we tell you one machine pays better than the others. So you choose one, press the button, and get instant feedback—you win or you lose. As you try to work out which machine is better, we monitor the patterns of activity in various parts of your brain. One of our goals is to find the part of the brain that represents the experienced value of the things we meet in the world—how good it feels to win, or how bad to lose.

We're also interested in how the brain changes its expectations. As you play the machines, you're constantly revising your estimate of which machine is better. We have computational models that we think represent how the brain internalizes feedback, and we're trying to find brain areas where the activity matches those models.

We think that understanding the neural circuits and computations that underpin our decision-making capacity may shed some light on certain psychiatric disorders, such as obsessive-compulsive disorder, depression, and addiction. On some level, all of these can be seen as decision-making gone wrong. Addiction, for example, involves a choice—voluntary or otherwise—to engage in a certain pattern of behavior.


Q: Setting aside clinical disorders, why do people make garden-variety bad decisions? What leads us to cross a busy road and almost not make it?

A: First, it's important to emphasize that humans are collectively pretty good at making decisions. That's why we've been so successful as a species. But there could be all sorts of reasons why an individual might make a poor decision. For example, you might underestimate how fast the traffic is moving.

My lab is particularly interested in how two distinct decision-making mechanisms may interact to produce bad outcomes. One mechanism is "goal-directed," in which you evaluate the consequences of your action in light of the goal you're pursuing. This requires a lot of mental energy. In contrast, "habit-controlled" decision-making is basically stimulus-response—you react to some cue from the environment. Habits can be very beneficial, because you can execute them quickly without thinking deeply. Once you learn to ride a bicycle, for example, you don't have to concentrate on keeping your balance. It becomes routine, and you can focus your mental energy on other things. Poor decisions can result when the habit system drives your behavior when you really should be solving things in a goal-directed manner. This may be how addiction becomes compulsive. The goal-directed system says, "I don't want to take this drug any more," but the habitual system overrides it.


Q: How did you get into this line of work?

A: Even as a kid I was interested in science and its unsolved mysteries. I was actually keen on astronomy as a teenager and really considered going in that direction. Then I started getting interested in how computers work, which led me to start wondering about how the most complex computer that we know of works, namely our brain. So I basically had a career choice between studying the universe or studying the brain, which are probably the world's two greatest outstanding mysteries. I decided to take my chances on the brain.

At the time, the field of cognitive neuroscience was based on the paradigm that the brain is like a digital computer, and brain processes were modeled in essentially in the same way. There were lots of studies of memory, such as recalling lists of words, but very little was known about how the brain assigns a greater value to some things than others. But it's a really fundamental question, because the ability to work out whether something is good or bad—and to maximize behaviors that lead to good things and avoid bad things—is critical for survival. Digital computers typically don't make value judgments of that sort unless they are programmed to do so. So that's what excited me, trying to unlock how it is that the brain assigns value to things in the world.


Named for the late Caltech professor Earnest C. Watson, who founded the series in 1922, the Watson Lectures present Caltech and JPL researchers describing their work to the public. Many past Watson Lectures are available online at Caltech's iTunes U site.

Douglas Smith
Home Page Title: 
Learning From Experience — How Do We Do It?
Listing Title: 
Learning From Experience — How Do We Do It?
Exclude from News Hub: 
Short Title: 
Learning from the Past
News Type: 
Research News
Friday, February 13, 2015
Center for Student Services 360 (Workshop Space) – Center for Student Services

Backpocket Barnburner: A Lightning Quick Overview of Educational Theory

Saturday, January 24, 2015
Center for Student Services 360 (Workshop Space) – Center for Student Services

The personal side of science

Wednesday, February 4, 2015
Center for Student Services 360 (Workshop Space) – Center for Student Services

Meet the Outreach Guys: James & Julius

Wednesday, February 18, 2015
Center for Student Services 360 (Workshop Space) – Center for Student Services

HALF TIME: A Mid-Quarter Meetup for TAs

Modeling Politics: An Interview With Alexander Hirsch

It is easy to assert offhandedly that the president made a particular decision in order to undermine Congress, or that Congress's latest bill is simply an attempt to bypass the Supreme Court. It is much harder to prove whether such arguments are accurate. After all, it is usually impossible to test such claims in a controlled manner. Still, some political scientists—such as Alexander Hirsch, a new associate professor of political science at Caltech—are using tools first developed by economists to do just that, creating mathematical models of these arguments. The models then serve as virtual laboratories that political scientists can use to test assumptions and identify implications.

Before coming to Caltech in August, Hirsch was an assistant professor of politics and public affairs at Princeton University. He earned his undergraduate degree in political science and economics at Yale University in 2003 and completed his doctoral work in political economics at Stanford Graduate School of Business in 2010.

Hirsch can often be found on campus with his dog, a goldendoodle named Baci, nearby. We recently sat down to talk with him about using models to study political behavior, why he is thrilled to be at Caltech, and where his research is headed.


What is the focus of your work?

I study simple, or toy, mathematical models of strategic behavior—game theoretic models as developed in economics, applied to political questions. The way I think about this kind of work is that when people study politics or they study elections, or they say that Obama chose to do this or the Republican Party is doing that, they're just making arguments about causal relationships. And those arguments are not really any different from the arguments that scientists make—you have assumptions, and you have implications. A and b implies c. If Obama wants a and thinks the Republican Party operates in b way, it suggests that c is the right strategic action for him to take. Those of us who use models are just subjecting those kinds of arguments to some mathematical rigor.


What do you gain from such modeling?

I think this is a valuable approach for a few reasons: First, people can be very sloppy about the arguments they make about why politics operates the way it does. Formalizing your logic forces you to be much more careful.

Writing down models also helps you discover the implications of your assumptions. People make assumptions about the preferences of political actors or how political institutions operate, and those assumptions have implications. The value of the model is not only to check whether those implications are valid, it is also to derive new implications that you didn't even imagine until you wrote down the model. Then you can use the data to actually see whether those implications come to pass.


What brought you to Caltech?

There are several things. One is that although many people don't know it, Caltech incubated this type of political-science modeling work early on, and the intellectual giants of the field were—or, in several cases are—here. It was an honor to have been invited to come and participate in that work.

One of the most appealing things to me about Caltech is that unlike many institutions of higher learning, Caltech lives and breathes research. There are other institutions that are wonderful at teaching and wonderful at research. But at Caltech it seems to me that to be wonderful at teaching means that you teach the students what the frontier of research is—that teaching and research are not different things.

An anecdote from teaching my first quarter here illustrates my point. It was around midterm time, and the students came in looking really exhausted. So I said, "Oh, I guess it's midterm time, so things are tough." And they said, "Well, actually, NSF grant deadlines are this week, so we were up all night."


Where does your passion for research come from?

My dad is a physicist and a theorist, so I grew up in a household in which academia was certainly respected as a profession. But the truth is that I wasn't really interested in pursuing an academic career for most of my childhood or even college. I gravitated a little bit more toward music and literature and stuff like that, none of which I pursued in a very serious way.

In college, I took some computer science courses and some math courses, but I was always interested in politics. Eventually, I started to take classes in economics, and I started seeing game theory taught by people who were interested in political applications. I found it very intellectually appealing that you could have these little mathematical toy models that look silly, but can say very surprising and in some cases deep things in this very controlled environment of the model.

So I saw economics and thought, "These tools are really cool and very rigorous and fun," and I saw political science and said, "These are interesting questions that I care about." So I started to gravitate in a direction where I could combine the two.


What are some of the specific topics you have researched using this approach?

I currently have a big research agenda on the incentives that legislators have to invest in developing expertise or developing new policy proposals. When will they want to put in a lot of effort versus very little effort to become an expert in an area or to develop new policies?

In one of my early papers, I also explored the idea that politicians, rather than disagreeing about what the aims of government should be, disagree about which policies will achieve those aims. I wrote down a model that said, let's imagine that politicians actually agree about what they want, but they disagree about how the world works—how to achieve those outcomes. So, for example, maybe Republicans and Democrats disagree about tax rates because they disagree about whether high taxes actually have a disincentive effect on work.


How do you write down a mathematical model about something like that?

Well, you're not trying to write a model that explains the world. You're just trying to write down the simplest possible structure that captures the types of political behaviors that you think are interesting.

In this case, the model that I wrote included just two possible policies: a and b. In the real world, of course, there are lots of possible policies, but we don't need to model all of those to try to understand what some of the political incentives are. In the model, there are two outcomes that can result from each of those policies—success or failure. I assign a utility of zero if the policy fails, and some number greater than zero if it succeeds. Then you need people to have the opportunity to learn, so you model a game in which people make the decision twice. They make their first choice and see what happened, then they use Bayes's rule—a simple mathematical theorem—to update their beliefs about which of the two alternatives is the right one, and then make the decision again.

The point is to predict patterns of what might come out of these incentives. It's not like modeling an atmospheric system; I'm not going to be able to make very precise estimates about what Congress is going to do tomorrow. It's to try to understand how these pieces fit together and what these assumptions might imply about the behavior of political actors.


What questions are you currently modeling?

There is an extensive literature in economics and in political science that tries to understand the nature of lobbying. Why does lobbying work the way it does? Why is lobbying effective? What are interest groups doing that legislators are responding to?

But there isn't a lot of literature on the lobbyists themselves. I am starting a project with Pablo Montagnes of the University of Chicago to try to understand the strategic incentives of lobbyists. Part of the focus is to try to understand the importance of the political ideology of a lobbyist. If you look at the available data, lobbyists look like passionate participants in the political process who have very real preferences about what government should look like, and not just like mercenaries who are willing to take sacks of cash to represent any interest group.

It's still very early, and we're working on the model. But we think that ideology plays a big role in what preserves a lobbyist's ability to represent you while also getting cash for it—it's what protects them from the incentive to take cash from anybody.


What do you do outside of work?

We have a very domestic life. My wife, Melanie, and I hang out with the junior faculty here. We like to eat at good restaurants. We've also been hiking a lot since we moved to L.A. We've hiked Echo Mountain maybe 12 times already. And we hang out with the dog.

Kimm Fesenmaier
Exclude from News Hub: 
News Type: 
In Our Community

Cake or Carrots? Timing May Decide What You'll Nosh On

When you open the refrigerator for a late-night snack, are you more likely to grab a slice of chocolate cake or a bag of carrot sticks? Your ability to exercise self-control—i.e., to settle for the carrots—may depend upon just how quickly your brain factors healthfulness into a decision, according to a recent study by Caltech neuroeconomists.

"In typical food choices, individuals need to consider attributes like health and taste in their decisions," says graduate student Nicolette Sullivan, lead author of the study, which appears in the December 15 issue of the journal Psychological Science. "What we wanted to find out was at what point the taste of the foods starts to become integrated into the choice process, and at what point health is integrated."

Since taste is a concrete, innate attribute—after all, people know what foods they like and do not like—the researchers hypothesized that it becomes factored into the food decision-making process first. A food's affect on health, on the other hand, is a more abstract attribute—one that you often need to learn about or do research on. In fact, there are such widely varying opinions about the healthfulness of nutrients like fats, calories, and carbs that you may not even be able to find a definitive answer. Therefore, the researchers assumed, the healthiness of a food likely is not factored into a person's food choice until after taste is. And for those individuals who exercised less self-control, they hypothesized, health would factor into the choice even later.

To test these ideas, Sullivan—along with her colleagues in the laboratory of Antonio Rangel, Bing Professor of Neuroscience, Behavioral Biology, and Economics, including Rangel himself—developed a new experimental technique that allowed them to evaluate, on a scale of milliseconds, when taste and health information kick in during the process of making a decision. They did this by tracking the movement of a computer mouse as a person makes a choice.

In the experiment, 28 hungry subjects—Caltech student-volunteers who had been fasting for four hours—were asked to rate 160 foods individually on a scale from –2 to 2, based on that food's healthfulness, its tastiness, and how much the subject would like to eat that food after the experiment was over. The subjects were then presented with 280 random pairings of those same foods and were asked to use a computer mouse to click on—to choose—which food they preferred from each pairing.

The researchers then used statistical tools to analyze each subject's cursor movements and, therefore, the choice process. They looked at how fast taste began to drive the mouse's movement—and how soon health did. For example, one subject's cursor trajectory might be driven by the taste of the foods very early in the trial, but soon after might be driven by health also—resulting in the selection of the healthier item, like Brussels sprouts over pizza. However, another subject's cursor trajectory might be driven by taste all the way to the selection of pizza—with health information coming online too late in the choice process to influence the selection of the food.

Sullivan and her colleagues found that, on average, taste information began to influence the trajectory of the mouse cursor, and thus the choice process, almost 200 milliseconds earlier than health information. For 32 percent of subjects, health never influenced their food choice at all; they made every single choice based on taste, and their cursor was never driven by the healthfulness of the items.

"What Nikki has shown is that a big factor here is how quickly you can represent and take into account different types of information when you are making choices," says Rangel. "People are making these choices very quickly—in a couple of seconds—so very small differences, even just a hundred milliseconds, can make an enormous difference in whether or how much health considerations ultimately influences the decision."

The researchers then wanted to find out if some people have an advantage in exercising self-control simply because they can factor health information into their choice earlier. Sullivan and her colleagues first split the subjects into two groups: those who exercised high self-control by often choosing the healthy option, and those who made their choices based almost entirely on taste—the low-self-control group.

On average, the low-self-control group began to factor in health information 323 milliseconds later than the high-self-control group. This suggests, says Sullivan, that the more quickly someone begins to consider a food's health benefits, the more likely they are to exert self-control by ultimately choosing the healthier food.

In addition, Sullivan says, it seems that those who calculate health earlier in the process also weigh it more heavily in their decision-making process.

These findings, she notes, mean it might one day be useful to encourage people to wait a bit longer before making a food choice. "Since we know that taste appears before health, we know that it has an advantage in the ultimate decision. However, once health comes online, if you wait—allowing the health information to accumulate for longer—that might give health a chance to catch up and influence the choice," she says.

Rangel adds that this work could also one day change the way health information is presented. "For example, if you go to the supermarket, does it matter how big the calorie count information label is on the yogurt?" he asks. "More visible information may affect how quickly you compute health information. We don't know, but this study opens such possibilities."

Sullivan and Rangel are next hoping to apply their cursor-tracking method to experiments beyond the refrigerator. They want to look, for instance, at how timing might affect self-control in choices involving saving money versus spending money, or deciding between an act of altruism versus an act of selfishness. They also plan to further explore the food-choice study in a larger and more diverse population of subjects through the Caltech Conte Center.

"In the past when psychologists and economists have thought about behavioral differences, they have thought of them as differences in preferences, like, 'Oh you make less healthy choices than her because you just value health less and that's the end of the story.' What our study is trying to say is that maybe part of these differences arise not from preferences, but from the amount of time it takes different people to represent information and feed it to the brain's decision-making circuit."

The Psychological Science study, "Dietary Self-Control Is Related to the Speed with Which Health and Taste Attributes Are Processed," was authored by Sullivan and Rangel along with Caltech postdoctoral scholar Cendri Hutcherson and former Caltech postdoctoral scholar and visiting associate Alison Harris, who is now an assistant professor of psychology at Claremont McKenna College. Their work was funded by the National Science Foundation.

Exclude from News Hub: 
News Type: 
Research News

Einstein Online: An Interview with Diana Kormos-Buchwald

The Einstein Papers Project, housed at Caltech since 2000, has worked in collaboration with Princeton University Press, the Hebrew University of Jerusalem, and the digital publishing platform Tizra to produce a digital edition of The Collected Papers of Albert Einstein. This new edition presents the world-renowned physicist's annotated writings and correspondence through 1923 on a free and publicly accessible website.

Upon its launch today, the digital papers will contain all 13 published volumes of The Collected Papers, in Einstein's original German and translated into English, along with an index volume. Additional volumes will be added to the site about 18 months after each new volume is published. The 14th print volume, covering the period from April 1923 through May 1925 and including Einstein's trip to South America, is scheduled for publication in February 2015.

We recently sat down with Diana Kormos-Buchwald, professor of history at Caltech and director and general editor of the Einstein Papers Project, to talk about the project's new digital endeavor.


The digital edition makes it so that anyone with access to the Internet can read Einstein's papers and correspondence from the first 44 years of his life for free. Why have you and your colleagues undertaken this massive project?

The Collected Papers of Albert Einstein is a unique project in and of itself. Einstein is the most revolutionary and famous scientist of the 20th century, and there is no similar integrated project that compiles and annotates a scientist's writings and correspondence. These scholarly volumes are addressed, in a way, to a specialist audience—the historian of science, the philosopher of science, the physicist who wants to read Einstein in his own words.

But Einstein is and always has been of great interest to the general public as well. His is the most recognized face on the Internet in all cultures. People are attracted to him because of his creativity, maybe because of his image as an unconventional scientist.

So we are now making available these volumes that have explanations and footnotes in English, introductions in English, bibliographies, plus full translations, along with the ability to see some of the original manuscripts in high-definition scans through links to the Einstein Archives Online, another project that we launched a few years ago in collaboration with the Hebrew University's Einstein Archives. We are presenting all of this in an integrated platform in which the user can search for words and phrases in both English and German.

Biographers and historians need to focus their attention and highlight a selection of documents. But we can present everything—his scientific papers, his letters to his children, his travel diaries, his impressions of foreign lands and cultures, etc.

I think it's a great achievement that we were able to put these volumes up without putting them behind a pay wall. The Press has done a wonderful job. Each volume is equivalent to something like 100 scientific papers, plus the translations. And we're making them free and open. This is a joint effort, and it furthers what I think of as an authoritative way of doing digital humanities.


What do you hope readers will take away from reading Einstein's papers?

What I would hope the reader would find is how extraordinarily hard working Einstein was. Things didn't happen with flashes of insight. In the famous year 1905, when he publishes his papers on the special theory of relativity, quantum theory, Brownian motion, and E = mc2, he also publishes 20 reviews of other people's work.

We're putting up 5,000 documents. Einstein is known for 5 or 10, maybe 15 major papers; the 5,000 documents provide a context for those well-known papers.  He was an extremely productive scientist who wrote two to three pieces per month for the rest of his career, between 1905 and the late 1930s. We have 1,000 writings, many of them unpublished. So the beauty of these volumes is also that they include drafts and writings on a variety of topics that were never published during his lifetime.

Also, Einstein was interested in a lot of fields of science. He started with great interest in physical chemistry and mastered that literature. And he continued through his entire career to be interested in applied physics, theoretical physics, experimental physics, chemistry, biochemistry. He has exchanges with doctors about physiology. So while Einstein is not a Renaissance figure the way let's say Helmholtz was—he is a specialized physicist—nevertheless, he is very curious.

We also hope to demolish some outstanding myths: Einstein was not the isolated theoretician working by himself in an attic with pen and paper. He was a modern, professional scientist, who earned his living through his work as a scientist and as a professor. He was not wealthy. He was the exemplar of the transformation, if you want, in academia at the end of the 19th century and early 20th century, when science expanded a lot in universities. And the correspondence shows he has this ever-growing circle of friends and colleagues in science and engineering, and young people whom he shepherds and advises.


How long have you been working on this digital project with Princeton University Press, Tizra, and the Hebrew University of Jerusalem?

We have been planning this for several years. We wanted to present an accurate rendering of our volumes, which are highly specialized. And we wanted to make these volumes searchable—not only the scholarly annotations but also the scans, facsimiles, and reproductions.


Einstein famously spent several winter terms here at Caltech in the early 1930s, but the published volumes of The Collected Papers only cover his life through 1923. Are there items referencing Caltech in those volumes that we can look for in the digital edition?

Yes, Einstein visited Caltech in 1931, '32, and '33, but his correspondence with scientists at Caltech goes back much further. For example, in 1913, Einstein wrote a letter to George Ellery Hale asking whether the deflection of sunlight in the sun's gravitational field could be observed in the daytime. Hale wrote back saying no, we cannot see that.

He also had contacts with Robert A. Millikan quite early on. In 1922, Millikan officially informed Einstein that the National Academy of Sciences had elected him as a foreign associate. They also discuss scientific work quite a bit, and Millikan and Einstein both serve on the Intellectual Committee for International Cooperation of the League of Nations.

Einstein was instrumental in recommending several prominent scientists for recruitment very early in the founding of the Institute. The volumes also show correspondence between Einstein, Millikan, and Richard Tolman, professor of physical chemistry and mathematical physics, who was one of the earliest relativists.

Einstein knows, right at the beginning, in the early 1920s, that Caltech is going to be an exciting place.


Was Einstein unusual in the size of his correspondence?

Yes, his correspondence is very large for a scientist. It amounts to about 30,000 items to and from Einstein. It's of the size of Napoleon's papers—orders of magnitude larger than any other modern scientist.

This amount of correspondence testifies to Einstein's centrality in the scientific life of Europe in the 1920s. He does become a nexus, at least in physics. And he is flooded by requests—everything from requests from indigent students up to requests from very famous people that he should endorse this or that appeal, contribute to this or that volume, or participate in this or that conference. He gets to be in great demand.

He also gets a lot of inquiries from the general public about general relativity.


Does he answer them?

Yes, he tries to respond to every letter he gets. He was extremely disciplined. He spent quite a lot of time answering correspondence.


Have any of your team's discoveries been particularly exciting for you?

I was excited when, a few years ago, we discovered some new letters from Croatia—from a Croatian physicist dating back to early in Einstein's career. These were letters dating to 1911 and '12, before Einstein finished general relativity. I'm always very pleased when we find material prior to 1915 or '16 because Einstein's path from special relativity to general relativity is one of the most exciting intellectual journeys. Whenever we uncover new material from that decade, it is quite significant, because we have so little material for the young Einstein compared to the older Einstein. Later, his correspondence grows exponentially.

Kimm Fesenmaier
Exclude from News Hub: 
News Type: 
Research News