fMRI Goes to the Movies

A group of cognitive neuroscientists in New York has brought a whole new approach to the psychology of film, with fascinating results and tantalizing promise. The group is Uri Hasson, Ohad Landesman, Barbara Knappmeyer, Ignacio Vallines, Nava Rubin, and David Heeger. The have placed people in an fMRI machine and asked them to watch film clips. Then they have statistically analyzed the fMRI results to find commonalities of responses among different viewers while they watched the clips. They call their method inter-subject correlation. High inter-subject correlations occur when the same parts of a number of viewers' brains are activated in the same way at the same time while they watch a film. In the newish journal of film and its psychology, Projections, the researchers review results they have obtained using this method, which they have reported in the neuroscience literature over the last few years.

In one study, Hasson and his colleagues compared responses to viewing films with responses of subjects as they listened to a story being read to them. Some areas of the brain, for instance the visual and auditory regions, were activated respectively by viewing and listening. Areas that, across a number of people, were activated both by viewing a silent film and by listening to a story were the superior temporal sulcus, the temporal-parietal junction, and the left intra-parietal sulcus. The researchers suggest that these regions, perhaps, are concerned with more abstract forms of processing, shared by film and audio-stories, for instance of understanding narrative sequences of human interaction.

In another series of experiments the researchers examined effects of scrambling shots of two silent Charlie Chaplin films. The researchers showed participants clips of the films in their ordinary sequence, and also in sequences in which the order of segments had been scrambled. The segments were of about 36 seconds (8 to 10 shots), or about 12 seconds (3 to 4 shots), or about four seconds (single shots). In the visual areas of the brain, the inter-subject correlations were relatively insensitive to scrambling of segments of any length, indicating that they were driven by instantaneous visual input. Some brain areas (the lateral occipital cortex, the para-hippocampal place area, the fusiform face area, the superior-temporal sulcus. and the precuneus) needed coherence across three to four shots to drive them with high inter-subject correlations. The authors propose that these areas are sensitive to arrangements of shots (that is to say to metonymic juxtapositions) of the kind that film editors carefully choose, for instance in montage. The anterior regions of the brain needed coherent segments of 30 seconds or more to drive them in what the authors call "more complex cognitive functions."

In a further set of studies, the authors compared effects of different genres of film, and found that some were able to control the brain systems of viewers rather closely, whereas others enabled responses to be more open to different viewers. They say: "The percentage of cortex exhibiting high ISC [inter-subject correlation] provided a measure of the overall effectiveness, or collective engagement power, of each movie to induce similar responses across viewers." With three film sequences by different directors plus a 10-minute unedited naturalistic shot of people in Washington Square Park, the researchers found overall percentages of the whole cortex that showed high inter-subject correlations as follows: for a film by Alfred Hitchcock over 65%, for a film by Sergio Leone 45%, for a film by Larry David 18%, and for the unedited Washington Square Park shot less than 5%. They offer a scale of collective engagement for different genres, with real-life shots being the least controlling, art films engendering a moderate degree of control, Hollywood films exerting a great deal of control, and (presumably) propaganda films (about which they speculate although they did not test the idea) being the most controlling.

These interesting results prompt a range of thoughts.

Uri Hasson, Ohad Landesman, Barbara Knappmeyer, Ignacio Vallines, Nava Rubin, and David Heeger (2008). Neurocinematics: The neuroscience of film. Projections: The Journal for Movies and Mind, 2, 1-26.