Learning efforts require feedback that (i) recognises the desired goal (ii) bears evidence about the present position and (iii) provides a way to close the gap between the two.
Beginning with Toy Story in 1995, Pixar Animation has made a total of 24 films, garnering 16 Academy Awards, 10 Golden Globe Awards, 11 Grammy Awards, and grossing more than $20b at the box office in the process. How are they able to sustain so much success over such a long period of time when, according to co-founder Ed Catmull, “early on, all of our movies suck”?
A Culture that Learns - Pixar’s Braintrust
… the only way to get a grip on the facts, issues, and nuances we need to solve problems and collaborate effectively is by communicating fully and openly, by not withholding or misleading…
Creativity has to start somewhere, and we are true believers in the power of bracing, candid feedback and the iterative process - reworking, reworking, and reworking again, until a flawed story finds its through line or a hollow character finds its soul.
- Ed Catmull
This candid feedback is provided by Pixar’s Braintrust - a gathering of people with deep understanding of the various aspects of storytelling and animation filmmaking, who have gone through the process of making a Pixar film themselves. The Braintrust meets every few months to assess each of the films that are in the pipe.
Braintrust meetings begin with everyone watching mock-ups (storyboards and test reels) of the in-progress movies. They then break for lunch before reconvening to discuss what works and what doesn’t. This assembly of smart and passionate people are charged with surfacing problems through deep analysis and candour. They offer suggestions of possible fixes but the director and producer have the final say in the changes they choose to adopt. These feedback are provided in the form of what Pixar calls notes.
A good note says what is wrong, what is missing, what isn’t clear, what makes no sense. A good note is offered at a timely moment, not too late to fix the problem. A good note doesn’t make demands; it doesn’t even have to include a proposed fix. But if it does, that fix is offered only to illustrate a potential solution, not to prescribe an answer. Most of all, though, a good note is specific. “I’m writhing with boredom,” is not a good note. - Ed Catmull
According to Catmull in his book Creativity Inc., the Braintrust often gets heated and voices are raised but it continues to work because there are no hidden agendas. The filmmakers are in it together to make good movies rather than get credit, please bosses, or win an argument. The Pixar leadership was careful to ensure that the Braintrust is not derailed by power dynamics and Steve Jobs was therefore not allowed to sit in to the meetings! Everyone is treated as a peer to be trusted and respected. The Braintrust is enabled and upheld by a culture of learning and growing.
A hallmark of a healthy creative culture is that its people feel free to share ideas, opinions, and criticisms. Lack of candour, if unchecked, ultimately leads to dysfunctional environments. - Ed Catmull
Believe it or not, this is Formative Assessment!
Formative Assessment
Formative Assessment is the generation of data and evidence of student performance for the purpose of informing teacher and students about the progress of the teaching and learning. Too often, it is regarded as a type of assessment which makes no sense at all. I was once quizzed by an auditor from the regulatory body how many percent of my school’s assessment was formative. I chose to reply with candour and gave him a lecture. I think I hear some of you giggling… But I digressed.
Formative Assessment begins with identifying the intended learning outcomes (often a curricular consideration) and deciding how successful achievement of the learning outcome is to be evidenced. This then makes possible the determination of assessment criteria and standards of performance, as well as, the setting of the assessment task by which the evidence may be generated. With the availability of the assessment criteria and task, student performance can then be evaluated and diagnosed for gaps or inadequacies. The teacher is then able to provide growth-producing feedback and students can make the necessary changes to improve their performance. When students become more proficient, they will be able to evaluate their own performance and direct their own learning. In other words, teachers are the students' Braintrust, providing them with good notes so that they may become successful directors and producers of the movies of their learning and ensuing futures.
Effective learning is only possible when learning efforts receive feedback that (i) recognises the desired goal (ii) bears evidence about the present position and (iii) provides a way to close the gap between the two, i.e. learning requires making mistakes!
A note of caution though. The codifications of the intended learning outcomes, assessment criteria, and assessment task are susceptible to ludic and narrative fallacies. The affordance of multiple narratives, along with, reflections on the limits of the assessment task will go a long to mitigate the pitfalls of the fallacies.
Comments