The need for context in lesson observation - at all levels…

“Do not let any unwholesome talk come out of your mouths, but only what is helpful for building others up according to their needs, that it may benefit those who listen”

Ephesians 4:29

Observation is a powerful tool if used appropriately; it enables dialogue and discourse, promotes reflection and critical thinking, and offers us an opportunity to learn from each other. It is also a place of hatred, prejudice, and the threat of ‘seeing someone better than me’...

Yet again, I will turn to Freire: “Dialogue cannot exist without humility. The naming of the world, through which people constantly re-create that world, cannot be an act of arrogance. Dialogue, as the encounter of those addressed to the common task of learning and acting, is broken if the parties (or one of them) lack humility. How can I dialogue if I always project ignorance onto others and never perceive my own? How can I dialogue if I regard myself as a case apart from others—mere "its" in whom I cannot recognize other "I"s? How can I dialogue if I consider myself a member of the in-group of "pure" men, the owners of truth and knowledge, for whom all non-members are "these people" or "the great unwashed"? How can I dialogue if I start from the premise that naming the world is the task of an elite and that the presence of the people in history is a sign of deterioration, thus to be avoided? How can I dialogue if I am closed to—and even offended by—the contribution of others? How can I dialogue if I am afraid of being displaced, the mere possibility causing me torment and weakness? Self-sufficiency is incompatible with dialogue. Men and women who lack humility (or have lost it) cannot come to the people, cannot be their partners in naming the world. 

Is it indeed possible to ‘measure’ teacher effectiveness through the lens of classroom observation in any way other than subjectively? Like a student, each observer arrives at a classroom experience with a set of preconceived ideas, unconscious bias, expectation built on reputation - they also arrive with expertise that may not be directly aligned to the domains being explored in the learning encounter being watched, and therefore with different ways of determining effectiveness. Even if given a rubric or checklist, the very nature of aligning subjective, context-heavy experiences to generic statements that lack nuance or empathy itself undermines the process. As Wiliam pointed out, it takes a number of different observers to give their perspective to ensure that any overall judgment arrived at is valid. Valid, not accurate. 

As far back as the 1960s Brown, Mendenhall and Beaver tried to to devise a rubric for classroom observation consisting of 62 statements that could be responded to with a simple ‘Yes / No’ by observers; the aim was of course to get summative judgments from potentially subjective scenarios, but observing a lesson with all 8 Teacher Standards constantly whirring in and out of your desired view is tricky enough, let alone looking for the potential presence or absence of 62 variables. We can’t, and should not look to, quantify teacher effectiveness, especially not based on observation alone.

In the early 2010s the Gates Foundation’s MET project also sought to objectify and - to an extent - quantify teaching by seeking to capture the variables; they stated that their aim was to provide school settings ‘with processes they can use to help ensure high-quality data collection during teacher observations’; quantifying the qualitative. Very well designed, the Measuring Effective Teaching (MET) project sought to offset or re-frame the huge variety and disparity of judgments given by designing an ‘observation instrument’ that ‘provides a shared conception of teacher effectiveness that standardizes the lens through which observers view teaching practice; provides teachers with meaningful data to improve their practice; and has the potential, ultimately, to help improve student learning’. The observation instrument would, ideally, ‘measure observable behaviors of teaching practice that are demonstrated in the classroom, such as student engagement, use of questioning techniques, classroom management, and accuracy of presentation of content’.

They stress the importance of training of all concerned: there is a vital need to ‘guide observers’ understanding of the dimensions of the instrument and its rubrics and to give them an opportunity to hone their skill in applying the rubrics accurately’, without which a ‘shared definition of teacher effectiveness cannot be realized’, and that ‘To provide consistent and accurate observation scores, all observers must have the same understanding of what constitutes each level of teacher quality the system describes’.

The key aspect? Calibration. Alignment. Shared understanding. Shared language. But, also, the validity (or not) of the prescribed rubric for drawing together these collective inferences in the name of ‘judgment’. 

The MET project acknowledges that there are more indicators of classroom effectiveness than are specifically ‘observable’ by onlookers - data, student voice - but that teacher behaviours are a strong litmus test. 

A study by the University of Bristol, reported by the EEF, found that (with strong evidence), a Teacher Observation programme had 0 impact on student progress. Why? Well, because it was no more effective than the system already in place. Was that system effective? Not sure.

So, why quantify at all? To refer to Kafka, ‘judgment does not come suddenly; the proceedings gradually merge into the judgment’. The message - one observation does not give an accurate picture. When we observe, do we have a specific diagnostic purpose or are we simply gathering an imperfect, context-less snapshot of a teacher at a particular point in time? Is the lesson like a poem, which, in the words of Philip Larkin, ‘is like a slot machine into which the reader (or in this case, observer) inserts the penny of his imagination’? A snapshot that has to be used as a basis for deeper inference, analysis, extrapolation? Challenging without dialogue, which engenders critical thinking. Not all ‘observation’ has to be a full lesson - indeed, why should it? Just as we cannot measure progress of students over one lesson, nor should we make judgment on teachers based on singular instances alone.

Well, let’s consider WHEN and WHY it is a worthwhile process. For those of use in ITE, it is a vital tool; it must be used as an instrument (one of them) to measure and understand trainee progress towards the Teacher Standards. But, as we have argued before, the Standards themselves are generic and in need of contextual and domain-specific application; the Scientist can see the nuance of the Biology trainee’s explanation far better than the English teacher, and so on round the merry circle, ad infinitum. Observation can be informative, but also damaging; it can uplift, and it can deflate; or at least the way in which the ‘findings’, such as they are, are interpreted and fed back to the observee.

I love observing lessons; I learn a great deal. However, I know that I’m not there as a subject expert; just an experienced pedagogue - there are ways to structure a lesson and ways to manage a classroom; ways to do the 5 areas that Kennedy cites in her considerations of ‘parsing the practice’ of teaching:

1. Portray the curriculum

2. Enlist student participation

3. Expose student thinking

4. Contain student behaviour

5. Accommodate student needs

Ways of doing all these things look very different, depending on the lens used to view them through; was that a good way to model the Triple Jump? I’ll ask a PE teacher. Was that the best way of working through the chronology of Tudor Monarchs? I’ll check with an Historian. To go back to the MET Project: ‘A teacher’s classroom instruction style is perhaps one of the most important and least well-understood factors contributing to teacher effectiveness’. As observers we look for what we already understand and can identify; if we don’t have the requisite prior knowledge of the context or content we can’t truly comment on how the teacher has chosen to interpret, or how they have chosen to approach the supporting of students in developing their knowledge base; we look for what we can see, but the learning is going on inside the heads of the students. Graham Nuthall points to the three ‘worlds’ of the classroom - the public world managed by the teacher, the semi-private world of ongoing peer relationships and the private world of the student’s own mind; he tells us that students live their classroom lives in the context of these ‘different, but interacting, worlds’. I would add to this that we as teachers manage three environments of the classroom - the physical one of seating plans and layouts; the cognitive one of pedagogy and instruction, and the socio-emotional one of the individual. I can not, and will not, be able to observe for all of this and make worthwhile comment that is of developmental benefit to the teacher who was observed. 

Professor Rob Coe’s advice was that we should ‘use multiple sources of validated evidence to support diagnostic and constructive evaluation of teacher quality’; agreed. Observation is just one part of the orchestra; an instrument that, when played well, adds depth and richness to the soundscape that is trainee assessment. But, as with all instruments, the players must be trained and they must practice, lest they emit little more than squeaks and discord. 

Coe (2013) goes on to give us a range of things to consider - mainly centred around the ultimate purpose of observation, how it is measured and who does the measuring; it comes down to what the centralized and agreed definition of ‘good teaching’ is - in itself a highly contentious topic and one that has the power to produce much bile and vitriol. As discussed earlier, Coe also reminds us of the power of the preconceived; as observers we know what we like and we like what we know; we think everything can be observed and we therefore focus on the observable - yet learning is invisible. The ultimate dichotomy.  

We also know that many proxies for learning are poorly considered; completion of a task in silence doesn’t mean the teacher has successfully taught anything at all - and the purpose of all learning experiences is to impart knowledge and promote its retention; it is not just controlling a range of disparate personalities for a specified period of time. 

An untrained or inexperienced observer with no prior knowledge of domain, subject or context might see an orderly classroom full of smiling students completing task upon task with barely a peep and assume there was a pedagogue-extraordinaire in the driving seat, but it could quite possibly be a simple combination of fear, boredom and apathy; no-one wants nihilistic students. 

We are in danger if we assume that an untrained observer can provide anything resembling valid comment or feedback that improves performance next time.

So, at the heart of worthwhile observation surely lies purpose, clear focus and support; badly-structured lesson observations conducted in a dialogue-free environment with simply applied, context-free labels of judgment are anathema; they are damaging, they are worthless. A shared understanding of the purpose of the observation, conducted by the expert in their field as part of a wider smorgasbord of decision-making and support, prefaced, accompanied and concluded by reflection through dialogue? Far better. Shared language increases efficiency of dialogue and therefore removes the illusion of assumed communication; both sides of the formative dialogue need to understand what’s being said, with no hierarchy beyond that of perspectives built on experience. And as for the humility and empathy of the observer? Back to Freire: Someone who cannot acknowledge himself to be as mortal as everyone else still has a long way to go before he can reach the point of encounter. At the point of encounter there are neither utter ignoramuses nor perfect sages; there are only people who are attempting, together, to learn more than they now know’. 

Is that not what effective teacher development is all about?

References:

Bob Burton Brown, William Mendenhall & Robert Beaver (1968) The Reliability of Observations of Teachers’ Classroom Behavior, The Journal of Experimental Education, 36:3, 1-10, DOI: 10.1080/00220973.1968.11011050 

http://www.metproject.org/downloads/MET-ETS_Foundations_of_Observation.pdf

Freire: Pedagogy of the Oppressed (1969)

Nuthall: The Hidden Lives of Learners (2007)

Kennedy - Parsing the Practice of Teaching (2016)