Becky Francis (@BeckyFrancis7) February 28, 2016
Ben Ward (@MrBenWard) March 09, 2016
Hélène Galdin-O'Shea (@hgaldinoshea) March 25, 2016
This post compares how we do differentiation with how it’s done elsewhere.
It explores the tension within education policy between autonomy and pedagogical prescription through the ‘forensic analysis’ of recent ministerial speech content.
It flags up the likelihood of further policy tensions following the recent (and welcome) shift from ‘no child left behind’ under the Coalition Government to ‘excellence for all’ under the new Administration.
It is a detailed critique built on evidence and should not be interpreted as critical of any government or office holder. Ministers of all political complexions in all recent governments have felt the need to influence pedagogy. This time round, however, a parallel emphasis on autonomy throws the issue into sharper relief, as does the present role of the EEF and the intended role of a College of Teaching.
This is the last in a set of four educational posts I delayed writing while otherwise engaged on the review commissioned by Oxford.
It was prompted by a recent foray by Schools Minister Nick Gibb into the secret garden of pedagogy.
Gibb is a competent and experienced minister but he often finds himself articulating a paradox at the heart of the government’s education policy, striving manfully to reconcile two fundamentally irreconcilable beliefs:
- Professional autonomy – especially pedagogical autonomy – will improve educational standards and
- The Government must encourage professionals to adopt the pedagogy it deems most effective, based on its interpretation of the evidence base. Indeed, our success in future rounds of PISA depends upon it.
It might be possible to resolve this paradox if there was a single answer to the question ‘what constitutes effective pedagogy?’, if this answer was entirely uncontested and if it could be presented accurately and objectively in rich detail, with all provisos and lacunae dutifully reported.
Unfortunately though, none of these conditions applies.
Much as we would like to believe otherwise, there are substantial tracts of uncertain and disputed territory. Worse, the interpretation and presentation of pedagogical research too often reaches us through an ideological filter.
If this is true of academic writing and ‘toolkits’, it is doubly true of political speeches whose lingua franca is the media soundbite. It is not credible for politicians to project the persona of a cautious academic. They must come across as committed, single-minded and prepared to defend a given position against all-comers.
I rarely open the gate to the secret garden and always with some reticence, largely because – like Gibb – I am not, nor have I ever been, a serving classroom teacher. Our understanding is essentially theoretical. (Not that I believe this should prevent either of us from making our position clear – and having it respected.)
But I don’t believe serious bloggers have the right to deal in soundbites. They should strive to present the substantive argument, with all its nooks and crannies laid bare, and lay themselves open to challenge by those with a different perspective (as well as by those who complain their posts are too long).
They should acknowledge openly that, despite their best efforts to be objective and evidence-based, some hint of ideological bias is bound to manifest itself, if only because they feel it necessary to compensate for the bias they detect elsewhere.
Two earlier posts have ventured into the secret garden on these terms and are directly relevant to this one. Both are set in the context of maths, the territory in which Gibb himself is most prone to intervene:
Other earlier material is also relevant to this discussion, especially ‘The Politics of Setting’ (November 2014). I have occasionally referenced this and other posts in the text below.
This time round I am focused particularly on what the OECD PISA and TALIS studies can and cannot tell us about effective differentiation. I shall deal only in specifics, setting aside some of the wider questions about the value of international comparisons studies, especially the risks associated with ‘edu-tourism’.
The post is organised into three main sections:
- The first part illustrates what I shall call the Gibbian (Gibbous?) paradox through the textual analysis of two recent speeches – and sets this in the context of several other ministerial speeches given since 2010.
- The second part scrutinises two statements in a February 2016 speech about the incidence of differentiated teaching elsewhere in the world.
- The third part lays out the underpinning evidence from the OECD’s TALIS and PISA studies.
A final section sums up the key points and offers a conclusion of sorts.
The Gibbian Paradox
The benefits of autonomy
In November 2015 Minister of State Gibb spoke about ‘How autonomy raises standards’.
Much of the speech makes the case for mass academisation but part-way through it changes tack, quoting Blair’s dictum that ‘structures beget standards’.
‘The fundamental premise for school autonomy has always been that the current mode of education, the orthodoxy that governs how schools are run and how lessons are taught, has not been good enough. For decades, too many English schools had been under-performing or coasting. The only way to challenge such schools is innovation through autonomy.’
And he continues:
‘At root, the one change that will drive up educational standards in this country is improved teaching methods. Changes to school structures and inspiring school leadership may be the midwives of such a change, but it is the fundamental interaction of what is happening in the classroom, between teacher and pupil, day in and day out, which will transform their potential to succeed.’
He celebrates the American Charter Schools movement, acknowledging a consensual view that they have had mixed success (and stating his certainty that the same will not be true of academies and free schools here).
However, he argues that the very diversity of charter schools’ education philosophies has meant that American researchers ‘are better able than ever before to see what is working in the classroom’.
He mentions some of these research findings, highlighting those most supportive of a ‘no excuses’ approach and naming King Solomon Academy as an English exemplar:
‘As such effective approaches to schooling become better known, we can expect improvements to cascade through the English education system….
…At root, it is not autonomy that really matters, it is what autonomy allows you to do differently that counts. The real genius of school autonomy is that when 1,000 flowers bloom, we are able, given time, to see which flowers bloom brightest. Current research points in the same direction – it is the traditional, academic, disciplined, ‘no excuses’ approach to education that will truly allow pupils to fulfil their potential.’
The logical sequence of that final paragraph demands deconstruction. It runs something like this:
- Autonomy is not really valuable in itself; what is valuable…
- …is the diversity of provision autonomy permits because…
- …when schools can choose their preferred approach…
- …we are more able, given time, to see what works best…
- …and (by implication) schools will then shift from their preferred approaches to adopt that best practice, so eliminating the diversity of provision, but…
- …contemporary research is already united behind a ‘no excuses’ approach…
- …so (by implication) that works best and that is the direction that autonomous schools should take.
There is no hint of contestability – of disagreement within contemporary research.
There is no recognition that best practice may be context-dependent; that ‘no excuses’ might work well in some settings but emphatically not in others.
There is no appreciation that the true value of autonomy lies in the flexibility it permits schools to adopt and adapt approaches to fit their very different settings, needs and circumstances.
There is no acknowledgement of a paradigmatic reality, in which a new orthodoxy (no excuses) replaces its predecessor, only to be challenged and replaced by a successor in its turn.
There is no admission of ideological bias.
Getting the basics right
Three months later, in February 2106, Gibb spoke about ‘Getting the basics right in mathematics’.
No reference to autonomy on this occasion.
Early on in the speech he uses the outcomes of the PISA 2012 maths assessment, to compare England unfavourably with the Asian Tigers – Hong Kong, Shanghai, Singapore and South Korea.
With admirable (and – for a minister – atypical) caution he adds:
‘The PISA survey which produced those results was carried out in 2012. Since then, the situation may, perhaps, have been changing for the better.’
For PISA 2015 assessments were administered late last year and results will be published in December 2016.
Gibb says his speech will:
‘…celebrate a renaissance in mathematics teaching that is taking place in our schools. Currently happening on a small scale, it has the potential to revolutionise the teaching of the subject in this country.’
He mentions the Maths Hubs, particularly their exchanges with Shanghai, King Solomon’s Academy (again) and the KCL Maths School (both of those links are to my posts examining their respective records):
‘Such exemplar schools show what can be achieved by pupils in this country. The challenge now is making sure that the approach to mathematics that characterises the best of our schools, can spread to the rest of our schools.
I do not believe that outcomes in mathematics are low for many pupils in this country because of bad teachers, or bad schools, or bad parents. Where pupil outcomes are low, I believe it is because of bad ideas.’
We are back to teaching methods. He lists some of these bad ideas:
- ‘sustained practice is too boring to engage pupils’
- ‘teacher-led instruction and worked examples…are passive’
- ‘memorising your multiplication tables is antiquated’
Then he outlines some research that supports these activities, linking them with practice in the ‘Far East’ (specifically Shanghai and Singapore).
We are advised of the benefits of whole class instruction (my emphasis):
‘But perhaps most importantly of all, Shanghai mathematics teaching is based upon the principle that, if taught well, all pupils can master the content of a lesson. Differentiated teaching is not common in Shanghai, as it reinforces the performance gap between pupils. Across the OECD as a whole, the use of differentiating by ability [sic] whilst teaching has a negative relationship with pupil outcomes – an insight provided by the maths teacher and education blogger Greg Ashman.
There appears to be no conception in Shanghai that some pupils can ‘do’ mathematics, whilst others cannot. Instead, the focus is on all pupils mastering a concept before moving to the next part of the curriculum sequence, allowing no pupil to be left behind’
‘The way that we are going to improve maths in this country is simple: improved curriculum, quality resources, and better teaching methods.’
So we are led to understand that ‘undifferentiated teaching’ is better pedagogy than ‘differentiating by ability’ because it does not reinforce ‘the performance gap’ and ‘allows no pupil to be left behind’.
A brief aside
I note in passing that a ‘no child left behind’ soundbite is rather out of kilter with the new political messaging from Secretary of State Nicky Morgan, who is proclaiming ‘excellence for all’ at every opportunity.
Are we witnessing paradigm shift? Is Gibb in danger of peddling yesterday’s mantra? Should he be performing a rapid tactical adjustment to ensure ‘no minister left behind’?
In this recent statement to the Education Select Committee, his Department outlines its three strategic goals, one of which is:
‘Educational Excellence Everywhere – every child having access to high quality provision, achieving to the best of his or her ability regardless of location, prior attainment and background’
I support this new position wholeheartedly, but it raises some awkward questions about elements of Gibbian orthodoxy and the education policy it has engendered.
Now back to the matter in hand.
The evolution of this position
Back in June 2010, in the early days of the last Coalition Government, Secretary of State Michael Gove set out his stall at the NCSL National Conference.
The case for greater autonomy was straightforward and easily made:
‘…over the last three years…Ministers decreased school autonomy, tried to drive improvement through bureaucratic compliance.’
‘It should be Government’s job to help, serve and support you – not direct, patronise and fetter you.’
‘At the heart of this Government’s vision for education is a determination to give school leaders more power and control. Not just to drive improvement in their own schools – but to drive improvement across our whole education system.’
By 2014 the first seeds of the Gibbian paradox are beginning to appear, even in the Govian voice:
‘In government we have applied a simple set of tests to help frame education policy. For us, what’s right is what works.’
‘The more autonomy enjoyed at the level of the school principal, the better.’
‘It’s been argued that our drive towards autonomy means more schools can go wrong more quickly, without adequate action being taken. That is the opposite of the truth.’
Now autonomy has to be set alongside ‘what works’, but is this the government view of what works, or is Gove encouraging schools to use their autonomy to develop alternative visions of what works for them?
Gibb wrestles with his paradox
Several of Gibb’s more recent speeches show him engaged with his paradox in a variety of settings:
The fruits of autonomy (November 2014) has him articulating the purist case for autonomy untainted by ‘what works’:
‘Autonomy is not about government directives, committees of experts, quango worthies or national strategies costing hundreds of millions of pounds. It is about associations of like-minded people, bound by a common purpose…It is with these little platoons of idealistic people that the future of our school system lies.’
‘We are seeing the beginnings of an academic renaissance in our education system. Autonomy is at the heart of that renaissance. The great liberal politician Lord Beveridge…specifically warned against monopolistic state provision, in which new ideas and new institutions are quashed instead of nurtured.
Beveridge contrasted a totalitarian state, or a state monopoly, with a free society. He wrote that in a free society, ‘discontented individuals with new ideas can make a new institution to meet their needs. The field is open to experiment and success or failure; secession is the midwife of invention.’
In March 2015 when speaking on the Government’s maths reforms he forces a direct collision between the twin imperatives:
‘Our approach to deliver greater autonomy for schools through academisation was based on clear evidence from around the world that in the most successful school systems schools are autonomous and accountable.
The Harris Federation is also part of another of the evidence-based approaches we have put at the heart of our education reforms – mathematics for mastery.’
‘Mastery is the model of the high-performing Asian systems such as Shanghai, Singapore and South Korea. ‘
‘We have introduced a new national curriculum, which is more detailed and more demanding, to reflect the mastery approach.’
‘We have made important changes to testing. The new key stage 2 tests will assess pupils’ mastery of mathematics and the first of these new tests will be taken in summer 2016.’
‘The next step to help spread and embed mastery is to develop a cadre of 140 primary mastery experts, who will support 3,500 teachers across primary schools to introduce teaching for mastery of mathematics effectively.’
Six months later, now part of the Tory Government, he is focused on the importance of the teaching profession. On this occasion he struggles mightily to resolve the paradox, beginning with the purist line on autonomy:
‘But, through granting unprecedented freedom to individual schools, we are creating an educational eco-system in which new ideas can flourish…
…I look upon the next 5 years with great excitement, anticipating the new practices that will emerge due to greater school autonomy, which will in turn influence government policy, leading to a virtuous circle of innovation and improvement.’
Then he moves on to ‘what works’:
‘We created the EEF due to a belief that high-quality, robust research could empower classroom teachers, and I firmly believe it can.
But he admits that evidence has its limitations:
‘Research can inform us about effective ‘means’, but it can never decide for us what our ‘ends’ should be… there is not and nor should there be a settled consensus on the purpose of school. This is a passionate and sometimes fierce debate, which research may inform, but will never answer.
For this reason, I am mistrustful of those who disdain lively debate, and defer all opinion making faculties to that omniscient being ‘the evidence’. ‘The evidence’ provided by the EEF suggests that school uniform has no impact on pupils’ performance. Should we abandon school uniform? I would argue no, because pupil performance is not the sole aim of a school.’
Evidently this mistrust does not extend to pedagogical evidence, for here he is later in the same speech, caught in the act of confusing correlation with causation:
‘The evidence which shows the effectiveness of Chinese teaching methods is unequivocal: according to the PISA tests, 15-year-old pupils in Shanghai are 3 years ahead of their English counterparts in mathematics. Whilst our pupils are in their first year of GCSE, Chinese pupils are doing A level work. In mathematics, the children of the poorest 30% of Shanghai’s population are outstripping the children of our wealthiest 10% in England.’
Finally, in January 2016 he moves on to the importance of school leadership. On this occasion he sets up a new dualism, pitching ‘what works’ against ‘the current orthodoxies’:
‘… I have been deeply impressed by those heads who have grasped the opportunities offered by today’s era of school autonomy to make a clean break with the current orthodoxies of how schools should be run, and plough their own furrow.’
‘I am confident that the education sector is moving towards becoming a mature profession, where evidence is finally allowed to trump orthodoxy and dogma. To speed this process along, it is the responsibility of all organisations involved in helping headteachers with school improvement to be absolutely rigorous in scrutinising the methods and approaches they promote.
Whether promoting the merits of project work or direct instruction, synthetic phonics or whole word, a knowledge-based curriculum or a thematic curriculum, educators must ask themselves: ‘Do I wish this to be true, or do I know this to be true?’
The very challenge I am mirroring back in this post.
Probing Gibb’s statements about differentiation
The Shanghai statement
No source is provided for the first statement that ‘differentiated teaching is not common in Shanghai’. No evidence is adduced to support the claim.
Of course ‘edu-tourists’ see what they want to see when they visit Shanghai – a point forcefully made in this IoE post by Chris Husbands.
The report of a government-sponsored 2013 study tour describes a system purporting to be equitable but which is actually highly selective and intensely competitive.
There is said to be no setting but ‘lessons appeared to be directed towards the high ability’.
In high schools ‘individual work predominated in classes for high ability students’
There is also ample differentiation through ‘catch-up provision’ for about 20% of pupils. At the top end:
‘Opportunities were provided for gifted students to pursue a research project in an area of interest in addition to their normal lessons. In one school the homework was differentiated so that the top five students received a harder piece of work than the rest of the class.’
Top-end differentiation is also secured through private tuition and participation in a hierarchy of local, regional, national and international competitions. Limited class contact time ‘generates opportunities for extension classes’.
A 2014 NCSL report notes that mid-year and end-year exam results are published, so children know their rank in class. Parents suffer ‘loss of face’ if their children fall behind.
One of the themes mentioned for regular teacher research groups is ‘differentiating the curriculum’.
During lessons, pupils undertake consolidation work while the teacher circulates to support those needing help. High-performing students ‘focus on high-grade questions’.
On the other hand another section reports:
‘Within-school variation in Shanghai is an issue as in England, but teachers work on a ‘rising tide’ principle. All pupils complete the same work. There is no differentiation of tasks in class. Children are given additional help before they can fall behind, in the belief that everyone is capable of learning and that there are no intellectual boundaries to knowledge.’
‘Teachers generally focused on the best and worst children providing the same work for all regardless of ability. Students reported that, if they became bored because work was too easy, they were not set harder tasks. Extra challenge was provided through homework or private tuition.’
A single paragraph remarks:
‘The pressures, however, are enormous and for many mar enjoyment of the subject.’
It is difficult to get a fully consistent picture but what this suggests to me is a system where differentiation through lesson content and setting is relatively limited so, as a direct consequence, extensive differentiation is necessary through:
- A highly selective school system with intense competition for places at the best performing schools
- Catch-up provision, consolidation work and homework
- At the top end, a rich blend of independent learning, research projects, extension classes and competitions
- Private tuition (for top and bottom alike)
- Effort, driven by a potent mix of shame, pressure and relentless workload.
This does not support the statement that ‘differentiated teaching is not common in Shanghai’.
‘Differentiated teaching… reinforces the performance gap’
I do not quite understand why differentiated teaching should necessarily ‘reinforce the performance gap’, or exactly what that means.
It seems to point in precisely the opposite direction to ‘excellence for all’, where the objective is to support all learners to achieve the very best of which they are capable, without imposing limits at any point in the attainment distribution.
This is fully consistent with a mission to close attainment gaps between advantaged and disadvantaged learners, wherever they may be on that distribution. It is not consistent with a mission to close attainment gaps between high and low attainers.
It may be the failure to grasp this subtle distinction that underlies the suggestion by Milburn and the Social Mobility and Child Poverty Commission that Morgan is no longer committed to ‘closing the gap’.
My earlier posts about NCETM-style maths mastery are built on the principle that ‘allowing no pupil to be left behind’ has to be balanced against ‘enabling all pupils to get ahead’.
I am not convinced that the strict diet of ‘differentiation through depth’ permitted within maths mastery is preferable to an earlier model which promoted a far richer blend of differentiation through breadth, depth and pace.
The NCETM has drawn on a rose-tinted view of Shanghai practice, while fostering an unduly negative perception of how differentiation is practised in England which highlights poor provision and ignores the good.
There are positive references in NCETM’s Assessment Materials to ‘differentiation through depth’ by means of extended problem-solving and independent investigations, but I have seen no substantive guidance on how to develop such activities. This is not straightforward to do well. It will be a tricky task to limit enrichment opportunities and exclude all accelerative material.
I am worried that this orthodoxy is unduly influencing wider assessment policy (witness the interim primary teacher assessment categories for 2016 which permit nothing beyond ‘working at greater depth within the expected standard’).
As I’ve noted elsewhere:
‘The critical point is that exceeding age-related expectations cannot be assumed to be synonymous with ‘working at greater depth within the expected standard’. There should also be in-built flexibility to raise the performance ceiling for high attainers who are ready and capable of working beyond that standard.’
Turning back to the comparison with Shanghai, the 2013 report cited above quotes the Director of the Shanghai PISA Centre:
‘We rank world best in Maths and Science, not because of the performance of our top students, but because of the small gap between high and low performers. High quality is matched by high equality’.
Here are the results for Shanghai and England in the PISA 2012 maths assessment.
From this it is evident that Shanghai has:
- a much shorter absolute ‘tail of underachievement’ (3.7% at level 1 or below compared with 21.7% in England) and
- a much larger absolute ‘head of overachievement’ (to coin a phrase) (55.4% at levels 5 and 6 compared with 12.4% in England).
So Shanghai has successfully lifted the distribution towards the higher levels. There are ten times as many level 6 performers as in England and a sixth as many at the bottom end.
But the span of attainment remains unchanged. In fact it might have increased because we know nothing about attainment beyond level 6. I’m willing to bet that, if the PISA maths assessment had a level 7, at least some of the Shanghai population already at level 6 would achieve it. (A possible reform for PISA 2018?)
And Shanghai still has a relative ‘tail of underachievement’ (24.3% at level 3 and below), which is broadly comparable with the percentage in England at level 1 and below.
Judged in this manner, Shanghai’s approach is most definitely not managing to ‘leave no pupil behind’. Nor is the Director’s statement strictly correct.
And we must beware of assuming that the absolute difference between Shanghai and England is attributable to the higher quality of the education system in Shanghai and specifically the classroom pedagogy it adopts.
It seems certain that the Confucian work ethic has a huge impact. Compare the distribution above with this breakdown of KS2 maths performance for Chinese and White British learners in 2015
N = Failed to register a level
B = Working below level of the test
X = data suppressed
It reveals a familiar pattern:
- The Chinese ‘tail’ (L3 and below) is some 4% of the population, while the White British ‘tail’ is three times the size.
- The Chinese ‘head’ (L5 and above) is 71% of the population compared with 42% of the White British population. And at level 6 the ratio of Chinese to White British pupils is more than 4:1.
These distinctions may not be as marked as in PISA but they are substantial nevertheless. These results have been obtained in the English education system, with our approach to differentiation, often in the very same schools. (Even if the Chinese pupils have typically benefited more from private tuition, doesn’t that rather prove the point about Confucian effort?)
The source of the statement about OECD-wide differentiation
Gibb attributes this statement to ‘maths teacher and education blogger Greg Ashman’,
I can find four substantive posts by Ashman that discuss the point:
- TALIS data on differentiation (July 2014)
- Your own personal PISA – what does the TALIS show us? (research ED, October 2014)
- Stand-out teaching – minus differentiation (TES, April 2015)
- The bad idea that holds teachers back (January 2016)
The centrepiece of three of these posts is a graph which plots the percentage of teachers in each TALIS participant country reporting such activity against the mean maths score for that country in PISA 2012.
I would like to be able to reproduce this graph, which reappears in three of the four posts, but it carries a prominent copyright notice.
So instead I’ve developed an alternative which plots the TALIS findings against the percentage achieving level 5 or 6 in the PISA 2012 maths assessment – one group that might be expected to benefit directly from the differentiation being practised.
My version omits Abu Dhabi, Alberta and Flanders but I believe it is otherwise comparable. It appears to show a similar – if not more pronounced – relationship between the two variables.
Chart 1: Percentage achieving L5/6 in PISA 2012 maths plotted against percentage of lower secondary teachers in TALIS 2013 reporting giving different work
Here is the equivalent graph fort the other end of the attainment distribution. This shows the expected trend, operating in the other direction (fewer low attainers; less differentiated work). This time the trend line is significantly shallower, suggesting a potentially weaker relationship.
Chart 2: Percentage achieving below L2 in PISA 2012 maths plotted against percentage of lower secondary teachers in TALIS 2013 reporting giving different work
I postpone further exploration of this relationship until the third section of this post.
Ashman’s earliest piece makes some important and sensible preliminary points about the complexity of the concept of differentiation, but argues that the data from TALIS:
‘…relates to the specific practice of giving different students different tasks to complete within the same classroom, either because they are ahead of their peers or because they are struggling.’
His second post (October 2014) refers directly to a newspaper article by ex-Schools Minister Liz Truss, who – very much in Gibb’s style – describes maths mastery as seeking (my emphasis):
‘…to learn from places that have cracked self-improvement. The top performers in international tests, such as Shanghai and Japan, are adept at building on years of experience…
….Differentiation of work within classes is much less prevalent in top-performing countries than it is in England, meaning that more students there can master the basics and that teachers can use strong core material. In England, 63 per cent of teachers reported often giving different work to pupils in the same class, depending on their ability. In the best rival systems, only 25 per cent of staff do this, making classes simpler to teach and making better use of great teachers’ time.’
Ashman does not endorse this leap of faith. Instead he injects a second note of caution in relation to his graph (which could equally apply to mine):
‘We have to be careful here, this is merely a correlation. We cannot even say that it is a particularly strong one although it’s not bad for the social sciences. I do not have the skill to be able to assign margins of error to the data and so we have to view it all with a large dose of healthy scepticism. Indeed, it is worth pointing out that England and France have wildly different responses to the survey question and yet possess similar PISA scores.’
But he goes on to propose one possible explanation as ‘opportunity cost’: the time spent planning such tasks might be better employed; the classroom time might be spent more fruitfully, including on whole class teaching.
His third piece (April 2015) takes a somewhat different tack. It says:
‘…attempts have been made to test the effect of differentiation and the results are distinctly underwhelming.’
But only two pieces of relevant research are cited, both anonymously (my emphasis):
‘A research team in the US conducted a classic “horse-race” study, testing three conditions: differentiated teaching, standard teaching with differentiated assessment and an undifferentiated approach. Despite running for a number of years, the study found little evidence to support differentiated instruction. The results were largely inconclusive, with the association between teaching approach and improvements in performance generally found to be weak.
The researchers captured a lot of qualitative information through interviews and suggested that a major reason for the results not being stronger was because differentiation was often poorly implemented. They also cited resistance from teachers and school leaders as a factor.’
‘A study of primary teachers in the US found that students placed in the lowest in-class groups spent more time “involved in non-instructional activities and [were] less likely to be asked critical comprehension questions”.’
Ashman concludes that ‘there is no overwhelming evidence to support efforts to teach multiple lessons within one class’. That is rather an extreme description of much classroom differentiation.
He suggests the preferable alternative might be ‘a more pragmatic approach’, perhaps combining whole class teaching with opportunities for ‘some students who already understand the concepts working independently as the same ideas are reinforced with others’.
But the final piece (January 2016) is still more bullish. The graph reappears, prefaced by the statement:
‘Differentiating work is difficult and time-consuming. It can also potentially lead to invidious outcomes as students who are identified as struggling get watered-down content, increasing the achievement gap. It is not something that higher performing states seem to do.’
The graph is followed by the statement:
‘The evidence just isn’t there to support the idea. It seems truthy [sic] enough that catering to individual needs will be better for students but it ignores the realities of the classroom; a teacher cannot simultaneously individually instruct 30 students…What is clear is that it is not an approach that is grounded in solid evidence.’
There is a link to a 2005 research study by NRC/GT ‘The Feasibility of High-end Learning in a Diverse Middle School’ (Brighton et al) which explored the benefits of professional development in ‘differentiated instruction’.
This seems likely to be the first of the studies described in the April 2015 article.
There is not space here to offer an extended critique of this long paper, so I will confine myself to five brief observations:
- Differentiation need not be synonymous with ‘simultaneous individual instruction’ of all class members. Indeed, that is not the model tested in this research.
- Differentiation need not be confined to the teaching of subject content in lessons, though that clearly predominates in this study.
- Note the emboldened statement above, suggesting that poor implementation and teacher resistance undermined the effectiveness of differentiated instruction.
- The study is focused on the radical transformation of practice in the US middle school system where mixed ability teaching was (is still?) particularly dominant.
- The study is conducted by gifted educators who have a radically different perspective and agenda. This extract is lifted from the study:
‘For much of the contentious history of gifted education and the middle school movement, middle school educators have opposed homogeneous grouping of students as vehemently as gifted educators have supported it.
…Proponents of the middle school movement argue that ability grouping defies the principle of equity by denying access to deeper academic content based on ability… According to the opponents of ability grouping, this practice confers little or no benefit to high ability learners and leads to increased segregation, limited educational opportunities for the majority of students, and damage to children’s social and political development… Middle school educators maintain that ability grouping works against our national ideology that all students are created equal and instead supports a racist and elitist division of educational opportunities, pointing to the fact that African American, Hispanic, and economically disadvantaged students are chronically over-represented in special education classes and lower educational tracks, while White, upper-class students dominate the population of advanced classes…
In contrast, various forms of homogeneous grouping have been supported as a necessary method of ensuring that gifted students are engaged with other gifted students in curriculum responsive to their advanced needs…Further, many gifted educators believe that advanced learners cannot be served appropriately in heterogeneous classrooms…Without homogeneous grouping and without a commitment from the middle school movement to maximizing the potential of all students within heterogeneous settings, advocates of gifted students believe the talents of our most able students are sacrificed.’
My overall assessment of Ashman’s position?
I support him where he is cautious about his findings – even up to the point where he advocates a ‘pragmatic’ approach to differentiation that is responsive to issues of challenge and workload.
I am less supportive of some of his more recent statements, which seem to me to demand more careful contextualisation than he gives them.
I want to provide that by reviewing the evidence from TALIS and PISA respectively, the former to check Ashman’s own source; the latter so this can be blended into a richer conceptualisation of system-wide differentiation.
The TALIS and PISA evidence base
There is actually very little about this issue in the main TALIS 2013 Survey Report.
It says that, in lower secondary schools:
‘Less than half of teachers (44%) on average report regularly giving different work to those students having difficulties learning and/or those who can advance faster. The use of this practice seems to vary among countries, with only 20% of teachers in Korea and the Netherlands using it frequently or in every lesson, while 67% of teachers in Norway and Abu Dhabi (United Arab Emirates) reported doing so.’
‘More teachers at a primary level also report giving different work to students who can advance faster or have learning difficulties (66%) than their peers in lower secondary schools (44%)….Across countries where data are available for both levels, fewer upper secondary school teachers report frequently giving different work to struggling or advanced students (35% vs 44%).’
So, across the sample, this approach to differentiation is significantly more common in primary schools than in lower secondary schools and somewhat less common in upper secondary schools. This has an important bearing on the findings (see below).
Unfortunately we can see how this relationship plays out in very few countries, because the data for all three settings is available for only five of them:
The TALIS Report describes this form of differentiation as both necessary and challenging (pedagogically and in terms of workload):
‘This can be a challenging – yet increasingly necessary – task for teachers. It also requires additional planning and preparation for each lesson to provide multiple tasks for students that progress at different rates.’
Here are selected results for lower secondary schools
The incidence is approximately three times as high in England as in Korea and Singapore.
There is more detail in England’s national report: ‘Teachers in England’s Secondary Schools: Evidence from TALIS 2013’ which deals only with lower secondary provision (KS3).
It also explores the frequency with which teachers reported the practice of letting ‘students work in small groups to come up with a joint solution’.
It gives an average of 32% for the nine high-performing TALIS countries on the ‘different work’ question, but notes that:
‘…this figure disguises a lot of variation. Four of the group – Japan, Korea, Singapore, and the Netherlands – have levels at around 20% while three are at about 45-50% – Australia, Alberta (Canada), and Estonia. The three high performers…close together in the bottom left corner of the graph are the three East Asian countries.’
In addition, within England:
‘This practice is reported by a somewhat lower percentage of teachers in independent schools, 54% compared to 62% of teachers in maintained schools and 67% of academy teachers. Within state-funded schools, it is more common in schools with lower average Key Stage 2 scores of their student intake – 72% of teachers in schools in the bottom quartile of average scores compared to 58% of teachers in schools in the top quartile. And it is more common in schools with lower Ofsted ratings.’
This is qualified by the important statement that:
‘The differences that we observe in the use of differentiation by ability [sic] between countries or across different schools in England may in part be driven by differences in the extent to which pupils in the same class (and school) vary in their abilities [sic]. Differentiation is likely to happen more where there is a wider range of ability [sic], or at least teachers may perceive that they differentiate more in this situation. In countries with school systems that tend to separate pupils by ability [sic] (whether or not deliberately), either between schools or between classes, teachers should have less need to differentiate within the classroom. Further analysis of the TALIS data is needed to shed more light on this.’
That would help to explain why the practice is far more prevalent in the primary sector – typically mixed ability – than in lower and especially upper secondary classes. But can PISA evidence throw any further light on the matter?
One volume of the main PISA 2012 Report ‘PISA 2012 Results: What Makes Schools Successful? Resources, Policies and Practices Volume IV’ deals with the selection and grouping of students.
It explains that whereas some jurisdictions:
‘…have adopted non-selective and comprehensive school systems that seek to provide all students with similar opportunities, leaving it to each teacher and school to cater to the full range of student abilities, interests and backgrounds’;
‘…respond to diversity by grouping students, whether between schools or between classes within schools, with the aim of serving students according to their academic potential and/or interests in specific programmes. Teaching in these schools or classes is adapted to students with different needs…’
- Vertical stratification: the ways in which students progress as they become older – principally associated with variation in the age of entry into primary school and especially grade repetition (‘redoublement’).
- Horizontal stratification: differences in instruction within a grade or educational level. These may operate system-wide or responsibility may be devolved to institutional level.
There are also between-school and within-school dimensions. The former incorporates five variables: number of educational tracks, prevalence of vocational and pre-vocational programmes, early selection, academic selectivity and school transfer rates. The latter is confined to ‘ability grouping’, which includes setting.
The OECD has developed standardised indices for vertical stratification and both types of horizontal stratification. This diagram shows how the three indices interact in different jurisdictions
The UK is in the bottom left-hand sector, surprisingly close to Finland (though with above average within school grouping). Shanghai is top-right, with higher vertical and between-school horizontal stratification than England.
Singapore and Korea are similar to the UK on vertical stratification but have higher between-school horizontal stratification.
The report gives OECD averages for a variety of approaches to ability grouping. With regard to between class grouping:
- 67% of students attend schools where students in maths classes study similar content but at different levels of difficulty, at least in some classes (A).
- 54% of students attend schools where maths classes vary in content and difficulty, at least in some classes (B).
With regard to within class ability grouping:
- 49% of students attend schools where students are grouped by ability within at least some maths classes (C).
- 79% attend schools whose teachers ‘use pedagogy suitable for diverse abilities’ at least in some classes (D).
These are clearly not mutually exclusive options.
Here – for a sample of jurisdictions – are the percentages for each of these categories and within selective schools. I’ve also added in the TALIS dimension explored above.
**Average for TALIS participants (lower secondary)
It is immediately clear that the evidence from Shanghai principals doesn’t entirely support the claim above that between class and within class grouping is exceedingly rare in that system. I am at a loss to explain this discrepancy.
If this data is reproduced as a chart one can begin to get a richer sense of the various dimensions of differentiation and how they interact in different jurisdictions. The example below includes all the elements above except D – the undifferentiated option.
Chart 3: Comparison for selected jurisdictions of the incidence of selection, stratification and giving different work in the classroom
This shows how differentiation profiles are subtly different – compare for example the four East Asian jurisdictions.
It would be good to see further analysis of this kind, ideally extended to include the substantial gifted education programmes operated in some of these systems, so statements about differentiation in the classroom can be placed in their wider context.
The NCETM hasn’t been shy of suggesting that its preferred approach to mastery might have positive implications for teacher workload.
Here is director Charlie Stripp in April 2015:
‘My NCETM colleagues and I are continually learning from the experiences of schools that are implementing a teaching for mastery approach to maths teaching. We will continue to promote the approach and to support schools and teachers who wish to adopt it. Key aspects of this support will be helping primary schools to address how to teach maths without differentiation of lesson content and how to record progress in maths without levels, both of which I believe will improve pupils’ learning. What’s more, if we get things right, this might even reduce teacher workload because you’ll only need to prepare one, whole class, lesson rather than several ‘differentiated’ variants, and recording progress should also become significantly simpler.’
The reference to ‘several…variants’ begs important questions about the number required and whether such differentiation necessarily demands detailed planning and the preparation of separate or additional content.
Moreover, it conveniently forgets the need to prepare opportunities for learners to demonstrate ‘mastery with greater depth’, which NCETM suggests should foreground mathematical problem-solving and independent investigations. I do not see why that should be any less onerous than the model I compared it with.
Ashman also raises the workload issue, mentioning the ‘opportunity cost’ in terms of both classroom teaching and preparation and planning time.
Given that TALIS includes a question about the number of hours teachers spend on individual lesson planning and preparation, I checked whether there was any correlation between that and the percentage of teachers providing differentiated work in lessons.
The answer seems to be a clear ‘no’, in that the trend line in Chart 4 below is almost horizontal. I suppose it is possible that a relationship might be evident if collaborative planning was also included, but it seems unlikely.
Chart 4: Correlation between percentage of lower secondary teachers giving different work in lessons and number of hours spent on individual lesson planning and preparation
Over in Shanghai, teachers use their extensive non-contact time (estimated in the 2013 NCTL report as 70% or so) to plan, mark and provide supplementary catch-up and extension sessions.
The analysis of teacher responses to our own Workload Challenge consultation identifies lesson/weekly planning as the third most mentioned task (38% of respondents) adding unnecessary burden to the general workload. The issue is not the need for planning but the level of detail and frequency required.
There is reference in the coding framework for the analysis to a ‘requirement for individualised learning and differentiation’ but the only occasion it is mentioned in the analysis itself is when noting the potential for reducing class sizes, so limiting the planning time needed ‘including for differentiation’.
This is as it should be. One hopes that the Group will confine itself to questions of detail and frequency, rather than raising the possibility of controlling differentiation itself.
Summary and conclusion
Summing up the key messages from this post:
- You might have observed a marked distrust on my part of the term ‘differentiation by ability’. That is because most of the differentiation that takes place in our schools is more accurately ‘differentiation by prior attainment’. The confusion of ability and attainment (not to mention the introduction of other misleading terms such as ‘aptitude’ and ‘potential’) lies at the root of much miscommunication. Differentiation by prior attainment is essentially about focusing teaching with reference to learners’ different starting points, as opposed to any assumptions about their capacity to perform well in the future.
- Differentiation is a complex and slippery concept. It can take place within the classroom, within other types of learning activity (out-of-hours activities, homework, independent study), within a school (setting, streaming) or between schools (selection). When it takes place within the classroom it can take several forms, some of which are within-class grouping, differentiation by task (different content or tasks for different students) or differentiation by outcome, resource or pace of learning. Some taxonomies are far more extensive than others. Differentiation can rest on two minor variants, or it can mean individualised learning, or all points between. So when differentiation is discussed, it is important to define the terms, the location and the span with great care and specificity.
- When Gibb says that ‘differentiated teaching is not common in Shanghai’, he might be correct in the narrow sense that there seems to be relatively little differentiation by content in ordinary lessons. But the consequence is that teachers and schools must rely on a variety of alternative methods, including much more between-school selection than operates in England and a more extensive programme of out-of-class provision than is typically available here. (While this might not extend as far as a full-fledged gifted education programme, it certainly does in most of the other Asian Tigers).
- When he says that ‘differentiated teaching…reinforces the performance gap between pupils’ he appears wedded to the idea that it is desirable to narrow the gap between high and low attainers (as opposed to between disadvantaged and advantaged learners). But that seems out of kilter with the ‘excellence for all’ mantra now adopted by Morgan and embedded in Departmental core objectives.
- Besides, the PISA evidence shows that Shanghai has successfully moved the distribution but it has not yet eliminated low performance. In that sense Shanghai’s performance gap remains as large as ever, and possibly larger if there are some students capable of performing above PISA level 6.
- Additionally, the exceptional performance of Chinese students in the English system suggests that it would be incautious to assume that Shanghai’s schools are entirely responsible for the shift in the attainment distribution there.
- When he says ‘across the OECD as a whole, the use of differentiating by ability whilst teaching has a negative relationship with pupil outcomes’ he is relying on Ashman’s plotting of a single TALIS response against PISA 2012 maths outcomes. The TALIS question relates to the practice of frequently or always giving different work in lessons to those struggling and/or capable of making faster progress. Ashman is initially cautious, making no claims for causation in the correlation he identifies (and which I have substantiated with specific reference to both high and low attainers). But recently he has become more bullish, while expanding the concept to encompass ‘differentiating work’ and even ‘simultaneous individual instruction’.
- Ashman cites important evidence that some negative effects picked up in research are attributable to ‘differentiation done poorly’ rather than differentiation per se. When comparing differentiation undertaken elsewhere with current practice in this country, it is critical that the comparison reflects our effective practice rather than our poor practice. One premiss being tested by the EEF-funded setting study is that, if the system is operated properly, it might improve the effectiveness of setting throughout the attainment spectrum.
- There is much variance within the sample, but the fundamental problem with the TALIS correlation is that it does not take account of wider dimensions of differentiation including within-class grouping, setting and between-school selection. Other things being equal, one might expect the need for in-class differentiation to be lower when the span of attainment within the class is smaller, potentially as a result of setting and/or selection. This probably explains why the TALIS variable is more prevalent in primary schools than in lower secondary schools.
- PISA evidence helps supply a far richer picture of differentiation in participating countries, placing the single TALIS variable in that wider context. Further work is necessary to encompass the full span of differentiation within different jurisdictions.
- The much higher incidence of selection outside the UK is telling. Across the OECD, 43% of PISA students are in academically selective schools. Over 80% are in such schools in Hong Kong, Japan and Singapore. In Shanghai the figure is 52.6%; in the UK it is only 28.2%. (For upper secondary schools only, Shanghai is at 74.4%, Singapore at 82.1% and the UK at 28.2%; the OECD average is 55.7%).
- According to PISA 2012, while both between-class and within-class ability grouping are more prevalent in the UK than the OECD average, they are by no means rare in most of the East Asian jurisdictions. Even in Shanghai, between-class grouping (studying different content but at different levels of difficulty) and within-class grouping by ability are reported to affect 92% and 84% of students respectively.
- My previous posts have expressed a preference for a rich model of top-end differentiation, for learning activity undertaken within and beyond the classroom to be built around the interplay between faster pace (acceleration), more breadth (enrichment) and greater depth (extension). This has conspicuous advantages over the mastery model which I have set out in those earlier posts and will not repeat here. I am not convinced by the suggestion that the mastery model is simpler to execute well or less work-intensive. Done properly, it wouldn’t be much less onerous, particularly if the older model is already familiar.
- The workload associated with differentiated classroom teaching depends on several variables including class size, the type of differentiation undertaken and the levels of differentiation applied. There is no obvious correlation in TALIS between average hours spent on individual planning and giving different work in lessons. It seems as though the Workload Challenge will confine itself wisely to considering how planning for differentiation is undertaken, rather than whether it should be done at all.
Like so much else in education, differentiation is all about striking the right balance between excellence and equity – and then sustaining equilibrium.
Gibb’s mastery-driven approach is pushing equity to the fore: education is about getting all learners to the required standard, eliminating the ‘long tail’ and reducing the gap between high and low attainers (often mistakenly assumed to be synonymous with the gap between advantaged and disadvantaged attainers).
Morgan’s alternative formula is giving excellence equal status: education is about helping all learners to achieve their best, regardless of prior attainment. Raising attainment at the top of the distribution is as important as raising it at the bottom, or in the middle. The gap between high and low attainers is of secondary importance (though the gap between advantaged and disadvantaged attainers should remain a priority, else equity suffers).
Gibb’s statements on differentiation rely on a selective interpretation of the evidence base, which is contested, uncertain and incomplete. Both presentation and publication of the evidence base are frequently clouded by (often undeclared) ideological bias.
The Gibbian paradox – an ongoing difficulty in reconciling the twin imperatives of autonomy and ‘what works’ – will, unless it is tackled, remain a problematic fault line within government education policy.
There are three broad choices:
- To continue to fudge the issue, because neither alternative below is politically viable.
- To reassert the right of schools to pursue their pedagogy of choice – and so back away from embedding a preferred approach through the assessment regime, the NCETM and the maths hubs.
- To assert the supremacy of the preferred government approach, bolstered by a (rather imperfect) evidence base – and so restrict schools’ right to pedagogical autonomy, perhaps by defining more clearly the territory in which it does and does not operate.
Maths and English constitute the territory of most interest to government; perhaps science too given the scope of PISA assessments.
Perhaps pedagogy in those subjects should be handed over to government, so recreating the central prescription inherent in the national strategies? But also, perhaps, this new national pedagogy should be fully consistent with ‘excellence for all’?
Such a proposition would not be attractive to the profession. It flies in the face of another strand of evidence from PISA, but it could gain traction if the outcomes of PISA 2015 are disappointing.
Postscript (March 2016)
The White Paper ‘Education Excellence Everywhere‘ has much to say about autonomy, using the word 28 times.
The government describes its general approach as ‘supported autonomy’:
‘We believe in supported autonomy: aligning funding, control, responsibility and accountability in one place, as close to the front line as possible, and ensuring that institutions can collaborate and access the support they need to set them up for success.’ (p. 4).
The revised national curriculum is seen as ‘giving teachers professional autonomy over how to teach’ (p. 89).
But there is a get-out clause, to be deployed sparingly (my emphasis):
‘We believe that outcomes matter more than methods, and that there is rarely one, standardised solution that will work in every classroom for government to impose.
The elected government should set out the outcomes – what needs to be achieved for the public money invested in education. But we start from the basis that the country’s best school leaders know what works, and that good, enthusiastic leaders should be able to use their creativity, innovation, professional expertise and up-to-date evidence to drive up standards.
So this government will very rarely dictate how these outcomes should be achieved – it will encourage and support teachers and leaders to develop the best possible solutions for their pupils, and will hold them to account for rigorous, fairly measured outcomes.’ (p9)
Ministers reserve the right to ‘dictate how…outcomes should be achieved’ but, for the most part, they will ‘encourage and support teachers…to develop the best possible solutions for their pupils’.
There are no criteria to explain precisely when autonomy can be overridden and a one-size-fits-all solution imposed. This might be attributable to sheer whim or ideology, rather than careful assessment of the evidence base.
But, these exceptional cases aside, the text is clear that teachers decide ‘what works’ for their pupils. They determine how to differentiate in their classrooms. The government wants to encourage and support them to develop the best possible solutions: it will hardly ever dictate to them the nature of those solutions.
So all the paraphernalia of ‘what works’ (a phrase used 17 times in the white paper) – the EEF and its toolkit, the NCETM and its maths hubs, the repeated endorsements in ministerial speeches – should not be taken as gospel. Unless, that is, the government has made it 100% clear that it is dictating a universal approach.
So when you read in the white paper:
‘According to the EEF, the use of mastery teaching methods, for example, can lead to an additional five months’ progress over the course of a school year compared to mainstream approaches.’ (p.38)
‘Maths hubs are popularising ‘mastery’ approaches to mathematics designed to ensure that no pupil’s understanding is left to chance, and each step of a lesson is deliberate, purposeful and precise.’ (p. 91)
take it with a pinch of salt. If you can achieve the same or better outcomes by doing things differently, go right ahead.
We should expect to see this line applied consistently in all ministerial speeches throughout the remainder of this government.
It is not quite the second of the three choices outlined above; the first remains in place, but it has been clarified to some extent.