Statistics and Public Policy

In common use, the term statistic is used to refer to a summary of a set of data (e.g. total rainfall in February was 15 cm), but in a more technical sense, a statistic refers to a function of a number of random variables. In the first use, no concept of probability is involved in the calculation of a statistic while, in the second, a statistic refers to a feature of a model which includes assumptions about the population and sampling process. Researchers often use summary statistics in an informal way to judge the importance of between group differences and to check for outliers. In principle, calculating a summary statistic is straightforward and in many areas of public policy summary statistics are sufficient to answer important questions (e.g. is GDP higher this year than last year). In other areas, in particular where questions of causation are important, it is usually not possible, however, to provide direct answers to policy questions by simply summarising observed data. In these situations, researchers need to calculate measures of uncertainty and conduct statistical tests and therefore need to resort to using statistical models.

For example, an article in Monday’s Guardian by Madeleine Bunting discussed whether ethnic diversity was a source of social cohesion or of social conflict. In order to examine the association between ethnic diversity and social cohesion, studies need to use statistical models rather than descriptive statistics partly because there are big differences in the characteristics of areas that are related to ethnic diversity and that might also influence social cohesion, in particular, area deprivation. For example, if we wanted to compare the level of trust that people in, say, Richmond-upon-Thames (where around 30 percent of the population are from non-White backgrounds) and people in Brent (where over 80 percent of the population are from non-White backgrounds) have for their neighbours there is little point in simply comparing the responses that people give to survey questions. We know that people living in more deprived neighbourhoods have lower levels of trust in their neighbours than people in better-off neighbourhoods, so we would expect that people in Richmond-upon-Thames had much higher levels of trust in their neighbours than people in Brent. Instead of relying on a summary statistic we can construct a statistical model of the relationship between diversity and cohesion, however, in which the assumptions of the model allow us to identify the effect of diversity on cohesion after adjusting for differences in deprivation between areas. There might be, however, only limited variation in ethnic diversity which is independent of deprivation (i.e. all areas with high proportions of the population from ethnic minority backgrounds tend to be deprived). If this is the case, we might still not be able to say anything meaningful about the relationship between ethnic diversity and social cohesion because the division between ethnically mixed and ethnically homogeneous areas is also a division between deprived and more affluent areas. Modelling might still produce estimates of the effect of diversity on cohesion but, in this situation, they would largely reflect the structure imposed on the data from the model rather than information in the data itself.

In order to illustrate the difficulty of separating ethnicity and deprivation at the area level, the figure below plots estimates of the social capital in neighbourhoods (measured using questions from the Citizenship Survey about trust, co-operation, shared values and belonging) against the proportion of the population from ethnic minority backgrounds separately for four English regions. Social capital in neighbourhoods was estimated for neighbourhoods grouped into deciles of the index of multiple deprivation giving ten observation in each region with the most affluent neighbourhoods in decile 1 and the most deprived neighbourhoods in decile 10 (the approach is described here). The figure shows that overall there is a strong negative relationship between the level of social capital in different types of area and the proportion of the population from an ethnic minority background. The points are coloured, however, according to the decile of the deprivation index and show that in each region there is also a significant negative relationship between social capital and neighbourhood deprivation. Studies have concluded that after adjusting for neighbourhood deprivation the association between ethnic diversity and social capital is actually positive (i.e. the reverse of the association actually observed in the data). The extent to which this result isolates the effect of varying ethnic diversity between neighbourhoods with similar levels of deprivation or is based on a comparison of what are fundamentally different types of area having different levels of both diversity and deprivation needs to be considered, however.

buntingIII

My point here is that although whether diversity is positively associated with social conditions in neighbourhoods is an important question, researchers can’t answer that or similar questions without recourse to using modelling techniques which require a number of assumptions. The reliability of the conclusion that diversity is positively associated with social cohesion does depend on whether those assumptions are reasonable. In most cases, modelling assumptions are made for convenience rather than realism and moving from the results of statistical models to public policy conclusions is something that we should only do with caution.

Do private schools educate children beyond their intelligence?

Michael Gove recently said that he wants state schools to do more of the things that private schools do. What struck me most about the ensuing hoo-hah was the idea that state schools might be able to provide the advantages of a private education if only the government provided more money. In most cases, parents send their children to a private school as a means of passing on privilege to their children rather than as a way of rescuing their children from the horrors of a state school. Because privilege can only exist if some people don’t have it, state schools can’t provide the same benefits to pupils as private schools. Politicians may see education as promoting social mobility, but it is also used by parents to try to make sure that their children get the advantages they need over other children to get on in life.

One argument (there are many) for how private schools help better-off families secure advantages for their children can be sketched as follows.

Firstly, children from private schools do achieve significantly better results than those from state schools at A level. The figure below (using data from the Department of Education) shows the proportion of A-levels awarded to pupils at private and state schools in 2013 that were in different grades. The difference in achievement of children at private and at state schools is clearly significant with around 50 percent of the grades awarded to pupils at private schools in the A* or A category in comparison to less than 25 percent for pupils at state schools. One consequence of the better A-level grades of children from private schools is that they are more likely to get into a higher status university (e.g. Oxford or Cambridge) than children from state schools. Figures from HESA show that in 2011 around 42 percent of students admitted to Cambridge and Oxford had been to a private school, while only around 12 percent of all children admitted to university in England had been to private schools.

Grades

Secondly, although private schools outperform state schools in terms of A-level results, students who have been to a private school are less likely to get a good degree when compared to those from state schools with the same entrance qualifications. On average, a student from a state school with ABB grades at A-level can be expected to get the same degree as a student from a private school with AAA grades at A level (HEFCE 2003). The reason that students from state schools outperform those from private schools at university is not well-established. The most reliable evidence suggests, however, that students from private schools benefit from teaching effects which increase their A-level grades in comparison to students from state schools but these then wash-out during university. The best evidence for teaching effects (and not student ability) as the cause of the better A-level grades at private schools comes from Ogg et. al. (British Educational Research Journal 2009) which used the results of an aptitude test administered to students who applied for admission to Oxford in 2002. The study shows that, on the basis of the aptitude test, students from private schools achieved the level of degree expected but, on the basis of A-level grades, they underperformed relative to students from state schools. If we assume that the aptitude test provides a true measure of student’s ability, then the better A-level grades of students from private schools can be interpreted as being due to the temporary influence of teaching conditions in private schools.

Thirdly, studies such as Naylor et al. (Bulletin of Economic Research 2002) have shown that the after leaving university graduates who have been to a private school get paid significantly more in comparison to graduates from state schools. I’m not aware of studies which show why students from private schools get better jobs than those who went to state schools. It seems likely to be due largely to social factors, however, such as the status of the school and university attended and other people’s perceptions of the abilities of students from different backgrounds. State schools can’t deliver the same benefits as private schools because the benefits of private schools run through social factors and not through education.

I’m not complaining that parents try to help their children get ahead in life. Whether it is acceptable for parents to try to give their children an unfair advantage over other children is something that it would be useful to have a public debate about, however, particularly given politicians current passion for social mobility. The debate would be improved if we could recognise that private schools do give some children unfair advantages over others, rather than allowing them to fulfill what seems to be their natural potential.

University Admissions in London

Over the New Year, I listened to a debate on the Radio 4 news about the future of cities involving Saskia Sassen and Ed Glaeser. It reminded me of how different the views of a geographer and an economist can be regarding the importance and origins of social inequalities in cities. Like most economists, Glaeser professed little concern about inequalities arguing that they were largely the consequences of the choices people make and of little public interest. Sassen, on the other hand, saw inequalities as being an important public issue and went out of her way to stress the relationship between inequalities and the distribution of power between different groups within the city.

In the UK, there has been a long debate about the origins of inequalities in education in urban areas. Although the proportion of young people who go to university has increased significantly over the last 20 years with around 40 percent of 18 year olds now entering higher education, inequalities in participation in higher education between children from different socioeconomic backgrounds has changed little. Research suggests that this is because differences in educational achievement at secondary school level between children from different backgrounds are very durable. Children from less well-off families have not had the same ability as those from better-off families to take advantage of the expansion of higher education. The numbers of children from less well-off families going to university has increased; however, the expansion of higher education has done little to close the relative gap in participation between children from different socioeconomic backgrounds.

Although inequalities in admission to higher education appear to be durable there are significant regional differences in the proportion of young people who go to university. In particular, London has a higher rate of participation in higher education than remaining regions and a higher rate of participation for young people from deprived areas which seems unusual given the extent of deprivation in many areas of the city. The finding that a higher proportion of young people from London are admitted to higher education is an observation about local areas, however, and it doesn’t necessarily follow that young people from less well-off families in London have been getting into university in higher numbers than in other regions. London is a very mixed city with rich people and poor people living in close proximity to each other. The observation that the rate of participation in higher education among children from poor areas is higher in London could therefore be a result of young people from affluent families who live in poor areas managing to get into university.

Recently it has become possible to link individual data from the school census to participation in higher education and the Department of Education has started to use this data to examine how the rate of participation in higher education varies between children from different socioeconomic backgrounds across local education authorities. The school census contains information on whether children received free school meals and receipt of free school meals has been commonly used in research as an indicator of family poverty. Overall, the figures for young people who took A levels and who left school in 2010 show that the rate of participation in higher education for children who did and did not receive free school meals was 48 percent and 46 percent, respectively. The regional differences in the rate of participation of children from different backgrounds are, however, quite striking. The figure below shows how the proportion of young people from different backgrounds going to university varies across regions. The figure shows in all regions except London children in receipt of free school meals have lower rates of participation in higher education, in comparison to children who did not receive free school meals. In London, however, the picture is dramatically different with children receiving free school meals having a slightly higher rate of participation in higher education than children who did not receive free school meals. If they manage to get to their A-levels, young people from poor families in London have a nearly 60 percent chance of going to university.

RegionFSM

One possible for why children from poor families who take A levels have much higher rates of participation in higher education in London might be that they go to different types of institution in comparison to children from better-off families. The availability of linked administrative data also allows examination of the type of university young people from different backgrounds go to and the Department of Education distinguish between institutions in the top third and remaining institutions. The figure below shows the proportion of young people from different backgrounds going to different types of institution (highly ranked vs. other) separately for Inner and Outer London. The figure shows that there is little difference in the type of university attended by children from Inner and Outer London who received free school meals with only around 13 percent going to a highly ranked institution. Among children who did not receive free school meals, however, around 20 percent of children living in Outer London went to a highly ranked institution in comparison to around 13 percent of children living in Inner London. Location does still matter for children’s education.

heifsmthird

 

Childcare and Post-Industrial Keynesianism

In a recent interview in the Guardian the shadow childcare minister, Lucy Powell, argued for the introduction of universal childcare for all pre-school children. It might seem unusual now but prior to the 1990’s childcare was of little interest to either social scientists or policy makers, largely because it was assumed that if women had children they would stop paid work, at least when their children were young. The number of women in employment has increased significantly over the last forty years, however, making childcare of major importance to both social scientists and policy makers. The figure below (using ONS data) shows that over the period since the early 1970s the proportion of women in paid employment has risen by around 15 percentage points, from around 50 percent to over 65 percent, while the proportion of men in paid employment has fallen by a roughly equivalent amount. The increase in the interest in the availability of childcare has therefore mainly been a result of changes taking place in the labour market and the need to make it easier for women to balance paid work and family life, particularly because both parents may now need to work in many families in order to reduce the risk of poverty. In the interview, Powell argued that governments should see childcare as an investment rather than as a cost. Labour is already committed to providing 25 hours of free childcare a week for working parents with three and four year old children and the argument is that by allowing parents to do paid work childcare will pay for itself through increased tax revenues, reduced benefit spending and improvements in child well-being.

Employment

There is significant uncertainty, however, about the size of the relationships between the availability of childcare and women’s labour force participation. The conventional approach to evaluating the effect of childcare on women’s employment would be either to compare the employment rates of women who are eligible for free childcare with the employment rates of women who are not, or to examine how employment rates for different groups are influenced by changes in the availability of childcare. The assumptions that are needed in such approaches to identify childcare as the main source of the difference in employment rates are always difficult to justify, however. An alternative approach has therefore been to examine how the availability of childcare influences the relationship between fertility and employment across countries. The extent of provision of childcare does vary quite a lot across countries with some countries, like Sweden providing universal childcare, while in other countries, like Germany, the availability of childcare is strictly limited. What studies comparing the relationship between fertility and employment across countries have shown is that increased availability of childcare does tend to be associated with higher levels of both fertility and female employment. For example, the figure below shows the relationship between female employment and total fertility for European countries using 2011 data from Eurostat. The figure shows that there is an overall positive association between total fertility and female employment with the countries that have high fertility and employment tending also to have high levels of provision of free childcare (e.g. Sweden) while those countries with low fertility and employment tend to have a more traditional view of women’s roles.

Fertility

Increased spending on childcare can also be understood as an example of what Esping-Andersen (Social Foundations of Postindustrial Economies 1999) has been termed post-industrial Keynesianism. The term Keynesian is usually used to denote the idea that growth is determined by demand while the term post-industrial is used to indicate that the source of demand driving growth is from services rather than from industrial output. In the modern service economy, job growth is mainly in various types of human services with recent data from the 2011 census showing that around 30 percent of women in employment are working in either ‘caring personal service’ or ‘caring, leisure and other service occupations’. The idea behind post-industrial Keynesianism is simply that growth in services, such as childcare, will not only help some families balance paid work and family life but will at the same time create jobs in the service economy.

There is a definite limit to what can be achieved through expansion of services such as childcare, however, not least because it is difficult to pay for wage increases in human services through increases in productivity. While computers might have changed manufacturing they don’t make it easier to care for children and jobs in areas such as childcare need therefore to remain fairly low paid for increased provision of childcare to be affordable for governments. In countries like Sweden there is a consensus that universal public services supported by high rates of taxation are desirable. In the UK this is not the case, so while provision of childcare might improve the efficiency of the labour market and help to reduce poverty it is not going to do much to reduce inequalities between families at different ends of the socioeconomic distribution.

Graphene and Growth

In 2010 Prof. Andre Geim and Prof. Konstantin Novoselov from the University of Manchester were awarded the Nobel Prize for physics for experimental work on the structure of graphene. Graphene is a layer of carbon one atom thick and has a range of fairly remarkable properties. For example, it is 200 times stronger than steel but is also stretchable and is the most conductive material in the world. Graphene’s properties mean that it has a wide range of industrial applications and it is already leading to innovations in the manufacture of electronic circuits and coatings, touchscreens and fibre-optic communication systems.

Graphene is still a new product, however, and because of the need for further technological developments the National Graphene Institute is currently being built next to where I work in Manchester. The building is being funded jointly by the Engineering and Physical Sciences Research Council (EPSRC) and the European Regional Development Fund (ERDF). The ERDF is a fund set up under EU regional policy which aims to reduce economic disparities between regions through investment in infrastructure and in companies (in particular SMEs). The total funding from the ERDF for England over the period 2007-2013 is €3282.5 million. Given the programmes aim to promote growth in less prosperous regions funding in England is mainly targeted on regions in the North, with the North West receiving €755.5 million and Yorkshire and the Humber €583.6 million while only €23.7 million has been allocated to the South East.

There is an open debate in economic geography and in government policy, however, as to whether trying to reduce regional imbalances in the economy is a good idea or even whether it is actually possible. One reason for the scepticism surrounding regional policies is that it is widely believed that growth in a city-region is driven by cumulative or circular processes. For example, regional growth in the well-known core-periphery model is determined by the number of firms that locate in a region but firms tend to locate in regions because they are growing. Regional differences in economic growth and investment tend therefore to be quite durable and self-sustaining, leading to the accumulation of different types of human, physical and financial capital in some regions and a lack of the same resources in others.

Cumulative processes certainly play a part in maintaining the North-South divide in economic prosperity in the UK. For example, the figure below shows the relationship between GDP per capita and the percentage of human resources in science and technology (HRST) for NUTS 2 regions in England (data from Eurostat). Inner London has over 40 percent of employment in HRST and has been omitted from the figure. The figure shows that there is a clear positive relationship between GDP and employment in science and technology, with areas in the South having higher levels of both GDP and employment in science and technology than areas in the North. Innovation is a major source of economic growth and firms in high-tech industries tend to locate in areas near to similar firms feeding back into regional growth. From one perspective, trying to intervene in these types of cumulative processes that underlie regional growth is likely to have little or no effect and instead we should be looking to reduce planning restrictions so that more firms and people can locate in the South of England. On the other hand, it can be argued that all regions have the potential to grow but that low levels of investment in regions with low growth reflect the higher risk attached to investing in a region that is not growing. It can therefore be argued that initiatives like the ERDF have an important role to play in helping to reduce the risks of investment and promoting growth.

GRapheneplot

It is worth noting that these different views of the effectiveness of regional policy are reflected in recent government reviews. The recent Witty review of links between industry and universities advocated a “sector not postcode” approach to investment. Innovation does have a distinct geography, however, which is not simply the geography of good ideas but partly the geography of whether those good ideas have market potential in a certain context. The Heseltine review, on the other hand, gave more credence to government intervention as a way of mobilising resources in regions and its recommendations included the development of regional innovation centres as a way of promoting growth in areas outside the South of England.

The Heseltine review has a striking portrait of Joe Chamberlain on the second page. Chamberlain made his fortune in screws, became Mayor of Birmingham and a Liberal politician and is credited with having been the force behind the late Victorian industrial development of the city. I don’t know whether there is a fortune to be made out of graphene but, if there is, it will partly be due to the effect of EU regional policy and not just the individual initiative of great men (or women).

Chamberlain

Geography and University Admissions

Although you don’t need to go to university to be successful in life (Bill Gates dropped-out of university, although it was Harvard), few people would argue that social mobility takes place through higher education (HE). The pathway into any of the professions requires a degree, for example. Most research on admissions suggests, however, that HE doesn’t play an important role in actually creating economic inequalities. The main factor influencing admission to HE is student’s prior level of academic achievement which is largely determined prior to entering HE. Inequalities originate in family and school factors, universities simply select students based on prior ability.

After reading the recent Higher Education Funding Council for England (HEFCE) report on trends in participation, I would doubt that this view of HE is entirely correct. When the report was published the BBC and various newspapers ran stories about London being the region with the highest proportion of young people who go on to HE with over 40 percent of young people from London entering HE in 2011/12. Regional differences in the proportion of young people from disadvantaged areas who are admitted to HE are even larger (astonishing, actually). The report measures area disadvantage using statistics on the number of people living in an area and in receipt of a low-income benefit. The figure below (from the HEFCE report) shows the change in the rate of HE participation for young people living in low-income areas between 1998/99 and 2011/12. In London, the proportion of young people entering HE who lived in areas where large numbers of people are in receipt of a low-income benefit, rose from 26 to 41 percent. In contrast, in the North East participation among young people living in disadvantaged areas rose from only 13 percent to 18 percent.

HEFCEII

It is difficult to understand these regional patterns as entirely reflecting differences in prior academic achievement, because the levels of achievement of young people from London don’t suggest that they are very much brighter than young people in other regions. The figure below shows the proportion of young people in each region who obtained AAB grades or better at A level in 2011/12 using data from the Department for Education. The figure shows that while Outer London and the South East did have the highest proportion of young people achieving grades AAB or better at A level, the East of England, the South West and the North West were not far behind and Inner London had the lowest proportion of entrants achieving grades AAB or better.

plot30

Young people at schools in deprived boroughs in Inner London (eg. Hackney) do have higher levels of school achievement than in deprived areas in other regions, but it doesn’t seem that differences in achievement can entirely explain differences in rates of admission to HE. For example, the HEFCE report shows that over 35 percent of young people from parliamentary constituencies in Hackney (Hackney North and Stoke Newington and Hackney South and Shoreditch) were admitted to HE in 2011/12, while in constituencies in central Manchester (Blackley and Broughton and Manchester Central) less than 30 percent of young people went on to university.

For young people from less well-off backgrounds the ability to stay at home while studying is an important way of reducing the financial risks attached to entering HE and it seems likely that this could be an important factor making it easier for students living in London to enter HE. The large number of places available in institutions across London would also seem to have a role to play as could the more mixed socioeconomic character of neighbourhoods in London. The opportunity that young people living in deprived areas of London have to get a degree is a good thing. It ought to be recognised, however, that while geography might not be that important nationally with around 80 percent of young people leaving home to enter HE, in some regions it would seem to be the key to understanding trends in participation particularly for students from less well-off backgrounds.

Evaluation of the Free School Meals Pilot

During the Lib Dem conference Nick Clegg announced that all children in reception, year 1 and year 2 in state-funded schools in England would be eligible for free school meals, a policy with a total cost of around £600 million per year. The policy is a little surprising because the Coalition has tended to favour targeted policies arguing that at a time of austerity we need to spend what money is available on those who are most in need. For example, child benefit is now a means tested benefit (in households in which someone earns £50000 or more per year) and, in education, the pupil premium provides extra funding to schools for each child in receipt of free school meals.

What was really interesting was hearing Menzies Campbell say that an evaluation of the policy had shown that children who received free school meals in schools in three pilot areas showed an increase in ability equivalent to two months learning in comparison to children in comparison areas. I suppose I thought it surprising because it is difficult to think of the immediate processes through which such a policy might have such a large effect. Policies that are area-based, such as Sure Start or labour market interventions such as the New Deal programmes, have often proved difficult to evaluate, however, because it is inherently difficult to select an appropriate control group where an intervention is area-based. Often it is argued that the control group is selected from areas similar to those of the treatment group but the combination of factors that give an area its character might be considered relatively unique. The initial Sure Start programme, for example, was restricted to the 20 per cent most deprived neighbourhoods in England so the evaluation couldn’t compare children on Sure Start to children in similarly deprived areas who had not had the opportunity of Sure Start. If the intended outcome of the intervention is influenced by types of area-based processes, so that children in deprived and non-deprived areas are not directly comparable, getting an estimate of the effect of the intervention that is separate from area-based influences becomes difficult.

In the areas in which the policy was piloted (Newhan, Durham, Wolverhampton), all children received free school meals between 2009 and 2011. The control group of children were chosen from schools in local authorities which were matched to the pilot areas using school characteristics such as school type (community vs. non-community), number of full-time equivalent teachers, proportion of students eligible for free school meals etc. (the details are in the appendix to the evaluation report). Children in schools in Newham (pilot A), for example, were matched to control schools in Enfield, Haringey, Manchester, Redbridge and Wandsworth. In order for the results of the evaluation to be credible it is important that the pilot and control areas are comparable. The authors of the report know that this is difficult to achieve and Figure 1 in the report (reproduced below) shows the proportion of the adult population receiving state benefits in each pilot and comparison area has varied over time. What is very striking about the figure is that in each of the pilot areas the proportion of the adult population in receipt of state benefit is significantly lower than in the control areas. For Newham (pilot A) and Wolverhapton (pilot C) the gap in benefit receipt in comparison to control areas is at least 5 percentage points.

fsmbenefits

The reason for running a study with an experimental design is that random assignment of the treatment should balance out unobserved characteristics between treatment and control units. The study estimates the effect of receiving free school meals by comparing the outcomes of children in pilot and comparison areas who are similar with respect to a wide range of observed characteristics, but because they live in different types of areas it is likely that there will also be unobserved differences between children. Is this likely to effect the estimate of the effect of free school meals on children’s ability? Yes. The gradient in children’s outcomes with neighbourhood deprivation is a durable feature of the patterning of social inequality so you would expect children in better-off areas to be doing better than those in deprived areas even after adjusting for other factors. The figure below shows the gradient in Key Stage 2 results across deciles of local area deprivation for each region using data from the National Pupil Database and shows that there are both regional and local differences between children in test results. In summary, because of the area component in child outcomes, children in pilot areas would be expected to do better than those in comparison areas simply because of the types of areas in which they lived. The difference in child outcomes between pilot and comparison areas might not have that much to do with free school meals.

fsmregion

Although the evaluation has its problems (and they are difficult to get around) it is probably a good policy and it is difficult to think of any government (other than perhaps the one that introduced it) trying to reverse it. I would expect, however, that providing free meals for all children will lead to quite modest, but positive, gains in ability for individual children. If you think that the reasons why some children struggle at school are embedded in the wider context and difficult to identify such a universal approach is likely to be far more effective than a highly targeted approach to disadvantage.