ESRC Society Today

The UK Strategic Forum for the Social Sciences was established in 2001 to provide an arena in which the major issues and challenges facing the social sciences can be debated at the highest level and to ensure that the needs of the social sciences are addressed and tackled. The Forum brings together representatives from the major research groupings and organisations with a strong interest in ensuring that there is a healthy and sound social science research base in the UK. The member organisations are shown at the end.

On 27 October 2009, the Forum discussed HEFCE's consultation on the Research Excellence Framework (REF), which sets out proposals for new arrangements to assess and fund research in UK higher education institutions. This Forum statement sets out the general areas of principle on which there was broad consensus at that meeting.

The Forum welcomes

• the recognition that high quality social science research plays an essential role in improving public policy and economic well-being, and that all these elements are recognised as contributing to the quality of life.
• HEFCE's commitment to recognise and reward high-quality research which contributes to social, cultural, public policy, and economic well-being. In particular, the Forum welcomes the principle that those doing high quality scientific work which has implications for policy or practice, or is high quality work that is of an applied nature or is geared towards achieving non-academic impact should be recognised. The Forum also welcomes the fact that the definition of impact will go well beyond economic value.
• the commitment to maintain the central focus of research assessment on the identification of excellence, with the greatest recognition being given to those units with excellent research activity as measured by outputs (research publications and other outputs) and research environment (the extent to which the environment supports a continuing flow of excellent research and its effective dissemination and application).
  the decision to retain peer review as the principal form of assessment (informed, where appropriate, by citation information and other quantitative data).
• the plan to have one overarching framework of assessment for all disciplines. It had been proposed in the 2006 consultation that there should be different methods of assessment for the 'sciences' and the 'non-sciences'. A division on these lines would have been unhelpful as many social science subjects, such as Psychology, straddle the sciences as well as the social sciences.
• the proposal to maintain the overall assessment at the level of the unit or department so that the assessment of impact will relate to the broad portfolio of activity across a department and not the activity of every individual researcher, as has been implied elsewhere.
• HEFCE's acknowledgement of the significant difficulties of measuring impact, such as time lags, attribution and corroboration. The Forum therefore welcomes the establishment of the impact pilot exercise to test and develop the proposals to assess impact, and the fact that the final decisions on the assessment of impact and its weighting will be taken in the light of the consultation exercise and the pilot outcomes.
• the commitment to ensure that there is greater consistency of assessment criteria and working methods across the exercise. In support of this aim, the Forum commends HEFCE's proposals to establish a main panel for the Social Sciences, along with a REF Steering Group with oversight of the conduct of the exercise. 

Areas for further consideration

The Forum identified five main areas for HEFCE to consider further.

1. The definition of impact. The definition of impact needs to be sensitive to the needs of individual disciplines and its potential uses, and should recognise that the types of impact will vary according to each individual discipline. While the Forum welcomes the broad definition given to impact in the consultation, covering social, economic, cultural, public policy and quality of life benefits, it considers that the definition should ensure that local, national and global effects are all captured and treated equally, not least because this is the most appropriate way in which the impact of particular disciplines is properly measured. For some disciplines, the nature of the area may mean that local and regional impacts are more appropriate; for others, the global is appropriate.
2. Linking impact to high-quality research. The Forum welcomes HEFCE's commitment to ensuring that impact should always be related to research quality. However, it is often not straightforward to identify the impact of high quality research, especially as excellent research sometimes has low impact, and poor quality research may have high, but not positive, impact.
   • Impact is often mediated through general reputation and expertise rather than being the result of an individual piece of work, which suggests that further consideration needs to be given to the assessment of esteem and its relationship to impact. For example, as a result of a portfolio of world-class contributions in a particular area, a social scientist might be asked to advise on the development of policy broadly, rather than on a particular project. This work may have great impact but could only be seen as indirectly drawing upon any particular piece of research.
   • Consideration should be given as to whether what were previously regarded as indicators of esteem should feature in the assessments of impact or environment as there is scope for ambiguity and possible duplication under the current proposals. 
   • As many submitting units within the social sciences [approximately 40-50% based on the RAE 2008 submissions] are relatively small (20 submitted staff or less), the assessment of the impact for a whole unit may be based on the work of only one or two researchers, as it is proposed that there should be one case study in each submission for every 5 to 10 full-time equivalent (FTE) staff submitted. This may lead to problems. Issues such as these will need to be considered carefully.
3. Sub-panel configuration. The Forum supports the proposal that there should be fewer sub-panels for the social sciences, grouping together a number of social science disciplines in order to reduce the number of social science sub-panels from 12 to 9.  As it is proposed that there should be standardised criteria, guidance and weightings for each unit of assessment, care must be taken to ensure that the new configuration of sub-panels maintains the intellectual integrity of the research base and does not have a distorting effect on the research undertaken in certain disciplines.
4. A Holistic approach. The Forum is pleased that HEFCE aims to build on, and learn from, the experience of the research councils of assessing impact. ESRC, in particular, has played a leading role in developing different approaches and methodologies for assessing research impact, and is evidently willing to continue to advise and collaborate. The Forum also encourages HEFCE to consider further the relationship between outputs, environment and impact, and the ways in which all three elements depend and contribute to each other and are mutually reinforcing.
5. Evaluation. The Forum urges HEFCE to undertake evaluation of the robustness of its policy initiatives, casting its expectations about the effect of those initiatives in testable form and making data available to interested researchers for analysis. It also urges HEFCE to ensure that the findings from the impact pilot exercise are carefully evaluated in a transparent manner, possibly subject to independent scrutiny, and that reference should be made to the experiences of foreign funding agencies in this area.

Representatives of the Forum are drawn from the following member organisations: The British Academy; Economic and Social Research Council; The Leverhulme Trust; Nuffield Foundation; Royal Economic Society; The Academy of Social Sciences; Joseph Rowntree Foundation; Association of Research Centres in the Social Sciences; Home Office; Office for National Statistics; Departments for Business, Enterprise and Regulatory Reform; The Treasury; Department for Children, Schools and Families; and Universities UK.

These organisations also have the opportunity to respond independently to the REF Consultation and the Forum does not seek to constrain any such submissions.