To make science reproducible, focus on prevention not cure

This post was written by Marcus Munafò, University of Bristol, Malcolm Macleod, University of Edinburgh, they are on the steering group of the UK Reproducibility Network and Malcolm Skingle, director of academic liaison at GlaxoSmithKline. 

Improved upstream quality control can make research more effective, say Marcus Munafò and his colleagues.

Science relies on its ability to self-correct. But the speed and extent to which this happens is an empirical question. Can we do better?

The Covid pandemic has highlighted existing fault lines. We have seen the best of scientific research in the incredible speed at which vaccines have been developed and trialled. But we have also seen a deluge of Covid-related studies conducted in haste, often reflected in their less-than-ideal quality.

Peer review, the traditional way of assessing academic research, occurs only after work has been done. Can we identify indicators of research quality earlier on in the process when there is more opportunity to fix things? And if we did, could scientific knowledge be translated into societal benefit more rapidly and efficiently?

In many ways, the cultures and working practices of academia are still rooted in a 19th-century model of the independent scientist. Many research groups are effectively small, artisanal businesses using unique skills and processes.

This approach can yield exquisitely crafted output. But it also risks poor reproducibility and replicability—through, for instance, closed workflows, closed data and the use of proprietary file formats. Incentive structures based around assessing and rewarding individuals reinforce this, despite the welcome shift to team-based research activity, management, dissemination and evaluation.

Lessons from industry 

Research needs a more coherent approach to ensuring quality. One of us has previously argued that one way to achieve this would be to take the concept of quality control used in manufacturing and apply it to scientific research.

Pharmaceuticals are one R&D-intensive industry that has worked hard to improve quality control and ensure data integrity. Regulatory frameworks and quality-assurance processes are designed to make the results generated in the early stages of drug development more robust.

Indeed, some of the early concerns about the robustness of much academic research—described by some as the ‘reproducibility crisis’—emerged from pharmaceutical companies.

For regulated work, major pharmaceutical companies must be able to demonstrate the provenance of their data in fine detail. Standard operating procedures for routine work, and extended description of less common methods and experiments, makes comparisons between labs easier and improves traceability.

Data constitute the central element of robust research. The integrity of the systems through which data are collected, curated, analysed and presented is at the heart of research quality. National measurement institutes, including the UK’s National Physical Laboratory and National Institute for Biological Standards and Control have a role to play, sharing best practice and developing protocols that contribute to international standards.

How well these systems perform depends on many factors: training in data collection and management; transparency to allow scrutiny and error detection; documentation, so that work can be replicated; and standard operating procedures to ensure a consistent approach.

Red-tape review

Academic researchers are increasingly keen to learn from industry, and vice versa—to identify best practice and ways to implement higher standards of data integrity. University and industrial research are very different, but academia can learn lessons and adopt working practices that might serve to improve the quality of academic research in the biomedical and life sciences.

Learning from other sectors and organisations is a central theme of the UK Reproducibility Network. The network, established in 2019 as a peer-led consortium, aims to develop training and shape incentives through linked grassroots and institutional activity, and coordinated efforts across universities, funders, publishers and other organisations. This multilevel approach reduces the cost of development and increases interoperability, for example, as researchers move across groups and institutions.

Given the likely future pressures on the UK’s R&D budget, effective and efficient ways to bolster research quality will be essential to maximising the societal return on investment. Simply encouraging, or even mandating, new ways of working is not sufficient—many funders and journals have data-sharing policies, for example, but adherence is uneven and often unenforced.

A coordinated approach will require a clear model of research quality; buy-in from institutions, funders and journals; infrastructure; training; the right incentives; and ongoing evaluation. Coordinating all these elements will be challenging, but it is essential to improving research quality and efficiency. We need to take a whole-system approach.

This also applies to the independent review into research bureaucracy recently announced by the UK government, charged with identifying how to liberate researchers from admin. This is laudable—academia should certainly not be regulated in the same way as the pharmaceutical industry—but the review should recognise that an ounce of prevention can save a pound of cure.

Developing and deploying systems that improve research quality might increase efficiency and reduce research waste, as well as securing greater value for our national research effort.

This blog post was originally posted on Research Fortnight, you can read the original article here.

Management consultants in healthcare do more harm than good, but keep getting rehired – new research

This blog post was written by Andrew Sturdy, Chair in Organisation and Management, University of Bristol and Ian Kirkpatrick, Chair in Management, University of York. This article is republished from The Conversation under a Creative Commons license, read the original article here.

The use of management consultants has grown enormously in recent years. In the UK, consultancy brings in around £10 billion a year in fees across the public and private sectors. And while not totally recession-proof, the numbers grew in the run-up to Brexit and then COVID-19. (Remember test and trace? Consultants played a major role.)

Consulting firms can provide advice and extra resources at short notice and can be very effective for the right task and client. But their use often brings controversy, especially when public money is at stake, over the value of outsourcing, for instance. This raises a number of questions. Does consultancy bring improvements such as increased efficiency? If not, how can we explain its huge growth?

In the NHS, there is a remarkable lack of clarity and transparency over how much consultancy is used and with what effects. This falls within broader concerns noted in a recent National Audit Office report on procurement across public services.

In our ongoing research on management consultancy in the NHS, we have started to address these issues.

Continue reading

Experts aren’t just for emergencies: How COVID-19 is changing evidence-based policy making for the better

 

 

This blog post was written by Anthea Terry, Interim Head of PolicyBristol and was originally published Universities Policy Engagement Network (UPEN) blog. Read the original article.

Michael Gove famously said in 2016 that ‘people in this country have had enough of experts’, and with social media ‘bubbles’, fake news, and the media desire to present opposing viewpoints – however marginal – it can often feel this way.

But the actual public perception of experts and their work is more nuanced. A 2018 survey by the Wellcome Trust found that 82% of people said they were fairly or very interested in health research, up from 77% in 2015, and 75% in 2012.

The value placed on experts by policymakers has always been variable and hard to measure, ‘evidence-based policymaking’ has been around for decades, and for almost as long, the perhaps inevitable cynicism about ‘policy-based evidence making’. We have incredible success stories about research influencing policy (my favourite being the research on CFCs that led directly to the Montreal protocol and recovery of the ozone layer), yet the combined weight of almost all the world’s climate scientists fails to enact sufficient policy change.

One of the many unique features of this time is the level of public discussion about research and the role of experts in policy making. I can’t remember another time when the membership of expert advisory groups such as SAGE was mainstream news. Similarly, a call for participants in a Covid-19 vaccine trial in Bristol was shared on neighbourhood WhatsApp and Facebook groups, and I’m talking to my family about R-numbers and logarithmic growth curves whilst lamenting the lack of supermarket delivery slots.

Continue reading

PolicyBristol and Covid-19

Written by Lindsey Pike

These are uncertain times. Both research and policymaking has been thrown into unknown territory, and anxiety is running high. The coronavirus situation is dynamic and is likely to change how we live and work for the foreseeable future. At PolicyBristol we are looking at how we can reprioritise how we work and what we do, to ensure we’re making the most of the resources we have within the team.  

The team are working from home, but are available to discuss policy engagement and related issues – related to COVID, or any other topic – over the phone or videoconferencing.  

We understand that both researchers and policy colleagues that we work with are under pressure to adapt to working from home, often while coordinating and managing childcare and other responsibilitiesHowever, we’ll continue to signpost you to relevant opportunities and information to make sure that, especially in times such as these, policy decisions are informed by a robust evidence base.  

We’re also aware that other policy priorities, while currently overshadowed, have not gone away. We’d like to reiterate that regardless of your area of research or policy, we’re here to support you. 

Many colleagues in our research community are already forging links with public health, clinical, social and third sector services to offer the resources we have. If you are a researcher whose work is relevant to Covid-19, we have listed some relevant engagement opportunities below: please get in touch with one of the team (emails below) if we can support you in any way; for example with support to edit or structure scientific summaries into policy/ lay friendly ones, horizon scanning, or any other task related to getting findings out there and used.  

Contact your faculty’s PolicyBristol Associate 

Meet the team and contact us here. 

We hope you stay safe and well during this time. 

Resources from the University of Bristol 

Guidance for researchers during the COVID-19 outbreak (internal). 

The University is keen to hear about the circumstances our partners and communities are facing, how they are responding and how we might work together to meet these unparalleled new challenges. Please contact us if you have suggestions for how the University, our staff and students can support your organisational or community activities in response to COVID-19. 

The University of Bristol’s researchers, staff and students are working together and with partners from across society to understand coronavirus (COVID-19) and its far-reaching impact on our lives. Find out more here. 

National consultations and inquiries related to COVID19 

Online harms (Home Affairs Committee) The inquiry seeks evidence on Online Harms arising from the Covid-19 lockdown period and the adequacy of the Government’s proposals to counter them. Deadline 21 May 2020

Covid-19 and the food supply (Environment, Food and Rural Affairs Committee)
This inquiry examines issues related to the food supply chain and access to healthy foods. Deadline: 22 May 2020

Economic impact of coronavirus (Treasury Committee) In this stage, the Committee will examine the operational effectiveness, cost and sustainability of the Government’s and Bank of England’s support packages. The Committee will also examine the impact on the economy and different sectors, the implications for public finances, and how the Government can work towards a sustained recovery. Deadline 27 May 2020

The impact of coronavirus on business and workers (Business, Energy and Industrial Strategy Committee)
The BEIS Select Committee has launched an inquiry into the impact of coronavirus on businesses and workers. Deadline: 29 May 2020

The impact of Covid-19 on education and children’s services (Education Committee) The inquiry will examine both short term impacts, such as the effects of school closures and exam cancellations, as well as longer-term implications particularly for the most vulnerable children.
Deadline: 31 May 2020 

Left behind white pupils from disadvantaged backgrounds (Education Committee) This inquiry will investigate the issues faced by disadvantaged groups, with an initial inquiry into the educational underachievement of white pupils from disadvantaged backgrounds including white working class pupils. This inquiry will examine the extent of the achievement gap between this group and their peers and how it is measured, alongside a consideration of the effects of the COVID-19 outbreak. It will also look at what the priorities should be for tackling this issue. Deadline 5 June 2020

Defence contribution to the UK’s pandemic response (Defence Committee) This inquiry will focus on the Ministry of Defence’s and the Armed Forces’ contribution to the United Kingdom’s response to the Covid-19 pandemic. The scope will include: assessing the MoD’s planning and preparedness for a pandemic; understanding how the Armed Forces have supported the civilian authorities during the pandemic; evaluating the effectiveness of the specific actions and activities undertaken by military and civilian personnel, and; exploring how the MoD has ensured that potential adversaries have not taken advantage of the need to focus on the pandemic response. Deadline: 15 June 2020

Impact of COVID-19 on DCMS sectors: (Digital Culture Media and Sport Committee) The DCMS Committee has launched an inquiry into the ‘Impact of Covid-19 on DCMS sectors’. It will consider both the immediate and long-term impact that Covid-19 and the related social and financial measures are having on the wide range of industries and organisations under the Committee’s remit. The Committee expects to hold a number of evidence sessions from late April onwards to hear directly from stakeholders, arms-length bodies and Government about what is being done and what further support is needed. Deadline 19 June 2020

Coronavirus and Scotland (Scottish Affairs Committee). Deadline 23 June 2020

Coronavirus: implications for transport: (Transport committee)The Transport Committee is asking transport workers, stakeholders and members of the public to write to them about the transport issues they face during the coronavirus outbreak. MPs will explore the impact felt by the industry, its workers and passengers in a rolling programme of work to monitor the impact of coronavirus on UK transport, sector by sector. Deadline: 29 June 2020

The science of COVID-19: (Science and Technology Committee (Lords) This inquiry will investigate the scientific and technological aspects of the COVID-19 pandemic, including the nature of the SARS-CoV-2 virus, its transmission and spread, the development of vaccines and treatments, and how digital technologies can be used for tracking and modelling. The inquiry aims to help Government and society learn from the pandemic and better prepare for future epidemics.Deadline: 30 June 2020

The Government’s response to Covid-19: human rights implications
(Human Rights Joint Committee)
The Committee is seeking evidence on how the Government is ensuring measures are human-rights compliant, the impact of these measures on human rights in the UK, and the groups who will be disproportionately affected. Deadline: 22 July 2020

UK Science, Research and Technology Capability and Influence in Global Disease Outbreaks (Science and Technology Committee)
Once the COVID-19 pandemic has passed its peak, the Committee will inquire formally into the place of UK research, science and technology in the national and global response, and what lessons should be learned for the future. Deadline: 31 July 2020 

Past COVID inquiries

Impact of Covid-19 on the charity sector
(Digital, Culture, Media and Sport Committee)
Deadline: 16th April 2020 

  • This is a short inquiry into the impact on the charity sector. Find out more here. 

Home Office preparedness for Covid-19
(Home Affairs Committee)
Deadline: 21 April 2020 

  • The Home Affairs Committee is undertaking a short inquiry into the Home Office’s preparations for and response to Covid-19 (Coronavirus). Find out more here. 

The Covid-19 pandemic and international trade
(International Trade Committee)
Deadline: 24th April 2020 

  • This wide-ranging inquiry seeks views on impact on UK businesses, supply chains, and access to essential goods. Find out more here 

Unequal impact: Covid-19 & people with protected characteristics
(Women and Equalities Committee)
Deadline: 30th April 2020 

  • The committee wants to hear about the different and disproportionate impact that the Coronavirus – and measures to tackle it – is having on people with protected characteristics under the Equality Act. Find out more here.  

Humanitarian crises monitoring: impact of coronavirus
(International Development Committee)
Deadline: 17 April/ 8th May 2020 

  • This inquiry is seeking evidence in two waves; current situation and immediate risks (17 April) and longer-term issues and implications (8 May). Find out more here.

Do development indicators underlie global variation in the number of young people injecting drugs?

Dr Lindsey Hines, Sir Henry Wellcome Postdoctoral Fellow in The Centre for Academic Mental Health & the Integrative Epidemiology Unit, University of Bristol

Dr Adam Trickey, Senior Research Associate in Population Health Sciences, University of Bristol

Injecting drug use is a global issue: around the world an estimated 15.6 million people inject psychoactive drugs. People who inject drugs tend to begin doing so in adolescence, and countries that have larger numbers of adolescents who inject drugs may be at risk of emerging epidemic s of blood borne viruses unless they take urgent action. We mapped the global differences in the proportion of adolescents who inject drugs, but found that we may be missing the vital data we need to protect the lives of vulnerable young people. If we want to prevent HIV, hepatitis C, and overdose from sweeping through a new generation of adolescents we urgently need many countries to scale up harm reduction interventions, and to collect accurate which can inform public health and policy. Continue reading

Universities need to do more to support impactful researchers

 [IMAGE: Caption text|/resources/images/blank.png]

Universities need to do more to support impactful researchers

For anyone who has worked in or on policy engagement, the image of the furiously busy policymaker will be all too familiar.

In training, case studies and even in the academic literature, this image persists: a policymaker, inundated with different priorities, brain saturated with information, inbox filled to the brim, running frantically from one meeting to the next, trying to get as much as possible done in difficult circumstances, and with limited resources.

Although I suspect it is less common, there is also an image of the academic: busy with research and teaching, they meet multitudes of students and they mark piles of essays. The academic has also has limited resources, (unless they have a handy grant), but great depth of expertise.

Much of the thinking about engaging with policymakers focuses – rightly, I think – on how to make life easy for our “furiously busy policymaker”. We write differently, more concisely, more simply, and more in stories than in facts. In short, we tailor what we do to the concerns and priorities of policymakers. There is nothing wrong with that – indeed, it seems a sensible thing.

CEOs make more in first week of January than average salary – pay ratios are the solution

Dr Tobore Okah-Avae, Assistant Teacher, University of Bristol

The typical FTSE 100 CEO will have earned as much as the average UK worker earns in a year by 5pm on January 6 2020 – £29,559 for 33 hours of work, according to data compiled by the High Pay Centre think tank. By the close of the year, the same CEO would have earned £3.46 million – roughly 117 times the average wage in the UK. This is a staggering differential.

If you believe that excessive executive pay is a problem, this statistic illustrates the point perfectly. These figures even represent a reduction from previous years, although this is due more to shrinkage in overall CEO pay than increases at the bottom. And UK CEO pay actually pales in comparison to their counterparts in the US, where levels topped US$14.5m (£11.5m), representing a 287-1 differential with the average worker. Continue reading

National greenhouse gas reporting needs an overhaul – it’s time to directly measure the atmosphere

Dr Matt Rigby, Reader in Atmospheric Chemistry, University of Bristol

How much greenhouse gas is emitted by any individual country? With global emissions of carbon dioxide hitting a record of 36.8 billion tonnes this year, and delegates gathering in Madrid for the latest UN climate talks, it’s a pressing question.

One might assume that we know precisely how much is emitted by any given country, and that such figures are rigorously cross-checked and scrutinised. And in some respects, this is true – countries are required to report their emissions to the UN, based on exhaustive guidelines and with reams of supporting data.

Yet these reports are based on what are known as inventory (or “bottom-up”) methods. To simplify, this means that governments figure out how much greenhouse gas is emitted by a typical car, cow, or coal plant, and then add up all the cows, cars and so on to get an overall emissions figure. Continue reading

Can Flying Go Green?

Justin Rowlatt BBC journalist in front of a plane

Can Flying Go Green? Panorama, BBC One

With BBC Panorama questioning if airlines are doing enough to go green, the University of Bristol We are Engineering blog asked Bristol’s engineers how they’re making aviation greener. We republish a shortened version below.

Viewers of BBC’s Panorama programme Can Flying Go Green? last week would have seen how our passion for flying is warming the planet. Aviation is a major contributor to global carbon dioxide emissions, burning more fossil fuels per passenger than any other form of transport. In the show, Justin Rowlatt investigates how the airlines are trying to clean up their act, but what is being done outside of the commercial sector to make aviation greener? Here’s what our engineers are doing to pave the way to a greener future for the industry. Continue reading

The ‘5 Ts’ of policy engagement: PolicyBristol’s approach to supporting academics

Supporting academics across the University of Bristol to achieve policy impact from their research is a diverse and fascinating job. In the process of doing this, our team at PolicyBristol is constantly learning about new topics; from the value of NHS managers to refugee rightsenhancing peace processes to the role of universities. Continue reading