To make science reproducible, focus on prevention not cure

This post was written by Marcus Munafò, University of Bristol, Malcolm Macleod, University of Edinburgh, they are on the steering group of the UK Reproducibility Network and Malcolm Skingle, director of academic liaison at GlaxoSmithKline. 

Improved upstream quality control can make research more effective, say Marcus Munafò and his colleagues.

Science relies on its ability to self-correct. But the speed and extent to which this happens is an empirical question. Can we do better?

The Covid pandemic has highlighted existing fault lines. We have seen the best of scientific research in the incredible speed at which vaccines have been developed and trialled. But we have also seen a deluge of Covid-related studies conducted in haste, often reflected in their less-than-ideal quality.

Peer review, the traditional way of assessing academic research, occurs only after work has been done. Can we identify indicators of research quality earlier on in the process when there is more opportunity to fix things? And if we did, could scientific knowledge be translated into societal benefit more rapidly and efficiently?

In many ways, the cultures and working practices of academia are still rooted in a 19th-century model of the independent scientist. Many research groups are effectively small, artisanal businesses using unique skills and processes.

This approach can yield exquisitely crafted output. But it also risks poor reproducibility and replicability—through, for instance, closed workflows, closed data and the use of proprietary file formats. Incentive structures based around assessing and rewarding individuals reinforce this, despite the welcome shift to team-based research activity, management, dissemination and evaluation.

Lessons from industry 

Research needs a more coherent approach to ensuring quality. One of us has previously argued that one way to achieve this would be to take the concept of quality control used in manufacturing and apply it to scientific research.

Pharmaceuticals are one R&D-intensive industry that has worked hard to improve quality control and ensure data integrity. Regulatory frameworks and quality-assurance processes are designed to make the results generated in the early stages of drug development more robust.

Indeed, some of the early concerns about the robustness of much academic research—described by some as the ‘reproducibility crisis’—emerged from pharmaceutical companies.

For regulated work, major pharmaceutical companies must be able to demonstrate the provenance of their data in fine detail. Standard operating procedures for routine work, and extended description of less common methods and experiments, makes comparisons between labs easier and improves traceability.

Data constitute the central element of robust research. The integrity of the systems through which data are collected, curated, analysed and presented is at the heart of research quality. National measurement institutes, including the UK’s National Physical Laboratory and National Institute for Biological Standards and Control have a role to play, sharing best practice and developing protocols that contribute to international standards.

How well these systems perform depends on many factors: training in data collection and management; transparency to allow scrutiny and error detection; documentation, so that work can be replicated; and standard operating procedures to ensure a consistent approach.

Red-tape review

Academic researchers are increasingly keen to learn from industry, and vice versa—to identify best practice and ways to implement higher standards of data integrity. University and industrial research are very different, but academia can learn lessons and adopt working practices that might serve to improve the quality of academic research in the biomedical and life sciences.

Learning from other sectors and organisations is a central theme of the UK Reproducibility Network. The network, established in 2019 as a peer-led consortium, aims to develop training and shape incentives through linked grassroots and institutional activity, and coordinated efforts across universities, funders, publishers and other organisations. This multilevel approach reduces the cost of development and increases interoperability, for example, as researchers move across groups and institutions.

Given the likely future pressures on the UK’s R&D budget, effective and efficient ways to bolster research quality will be essential to maximising the societal return on investment. Simply encouraging, or even mandating, new ways of working is not sufficient—many funders and journals have data-sharing policies, for example, but adherence is uneven and often unenforced.

A coordinated approach will require a clear model of research quality; buy-in from institutions, funders and journals; infrastructure; training; the right incentives; and ongoing evaluation. Coordinating all these elements will be challenging, but it is essential to improving research quality and efficiency. We need to take a whole-system approach.

This also applies to the independent review into research bureaucracy recently announced by the UK government, charged with identifying how to liberate researchers from admin. This is laudable—academia should certainly not be regulated in the same way as the pharmaceutical industry—but the review should recognise that an ounce of prevention can save a pound of cure.

Developing and deploying systems that improve research quality might increase efficiency and reduce research waste, as well as securing greater value for our national research effort.

This blog post was originally posted on Research Fortnight, you can read the original article here.

To address ethnic inequalities in COVID-19, we must acknowledge the multifaceted influence of racism

This blog post was written by Dr Saffron Karlsen, (Associate Professor in Sociology, University of Bristol). 

The evidence of ethnic inequalities in the number of COVID-related infections and deaths in the UK is compelling – yet discussions about how to address them remain somewhat simplistic. Dr Saffron Karlsen discusses five key issues that must be acknowledged if we are to establish a more complete picture of these inequalities and their drivers.

Continue reading

Management consultants in healthcare do more harm than good, but keep getting rehired – new research

This blog post was written by Andrew Sturdy, Chair in Organisation and Management, University of Bristol and Ian Kirkpatrick, Chair in Management, University of York. This article is republished from The Conversation under a Creative Commons license, read the original article here.

The use of management consultants has grown enormously in recent years. In the UK, consultancy brings in around £10 billion a year in fees across the public and private sectors. And while not totally recession-proof, the numbers grew in the run-up to Brexit and then COVID-19. (Remember test and trace? Consultants played a major role.)

Consulting firms can provide advice and extra resources at short notice and can be very effective for the right task and client. But their use often brings controversy, especially when public money is at stake, over the value of outsourcing, for instance. This raises a number of questions. Does consultancy bring improvements such as increased efficiency? If not, how can we explain its huge growth?

In the NHS, there is a remarkable lack of clarity and transparency over how much consultancy is used and with what effects. This falls within broader concerns noted in a recent National Audit Office report on procurement across public services.

In our ongoing research on management consultancy in the NHS, we have started to address these issues.

Continue reading

Maximum cigarette pack size: a neglected aspect of tobacco control

Written by Anna Blackwell, Senior Research Associate

Follow Bristol Tobacco and Alcohol Research Group (TARG) on twitter

The manufacturing or importing of packs of cigarettes with fewer than 20 cigarettes per pack was prohibited in the UK when the EU Tobacco Products Directive and standardised packaging legislation were fully implemented in May 2017. This change was aimed at reducing the affordability of cigarettes and thereby discouraging young people from smoking. This directive also required the removal of branding and established a standard shape and dark green colour for packaging, including pictorial health warnings, which prevented the use of packaging for promotion and reduced its appeal.

Continue reading

The ties that bind: what the killing of George Floyd can tell us about ethnic inequalities in COVID-19 (and why we should listen)

 

This blog post was written by Dr Saffron Karlsen, (Senior Lecturer in Social Research, University of Bristol) 

On the last weekend of May 2020, much of the world watched with horror scenes of US urban disturbances in response to the death of George Floyd – another Black person killed in police custody. On the other side of the pond, many in the UK also awaited the release of an official report into the higher rates of infection and death of Black and other ethnic minority people from COVID-19.

Photo by Thomas de LUZE on Unsplash

Delays and disappointment

This Public Health England (PHE) report was heralded as an opportunity to finally provide answers to questions we’d had since evidence of these inequalities first emerged. The inquiry’s lead, Professor Kevin Fenton, described the pressing need for open discussion, to listen to the views of people from Black communities and those who worked with them to find out what was producing these inequalities.

Unfortunately, the report which was finally released is very far from fulfilling these ambitions. It does not provide a detailed investigation of the drivers of these ethnic inequalities and includes very little new information from which to make sense of these patterns.

Continue reading

Coronavirus: flying in fruit pickers from countries in lockdown is dangerous for everyone

Dr Denny Pencheva
Senior Teaching Associate, Migration Studies and Politics, University of Bristol

In the wake of the COVID-19 pandemic, major agricultural companies and charities have chartered flights to urgently bring in tens of thousands of Bulgarian and Romanian agricultural workers. Flights have headed to places like Karlsruhe and Düsseldorf in Germany, along with Essex and the Midlands in the UK.

This comes after farmers in both countries warned there is a real risk that thousands of tons of produce might be left to rot – further affecting food supply chains – if vacancies for agricultural workers go unfilled.

The excessive demand for food during lockdown has meant that farm labourers are classed as key workers, which is why they are being flown to the UK and other Western European countries.

Continue reading

What impact will COVID-19 have on drug markets and users?

Josh Torrance (PhD student and Assistant Teacher, School for Policy Studies)

Much of this research is based on personal emails and conversations with the police and other agencies. As such, not all of the facts presented are referenceable.

Covid-19 will present a major challenge to both drug users and drug treatment agencies over the coming months. There are 320,000 problematic drug users in the UK, many of whom have weaker immune systems than the general public – and therefore a diminished chance of recovery from the virus. People who inject drugs and street homeless communities are at particular risk; viral infections spread quickly through these populations. On the face of it, the pandemic might seem like a fantastic opportunity for problematic users to become drug-free, but the reality is much more complex.

A used needle left on grass.

Continue reading

Do development indicators underlie global variation in the number of young people injecting drugs?

Dr Lindsey Hines, Sir Henry Wellcome Postdoctoral Fellow in The Centre for Academic Mental Health & the Integrative Epidemiology Unit, University of Bristol

Dr Adam Trickey, Senior Research Associate in Population Health Sciences, University of Bristol

Injecting drug use is a global issue: around the world an estimated 15.6 million people inject psychoactive drugs. People who inject drugs tend to begin doing so in adolescence, and countries that have larger numbers of adolescents who inject drugs may be at risk of emerging epidemic s of blood borne viruses unless they take urgent action. We mapped the global differences in the proportion of adolescents who inject drugs, but found that we may be missing the vital data we need to protect the lives of vulnerable young people. If we want to prevent HIV, hepatitis C, and overdose from sweeping through a new generation of adolescents we urgently need many countries to scale up harm reduction interventions, and to collect accurate which can inform public health and policy. Continue reading

Baby box: child welfare experts say use of sleep boxes could potentially put infants’ lives at risk

Authors:
Debbie Watson, Professor in Child and Family Welfare, University of Bristol
Helen Ball, Professor of Anthropology and Director of the Parent-Infant Sleep Lab, Durham University
Jim Reid, Senior Lecturer, Department of Education and Community Studies, University of Huddersfield
Pete Blair, Professor of Epidemiology and Statistics, University of Bristol

The baby box in Finland is embedded as part of the maternity system. Kela

Having a baby can be expensive. So it’s maybe not surprising that many retailers around the world have cottoned on to the success of Finland’s baby boxes – a package aimed to set up new parents and their bundle of joy. The Finnish boxes include baby clothing, sleep items, hygiene products and a parenting guide –- as well as a “sleep space” for the baby.

Many retailers around the world are now offering similar boxes for expectant parents. Indeed, research conducted at the University of Tampere in Finland suggests there are variants in over 60 countries. This includes Scotland’s baby box scheme – with all newborn babies getting a free baby box from the Scottish government.

But as a group of child welfare experts, we believe imitations of the Finnish boxes could be placing babies at risk. This is because it has become common to believe that if babies sleep in these boxes, it will help protect them from sudden infant death syndrome (SIDS). Unfortunately, the research does not back this up. Continue reading

Using evidence to advise public health decision makers: an insider’s view

This blog post reviews a recent seminar hosted by the MRC IEU, PolicyBristol and the Bristol Population Health Science Institute.

Public health is one of the most contested policy areas. It brings together ethical and political issues and evidence on what works, and affects us all as citizens.

Researchers produce evidence and decision-makers receive advice – but how does evidence become advice and who are the players who take research findings and present advice to politicians and budget-holders?

We were pleased to welcome a diverse audience of around 75 multidisciplinary academics, policymakers and practitioners to hear our seminar speakers give a range of insider perspectives on linking academic research with national and local decisions on what to choose, fund and implement.

In this blog post we summarise the seminar, including links to the slides and event recording. Continue reading