ASSESSING CLINICAL TRIAL TRANSPARENCY AT UK UNIVERSITIES (version 3)

UAEM-UK and TranspariMED, London and Bristol, 28 June 2018

FORTHCOMING RANKING OF UK UNIVERSITIES

The 2018 Global Health Ranking

This year, Universities Allied for Essential Medicines (UAEM) UK will compile and publish the
Global Health Ranking of leading UK universities.

UAEM-UK is the national branch of a global network of university students that advocate for the
maximal public health impact of health products, by promoting access to essential medicines. UAEM
periodically publishes national Global Health Rankings of the top universities in key countries to
make visible their contributions to urgent global health research and access to treatment worldwide.
These rankings typically attract strong media attention, and are widely cited by highly performing
universities themselves. The next UAEM-UK Global Health Ranking will be published later this year.

TranspariMED, a Bristol-based global initiative to end evidence distortion in medicine, is assisting

UAEM-UK in developing and compiling the clinical trial transparency metrics. These metrics will be
incorporated in the overall UK Global Health Ranking. Thus, scoring results on these metrics will
affect universities’ performance in the overall ranking.

Why clinical trial transparency?

For the first time in the UK, the forthcoming Global Health Ranking will assess universities’ clinical
trial transparency. Failure to pre-register trials, post their summary results, and completely and
accurately report outcomes is not a ‘victimless crime’. It harms patients, weakens public health, and
slows down the development of new treatment and cures. Please see this short presentation for an
overview.

Additional reasons why UK universities urgently need to tackle clinical trial transparency include:
 Trial registration and results reporting are ethical and scientific imperatives
 Trial registration and the timely, appropriate and accurate reporting of trial outcomes are an
integral part of overall research quality
 Major research funders, including NIHR, MRC and NIH are moving to cut off funding to
researchers who fail to comply with global best practices

Methodology and assessment criteria

The Global Health Ranking will measure:

 University policies on trial registration and reporting
 University performance on summary results posting on two major trial registries

The assessment criteria used are based on global best practices set out by the World Health
Organization and the Declaration of Helsinki. Therefore, they measure university compliance with
global best practices, which typically (though not always) go beyond regional and national legal and
regulatory requirements. This focus on best practices is fully aligned with the existing policy positions
of the World Health Organization, Transparency International, Cochrane, TranspariMED, and the
AllTrials campaign. Please see further below for more details on the criteria we will use.

1
NEXT STEPS AND FUTURE OUTLOOK

Next steps

UAEM-UK and TranspariMED will jointly review the links/documents provided by universities in their
FOI responses and assess them in line with the draft criteria outlined above. We will then share our
assessment results with each university to give it the opportunity to dispute our findings and
highlight any relevant information we may have overlooked. Tentatively, this process will take place
during the second half of August 2018.

If the publication of the timeline of the EBM Data Lab’s EU Trials Tracker permits, we will also seek to
validate the data on summary results posting performance with universities prior to publication.

Future outlook

The transparency metrics used in this year’s ranking are solidly grounded in widely accepted global
best practices. However, they do not capture the full range of best practices.

In future years, we anticipate that we will:

 Assess universities against the full range of best practices set out in the WHO Joint Statement
 Include university performance on additional trial registries, notably ISRCTN

We strongly encourage universities to familiarize themselves with the WHO Joint Statement now
and use this audit tool to self-assess how they measure up to its criteria, for internal use. This will
enable universities to identify any remaining gaps in their policies and practices and adopt the full
range of global best practices prior to future rankings.

We also strongly encourage universities to include ISRCTN and other WHO-approved trial registries in
their efforts to locate trials with missing summary results and retrospectively post these onto
registries. Please note that WHO best practices state that summary results should be posted onto
every registry where a trial was originally registered.

ASSESSMENT METHODOLOGY

Definitions

The definition of clinical trials used will be that set out by the WHO:
“[A] clinical trial is any research study that prospectively assigns human participants or
groups of humans to one or more health-related interventions to evaluate the effects on
health outcomes. Interventions include but are not restricted to drugs, cells and other
biological products, surgical procedures, radiological procedures, devices, behavioral
treatments, process-of-care changes, preventive care, etc.”

Thus, the definition of clinical trials used here applies to all interventional trials, regardless of
location, date, or regulatory context.
 It includes all CTIMPs, and thus all trials registered on EU CTR, but is not limited to CTIMPs.
 It includes all interventional trials registered on Clinicaltrials.gov, but excludes non-
interventional trials registered there.

2
DRAFT ASSESSMENT CRITERIA

Trial registration policies

Criteria Yes/No

An audit of trial registration performance has been made public within the last 12
months

The university has a publicly available policy that requires registry entries to be
regularly updated

The university has a publicly available policy that requires clinical trials to be
registered before the first subject receives the first medical intervention

The university has a publicly available policy that requires all clinical trials to be
registered.

The university has a publicly available policy that requires some clinical trials (e.g.
CTIMPs) to be registered.
[We will score this as well if the university requires all trials to be registered, to
provide additional points.]
Note: Where universities have shared policies with external parties, such as local NHS bodies, such
policies are scored as university policies.

Summary results posting policies

Criteria Yes/No

An audit of summary results posting performance has been made public within the
last 12 months

The university has time-specific plans to retrospectively post missing summary results
for university-sponsored clinical trials completed in the past

The university has a publicly available policy that requires clinical trials to post
summary results within 12 months of their primary completion date.

The university has a publicly available policy that requires all clinical trials to post
summary results.

The university has a publicly available policy that requires some clinical trials (e.g.
CTIMPs) to post summary results.
[We will score this as well if the university requires all trials to post results, to provide
additional points.]
Note: Where universities have shared policies with external parties, such as local NHS bodies, such
policies are scored as university policies.

3
Policies on undeclared ‘outcome switching’, HARKing, p-hacking, etc

After consultation with experts, UAEM-UK and TranspariMED have decided to review all related
university policies received before developing a scoring methodology. We may conclude after this
review that a quantitative scoring of university replies to this question would not add value to the
ranking. In that case, we will produce a stand-alone publication highlighting examples of good
universtity policies in this regard, giving public credit to strong performers.

Note: Useful background info on ‚outcome switching‘ can be found at http://compare-trials.org/. See
also items 6a+b and 17a in the CONSORT checklist.

University performance on summary results posting on two major trial registries

How well does the university perform in posting the results of clinical trials onto trial registries?

Scoring for this component will be based on data collected from two major trial registries,
Clinicaltrials.gov and EU CTR (EudraCT):
 The performance metric used for Clinicaltrials.gov will be the % of interventional trials with a
primary completion point >12 months in the past that have posted summary results, as per
WHO best practices.
 The exact performance metric used for EU CTR will depend on the methodology used by the
EBM Data Lab’s forthcoming EU Trials Tracker, as this new tracker will be used to collect the
data for that registry. If possible, we will use the same metric as for ClinicalTrials.gov.

Important: According to WHO best practices, posting of summary results onto trial registries should
occur within 12 months of the primary completion point of a clinical trial, regardless of whether this is
required by law or not. Please note that publication of trial results in academic journals is not a
substitute for posting summary results onto clinical trial registries. Due to prolonged academic
publication timelines, WHO’s 12 month timeframe means that summary results posting in registries
will typically precede academic publication. According to ICMJE policy, such posting of summary
results will not prejudice subsequent acceptance for publication by an academic journal.

FOI Questions

Q1. Does the university have publicly available policies and/or SOPs that require its staff to follow
global best practices in clinical trial registration and registry entry maintenance?

Q2. Does the university have publicly available policies and/or SOPs that commit it to following
global best practices in clinical trial summary results posting?

Q3. Has the university within the last 12 months made public an internal audit of trial registration
and/or summary results posting performance? If yes, please share a link to that document.

Q4. Does the university has time-specific plans to retrospectively post missing summary results for
university-sponsored clinical trials completed in the past? If yes, please share a link to that
document. If the document is not online, please provide a copy of the document.

Q5. Does the university have a publicly available policy that prohibits its staff from undeclared
‘outcome switching’, HARKing, p-hacking and/or similar (mal)practices when they report clinical trial
outcomes?

4
FURTHER READING

This study by Transparency International, Cochrane, TranspariMED and CRIT sets out global
standards and best practices in clinical trial transparency and their underlying rationale (see
especially the sections on trial registration, summary results posting, and academic publication):
https://docs.wixstatic.com/ugd/01f35d_def0082121a648529220e1d56df4b50a.pdf

This blog addresses some common misperceptions by UK universities:

https://www.transparimed.org/single-post/2017/09/11/Universities-and-clinical-trials-Common-
myths-debunked

This blog discusses the growing pressure on UK universities to post missing summary results:
https://www.transparimed.org/single-post/2018/05/15/Sort-it-out-Pressure-grows-on-UK-
universities-to-report-all-clinical-trial-results

This article explains how many leading U.S. universities have recently improved their clinical trial
governance and Clinicaltrials.gov results posting performance:
https://www.statnews.com/2018/01/09/clinical-trials-reporting-nih/

This blog explains how universities can begin to tackle their EU CTR (EudraCT) registry entries:
https://www.transparimed.org/single-post/2018/06/01/Guest-blog-Two-steps-for-universities-to-
get-all-clinical-trial-results-posted-on-the-EU-registry

This article explains how weak clinical trial governance may threaten universities’ future research
funding:
https://www.insidephilanthropy.com/home/2017/7/28/transparency-clinical-trials-gates-wellcome

For questions and comments, please contact Till Bruckner of TranspariMED.

Email here:
https://www.transparimed.org/about