A group of students sat around a table writing notes and smiling.

How Open Research practices can increase impact

Intro

There are a range of open research practices that you can apply throughout the research process from the earliest stages to the publication of outputs. Look for more details in the other pages.

Jump to each section of the page on:

How making research plans, methods, data and papers open increases impact

Two male figures working at their laptops making notes on the piece of paper

Pre-registration and registered reports

There are advantages to depositing a study design in a repository—and even submitting it for peer review —before the study has even begun gathering data. 'Pre-registration' is a way of building high levels of research integrity into the research process through identifying in advance, for example, what theory will be tested or if the study will be purely inductive. Peer review feedback from experts in the field can ensure the best possible design and methodology.

Registered Reports are research designs that have been peer reviewed by a journal and published. If the researcher follows the plan, then there is a promise to publish the final paper, whatever the outcome, even if the outcomes fail to support existing theories or even the pre-study hypotheses of the researchers themselves. Much of this kind of data never finds its way to publication through the usual channels because of this, but such data is essential to fully understand any phenomenon being investigated.

A data analyst at a PC in a shared workspace

Open Data

There are many advantages and an increasing number of demands for researchers to make their data open access and available (UKRI website). Other researchers can build on your work or work to reproduce your findings.

Funders often insist that not only papers based on funded research, but the data sets themselves are made available for reuse.

A group of students standing and working on a task together, while one is plugging something into a socket.

Research papers

Even before you submit your paper for peer review, consider uploading it to a ‘preprint server’ where it can be read and acknowledged and obtain feedback. You immediately claim the subject of your study as an area of your expertise and initiate discussions with others in your discipline. More details in the pre-print section.

It’s vital that you not only follow UWL's open access policy, and your published papers' research funder, but you take all possible open access routes to maximise the availability of your findings to the research community and the public at large.

How can I use Open Research practices to increase impact?

  • Share your methods and materials

    Explore the potential of online workflow and collaboration tools, such as Electronic Lab Notebooks and platforms such as Zooniverse, which can enable you to share your methods and materials, develop new research possibilities, and allow stakeholders – including those from outside UWL – to contribute to the design and implementation of research.

  • Pre-register your research plans

    Pre-register your hypotheses, study design and materials using a public registration platform such as the Open Science Framework or consider publishing your study as a registered report - visit the Center for Open Science's journal article which explains how methods and proposed analyses are peer-reviewed and the results accepted for publication prior to research being conducted.

  • Make it FAIR

    If you collect or create primary data, make them FAIR (Findable, Accessible, Interoperable and Re-usable) by depositing them in a data repository under an open licence, in usable formats and with appropriate documentation and metadata, and cite the data using the DOI or other unique permanent identifier in your publications. At the moment, UWL doesn’t have data depositing facilities, but we can advise on where to deposit. Don’t forget to make a Data Management Plan to help you manage your data creation and storage processes.

  • Promote your data or software as a research resource

    If you create a dataset or software that has the potential to be re-used, publish a peer-reviewed data paper or a software paper to describe and contextualise its value.

  • Use meaningful and effective keywords in your publications

    Make sure your abstract and title are keyword-rich. The impact of your paper relates to the frequency and reliability of it being located by researchers in the first place through a relevant keyword search in a database. A clear title and well-structured elucidating abstract can also attract the eyes of those researchers who will be most likely to adapt, cite and apply your work.

  • Make your research publications Open Access

    This can be done either via the publisher (Gold Open Access) or through the University’s repository (Green Open Access), under a suitable open copyright licence. A whole range of formats can be published by Open Access means, including monographs and book chapters. UWL has a number of agreements with publishers allowing papers to be published via the gold route without incurring fees.

  • Use a preprint server or open journal submission system

    Use a preprint server or open journal submission system (where the preprint is published by the journal prior to and during peer review) to get your research findings into the public domain as soon as possible; the is particularly important if your discipline is fast moving and crowded. If you are using a preprint server, make sure that your journals of choice allow posting of preprints, and check with the library Open Research team.

  • Get an ORCID ID

    ORCID is a way of fixing your identity to all of the outputs you will create throughout your career. There may be more than one researcher with your name out there, even if your name isn’t a common one. It’s a simple process to create your ID.

  • Learn how to summarise the findings of your research

    Learn how to summarise the findings of your research for ‘lay’ audiences and social media use. It’s more difficult to summarise your findings in short simple sentences than you may think.

How do I assess the research literature's impact?

How can we determine the impact of a research output? What do we mean when we say Open Research increases impact? To properly understand the impact and significance of outputs identified in a literature review also helps us understand the value of Open Research practices, as well as the future impact of our own research. It's not just about citations.

Traditionally citation count and journal impact factor (JIF) are taken as absolute measures of significance, and greater openness and dissemination of plans and outputs will likely increase citations. Sometimes, however, papers are not just cited based on the quality and significance of the study but out of habit or for trivial reasons (a common definition, for instance). And do citation count and JIF really measure all of 'impact' and 'knowledge exchange'? If research is discussed in the public forum or used to inform policy outside the academy, this may not register in the form of citations.

Man with glasses writing and looking at a piece of paper

Bibliometrics and Altmetrics – a complementary duo

Bibliometrics, the statistical measures used to assess the frequency of citation of research papers, remain key tools in assessing research significance and value.

However, there is growing awareness of our collective need to improve the ways in which research outputs are evaluated by funding agencies, academic institutions, and other parties.

Our research outputs are diverse and diffuse, including research articles reporting new knowledge, data, reagents, software, intellectual property, and creative outputs. Funding agencies, institutions that employ researchers, and researchers themselves, all have a desire, and need, to assess the quality and impact of our outputs. Therefore, it is imperative that output is measured accurately and evaluated wisely.

In judging what is the ‘best’ research, some have relied on bibliometrics only. In part, this is why the Declaration on Research Assessment (DORA) was created and is now supported by the majority of funders including UK Research and Innovation (UKRI).

Two people looking at sheet of paper with calculator

Metrics that rely purely on citations are likely to be inaccurate as a measure of overall impact for several reasons. Citations are not measures of significance per se. A citation may be to the literature review of the paper, for instance, or to a basic definition included in the paper. Citations can become ‘habitual’; the same papers cited repeatedly when setting out the basic concepts or developmental stages of a particular subject while more recent and insightful papers aren’t included. Qualitative studies aren’t usually heavily cited, mainly due to the the great variation in the foci of qualitative research; impactful studies may not necessarily be closely relevant to large numbers of future studies, while still breaking new ground and introducing new ways of understanding. The low level of citations for qualitative studies gives a misleading impression of quality and significance.

a woman working on her laptop at home

Alternative Metrics, or ‘Altmetrics’, identify additional meaningful measures of ‘impact’, especially to be found in the contemporary digital environment and in environments outside strictly academic contexts; the environments in which practitioners and policymakers operate, for example. In a research environment in which ‘Knowledge transfer’ to, and ‘impact’ in, the wider community is required by public and private funders alike, this kind of analysis is essential.

Altmetrics also tend to accumulate faster than traditional citations, and can be tracked for not just articles, but books, chapters, conference papers, datasets, software and presentations.

Bibliometrics retain their key value if properly understood and evaluated, but they are best used in conjunction with the new Altmetric tools to provide a wider context of research impact.

The sections below give more detail on the nature and structure of Bibliometrics and Altmetrics, and how to use Altmetric Explorer to investigate the wider impact of research outputs. 

  • Bibliometrics

    Bibliometrics are quantitative measures of academic impact of research.

    Advantages

    • Easy to find out who are the most influential researchers and journals
    • Simple to understand for non-subject experts
    • Reasonably objective compared to peer review

    Disadvantages

    • Bibliographic databases are limited in scope
    • Number of citations can be influenced by subject area and the journal the paper is published in, rather than the quality and significance of the research
    • Book chapters poorly tracked
    • Citations need to be considered in context
    • Potential for ‘gaming’.

    Metrics are either at:

    • Article level (eg citation count)
    • Author level (eg publication counts, h-index, i10 index)
    • Journal level (eg Journal Impact Factor)

    Citation: measures a paper’s influence based on how many other papers have cited it

    • How to measure: Google Scholar, Scopus

    H-index: a widely used author-level metric:

    • Accounts for productivity (no. of papers) and impact (no. of citations)
    • Definition: H-index = x number of scholar’s papers cited x number of times
    • How to measure: Google Scholar, Scopus

    Journal Impact Factor: Refers to average citation counts of papers published in a journal over the last two years:

    • Tends to be higher in medical journals
    • Can often be found on the journal's or publisher's websites.
    • Regarded more and more as of dubious value (sometimes 'gamed' by publishers. Can research be measured by where it's published?)
    • Scopus contains several tools that try to improve on standard JIF: CiteScore (Elsevier website); SCImago Journal & Country Rank and Source Normalised Impact Per Paper. The latter accounts for field-specific differences in citation practices by comparing each journal’s citations per publication with the citation potential of its field.

  • Altmetrics

    Altmetrics are the study of scholarly impact using measures based on activity on digital communication tools like:

    • social media,
    • blogs and
    • news websites.

    They have emerged as an additional tool that takes into account the wider range of ways research outputs can be reviewed across the internet (eg twitter comments, blog mentions, news reports).

    How do Altmetrics work?

    Altmetrics are data that can explain both the volume and nature of attention that research receives online.

    Altmetrics weight mentions, so that, for example, a policy document mention provides outputs with more points than a single tweet (which is considered a lower value level mention).

    Outputs are then ranked and benchmarked so it is possible to determine which researchers have received the most attention for their work, and how they compare based on articles published in the same journal, for example.

    Plum Metrics

    Plum Metrics can be accessed via Scopus.

    Once you've identified the article, click 'view all metrics'. Scroll down to find the basic Plum metrics. Click 'view Plum details' for more.

  • How to use Altmetric Explorer

    The Altmetric Explorer (login using your UWL username and password) can identify specific mentions and general attention around your own and others’ work and explore enhanced visualisations of altmetrics data. Click the link and log in using your UWL credentials.

    • The search bar at the top of the page can be used to search publications from UWL (‘my institution’), or from all institutions (‘full database’).
    • You can search by title, keyword or author.
    • Results can be sorted in numerous ways: by attention score, by attention source type, and by publication date.
    • Research from UWL has a blue tick next to it, so when searching for keywords across all institutions, you can easily identify work that has come from UWL.

    Results Analysis Panel

    The results analysis panel offers various visualisations. If you click ‘analyze these results’, you can see visualisations for the dataset (ie UWL publications): summary tab (mentions over time), highlights tab (for recent or notable attention), demographics tab (Twitter/X reach), mentions tab (reviewing mentions by source and over time). The highlights tab is particularly useful for seeing what UWL research is popular at the moment.

    Clicking on a particular article will display an article summary page, in which you can explore specific mentions of an article by clicking through the various tabs. For example, clicking the ‘blogs’ tab shows links to the blogs in which this particular article has been mentioned.

    Other Features

    • If you sign up for an account, you can set search alerts, so when someone mentions your work, you’ll receive an email about it
    • Use the institution icon on the left-hand side menu to browse publications by UWL author and by UWL Schools.

    Watch our Altmetric Explorer video on how to use Altmetric Explorer and how it can help you assess the wider impact of research.

Get in touch

A group of students in discussion

Open Research team

Get in touch with Dr Marc Forster and Eilish Purton from the Open Research team for help and advice at open.research@uwl.ac.uk.

Library team

In-person: Visit the Help Zone, ground floor at our Ealing & Reading sites. Find out the library opening hours.

Email us: library@uwl.ac.uk

Telephone:  Ealing: 020 8231 2405  /  Reading: 020 8209 4434  (Mon-Fri 9am-5pm)

Need help? Chat with us.