Q&A: How novelty scores can affect pharma

Scientific breakthroughs are the catalyst to developing new treatments for diseases. From the rapid development of COVID-19 vaccines to a single-dose breast cancer treatment, novel science has been central. After releasing its novelty scores feature, DeSci Labs has enabled the public to compare the novelty scores of more than 250 million articles in the scientific literature. Here, Philipp Koellinger, co-founder and CEO, explains what the novelty scores are, and the effect they can have on the pharmaceutical sector.

First, what are the novelty scores?

The novelty scores are a mathematical measure of how surprising a scientific study is. Currently, the novelty scores look at two different things: content novelty and context novelty.

Content novelty examines a manuscript’s combination of keywords, topics or concepts and compares them to previously published work. If the combination of keywords and topics are already frequently observed across science, a low content novelty score reflects this, representing marginal progress or consolidation of knowledge in a well-defined field or topic. High content novelty scores flag articles that combine keywords, topics, and concepts in unexpected ways.

High content novelty scores flag articles that combine topics and concepts in unexpected ways. Really surprising research is more likely to make a difference, but it also has a higher chance of being wrong. The old saying “extraordinary claims require extraordinary evidence” often applies to research with high content novelty scores.

Context novelty scores are a mathematical measure of how surprising the combination of cited background literature is: Are the authors making connections between different fields of literature not previously made, combining various disciplines and approaches that nobody has thought about before? The most meaningful breakthroughs often happen at the intersections of very different fields.

Surprising research is more likely to make a difference, reflected in a higher correlation between novelty scores and future citations (see here). However, highly surprising research also has a higher chance of being wrong. The old saying “extraordinary claims require extraordinary evidence” often applies to research with high content novelty scores.

What is their practical effect on science?

Novelty scores make it much easier and quicker to filter and find research that is surprising – and, therefore, more likely to make a difference. The traditional peer review process of many journals often puts a high premium on articles that are viewed as novel or surprising. However, the traditional peer review process is not only frustratingly slow, it’s also highly subjective and prone to biases. Now there is an objective measure of novelty available immediately when the research is first shared publicly – even if “only” on a preprint server.

Perhaps novelty scores can play a role in encouraging scientists to take higher risks in the research questions they tackle, to think outside the box, and do unusual things that can lead to breakthroughs.

What effect can novelty scores have on the pharmaceutical/medical sector?

Novelty scores are a quick, effective way for researchers, pharmaceutical companies, and investors to find the most surprising, new research results. Thousands of medical research articles are published every week – it’s almost impossible for anyone to stay on top of what’s new and relevant. In that sense, novelty scores can help to save time and money, and potentially unlock new opportunities that would have been missed or spotted too late.

High-impact journals typically play the role of picking the most promising, novel research. However, extensive empirical evidence shows they are not particularly good at that task. Novelty scores are an alternative, quicker, and more objective way to identify surprising research.

How can the pharmaceutical industry use them?

Measuring surprise objectively allows companies to quickly discover the most novel research around the most pressing diseases and conditions that require treatment. For example, pharmaceutical companies can look for novel findings that are relevant to the treatment of cardiovascular conditions, cancer, chronic illnesses, and other conditions.

Anything else you’d like to add?

Of course, novelty is nothing without rigor and replicability. Just because a study reports surprising, promising results does not mean those results are true. Pharma companies and researchers in the life sciences are all too familiar with the frustration of being unable to replicate some results. Much time and money has been wasted on research agendas and drug development projects based on scientific studies that turned out to be false. Ideally, highly promising research results should be independently replicated much more frequently - before lots of time and money is spent on follow-up work that just assumes those results were correct. Real scientific progress happens when novelty, rigor, and replicability coincide.

That’s why we will launch a crowdfunding mechanism for replication studies on DeSci Publish early next year. This will allow the scientific community and R&D-intensive companies to identify the studies they think should be replicated and provide researchers in the field with the funding required for carrying out those replications. I invite anyone to explore our novelty scores by visiting desci.com and experiment with the fields that interest you.