Written by Peter Keay

Full-stack dApp developer || Host, Crypto Philosophy Podcast (bitgenste.in) || Host, Everything EOS Dev Series || Co-host, Inside IOST (leobi.io/st) || Writer.
Find me on:
October 19 2018

Bitgenstein's Table Interview with Manuel Martin Marquez of Orvium & CERN

Fixing Scientific & Medical Publishing with Blockchain

Podcast-disclaimer

 


Academic publishing in scientific and medical fields is central to much of society.

You might not think about it often, but legislation, corporate policy, litigation, ethics, research direction, guidebooks and textbooks of all kinds all rest on the foundation of peer-reviewed academic studies.

But the ancient peer-review system is opaque, clunky and perhaps even prone to error. How can transparent, decentralized networks revolutionize the way we as humanity know things?

I’m Peter Keay, and this is Bitgenstein’s Table.

Manuel MartinI’m very excited to have Manuel Martin Marquez – co-founder of Orvium and data streaming project leader at CERN, the European Organization for Nuclear Research, famous for the Large Hadron Collider – on the podcast today. Manuel is an expert on both blockchain and scientific publishing.

Before we get started, I want to make it clear this podcast is not sponsored in any way by Orvium or Manuel or anyone else. I have received no compensation of any kind and only scheduled this interview because I personally find that the problems Manuel is solving are critical for our future.

And of course, although I’m the Senior Architect and Director of Globalization at ICO Alert, and Manuel is the Co-Founder of Orvium and Data Streaming Project Leader at CERN, the opinions we express are our own and not the opinions of the companies or organizations we work with. None of this is specific professional financial or investment or other advice of any kind. This podcast is purely for informational and entertainment purposes. It helped me learn and enjoy life, and I hope it does the same for you.


For any who don’t know, CERN is the European Organization for Nuclear Research, most famous in recent years for its Large Hadron Collider: the largest particle collider and indeed the largest machine in the entire world. You’ll often hear Manuel refer to the Large Hadron Collider as the LHC. The collider took ten years to build, and its circular tunnel, which is actually repurposed from an older collider, is 27 kilometers (or 17 miles) around its circumference. Underground. And filled with a massive particle accelerator. In places, it’s over 175 meters (or 500 feet) deep.

As we chatted before the podcast, I remarked how I remember hearing that maybe the collider would create a black hole that would end the earth. Manuel was quick to reassure me that the kinds of collisions performed in the Large Hadron Collider happen all the time in nature. So unless we think cosmic rays are just going to create black holes in the sky, we don’t have anything to fear from CERN’s experiments.

A lot of fascinating scientific data has come out of CERN over the past decade and Manuel has been a vital part of the team managing that data.Photo of CERN collider

CERN’s biggest collider is seriously large.

As we chatted before the podcast, I remarked how I remember hearing that maybe the collider would create a black hole that would end the earth. Manuel was quick to reassure me that the kinds of collisions performed in the Large Hadron Collider happen all the time in nature. So unless we think cosmic rays are just going to create black holes in the sky, we don’t have anything to fear from CERN’s experiments.

A lot of fascinating scientific data has come out of CERN over the past decade and Manuel has been a vital part of the team managing that data.


Peter: So excited to have you on the podcast, Manuel, and as I’m generally in love with all things nerd, from space exploration into tabletop RPGs to linguistics to philosophy (obviously) to quantum physics. You know, anything. So I have to ask first: What have you been doing at CERN for all these years?

Manuel: So yeah, I’m kind of nerdy, as you said, so this was my first motivation when I came to CERN 12 years ago as a student. I came here because I wanted to know, in the first person, what such a big institution, one of the biggest research labs, was doing. What kind of things they were doing. So I’m not a physicist, I’m an engineer. So coming to a very pure physics place, that was a challenge for myself and a challenge that I wanted to embrace.

CERN accelerator complex diagramSo at the time, the first task I was given was mainly regulating the control systems. So we wanted to take over all the responsibilities regarding the data management for the control system. And that was a very exciting time because the LHC was just launched in 2008. About a year after that, I arrived at CERN. So we were pretty busy at the time.

I had the chance to collaborate on such an amazing experiment that was extremely motivating at the time. After that, I kept for a while the same task, and seven years ago I was given the responsibility to understand, or to assess, how all these new technologies that were coming at that time, integrated with big data, how they can be used or could be used in our environment.

So I was in big data at that time, for twenty years. This was a new branch of technology, and I wanted to understand how those technologies can make us more efficient in our database operations.

I did actually understand the technology, and I made the machine. So nowadays all this big data, machine learning techniques that were coming in at that time are widely used in the institutions for the control system to physics analysis, and so on and so forth. That was quite a good achievement as well from my side.

Peter: So you began to work with managing data there at CERN, including data from the Large Hadron Collider. I assume that’s what eventually lead you into blockchain, right? It’s kind of a branch of data science, databases, data management.

Manuel: Like a year ago, a little bit more than a year ago, I was given another responsibility. I was still keeping the previous one, but I was given another responsibility which is to understand how blockchain can be used in our institution. And this is exactly what I’m doing now, how we can utilize Blockchain in our environment. But actually, CERN, at the end of the day is a wide collaboration with people and institutions from more than 110 different nationalities working together to achieve the same goals. And blockchain is kind of the same. It’s like how to put many different people to work together to achieve the same goal.

Subscribe here for instant podcast notifications

Peter: Fascinating. So, you’re focused on scientific publishing. Scientific, academic, peer-reviewed publication is a large industry. Tens of billions of dollars at least.

I worked in the non-profit sector for a good amount of time myself, dealing with medical ethics, and it’s amazing the number of articles you can find that have problems with their methodology – or their conclusions are simply wrong and don’t even follow from the actual study.

There’s often a profit motive you can find, whether direct kickbacks, or maybe avoiding legal liability, or maybe undisclosed conflicts of interests that the authors have. This leads to unnecessary and harmful medications and even surgeries continuing because they make somebody’s wallet fatter.

Tobacco companies paid people to publish studies finding that cigarettes weren’t so harmful decades ago, and often there wasn’t anyone on the other side incentivized to fork up the millions of dollars required to counter all of those studies with legitimate studies. That took decades for us to get past, and it still continues with the tobacco industry today, to an extent, and to a greater extent with many other, more controversial topics.

So I’m sure that, in the scientific field, you also saw problems with scientific publishing. What were the first problems you saw with the academic peer review process?

Manuel: So you’re totally right. At CERN, a scientific institution, one of the biggest in scientific collaboration worldwide, something we have to do is to publish our papers, our results. I used to work for the IT department at CERN, and I worked pretty close by to the office where Tim Berners-Lee was working at the time that he was working at CERN and creating the Web.

Tim Berners LeePeter: Tim Berners Lee, basically the man who invented the World Wide Web. Right?

Manuel: What was his purpose in creating the World Wide Web? To make possible communication between scientists: How do they can share their results from one side of the world to the other. And actually, the Web has changed everything. Changed almost everything that we know. It changed our way to communicate, our way to work, to live, everything that we know. It’s a different world. But, what it didn’t change in the wider scope is scientific publication itself. So, scientists already, they are communicating a different way now, but the results still are going through these publishers. So, that was really the motivation at that time. That was early 80s or early 90s. But today we are facing exactly the same problem, no matter whether this is at CERN or some small university.

So myself, I have to struggle, over that time, to publish some papers, and then I have to wait two years before they give me an answer: “You have to change this part,” or “you have to change that part before it can be published.” And then, after those two years, they told me, “If you change that, it will be published.” You make the change, and then you have to wait a little bit more than six months because they don’t have any slot in the next issue of the journal. Which is something kind of stupid, because at the end of the day they can publish the paper today. Why do I have to wait for a slot? Yeah. That makes no sense at all.

I mean, the business model itself makes no sense at all nowadays. All this new technologies that we have been embracing for the last decades.

So, that was my motivation. Personally, this is something that I faced by myself. The rest of the team, as well. This is something that we face in the lab, as well, and this is why we decided to get to finding out a solution for this problem.

Peter: I haven’t ever published myself, but in both humanities fields and in scientific fields, it’s my understanding that the peer review system might be better than anything else out there currently, but it’s hard to investigate, hard to probe and try to find things wrong with, and easy to rig because the viewers are anonymous and their reviews aren’t published or analyzed or anything. In fact, the editor of any given journal can arguably outright block articles whose conclusions he or she doesn’t like. Or whose conclusions go against the editor’s financial or legal interests. And one way to do that is the very peer review process designed to protect the system.

This peer review system hides all kinds of information, like:

  • the identity of the reviewers,
  • the content of their reviews,
  • the selection process for reviewers,
  • communication between the reviewers, and
  • communication between the reviewers and the editor.

All of that’s hidden information. Who knows what kickbacks the peer reviewers might be getting? Either explicitly or implicitly. I mean, we’re just trusting them to be honest, and I respect scientists and doctors, but they’re still human, you know?

They’ve got flaws. They have debts, often massive debts. Perhaps they’re looking for government grants or corporate funding for projects they care about – and the committees that assign those grants and funding are pretty selective. So there are clear pressures for them to publish things that will advance their own goals. Perhaps the peer review system shouldn’t be so secretive.

Digital Science Doodle

A Digital Science Doodle by Dana Cairns

I have doctor friends who have their studies rejected for absurd reasons, and there’s just no accountability when that happens. There’s no auditing of the system.

It happens and you have to shrug and say, “Oh, well.”

Manuel: Yeah, I mean you are totally right on the point that you describe. I think that this is one of the major problems of the current system, the lack of transparency. It’s a very opaque process.

There is a good part of that, we have to be honest with this. So they wanted to try to keep what is called in science “double-blind.” So if I don’t know you, I can be more, how to say, unbiased. I can be more fair in my review.

But at the same time, this is a small world. So what is clear – and this is known by the publishers – that this double-blind, where you don’t know the reviewers and the reviewers don’t know the authors, does not work at all. And this is because, mainly, when you review something, it’s something that is very close to the field that you are working with.

So actually, it’s pretty easy for you, based on the way that the author is writing and things, to understand who is the writer. And if you know the authors, you are already biased, and the system breaks.

Peter: So, while the system is designed for peer reviewers to not know the identity of the authors of the articles they are reviewing, in practice, because these fields are small groups of people, relatively speaking, reviewers can generally tell who the authors are. And that defeats the entire purpose.

Manuel: Yeah, I mean, in some situations, a part of which is the important situation that you mentioned,  as you said, they are competing for grants and many other things. So I’m asked for free to review a paper that actually is working in kind of the same field, or in a very close field. So it may happen that this is something that I’m working on, or this is something I’m waiting on to be published. So I don’t have any pressure on how long should I take for my peer review. Should I not do it today? I can just wait for the end, and keep it there for a while, until my paper is published. And this is a situation that is happening every day.

Actually, this a very old problem. This is not something new.

And why didn’t it happen to be solved before? We strongly believe – meaning myself and the rest of the Orvium team – that this was mainly because there was not a way do that. Nowadays, with blockchain, there is a way to do that. There is a way to create a system that is not owned by anyone, or that is owned by everyone, actually, as you want to see it.

And this system is capable: to do whatever the publishers were doing in a more efficient manner, in a more transparent manner, in a manner that actually empowers scientists with the results that they are doing. So it can change everything.

Peter: A journal article fairly recently, in the past few years, dropped in the United States and hit a number of media outlets like a bombshell. It was written by the editor, former editor, a long-time editor, of The Lancet, the very esteemed British medical journal.

PLOS medicine screenshot

I said “in the past few years.” Apparently I meant 2005.

And it said that “half of all published journal studies are false.”

This was an estimate, obviously. You and I and listeners could easily fight over exact numbers. But it’s clear that, whatever the percentage, the creation of false knowledge, at least in the medical industry, happens regularly in academic publications. False studies, of course, once they’re out there, will get cited by other studies, creating a kind of snowball effect.

In the medical cases that I mentioned before, there were many meta-analysis articles. In some cases, these articles cited dozens of other articles by the same authors as the authors of the meta-analyses, creating feedback loops.

And these feedback loops just create and then reinforce new “knowledge,” and that’s in air quotes, bad knowledge. And that stuff is really difficult to root out. As someone from Europe, you would probably be amazed at some of the destructive medical practices that persist over here in the United States, and in a few other countries in the world, despite those practices being thoroughly debunked by medical science.

And then, of course, the media doesn’t make things better. It often summarizes up legitimate studies with headlines that are basically the opposite of the studies’ actual conclusions and that just furthers the confusion. How can we work on undoing this damage of potential bad knowledge reinforced by years of bad studies?

Manuel: Actually, as you said, I mean, we can argue on the numbers. So I wouldn’t say that a large number of, or a large percentage, of the articles are totally fake, or biased, or driven by private interests or whatever. So I’m a believer, I think that scientists are doing a good job.

However, don’t misunderstand me: I’m not saying that this doesn’t exist. I’m just saying that I try to believe that the number is not 50%. It should be lower. So, however, this is a problem. This is a problem in every single field of science, not only in the medical one. So, as I said before, this is how we end up basing our next study on this snowball that you mentioned.

Pete: Let me ask you, what specific measures are you pursuing to fix the problems in academic publishing?

Manuel: So, the problem is that, today, we have only access to the final result of our research. So researchers have done work for years, and then they write it down. They summarize everything in two, thee, ten pages sometimes, that’s it. You don’t have any other tools today.

Do you have access to the data that helped generate this article? No.

Do you have access to who did the reviews of this paper? No.

Do you have access to the reasons why the publisher just published this paper and not others that contradict that one? No.

You don’t have this information at all.

What blockchain brings – the transparency that blockchain brings can change the model. So if the peer review’s public, the whole community can evaluate whether this review is valid or not. Then they also have to make decisions, whether they should accept this review or not. But this is actually – everybody can put their eyes on this, so that can change the model.

Today, when you read a paper, you read just a result, you read a history. You read just a few pages that summarize maybe ages, years of work. So the idea with Orvium is totally different. You, as a reader, you can see the whole history of the paper. What time it was born, what was the first version, how it was modified, who did the peer review, what was the content of the peer review, what was modified or not.

But as well, that is something we didn’t mention here, is the data that’s being used for that. Today, you have just access to the last version of the paper, to the history, to the final history. But with Orvium, you have the whole set of tools to determine by yourself whether the paper is a valid one or not.

And if you’re not somehow in the field, you cannot do that, then you will see the result of the whole community: How the whole community has reacted to the paper, because they have all the data, they have all the results, they have the whole life-cycle of the paper. And this is where we wanted to take science.

I mean, if you asked any scientist, they are not afraid to make things transparent. Because actually this is the only way that we have to do science, is to base it in previous results. So if we’re basing our next science on previous results that are not consolidated, we are in trouble. And this is something that is happening. So we have to be sure that the work that I am starting today is based on work that is already validated. And giving the tools to validate to this whole community, to not only two or three reviewers, this is key to make that possible.

And then something that Orvium integrated, is that we have a lot of experience in big data and machine learning itself. We try to find some hints that will determine how good is the review that we’ve been doing. For instance, there is a problem with references. At the end of the day, references is one of the biggest factors to know how important a paper is. So there is this kind of loop with references. You are referencing the paper of a colleague because he would reference [you back].

And that is pretty easy to find it out. I’m not saying that they are not valid, as I said: the weight of this reference should be less than the other, and that will need to be taken into consideration.

Pete: So it’s not just a struggle to publish papers, it’s also a struggle once the papers are published for readers to see deeper and see what’s actually going on. You want to make the review process more transparent. What about the data itself, behind the articles?

Manuel: What we wanted to do, or wanted to allow the people to do, is to check the data that we have generated instead. Today everything, every single study, no matter the field, is based in data. And then when you get the paper, you get the results in tables about the data, but you don’t have access to the data itself. At Orvium, we wanted to say, you have the chance to make this data available to the rest of the people. So if you really want the people to validate your article, make it easier for them.

And then this kind of a very easy system that would allow the people to understand how good the paper is, or how fair and how accurate is the paper itself.

And then there is an advantage to sharing the data as well. Today, most of – a majority in some fields – the research, as I said, is based in data. But the data itself is very costly to produce, and it’s being produced in the same manner in many different things, because there is no communication going between the different people to say, “This dataset is already created, why should we have to create it again, when it costs millions?”

So I can just base my study in data that was already created by some other [study]. With that, you will allow this communication, not double the spending and the cost and the effort. But as well, the most important aspect here, is that you have access to every single piece of the research. That will allow you to understand whether what you are reading is correct or not.

Pete: Of course many people aren’t really in a position to evaluate very specialized articles themselves. They’ll want to see what the community says. They’ll want to see what a number of experts say.

Manuel: And as I said before, it maybe is not you yourself who has to make the decision, because you are not prepared, you’re just a reader in that field. But you will see all the thoughts of the community. Right now, the importance of a paper is based on the impact factor, and the impact factor is based on the references. So there are many ways to get these references.

Right now, there is on the table – and this is not Orvium-specific – there is on the table, a discussion, a very deep discussion, on how these kinds of metrics should be calculated and how they have to be defined.

At Orvium, we are part of these discussions because we strongly believe that as you change model, the metrics should change as well.

I didn’t ask who will change the metrics. Because we can have a vision today, but the metrics should change over time.

So what we wanted to do at Orvium is to have kind of open every single piece of data, every single bit of the research, so the metrics can be calculated by the community itself. Because everything is transparent, everything is open, they can calculate the metric in a very easy manner.

I think the core aspect of Orvium is that possibility to give tools to the people, tools that don’t exist today, so I can determine whether the results presented to me are right or not.

Pete: I love studying cognitive biases. You know, those problems that we all experience with our thinking because of the mental shortcuts we have to use to live. What is the biggest tip you would give me if I’m looking to improve my critical thinking? Maybe I need to improve my judgment of scientific studies, for example. What would you say I should do?

Manuel: So I don’t have a tip that can work for you, but I can tell you what worked for me at the time. So the first thing is to be humble, to understand that we don’t know anything about many different things.

Sometimes we are afraid to say “I don’t know.” You have to face some people, and they ask you something, and then you have to say “I don’t know. You can find someone who knows better about this thing, but this is not my topic.” But we tend to understand, we tend to know, about everything. But this is not it – you have to be humble.

But what I wanted to say here is that, I don’t have a tip for you, but I can tell you what worked for me. And what worked for me was that when I arrived at CERN, more than 12 years ago, something that I found out is that CERN is wide collaboration. So you can find people from all over the world, with many different backgrounds. From engineering to physicists, mathematicians, equipment experts, and many different things. At the end of the day, you have lunch with them every single day. And then you have two options: To listen to them and learn, or to just think by yourself that you know all the answers.

So you need to be humble, sit down with them, and then embrace whatever is different to you. Because that would bring some benefit to your thinking. That will open your mind to many different ways of thinking, to many different ways to improve yourself, to many different ways to understand life and science in that way. Which way could be better than the one that you have in your mind.

Pete: The first major philosopher in the Western traditions, Socrates, reportedly said, “The only thing I know is that I know nothing.”

Meditations bookMaybe we wouldn’t go that far, but it’s like the stoic emperor Marcus Aurelius later said – he was a philosopher in his own right, he wrote a book that’s still with us called Meditations – “I have learned that a man can be both resolute and yielding.” It’s kind of that same wisdom that we hear today when we hear “strong opinions weakly held.”

And in fact the philosopher my podcast is named after, Ludwig Wittgenstein, was so willing to change his views, and underwent such a dramatic shift in his views as his life went on, that we have to split him into two thinkers, basically: “Early Wittgenstein” and “Later Wittgenstein.”

It does us good to remember that we really could be wrong. Even on fundamental things. Isaac Newton’s theories, as far as gravity is concerned, seemed correct, and they matched up with observations people made about the world. But they were completely wrong. Newton was completely wrong about gravity. Yet for centuries everyone just took Newtonian gravity for granted.

And we could be completely wrong about gravity. There’s still things, obviously, about gravity we don’t understand. And we could be completely wrong about its fundamental nature. We could be completely wrong about the nature of a number of other things. I’m not saying we are. I’m not saying that we don’t know anything. But that humility you describe, that willingness to be wrong, is always a crucial part of progress. Without humility, there can’t be any progress.

Manuel: Alright, I want to close this interview with that, because in that line I have two stories that I think worth mentioning here.

So the first one is that in 2012, when CERN just made the announcement that we discovered something that was very similar to the Higgs boson. We didn’t say that it was the Higgs boson. We said that it was something similar to the Higgs boson, corresponded to the properties that we were expecting to see for a Higgs boson. So it was a groundbreaking thing.

The people at CERN were kind of nervous, because there was a press conference. We didn’t know – no one at CERN knew – what was going on during this press conference, because we wanted to keep this double blind between the two experiments that were doing the numbers. So at the end of the day, what we were expecting to see is that those numbers matched.

But actually that was very funny, because I worked very close by to Alvaro De Rujula, one of the most reputable physicists here at CERN – who happen to be Spanish as well, as myself. I sit down with him and I say “Well, you know we discovered the Higgs boson, so what do you think, are you glad?” And he says, “You know, I do prefer that it was not the Higgs boson. Because if it is a Higgs boson, it means that we are on the right track. And this is not exciting. But if this is not the Higgs boson, it means that we are not on the right track, so myself” – he’s like 60 years old – “I’m done, and we have a mystery for the next generation, in that sense.” I think that was a humble opinion, a very good one.

Alvaro De Rujula veteran physicist at CERNAlvaro De Rujula, veteran physicist at CERN

And then there is another history. In two thousand and – I don’t recall the year – but at a moment in time we announced that we found particles traveling faster than the speed of light. That we observed particles traveling faster than the speed of light.

That was something that we didn’t understand at all. I was working on the team that made the measurements at the time of the timing. So measuring how fast they were traveling, the particles. The neutrinos. So we were pretty humble as well, to say “we observed that the neutrinos are traveling from here, in Geneva, to Gran Sasso in Italy, faster than the speed of light. We did not understand that, because if that is happening it’s breaking everything we know about physics. It will break laws, it will break everything. Relativity and so on and so forth.

So [we said,] “We will make available every single piece of data that we have, and then it’s up to the community to try to find out what is wrong. Because there must be something wrong.” Imagine such a large institution, reputable institution as CERN saying “We observed this thing that could change everything we know, but there might be something wrong.”

But of course, as you said, that hit the news and then in news it said “CERN observed particles traveling faster than the speed of light.” Which is not the message that we passed.

So at the end of the day, it happened to be that we made a mistake in how we were calculating the timing. That was kind of embarrassing, but this is how science should move, and how we have to be humble, to say, “Maybe we are at the top of field, maybe we are at the top of our knowledge, but I’m pretty sure that there might be someone who is higher than myself, or better, so please check whatever I did, because maybe I did something wrong.”

Pete: So what does your general timeline look like for the Orvium project? Are there any exciting announcements or developments coming soon?

Orvium beta platformOrvium’s beta platform is live

Manuel: Actually, we have a very big step this week. We are launching our beta system – now it’s publicly available. It took some time because we were validating the ideas, validating the platform, together with the rest of the team and our partners.

But as of yesterday we have it public, so this is open to anyone that wanted to see the majority of our ideas already implemented. So we are ahead of our road map. This is something that makes me really proud, for myself and the rest of the team. And something that is good as well is that this platform has been developed together with us partnering with Amazon, so they are doing a lot of work in order to support that, support from a technology aspect, in every single aspect that they can. So this is a very valuable partnership as well.

So just go to our website, you have the link. Test it, and we are pretty open and working 24/7 nowadays. So if you have any feedback, or you wanted to give us any comments, or you have in mind any improvements, do not hesitate to contact us through any of the channels – Telegram, for instance – because we would be more than glad to hear from you.

Pete: Fantastic. It’s been great having you on. How can people best get in touch with you to learn more about what you’re doing?

Manuel: So my recommendation is to visit our website, orvium.io. There you will find all the information about us. But I wanted to say here: please go to Telegram. Telegram is a great chance to get with us. So you will find myself or my work team online almost 24/7. We have non-stop working hours these days.

Any questions you have, or just give us any feedback, because we would be very glad to hear it. Most probably you will get an answer very very soon.


Next week we’ll be discussing Universal Basic Income and variants, including “Universal Resource Inheritance,” any how whether people love it or hate it doesn’t depend on their political standing.

So make sure you tune in, and I’ll see you all next week on Bitgenstein’s Table, The Crypto Philosophy Podcast.

Topics: Blockchain, Data, Podcast, Blockchain Technology, Podcasting, Decentralized, Bitgenstein