Categoría: Blog posts

The seminar discussed several topics related to the goals and roadmaps to Open Science, Open Knowledge and Career Assessment. Transitioning to open science is proposed mainly because of the widely known issues we can encounter nowadays in the ways scientific knowledge is spread and the criteria under which researchers and scientists are evaluated to build up a successful career. Probably the most visible issue regarding the dissemination and access to scientific knowledge comes from the conventional model of publication in academic journals. Usually, research articles and papers are published behind paywalls, which limits the wide access of scientific knowledge. Moreover, a publication fee might also be requested to authors in order to proceed with the publication after peer-reviewed and accepted. Peer-review is usually done by other researchers without recompense, and in some cases even the editorial board of a journal is not recompensed either. This is a publishing workflow which in my opinion is economically inefficient (for science): Unpaid work and fees are invested into the publication of a paper (which is in principle done for the benefit of humanity’s scientific knowledge), but then that knowledge is not available for anyone. Open access publishing might be considered a more efficient alternative, but like the conventional model, it still relies on the valuation of a journal by simplistic indicators such as the impact factor. The wide acceptance that such indicators of a journal are strongly correlated with the quality of any paper published in it, entails a series of issues regarding the assessment of researchers that pursue an academic career. I believe a paper is more likely to be of high quality if it is published in a top ranked journal, than if published in a low ranked one. However this cannot be taken as a strong or even reasonable evidence about the quality of a paper. Thus, universities and regulating institutions judging the quality of work through indicators such as the impact factor of a journal are probably not deciding objectively in most cases. Moreover, university departments and regulating institutions sometimes score the work of a researcher by using formulas involving the impact factor of the publishing journals. I personally find it hard to believe those quite arbitrary scoring systems lead to fair conclusions strongly correlated to the quality of work.

Besides research, other aspects must be considered to assess careers in academia, such as e.g., education, leadership and impact. It has been suggested to be academically more efficient if instead of weighting all of these aspects in the career assessment of one individual, different career paths are considered based on the strengths of the individual. Thus, specific evaluation criteria are followed based on the path followed by the pursuant of an academic career.

I personally believe that solving the aforementioned issues is very difficult for two main reasons: i) There are no widely known indicators that can objectively assess the quality of work, and ii) there is no widely accepted way to certify a paper has been subject to rigorous and high quality peer-reviews, other than publishing in journals with some degree of prestige.

Here’s a bunch of crazy ideas: what if we develop digital platforms to help address both issues above?

Imagine changing the concept of paper or scientific publication, and let’s call it «e-paper» for further reference below. I will focus on features of technical papers which is the area I am experienced with, but the concept can probably be generalized to any field. Instead of sticking to a pdf document where static text, equations and figures are included, let’s have an online open platform where interactive content of a research work can be included. We can think about this as an arXiv on steroids. Plots in the e-papers can be, for example, manipulated interactively against various parameters in a way that might resemble Wolfram Mathematica’s notebooks. Notes can be added to the document, with links to other e-papers or other works. Other papers or e-papers can be easily referenced pointing to specific sections. Raw data, and processing algorithms can be also incorporated to be readily available to anyone, so others can replicate experiments and results. Edits to the e-paper are optionally tracked in a «git»-like way. The e-paper is interactive and anyone can request to make comments or ask questions on specific parts (stackexchange-like feature?). The authors can read the comments or questions and maybe decide which ones are worth making visible (or highlighting) to anyone because they contribute to the e-paper content or discussion. Edits or contributions could also be done by other authors, so the main authors can decide to make them public to anyone if they think they are valuable. This promotes and enhances collaboration between authors and research groups, and generalizes the concept of co-author! Now a third-party can make significant contributions to an already published e-paper, and upon agreement of the original authors, the contributions are published and their contributors added as co-authors, specifying their exact contributions.

E-papers would generate tons of metadata which can be arranged for each author and be readily available. Hence, an author can have a set of numerical variables such as e.g., number of publications; number of comments, reads and interactions per publication; number of contributions as co-author to other e-papers; number of citations, etc. After such large set of variables is readily available in the platform, it can be used by any institution or university to create their own indicators based on their needs, and then rank and assess individuals. After papers are published in the platform by anyone and for free (like it happens now with arXiv), then authors can ask publishing companies (or vice-versa!) to start a certified peer-review process. If the paper passes the requirements of the reviewers, then the paper is stamped as certified. This stamp is just another metadata item, and has the weight the university or institution considers as appropriate when creating their indicators.

Gabriel Arturo Santamaría Botello

Open science is a belief grounded on an approach to scientific contribution by the means of sharing, openness, transparency, and peer collaboration. It aims to facilitate the access to scientific research to everyone: from researchers to general public.

The spreading media for scientific work has been based on specialized journal publication since the proliferation of science after the Second World War. Despite of its long tradition, this method is not free from flaws. Open science proposes different approach to tackle the main issues.

First and foremost, the most remarkable drawback of the current system is the limited accessibility to scientific publications. Briefly explained the system is simple and reasonable: researchers submit a manuscript to a journal, which acts as guarantors of quality and correctness of the work by the review of other researchers in that field. If the work meets some standards, it is finally published and accessible by a fee that can range 40€ per publication online.

However, research is mostly founded by taxpayers. Why does science then not pay back this patronage as knowledge, public reports and popular science to the society? Why does the public money end up instead in private shareholders’ pockets? One can think that the journal is entitled to charge a fee for its edition management, but the business model seems to be focused on profit rather than scientific contribution. Researchers does not economically benefit as writer of the publication, the reviewers ‘s job is altruistic, and researches commonly commit its manuscript already with the template of the journal. In short, the business is fed by some else’s job, which is usually founded with public money and public institutions like universities or researched centers have to pay a sustainable fee to access to colleague’s job and even they own work.

Open since proposes a straightforward solution: cut off these science retailers and grant open access to these reports. Review can be still done under the same peer to peer revision and work can be easily spread nowadays with the framework of internet.

Open science points another flaw: lack of transparency of the results. It is worth noting that the current publication scheme only requires a brief description of methods, figures and graphs to back up the results. This obviously lacks from transparency, reusability and reproducibility. Up to some extent, under these conditions, reviewer can only acknowledge whether results make sense or not and in some cases, they are merely grammar reviews rather than scientific critic.  Furthermore, the role of reviewers is not always free of controversy. In a highly specialized scientific environment, the number of potential reviewers is not large and, it is not rare to fall into conflict of interest and many cases of biased reviews have been reported.

One of the pillars of open science is reproducibility and reusability. The need of the first is obvious in terms of evaluation and the second can set a solid ground for potential progress in a given field. Computer science poses a clear example in this sense. In certain occasions, data is generated by high performance computers that are only accessible for members of some institutions. It would be more profitable for the society if this data were freely available to extract more results, completeness of the study or just merely for education and self-teaching purposes.

Open science has also recognized that the current situation is leading to a system in which scientific career is driven by number of publications and impact index, two metrics of doubtful fairness. The motto “publish or perish” has made lot of damage to science. The drawback of the pressure that some researches experiences to get theirs results published can be translated in poor results, publication of research in parts and submission of manuscripts with minor changes with respect to previous results. Nowadays it seems more important where and how much a research publish rather than what.

To the inherent pressure, one should also add the long review time, months can become a year. A doctoral program last for three years and most of the results are achieved at the end of this period. This leaves students without publications at the end of their Ph.D. programs, which compromises the options of further scholarships and job positions. Once the reviews are ready, the researcher might have left the scholar career, leaving the work in ungrateful anonymity.

This concern can be tackled by setting a system in which scholar have to review someone else’s manuscripts to get their own work reviewed. Due to the large number of publications and the common interest to get results done, it is feasible that manuscripts can be reviewed in short time. Furthermore, it could be accepted just as another step towards publication and it could be easier to set this duty in the researcher’s agenda and so get it done in shorter time. 

In my personal opinion, it is still a long way to go for these goals to be achieved. Journals made a step forward to perpetuate their model with the introduction of the impact factor and signing highly recognized researchers in particular fields. Scientific community has bought this lunch box and it seems difficult to promote nowadays if publications does not catch the attention of these elite researchers. In an open and diverse database of publications, good and meaningful work can make its way without the current inherent bias. On the other hand, transparence and openness is difficult to address up to the highest point. In a world dominated by ego, it is not clear to me that everyone is so keen to share their methods because its work is exposed to reanalysis which might point errors or misleading conclusions. Finally, the success of open science required a change in mindset not just in the research field but also in society. Public money has to be understood as a mean to a common goal, not just as a resource. In this regard, society has to demand the scientific community to digest and promote their results as popular science. This demand might be a bit of a detour but science is part of society and as such, society must claim its rights. Change a system and its inertia has never been easy.

#IamAnOpenScientistBecause money cannot be a wall towards knowledge. Public money, public science.