Science: Same data – different results?

When it comes to complex facts or new phenomena, to work, scientists often only step-by-step to an understanding of the whole picture. The current Corona pandem

Science: Same data – different results?

When it comes to complex facts or new phenomena, to work, scientists often only step-by-step to an understanding of the whole picture. The current Corona pandemic illustrates this typical process of Knowledge production very significantly: in the Beginning, only a little was about the Virus and its effects are known and the data are based on small, often local, studies. To the extent that adding more data have been added, and have hardened some initial guesses, others had to be revised or withdrawn. In the course of the pandemic keyword masks have changed this also the statements and recommendations of the experts in Parts. In the population, this provided in part for a lack of understanding and uncertainty. However, the ability to check on the Basis of new findings and old assumptions, and to correct, where appropriate, is ultimately the engine of scientific progress – and an important pillar of the internal quality control.

108 brain scans and nine hypotheses

But what happens when 70 teams of researchers analyze independently the same records to test the same hypotheses? The has tried an international team of researchers is now quite practical. "The scientific process involves many steps: A theory is developed, hypotheses are created, after all data is collected and evaluated," explains Co-author Simon Eickhoff, research center Jülich. "Each of these steps can influence the final conclusions potentially, but to what extent? Be, for example, different researchers on the basis of the same data and hypotheses to different conclusions?" To test this, selected the researchers, under the direction of Tom Schonberg Tel Aviv University, recordings of brain activity from 108 subjects who had participated in a neuropsychological study. This with functional magnetic resonance imaging (fMRI) recordings showed which brain areas were active when the participants met certain financial decisions.

The scientists sent this 108 data records to 70 teams of researchers all over the world. Each Team analyzed these data with their respective Standard methods and verified on the basis of the results of nine all pre-specified hypotheses. As a final result, you should answer the hypotheses, each with a Yes or a no. "In each of these hypotheses was asked how certain aspects of decision-making have an impact on brain activity," explains Eickhoff. The analysis teams had three months to analyze the data. After that, they delivered Schonberg and his Team are your results for the different hypotheses, their results, as well as detailed information about your procedure in the analysis.

deviations in the conclusions, despite similar results of analysis

The comparisons of all the results showed some significant differences: In the case of five of the hypotheses, the conclusions of the Teams were significantly different from each other, for the remaining four there was agreement tends to be. "Interestingly, the analyses underlying the Teams compiled records still show a relatively large consensus between all the Teams,“ explains Co-author Felix wood masters from the University of Innsbruck. This is also a meta-analysis, which compared all of the Teams carried out the data analysis and the intermediate results are confirmed. They revealed a high convergence of the analyses and the data-based brain activation maps. However, because the research teams had to reduce this complex results in a simple Yes-or-no decisions with respect to the nine hypotheses were the differences. "This special issue of the analysis relates to all areas where highly complex data, which need to be reduced at the end of a bare Yes-no result," explains Eickhoff.

Thus, this study highlights that the type of analysis and evaluation, particularly in the case of complex data influence the result. "The reason for this is that researchers of such complex data sets need to meet on the way to the result of many individual decisions of how the available data is processed, sorted, are modeled, analyzed," says Eickhoff. At the same time, the study confirms how important it is that raw data is shared and the analyses and conclusions of colleagues to be repeated and reviewed in many specialist areas of common process. "This process of self-reflection and the continuous improvement of the methods is unique, and features the science," emphasize the Co-authors, Michael Kirchler and Juergen Huber, University of Innsbruck. The fact that almost 200 scientists were willing to invest hundreds of hours in this Experiment, show how strong the willingness is. "This study underscores how important it is that we are the possible discrepancies in the case of complex issues, scientists are always aware of," says Huber.

source: Tom Schonberg (Tel Aviv University) et al., Nature, doi: 10.1038/s41586-020-2314-9

*The post "science: Same data – different results?" is published by Contact with the executives here.
Updated Date: 21 May 2020, 18:30

You need to login to comment.

Please register or login.