RT Journal Article
T1 Variability in the analysis of a single neuroimaging dataset by many teams
A1 Botvinik-Nezer, R.
A1 Holzmeister, F.
A1 Camerer, C. F.
A1 Dreber, A.
A1 Huber, J.
A1 Johannesson, M.
A1 Kirchler, M.
A1 Iwanir, R.
A1 Mumford, J. A.
A1 Adcock, R. A.
A1 Avesani, P.
A1 Baczkowski, B. M.
A1 Bajracharya, A.
A1 Bakst, L.
A1 Ball, S.
A1 Barilari, M.
A1 Bault, N.
A1 Beaton, D.
A1 Beitner, J.
A1 Benoit, R. G.
A1 Berkers, R. M. W. J.
A1 Bhanji, J. P.
A1 Biswal, B. B.
A1 Bobadilla-Suarez, S.
A1 Bortolini, T.
A1 Bottenhorn, K. L.
A1 Bowring, A.
A1 Braem, S.
A1 Brooks, H. R.
A1 Brudner, E. G.
A1 Calderon, C. B.
A1 Camilleri, J. A.
A1 Castrellon, J. J.
A1 Cecchetti, L.
A1 Cieslik, E. C.
A1 Cole, Z. J.
A1 Collignon, O.
A1 Cox, R. W.
A1 Cunningham, W. A.
A1 Czoschke, S.
A1 Dadi, K.
A1 Davis, C. P.
A1 Luca, A. D.
A1 Delgado, M. R.
A1 Demetriou, L.
A1 Dennison, J. B.
A1 Di, X.
A1 Dickie, E. W.
A1 Dobryakova, E.
A1 Donnat, C. L.
A1 Dukart, J.
A1 Duncan, N. W.
A1 Durnez, J.
A1 Eed, A.
A1 Eickhoff, S. B.
A1 Erhart, A.
A1 Fontanesi, L.
A1 Fricke, G. M.
A1 Fu, S.
A1 Galván, A.
A1 Gau, R.
A1 Genon, S.
A1 Glatard, T.
A1 Glerean, E.
A1 Goeman, J. J.
A1 Golowin, S. A. E.
A1 González-García, C.
A1 Gorgolewski, K. J.
A1 Grady, C. L.
A1 Green, M. A.
A1 Guassi Moreira, J. F.
A1 Guest, O.
A1 Hakimi, S.
A1 Hamilton, J. P.
A1 Hancock, R.
A1 Handjaras, G.
A1 Harry, B.B.
A1 Hawco, C.
A1 Herholz, P.
A1 Herman, G.
A1 Heunis, S.
A1 Hoffstaedter, F.
A1 Hogeveen, J.
A1 Holmes, S.
A1 Hu, C. P.
A1 Huettel, S. A.
A1 Hughes, M. E.
A1 Iacovella, V.
A1 Iordan, A. D.
A1 Isager, P. M.
A1 Isik, A. I.
A1 Jahn, Andrew
A1 Johnson, Matthew R.
A1 Johnstone, Tom
A1 Joseph, Michael J. E.
A1 Juliano, Anthony C.
A1 Kable, Joseph W.
A1 Kassinopoulos, Michalis
A1 Koba, Cemal
A1 Kong, Xiang-Zhen
A1 Koscik, Timothy R.
A1 Kucukboyaci, Nuri Erkut
A1 Kuhl, Brice A.
A1 Kupek, Sebastian
A1 Laird, Angela R.
A1 Lamm, Claus
A1 Langner, Robert
A1 Lauharatanahirun, Nina
A1 Lee, Hongmi
A1 Lee, Sangil
A1 Leemans, Alexander
A1 Leo, Andrea
A1 Lesage, Elise
A1 Li, Flora
A1 Li, Monica Y. C.
A1 Lim, Cheng Phui
A1 Lintz, Evan N.
A1 Liphardt, Schuyler W.
A1 Losecaat Vermeer, Annabel B.
A1 Love, Bradley C.
A1 Mack, Michael L.
A1 Malpica, Norberto
A1 Marins, Theo
A1 Maumet, Camille
A1 McDonald, Kelsey
A1 McGuire, Joseph T.
A1 Méndez Leal, Adriana S.
A1 Meyer, Benjamin
A1 Meyer, Kristin N.
A1 Mihai, Glad
A1 Mitsis, Georgios D.
A1 Moll, Jorge
A1 Nielson, Dylan M.
A1 Nilsonne, Gustav
A1 Notter, Michael P.
A1 Olivetti, Emanuele
A1 Onicas, Adrian I.
A1 Papale, Paolo
A1 Patil, Kaustubh R.
A1 Peelle, Jonathan E.
A1 Pérez, Alexandre
A1 Pischedda, Doris
A1 Poline, Jean-Baptiste
A1 Prystauka, Yanina
A1 Ray, Shruti
A1 Reuter-Lorenz, Patricia A.
A1 Reynolds, Richard C.
A1 Ricciardi, Emiliano
A1 Rieck, Jenny R.
A1 Rodriguez-Thompson, Anais M.
A1 Romyn, Anthony
A1 Salo, Taylor
A1 Samanez-Larkin, Gregory R.
A1 Sanz-Morales, Emilio
A1 Schlichting, Margaret L.
A1 Schultz, Douglas H.
A1 Shen, Qiang
A1 Sheridan, Margaret A.
A1 Silvers, Jennifer A.
A1 Skagerlund, Kenny
A1 Smith, Alec
A1 Smith, David V.
A1 Sokol-Hessner, Peter
A1 Steinkamp, Simon R.
A1 Tashjian, Sarah M.
A1 Thirion, Bertrand
A1 Thorp, John N.
A1 Tinghög, Gustav
A1 Tisdall, Loreen
A1 Tompson, Steven H.
A1 Toro-Serey, Claudio
A1 Torre Tresols, Juan Jesus
A1 Tozzi, Leonardo
A1 Truong, Vuong
A1 Turella, Luca
A1 van ‘t Veer, Anna E.
A1 Verguts, Tom
A1 Vettel, Jean M.
A1 Vijayarajah, Sagana
A1 Vo, Khoi
A1 Wall, Matthew B.
A1 Weeda, Wouter D.
A1 Weis, Susanne
A1 White, David J.
A1 Wisniewski, David
A1 Xifra-Porxas, Alba
A1 Yearling, Emily A.
A1 Yoon, Sangsuk
A1 Yuan, Rui
A1 Yuen, Kenneth S. L.
A1 Lei Zhang, 
A1 Zhang, Xu
A1 Zosky, Joshua E.
A1 Thomas E. Nichols, 
A1 Poldrack, Rusell A.
A1 Schonberg, Tom
A1 Melero Carrasco, Helena
AB Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses1. The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset2–5. Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed.
SN 0028-0836
SN 1476-4687
YR 2020
FD 2020-05-20
LK https://hdl.handle.net/20.500.14352/100828
UL https://hdl.handle.net/20.500.14352/100828
LA eng
DS Docta Complutense
RD 20 ago 2024