BrainPower: resources for power analysis in neuroimaging

About

At some stage in an empirical neuroimaging research project that aims to find evidence for or against an effect, sample size and statistical power calculations should be considered. This process aims to shed light on questions like:

• how big is the effect size of interest?
• how many study participants, data acquisition runs and trials are needed?
• how likely will the effect be found in the data, given practical constrains?
• how should confidence in the findings be balanced with the likelihood of false positives?

Conducting power analysis to provide tentative answers to these questions could appear relatively straightforward for simple designs with intuitive behavioural variables, especially with the help of programs such as G*Power and standard effect size measures such as Cohen’s d.

However, the complex nature of neuroimaging data and designs (processing three-dimensional data over time with mass univariate and multivariate approaches) requires additional steps, e.g. cluster correction, to improve reliability of inferences. And our understanding of “effects” based on these data and methods is not necessarily as intuitive: how strong should the level of activation be, or how large should the cluster extent be?

Experts in these domains have created several primers, tools and software to help researchers approach power analysis appropriately. In a hackathon at the SIPS2019 conference, we set out to start collating a variety of such resources to make neuroimaging power analysis more accessible to researchers. We collected:

1. Informative journal articles and blog posts on the underlying principles of power analysis
2. Software toolboxes for conducting power analysis in neuroimaging
3. A list of useful questions to ask or heuristics to consider when planning an experiment
4. Tools for simulating fMRI data with which to pilot power analyses

This content is still growing. Our short-term goal is make the improve the quality of the current content and to expand the resources with tutorials and worked examples of conducting power analyses on real and simulated fMRI data.

We invite and welcome any and all contributions from the community!

Getting involved

New contributors are very welcome and encouraged to join!

To get started, join the discussion on our Mattermost channel!

Do you have a specific question or comment about existing or missing content? Open or comment on an issue!

If you want to help develop and are familiar with the standard developer workflow, fork the project on Github and send us a pull-request.

We ask that all contributors adhere to our code of conduct.

Things to consider

Why conduct a power analysis?

• To determine your sample size
• To determine whether it is even possible to find an effect with the given means (technical settings, available budget for subjects)
• To optimize your pre-processing software given your scanner settings

How to define an effect size?

• Test statistic
• Cluster extent
• Relate one contrast of interest A to the strength of another contrast B → see Matan Mazor and David Mehler
• Length of recording → number of trials

Tools for univariate power analysis

• GPower, Pangea
• Suited if clearly defined ROIs/ localizers are available
• How to clearly define ROIs? How to define ROIs on a subject-specific level?
• G*Power is suitable for very particular questions/ analyses, but require many decisions a-priori → defining ROIs, defining summary statistics on ROIs
• Can even increase power (e.g. for very small volumes) because you avoid correction for multiple comparisons

Tools for 3-dimentional power analysis

• Simulation: take available data set (resting-state) with a realistic noise profile, add an effect of a certain extent and strength, see if your (preprocessing + analysis) finds it

• What are the parameters to vary?

  • Sample size
  • Effect strength/ extend
  • Noise profile
  • Using nuisance regressors or not

Useful publications

Journal articles

• Mumford 2012: A Power calculation guide for fMRI studies
• Durnez et al 2016: Power and sample size calculations for fMRI studies based on the prevalence of active peaks
• Schönbrodt et al 2017: Sequential Hypothesis Testing With Bayes Factors: Efficiently Testing Mean Differences
• Schönbrodt and Wagenmakers 2018: Bayes factor design analysis: Planning for compelling evidence
• Durnez et al 2018: Neurodesign: Optimal experimental designs for task fMRI
• Chen et al 2019: Handling Multiplicity in Neuroimaging Through Bayesian Lenses with Multilevel Modeling
• Ostwald et al 2019: Power, positive predictive value, and sample size calculations for random field theory-based fMRI inference
• Ellis et al 2019: Facilitating open-science with realistic fMRI simulation: validation and application (code on github, site)

Blog posts

• Lakens 2017: How a power analysis implicitly reveals the smallest effect size you care about
• Mumford 2016: A power analysis approach that needs to end

Software tools

Tools for conducting power analysis for neuroimaging data.

G*Power

• A flexible statistical power analysis program for the social, behavioral, and biomedical sciences
• Standalone application
• GPower Website
• GPower Paper

Pangea

• Power ANalysis for GEneral Anova designs
• Pangea paper
• R source code

fMRIpower

• A Power Calculation Tool for 2-Stage fMRI models (ROI effects and group studies)
• Matlab and SPM toolbox (works with output from FSL as well)
• fMRIpower Website
• Initial OHBM poster
• fMRIpower Paper

NeuropowerTools - NeuroPower

• Sample size calculation tool for “whole-brain analyses, as well as for region-of-interest analyses.”
• NeuroPower website

NeuropowerTools - NeuroDesign

• A genetic optimization algorithm tool that searches for an experimental design to help researchers obtain the optimal detection power or estimation efficiency.
• NeuroDesign website
• NeuroDesign web app
• NeuroDesign paper

PowerMap

• Power and Sample Size Calculation for Neuroimaging Studies
• PowerMap website
• PowerMap paper

Simulating data

Tools for easily simulating data, i.e. “placing” effects of certain strength and certain temporal and/or spatial extent into a brain.

fmrisim

A python module, part of the Brainiak package, that contains a set of functions necessary to produce realistic simulations of fMRI data for a single subject. Steps include haracterizing the signal and generating the noise model, which are then combined to simulate brain data.

• bioRxiv preprint: Ellis et al 2019 - Facilitating open-science with realistic fMRI simulation: validation and application
• fmrisim code on GitHub
• fmrisim documentation
• fmrisim example code
• Brainiak website

neuRosim

An R package to simulate fMRI Data, including activated data, noise data and resting state sata.

• Journal article: Welvaert et al 2011 - neuRosim: An R Package for Generating fMRI Data
• neuRosim package on CRAN
• neuRosim code on GitHub

STANCE

The MATLAB-based Spontaneous & Task-related Activation of Neuronally Correlated Events simulator

• Journal article: Hill et al 2017 - A task-related and resting state realistic fMRI simulator for fMRI data validation
• STANCE code on GitHub

SimTB

MATLAB Toolbox for flexible generation of fMRI datasets under a model of spatiotemporal separability

• Journal article: Erhardt et al 2012 - SimTB, a simulation toolbox for fMRI data under a model of spatiotemporal separability.
• simTB website

Tutorials

– to be updated, contributors welcome! –

Indices and tables

• Index
• Module Index
• Search Page