data for RSA tutorial

[samanthareisman]

The RSA tutorial asks us to generate a file list from the data written in the single subject GLM tutorial that gets files that don't include the word 'betas' and extracts the conditions. However, in the earlier tutorial, we write a single file.

smoothed.write(f'{sub}_betas_denoised_smoothed{fwhm}_preprocessed_fMRI_bold.nii.gz')

I'm confused as to how to use the below code in the RSA tutorial, when the written file has the word "betas" and doesn't seem to contain specific condition info

sub = 'S01'

file_list = glob.glob(os.path.join(data_dir, 'derivatives','fmriprep', f'sub-{sub}' ,'func','*denoised*.nii.gz'))
file_list = [x for x in file_list if 'betas' not in x]
file_list.sort()
conditions = [os.path.basename(x).split(f'sub-{sub}_')[1].split('_denoised')[0] for x in file_list]
beta = Brain_Data(file_list)

[ljchang]

Hi @samanthareisman, thanks for your question.

The first line of code writes out a nifti file that contains all of the betas estimated from your first level model that will include the task regressors and the covariates of no interest.

The next chunk of code seems to have an issue and might be from an outdated tutorial. Sorry for the confusion!

I've updated the code on the website with this:

sub = 'S01'

file_list = glob.glob(os.path.join(data_dir, 'derivatives', 'betas', f'{sub}*'))
file_list = [x for x in file_list if f'{sub}_betas.nii.gz' not in x]
file_list.sort()
conditions = [os.path.basename(x).split(f'{sub}_beta_')[1].split('.nii.gz')[0] for x in file_list]
beta = Brain_Data(file_list)

Which uses files generated from code in the group analysis tutorial:

import os
from glob import glob
from tqdm import tqdm
import pandas as pd
import numpy as np
import nibabel as nib
from nltools.stats import zscore, regress, find_spikes
from nltools.data import Brain_Data, Design_Matrix
from bids import BIDSLayout, BIDSValidator
from nltools.file_reader import onsets_to_dm
from nltools.data import Brain_Data, Design_Matrix
from nilearn.plotting import view_img, glass_brain, plot_stat_map

data_dir = '../data/localizer'
layout = BIDSLayout(data_dir, derivatives=True)

tr = layout.get_tr()
fwhm = 6
spike_cutoff = 3

def load_bids_events(layout, subject):
    '''Create a design_matrix instance from BIDS event file'''
    
    tr = layout.get_RepetitionTime()[0]
    n_tr = nib.load(layout.get(subject=subject, scope='derivatives', suffix='bold', return_type='filename', extension='nii.gz')[0]).shape[-1]

    onsets = pd.read_csv(layout.get(subject=subject, suffix='events')[0].path, sep='\t')
    onsets.columns = ['Onset', 'Duration', 'Stim']
    return onsets_to_dm(onsets, sampling_freq=1/tr, run_length=n_tr)

def make_motion_covariates(mc):
    z_mc = zscore(mc)
    all_mc = pd.concat([z_mc, z_mc**2, z_mc.diff(), z_mc.diff()**2], axis=1)
    all_mc.fillna(value=0, inplace=True)
    return Design_Matrix(all_mc, sampling_freq=1/tr)



# Create output folder if it doesn't exist yet
if not os.path.exists('../data/localizer/derivatives/betas'):
    os.mkdir('../data/localizer/derivatives/betas')
        
for sub in tqdm(layout.get_subjects(scope='derivatives')):
    data = Brain_Data(layout.get(subject=sub, scope='derivatives', suffix='bold', extension='nii.gz', return_type='file')[0])
    data = data.smooth(fwhm=fwhm)
    dm = load_bids_events(layout, sub)
    covariates = pd.read_csv(layout.get(subject=sub, scope='derivatives', extension='.tsv')[0].path, sep='\t')
    mc_cov = make_motion_covariates(covariates[['trans_x','trans_y','trans_z','rot_x', 'rot_y', 'rot_z']])
    spikes = data.find_spikes(global_spike_cutoff=spike_cutoff, diff_spike_cutoff=spike_cutoff)
    dm_cov = dm.convolve().add_dct_basis(duration=128).add_poly(order=1, include_lower=True)
    dm_cov = dm_cov.append(mc_cov, axis=1).append(Design_Matrix(spikes.iloc[:, 1:], sampling_freq=1/tr), axis=1)
    data.X = dm_cov
    stats = data.regress()

    # Write out all betas
    stats['beta'].write(f'../data/localizer/derivatives/betas/{sub}_betas.nii.gz')

    # Write out separate beta for each condition
    for i, name in enumerate([x[:-3] for x in dm_cov.columns[:10]]):
        stats['beta'][i].write(f'../data/localizer/derivatives/betas/{sub}_beta_{name}.nii.gz')

[ljchang]

I didn't test the rest of the notebook yet. Let me know if you run into more issues. thanks for letting us know!

[samanthareisman]

This works great. Thank you for the quick response!

The other issue I've run into which may be of interest (but perhaps not solvable) is with downloading the data using git-annex in the same virtual environment where I had nltools imported. I am working on a remote RHEL as part of a shared cluster (so I do not have sudo access and have to use a virtual environment). Our administrators recommend using a python venv, but the the git-annex installation did not work in my python venv. Though I was told explicitly to avoid conda, I was able to import and use datalad & git-annex in a conda venv (using miniconda3)-- (I have opened an issue on the datalad GitHub about this). The issue I ran into is that I could not successfully install nltools in the conda venv. I ended up downloading all the data in my conda environment and switching to a python venv in order to run through the tutorial. Again, not sure what the solution is here but thought it might be helpful to mention/to document for any others who may be running into a similar issue.