How are waveforms in CellExplorer being extracted?

[ajcheng1]

I have a binary dat file that I sorted with kilosort and then manually curated with phy. just as a test, I labeled all the waveform units as good and it saved in cluster_group.tsv as good. But CellExplorer kept saying all the channels were recorded as bad.
I realize CellExplorer was extracting the bad channels from getBadChannelsFromDat.m. When I comment several lines of code from that matlab file (line 97-100) to open the CellExplorer GUI, the waveforms sorted looked nothing alike to what was sorted and displayed in kilosort. Are the waveforms being extracted straight from the dat file and not the sorted spike data?

%newBadChannels = find(wfF_std2 > params.noiseStdThreshold);
%newBadChannels2 = find(wfF_range > params.noiseRangeThreshold);
%newBadChannels = unique([newBadChannels,newBadChannels2]);
%session.channelTags.Bad.channels = unique([badChannels,newBadChannels]);

[petersenpeter]

Hi

The waveforms are indeed extracted from the binary data. This is to make them consistent among users and various pipelines. By default, the pipeline will try to detect noisy/bad channels. This feature can be turned off, but it matters for the estimation of the peak spike waveform across your channels: https://github.com/petersenpeter/CellExplorer/blob/main/calc_CellMetrics/getWaveformsFromDat.m#L61

Here is the function detecting the bad channels: https://github.com/petersenpeter/CellExplorer/blob/main/calc_CellMetrics/getBadChannelsFromDat.m

Are you saying that all your channels are labeled as bad? If so, you could have a scaling issue.

[ajcheng1]

Yep, you are right! I was using int16 instead of single since my .dat file was in float32. However, the waveforms still don't look right, but it may be how I set up my .dat format.

On a side note, I am using kilosort4 which doesn't use rez.mat anymore. Do you suggest I just manually input the electrode features?

[petersenpeter]

We are working on an implementation for Kilosort4. Let me get back to you on this. I believe we can get the electrode features from phy instead.

You can find more information on the format here: https://cellexplorer.org/datastructure/data-structure-and-format/#raw-data-file-format. The simplest test is to open your data with NeuroScope2: https://cellexplorer.org/interface/neuroscope2/

[ajcheng1]

Is there a way to extract waveforms from phy and not from dat?

The waveforms present do not resemble what I see when I curate it with Phy.

I did specify the numeric type to be single, but does the numeric type impact the quality of the waveform appearance?

session.extracellular.precision = 'single';  % Numeric data type

[petersenpeter]

Alvin, I would be happy to try to troubleshoot it if you shared a session with me? If Kilosort can read your data, so can CellExplorer.

If so I would need the binary data, relevant metadata and your spike sorted output from phy.

[ajcheng1]

I sent the files to your email address. Thanks!