Hi,
There seems to be invalid_times not annotated in session 787025148 of the Neuropixel Visual coding dataset, they look like “Gain fluctuations” as described in the white paper, see event 1 and 2 in the following video. You can see the timing of the events in the box up right.
PS: There are more of these events in this session.
Cheers,
Roberto
Hi Roberto,
Thanks for reporting this.
This appears to be a different phenomenon than the “gain fluctuations” for two reasons:
This dataset was collected with a different recording system (Neuropixels 3a) than the one that showed the gain fluctuations (Neuropixels 1.0).
Except for one very large event around 3590 s, these artifacts appear to only impact the LFP band. During the gain fluctuations, the AP band gain would temporarily drop to zero, and no spikes would be detected during that interval.
I’m not entirely sure what’s causing the artifacts in this session, but my best guess would be that there is some movement artifact or static discharge that is saturating the LFP band, but not the AP band.
For now, you can set an amplitude threshold to detect these events and exclude the corresponding intervals from analysis. We will work on a systematic way of detecting them across the entire dataset.
Also, please let us know if you find them in any other sessions.
Josh
I found another puzzling thing in session 799864342.
print("regions recorded: ", session.metadata["structure_acronyms"])
returns
regions recorded: ['VISam', nan, 'COAa', 'CA1', 'LP', 'DG', 'VISmma', 'VISp', 'ProS', 'SUB', 'CA3', 'CA2', 'VISl', 'VISal', 'LGd', 'VISrl', 'APN', 'POL']
however COAa is incredibly deep in the brain… and actually when I plot that section of the probe it resides far outside the brain.
This is a result of negative CCF coordinates wrapping around to the bottom of the brain. The units in COAa are artifacts and should be ignored.
Hi Josh,
I put together a python list with all the artifacts I was able to detect in this repo, I found 3937 across 63 probes (noise_artifact). I didn’t check this thoroughly but I think the vast majority do not affect the AP band. Still if somebody is analysing the LFP (like me) it might affect some results.
I found also a probe where the time is not monotonically increasing (probe_id=769325786).
Last point, In two sessions the lfps seems to be duplicated: the same exact lfp is present in two different probes.This happens in sessions 746083955 and 756029989. You can see an example of this in the notebook at the end. Lfps coming from channels 849734700 and 849739612 are the same, the same happens to 46 other couples of channels in that session.
PS:
Thanks for making this amazing huge dataset open to everybody!!
Thanks for characterizing this in detail! Would it possible to send a CSV file with columns of probe_id, start_time, and stop_time for the artifacts? We can make that available to others performing LFP analysis.
We will also add some warnings about the duplicated data, as well as a link to your notebook, in the LFP analysis tutorial.
I’ll do it tomorrow! And also add some more markdown to the notebook then!
I updated the code, it was quite ugly before. I added a csv in the data folder, I can’t attach it here directly GitHub - RobertoDF/Allen_visual_dataset_artifacts: tables with dataset artifacts .
