CSD: session.get_current_source_density() hanging

Technical
, , ,
1

Hi, I’m trying to access the current source density data from the visual coding neuropixels datasets. The session.get_current_source_density() is consistently hanging on me, eventually resulting in time out errors. I’m using the following code to run it:

df = session.probes
df = df.loc[df['has_lfp_data'] == True]
probe_id = np.array(df.index)

csd = session.get_current_source_density(probe_id[0])

I have the .nwb file stored locally and don’t have any issues accessing other data from the files.

Any ideas how to get this to work?

Thanks

2

Hi @eabhorrocks,

Thanks for your question! To get started, could you let me know:

  • Is it a particular visual coding neuropixels session where calling get_current_source_density results in hanging or all sessions?
  • What version of AllenSDK you’re using
  • Python version

That should be enough info for initial troubleshooting!

Best,
Nick

3

Thanks for your reply.

  • It seems to happen with all sessions I test (all functional connectivity, all locally stored)
  • Allensdk version: 1.5.1
  • Python 3.6.6

Let me know if there’s anything I can try on my side.

Best
Edd

4

@eabhorrocks

Thanks for your patience!

I tried session.get_current_source_density() with session 719161530 and for probe id 729445648 using AllenSDK 1.6.0 and Python 3.7.7 and was able to get CSD data from the NWB file from my home computer. What follows is a guess as to what may have gone wrong as well as some additional troubleshooting tips:

  1. The CSD data is stored on a NWB file (which are typically multiple gigabytes) that is separate from the session NWB data. So the ‘hanging’ you experience, may be the AllenSDK trying to download a separate probe NWB file.

  2. If your download speed is not great, you may also want to set a longer duration for the NWB download timeout (which defaults to 10 minutes before giving up on a download). You can find an example code snippet for setting a custom download timeout here: Failed to get session data

  3. You can add the following snippet at the start of a script or notebook to get more information about what the AllenSDK is doing in the background. As mentioned in 1. I suspect when you call get_current_source_density() the logging will show the AllenSDK taking a while to download the separate probe NWB file that I mentioned.

import logging
logging.getLogger().setLevel(logging.DEBUG)

Let me know though if you are still encountering issues!

5

Thanks for your help. I thought the CSD data was stored in the .session nwb rather than the probe files. The issue will almost certainly be downloads taking too long - the d/l speed for these files seems to be capped at 500 kb/s for me. I’ll manually download the required probe files and go from there.

One more thing, what is the directory structure I should use for the probe files to access the CSD data with session.get_current_source_density()? Should they be stored in the relevant session folders, in their own individual probe folders, or in some other structure?

Thanks for your help!

6

Hi @eabhorrocks,

The directory structure looks like the following for the example session I tried:

Data Directory
| - channels.csv
| - manifest.json
| - probes.csv
| - sessions.csv
| - units.csv
| - session_719161530
|    | - probe_729445648_lfp.nwb
|    | - session_719161530.nwb

So the manually downloaded probe nwb file should be stored in the relevant session folder.