I’m trying to download the session data for the virual coding experiments using allensdk, but getting really slow download speeds (regardless of internet connection). Here’s the code I’m using:
from allensdk.brain_observatory.ecephys.ecephys_project_cache import EcephysProjectCache
from allensdk.brain_observatory.ecephys.ecephys_project_api import EcephysProjectWarehouseApi
from allensdk.brain_observatory.ecephys.ecephys_project_api.rma_engine import RmaEnginedata_dirname = “./DATA/”
#!mkdir {data_dirname}
#!lsdata_directory = data_dirname + ‘ecephys_cache_dir’ # must be updated to a valid directory in your filesystem
manifest_path = os.path.join(data_directory, “manifest.json”)#Timeout fix from Failed to get session data - #4 by wayne
cache = EcephysProjectCache(
manifest=manifest_path,
fetch_api=EcephysProjectWarehouseApi(RmaEngine(
scheme=“http”,
host=“api.brain-map.org”,
timeout=120 * 60 #set timeout to 60 minutes
))
)sessions = cache.get_session_table()
#fetch a specific experiment, this is where the issue is
session = cache.get_session_data(sessions.index.values[20])
This screenshot is from my own machine - Notice the ETA which is >2hours for a single 2GB file. I’ve run the same thing on google colab thinking it was an issue with my internet connection, but same issue, slow download.
Is there a way to improve the download speed?
(there is a github issue on the allensdk repo regarding this: Speed download neuropixel NWB files extremely slow · Issue #1505 · AllenInstitute/AllenSDK · GitHub
No solution there either)
PS: I don’t yet have access to any AWS unit, so looking for alternatives