Download Section Dataset Images from Brainspan ISH with gene/probe and donor age metadata

Technical
, ,
1

Hi,

I am trying to get all the Brainspan ISH images for all Section Datasets belonging to prenatal ages (3 donors) and only from 3 strcutures: visual cortex, prefrontal cortex, and ‘brain’.

i.e., the experiments listed in this search:

http://www.brainspan.org/ish/search/show?page_num=1&page_size=100&no_paging=false&search_term=&search_type=advanced&facet_age_tags=prenatal&facet_structure_name="Medial%20Prefrontal%20Cortex","Visual%20Cortex","Brain"

I’ve followed the instructions here:

https://allensdk.readthedocs.io/en/latest/_static/examples/nb/image_download.html#Downloading-all-of-the-images-from-a-section-data-set

brainspan_datasets = image_api.get_section_data_sets_by_product([22]) 
brainspan_datasets_df = pd.DataFrame(brainspan_datasets)

section_dataset_ids = [ds['id'] for ds in brainspan_datasets

# Iterate through `section_dataset_ids` to get list of `section_image_ids` and download the images.

# disable logger
logging.getLogger('allensdk.api.api.retrieve_file_over_http').disabled = True
  
downsample = 2
format_str = '.jpg'

for section_dataset_id in section_dataset_ids:
  dir_name = str(section_dataset_id) + '_section_images'

  # Get list of image ids from field `id` in each element of `section_images`.
  section_images = image_api.section_image_query(section_dataset_id)
  section_image_ids = [si['id'] for si in section_images]
  print(len(section_image_ids))
  # Download
  for section_image_id in section_image_ids:
      file_name = str(section_image_id) + format_str
      file_path = os.path.join(dir_name, file_name)
      Manifest.safe_make_parent_dirs(file_path)
      image_api.download_section_image(section_image_id, file_path=file_path, downsample=downsample)
      
# re-enable the logger
logging.getLogger('allensdk.api.api.retrieve_file_over_http').disabled = False

and have successfully created a directory for each SectionDataSet with the corresponding images.

image
image678×1522 242 KB

However, I had to manually filter the specimen_id from which to download images to restrict to prenatal.

section_dataset_ids = [ds['id'] for ds in brainspan_datasets\
if ds['specimen_id']==708383 or ds['specimen_id']==708307 or ds['specimen_id']==708787]

I have not been able to figure out how to get the corresponding metadata for the specimen/donor associated with each experiment.

i.e., for experiment 100135095, I know that the donor is a 21pcw F from the results table via the browser, but that is not returned in my API query.

I would like to be able to add the gene/probe and donor age/sex to the directory name which currently only has the experiment id.

There is a similar question on Stackoverflow (can’t post more than 2 links), which almost lets me address this, except I don’t know how to request the donor age and sex too.

section_datasets = pd.DataFrame(api.model_query('SectionDataSet', 
criteria='[id$eq%d]' % brainspan_dataset_id,
include='section_images,treatments,probes(gene),
specimen(structure),
specimen(donor)'

This is another hint I was able to find but I don’t know how to translate this regular GET request into an api.model_query like above.

section_datasets = requests.get("http://api.brain-map.org/api/v2/data/query?criteria=model::SectionDataSet,
rma::criteria,[failed$eqfalse],products[id$eq22],treatments[name$eq'ISH'],
rma::include,probes(gene),specimen(donor(age)),section_images")

Thanks for your help! :slightly_smiling_face:

3

Hi @carmen.se

This snippet worked for me. Does this help you get what you need?

import pandas as pd
from allensdk.api.queries.rma_api import RmaApi
rma = RmaApi()

brainspan_dataset_id = 100135095

section_datasets = pd.DataFrame(
    rma.model_query('SectionDataSet', 
                    criteria='[id$eq%d]' % brainspan_dataset_id,
                    include='section_images,treatments,probes(gene),specimen(structure),specimen(donor(age))'
                    ))  

section_datasets