Hi,
the human micro-array data provides the z-score and according to http://help.brain-map.org/display/humanbrain/API
“* a vector of z-score values in the same order as the “samples” array. Note: z-score is computed independently for each probe over all donors and samples.”
i re-calculated the z-score for each probe with this formula, but i am unable to obtain the same z-score-values as provided in the dataset:
(expression_level - expression_level.mean())/expression_level.std()
I obtained data using this rma-query:
“http://api.brain-map.org/api/v2/data/query.json?criteria=”
f"model::Probe,rma::criteria,gene[acronym$il{self.geneAcronym}],rma::options[num_rows$eqall],"
“pipe::list[probes$eq’id’],”
“service::human_microarray_expression[probes$eq$probes]”
how do I correctly calculate the z-score on behalf of this data?
some background: we are trying to compare z-scores for human-microarray data with mouse ish-data , which is unfortunatelly not providing z-scores. alternatively: how can we obtain z-scores for ISH using the api/sdk that are consistent with the z-scores from microarray-data?
Hi @christoph-hue, Z scoring of the microarray data has already been address in this community forum post. Can you take a look at that and reply to this thread if it does not address your issue? I don’t know how to calculate Z scores in the mouse ISH data so if that ends up being necessary, someone else will need to reply.
hi @jeremyinseattle , I’ve already read it but it doesn’t really address the issue. my questions basically are: can I calculate the z-scores provided for one probe using the expression-levels of this probe and if so, how? because I don’t get the same values with the formula stated here: What is the Z score reference value in RNAseq GBM data?
you might want to clarify what the x is in your formula, in case it isn’t the expression-level (which I assumed). thanks!
If the Z score formula still doesn’t work, it may be the difference between downloading the data at the level of samples vs. structures. Can you try following this steps from this post for downloading the data at the sample level and then calculating the Z scores to see if you get the expected answer? You are correct that “x” is the vector corresponding to expression levels for a given probe when calculating Z scores. If this still doesn’t solve your problem, or if you would prefer to use the API, please post again and I’ll see if I can find someone else to address the question.
I compared the data
a) using the rma-api and its provided z-scores
b) calculating z-score manually using the expression-levels obtain by b)
c) downloading expression-levels as you described and recalculating z-scores
d) same approach as c) but downloading z-scores directly
regardless of whether I download the log2-intensities or obtain them (then they are called expression-levels) using the rma-api, recalculating z-scores gives me the same results - but they differ from the pre-computed z-scores.
I checked this by flattening all probe-values and sorting them. if all values for each approach are equal, then the value-lists should be the same as well.
so:
pre-computed z-scores of a) do not match the manually calculcated ones of b) and c)
pre-computed z-scores of d) match the pre-computed ones of a), but hence also not b) and c)
b) and c) are the same
you can check out my the zip-file I provide in the following. it contains the results and the code / pseudo-code I used:
Could you please check how to retrieve the same z-scores you provide using the expression-levels and expand on how you managed to do this? Thank you very much!
Furthermore, I wonder why the z-scores are calculated on a per-probe basis. according to the normalization white-paper (http://help.brain-map.org/download/attachments/2818165/Normalization_WhitePaper.pdf) expression-levels have been normalized across brain-regions and donors. wouldn’t the fold-change be more robust if expression-levels from all probes were taken into account? having high-variance expression-levels in a specific well, representing tissue from a specific region, would lead to skewed z-scores for the other samples of this probe, right?
Hi @christoph-hue. Thanks for the thorough response. Let me ask the scientist who did these calculations and someone will get back to you in the next week or so.
After some digging, I think we have at least a partial answer. First, what I meant above was that probe sets are normalized independently, not that probes are. Each probe set contains multiple probes that go into the single intensity value. Some genes have multiple probe sets, which are (and should be!) treated as independent values. This post discusses which probe set to choose per gene.
As for the z-score, these appear to be calculated separately for each donor. You can see this by looking at gene expression for XIST , which is on the X-chromsome and dramatically higher in females than males. You can see that all donors have both negative and positive Z score values, even though the log2 expression in the female donor is much higher. My suggestion would be to try recalculating Z scores from your b or c separately by donor to see if that reproduces the results of a and d. These analyses were all done more than a decade ago and many of the people who worked on this project have left the Allen Institute, so thank you for your patience in sorting this out!
Hi Jeremy,
I have looked on the forum a bit and read through the documentation and haven’t found an answer to this question, so I assume the answer is very obvious. What do z score values of exactly 0 in the IVY GAP downloadable data mean? Do they either mean that there is a missing data point or that the z score was actually exactly zero?
Thank you,
Austin Kunch
The Ivy GAP downloadable data is FPKM-normalized and not z-scores, and therefore a 0 value means that no RNA for that gene was detected. Here is a good summary of different RNA-seq normalization strategies, but it short, FPKM (fragments per kilobase per million) accounts both for the total number of RNA measured in a sample as well as the gene length.
In the heatmap view (e.g, here) I would suggest treating the 0 z-scores either as “NA” or as whatever the smallest non-zero value is. It appears that z is set to 0 whenever the log2 intensity value is 0, which does not make sense, as it always should be negative. More generally, I would recommend using read counts or log normalized FPKM values rather than z-scores whenever dealing with RNA-seq data programatically.