___________________________________________________________________________________
Frequently Asked Questions about DeSoto and Hitlan (2007)
Blood levels of mercury are related to diagnosis of autism: A reanalysis of an important data set.
Journal of Child Neurology
___________________________________________________________________________________
This interactive web site has been created to address a number of questions that have arisen about the mistake we found in the Ip et al (2004) paper and the subsequent erratum that was published by the journal in November 2007. The DeSoto & Hitlan (2007) article has since been featured on dozens of blogsites, has been picked up by various newspapers, has even had portions read on the floor of the US House of Representatives. For the month of December, it is the most read article published in the Journal of Child Neurology and is in the top 10 most widely read articles for ALL OF BIOLOGY as listed by Faculty 1000 Biology. We appreciate that our research has garnered so much attention, but have become increasingly aware that a great amount of misunderstanding exists and much misinformation is being repeated. For example, one blog site has stated that we did not do analyses which we clearly did do, and made it appear that the only problem with the Ip 2004 article was that a one-tailed test should have been used. We are troubled that this information about what we wrote is being repeated, since it is not accurate on several counts. It is our sincere hope that interested parties will try to keep an open mind and carefully read our article for what it says -- and what it does not say.
Recent Questions
JULY 2008 Epiwonk query responded to-Click here : EpiWonk asks for our comments on his website's critique.
June 2008 question about the data set: could you provide us with the 95% confidence interval and the odds ratio and 95% confidence interval?
FEB 2008 NEW QUESTIONS SUBMITTED--CLICK HERE:
Do you think it was wise to build on the assumptions of the Holmes study, in light of the findings of the Gundacker study and the NHANES mercury level measurements?
Do you disagree with Autism Street's assertion that the Chrysochoou paper fails to support your contention?
Your paper acknowledges mercury in hair having being found to be both both higher and lower among autistics, do you believe a one-tailed test was warranted?
Does your study prove that vaccines cause autism?
why did you take the time to write about a mistake that had already been corrected by the authors?
Why are you taking the time to write all of this?
What was really so wrong with the Ip et al 2004 article on Autism and Mercury?
What do you think of some blog sites writings about your hair analysis results?
What is wrong with bloggers posting anything they want?
How can I check the original numbers for myself?
How can I check the full data set (the one provided by Wong in 2007)?
Why did you cite a non peer-reviewed journal?
Dr. DeSoto, you are a junk scientist with no hope of getting tenure.
FAQ
Q. I read you have never published anything related to autism, Why would a hormone researcher develop an interest in autism?
A. Yes, there are a couple of blogsites that have made much of this. Of course this should not matter (who says something should not matter and the merit of their argument and logic should be the sole basis of judgment). But, this is false anyway. Here is an excerpt from my most recent publication, “The idea that ToM may be a module that varies among individuals and is sexually dimorphic has generated a lot of interest among basic scientists as well as those studying disorders like Asperger Syndrome and autism. Specifically, Baron-Cohen’s idea that Autism may be an extreme form of a normal male type brain is supported by a relationship with testosterone levels to ToM skills....” (p.542). There is actually a large body of research connecting autism and testosterone. There is a prominent theory that central deficit of autistic persons is a lack a "theory of mind" (ability to think about and infer what others are thinking), and Simon-Baron Cohen has proposed autism can be thought of as an extreme form of the male brain (See the cover story on Newsweek Sept 8, 2003) or for a recent peer-reviewed article
Knickmeyer, R. & Baron-Cohen S (2006). Fetal Testosterone and Sex Differences in Typical Social Development and in Autism. Journal-of-Child-Neurology. Vol 21(10)
Experiments in animals leave no doubt that androgens, including testosterone, produced by the testes in fetal and/or neonatal life act on the brain to induce sex differences in neural structure and function. In human beings, there is evidence supporting a female superiority in the ability to read nonverbal signals, specific language-related skills, and theory of mind. Even more striking than the sex differences seen in the typical population is the elevated occurrence of social and communicative difficulties in human males. One such condition, autism, occurs four times more frequently in boys than in girls. Recently, a novel theory known as the "extreme male brain" has been proposed. It suggests that the behaviors seen in autism are an exaggeration of typical sex differences and that exposure to high levels of prenatal testosterone might be a risk factor. In this article, we argue that prenatal and neonatal testosterone exposures are strong candidates for having a causal role in sexual dimorphism in human behavior, including social development, and as risk factors for conditions characterized by social impairments, particularly autism spectrum conditions. (PsycINFO Database Record (c) 2007 APA.
This is a secondary interest, but a long standing one.
DeSoto. M.C., Bumgardner, J., Close, A., & Geary D.C. (2007) Investigating the role of hormones in Theory of Mind. North American Journal of Psychology, 9 (3).
DeSoto, M.C., Close, A., Bumgardner, J., & Collingwood, M. (2006). Testosterone and ToM. Poster presented at the Annual Meeting of the Midwestern Psychological Association, Chicago, Il. May 2006.
DeSoto, M.C., Geary, D.C. & Renfrow, A. (2004). Theory of Mind and reading people's intent: A possible role for Estrogen? Poster presented at the Annual Meeting of the Society for Personality and Social Psychology, January 2004.
Q. Does your study prove that vaccines cause autism?
A. No, I would not say this. As a matter of fact, the word vaccine is not even mentioned in our article. Mercury exposure can come from multiple sources, one of which is certain vaccines. In terms of the vaccine and autism hypothesis, DeSoto and Hitlan's results (2007) could correctly be said to prove that a widely cited study that some have used to rule out this hypothesis was incorrect. This is important; but, it should not be over-stated. Again, it is not proof that vaccines cause autism. This question is not fully settled, a typical statement by scientists who do not support the hypothesis but do not wish to overstate the state of science either reads like this, "The US Institute of Medicine has reviewed this issue and concluded that, although it is biologically plausible, there is presently insufficient evidence to support or refute the hypothesis that ethyl mercury in vaccines and autism spectrum disorder prevalence are associated" (Davidson et al, 2004, p. 1027) full text : http://pediatrics.aappublications.org/cgi/reprint/113/4/S1/1023 and the need for more research is stated. Our article is important because it documents a research finding that has been said to show there is "no connection between mercury and autism" was wrong; thus, some may have written off any connection too hastily. Certainly, if persons concluded that mercury and autism could not be linked based on Ip 2004 stated results, they would want to reconsider.
Ip et al (2004) has been cited as support for the following conclusions:
" By and large, biological studies of ethylmercury exposure have also failed to support the thimerosal hypothesis." (Fombonne et al, 2006).
"The majority of recent studies have failed to establish a connection between measles-mumps-rubella vaccination or the use of mercury-based vaccine preservative and autism." (Shattuck, 2006)
"However, more recently it has been reported that the association between mercury-containing vaccines and autism might have no firm basis." (Bondy and Campbell, 2005).
In all, Google Scholar reports there have been 21 citations of the original Ip et al study.
Q. Why did you take the time to write about a mistake that had already been corrected by the authors?
A. We didn’t. The mistake had not been found until we found it. We are the correctors. Again, some blog sites have unfortunately served to confuse this issue.
Q. Is your point is that they should have used a one-tailed test?
A. No, this is really a side issue. This was part of our article's "discussion section", where we attempted to reconcile the original data analysis reported by Ip in 2004, the correct analysis of the 2004 numbers, the currently reported correlation and F test, and Wong's analysis contained in the current erratum. It was not part of the main "results" section. Persons who have been been confused by this should realize:
1. the means published be Ip et al in 2004 yield a statistically significant t test, one-tailed or two tailed.
2. the one-tailed/two-tailed is relevant only using the original analytic technique on the new numbers provided in 2007.
3. proper analysis of the data set provided in 2007 is statistically significant using with a one-tailed or a two-tailed test of significance.
By picking points of the article out of context, it is possible to lose sight of these central facts. The articles should be read as a whole by objective persons, and those who have some understanding of statistics should recognize that 1,2, and 3 above are true regardless. To the extent that points 1, 2 and 3 become lost, it serves to prove that focusing on the side issue of one-tailed testing only serves to confuse readers.
Q. What was really so wrong with the Ip 2004 article?
A. Based on their retraction
which appeared in the same issue issue as our article, the mean for the autistic
group was wrong, the standard deviations were wrong for both groups, the stated
statistical significance in 2004 was way off.
The means as they reported them in
2004 result in a significant t test by any standard…meaning that the autistic
group had significantly more mercury in their blood than the control group. This
is indisputable (or should be). It would not matter if a one tailed or a
two tailed test was used. All interested parties should use their original data
from the 2004 article and calculate the t value and p value (or put the numbers
into an online t test calculator-- see "how can I check the
original numbers myself?"). Their original stated level of
statistical probability was off by almost 10 fold.
The data set they provided in 2007 misses conventional significance by a hair
using their original statistical technique.
Some blog sites such as Age of Autism have also pointed out that Ip et al overstated their findings in 2004. This means that the conclusions they made reached way beyond their findings. This is less serious compared to flubbing your stats, but I will note it for completeness.
Q. How did you find the mistake?
A. I put the numbers into an effect size formula and realized something was amiss. I then redid the t test using their numbers. I checked with another stats expert because I was somewhat doubtful this could actually be such a serious mistake. But it was. Again, check the numbers in a t-test calculator from the 2004 article, this is essentially what I did.
A. She was second author listed on the Ip et al 2004 study. She was the corresponding author. Virginia Wong is herself on the editorial board of JCN.
A. This may be hard to follow if you are not knowledgeable about statistics and hypothesis testing.
The answer is because the data set provided by Wong ( in 2007 ) does not match the numbers published in 2004. Wong states that the mean for autism group was wrong and that the Standard Deviations were wrong for both groups as well as the p value. If the means and SD’s were correct in 2004, the p value would be .02. Again, Wong says that ALL of these numbers were typos. The data set that was requested in 2007 and provided by the author result in a p value closer to the value that should have been published in 2004, but not exactly the same. It is far from the p value reported in 2004.
A. Yikes and/or C R I N G E
a.. Since I read the article (and obviously saw the numbers when I read it) would it have even been possible to think about what test to use before seeing the numbers?
b. I have long advocated for one-tailed tests when the hypothesis is directional. I am not of the opinion that one-tailed tests should never be used…I believe such blanket statements embody a misunderstanding of the true purpose of a hypothesis test and are out of sync with statistics texts and standard definitions. People who advocate this extreme position generally misunderstand why we use hypothesis testing in the big picture. I have read dozens of stats books, and none have ever said a directional t test is never appropriate… although I have of course heard individuals state this. The wording used by Ip in 2004 and the literature review (they were investigating whether "increased mercury exposure" was related to autism p.431 ) immediately imply a directional hypothesis. But in any case – it is unreasonable in this particular case to suggest it would have even been possible for us to have proposed a one tailed or two tailed test before seeing the numbers. I would urge interested persons to read a couple of standard stats book on this subject (you don’t have to read a whole book—just the part about one and two-tailed tests and when they are used).
Here are some links that define when a one-tailed test is used:
http://www.stats.gla.ac.uk/steps/glossary/hypothesis_testing.html
http://faculty.vassar.edu/lowry/ch11pt1.html
http://www.statpac.com/surveys/statistical-significance.htm
and if you don’t want to click on these sites or read a book, at least note that even the definition for a two tailed test reads thus: “ Two-tailed test: Statistical test used to detect differences without regard to direction. If a prediction regarding the direction of differences is made beforehand, a one-tailed test should be used.” http://www.dimensionresearch.com/glossary/glossary_pop.html
By far, the best thing is to refer to a standard text of your own choosing. More on this later.
Essentially, it comes down to the area under the curve (as in Calculus). The question is where you want to put your allowed 5%... all on end or split it between two ends (the two "tails").
Q. What do you think of some blog sites writings about your hair analysis results?
A. To be honest, I think it is a text book case of “Argument of Selective Reading” noted on A List Of Fallacious Arguments (retrieved from http://www.don-lindsay-archive.org/skeptic/arguments.html#selective_reading)
************************************
" Argument By Selective Reading: making it seem as if the weakest of an opponent's arguments was the best he had. Suppose the opponent gave a strong argument X and also a weaker argument Y. Simply rebut Y and then say the opponent has made a weak case. This is a relative of Argument By Selective Observation, in that the arguer overlooks arguments that he does not like. It is also related to Straw Man (Fallacy Of Extension), in that the opponent's argument is not being fairly represented. "
************************************
The hair analysis is NOT the main point of our article ("Although not the central focus of this report..."p. 1309). Even if it is wrong (which I am not at all conceding), it still would not have any effect on the fact that 2004 Ip et al report that there was no link between mercury blood levels and autism diagnosis was falsely stated, widely repeated, and had to be retracted. In some regards, it seems a bit like Autism Street is hoping that anyone who can be made to agree that the hair excretion hypothesis is wrong will think that everything I have said as wrong (and then than Ip must have been right). It is the responsibility of readers who want to understand and be informed to separate each part of the argument…. There is, in fact, no possibility that Ip’s 2004 analysis was done correctly, although it is possible that hair excretion hypothesis is wrong.
Q. Can you or can you not support that mercury is "excreted" into the hair?
A. OK... but this is going to be long. Anyone reading this FAQ Webpage will need to know the background. Autism Street bloggers penned the following within their online critique and associated discussion:
“Mercury is not excreted in the hair...If there were any mechanism of “excretion” for mercury in the hair, maybe, but we’re not aware of any such mechanism. Call us spoil sports, but it would also seem that forming hair follicles’ relative position in the circulatory system (closest to peripheral capillaries, after circulatory delivery of blood to major organs like the brain, and before venous collection of cellular waste products and toxins) make it a very unlikely candidate for containing a mechanism for active excretion of any kind.” And it goes on to imply that hair mercury levels are not even related to mercury elimination, they say this “has not been shown in any way shape or form that we are aware of,” and culminates with “Only one small problem with (the) hypothesis - mercury is not “excreted” in the hair.”
BTW... I found this on line at the same critical thinking web site referred to above.
*******************************
" Argument By Prestigious Jargon: using big complicated words so that you will seem to be an expert.
Why do people use "utilize" when they could utilize "use" ?"
*******************************
Putting the jargon aside, what is this saying? The authors are saying that they don't know of any way that mercury can be excreted into hair; that the physiological processes make it impossible for hair to be a mechanism of excretion. They are confident that mercury can not be said to be excreted via the hair.
This then has been repeated across various websites, that mercury can not be excreted into hair, that the idea that mercury is excreted into hair is absurd (based on Autism Street as a source!) I secretly hope that persons who have trusted these authors as a legitimate source of correct information will start to think independently.
**If you have not visited these sites, bare with me... It does seem that many persons have rather uncritically accepted that there is nothing other than blood level mercury at the time the hair was formed (it is stated that "dozens of studies that have shown that the hair mercury content follows blood mercury content at the time the hair is formed.") that could potentially influence the amount found in hair. Others have even opined that mercury can not be tested by hair. I have actually been taunted, "Can you or can’t you present evidence for the proposition that there is an active process of mercury excretion into hair which could be impaired in certain individuals?" No problem.
First below are some sources that establish hair is considered a route of "excretion", illustrating this is a proper term to use to describe it. Next are some sources that use hair to index mercury exposure, finally some sources that document individual variation and things other than blood level that influence hair excretion rate. The final one speaks to all three. Readers, please keep in mind this is a side issue in terms of the article as a whole, but one I have been asked to address/support.
Carrier G, Bouchard M, Brunet RC, Caza M (2001). A toxicokinetic model for predicting the tissue distribution and elimination of organic and inorganic mercury following exposure to methyl mercury in animals and humans. II. Application and validation of the model in humans.Toxicol Appl Pharmacol. 2001 Feb 15;171(1):50-60. *click for pub med link to full text The authors are presenting detailed information about how mercury moves through the body as well as a set of differential equations to describe it. The main routes of excretion of mercury are hair, urine and feces (with urine a lesser route fyi)... here are some quotes, "excretion rates of organic mercury from the whole body into feces and hair were 100 and 40 times smaller in humans, respectively, and urinary excretion of organic mercury in humans was found to be negligible." (p. 50). and "These values clearly indicate an important bioaccumulation of both organic and inorganic mercury in hair as well as inorganic mercury in kidney..." "excretory compartments, namely: feces, urine, and hair, had to be reduced...."( p. 58)
See also: Farris FF, Dedrick RL, Allen PV, Smith JC. (1993). Physiological model for the pharmacokinetics of methyl mercury in the growing rat Toxicol Appl Pharmacol. 1993 Mar;119(1):74-90. pub med link
Studies that measure mercury via hair:
Mortada WI, Sobh MA, El-Defrawy MM (1993). The exposure to cadmium, lead and mercury from smoking and its impact on renal integrity. Med Sci Monit. 2004 Mar;10(3):CR112-6.
Feng Q, Suzuki Y, Hisashige A. (1998) Hair mercury levels of residents in China, Indonesia, and Japan. Arch Environ Health. 1998 Jan-Feb;53(1):36-43
Dolbec J, Mergler D, Larribe F, Roulet M, Lebel J, Lucotte M. (2001). Sequential analysis of hair mercury levels in relation to fish diet of an Amazonian population, Brazil. Sci Total Environ. 2001 Apr 23;271(1-3):87-97
Passos CJ, Mergler D, Lemire M, Fillion M, Guimarăes JR. (2007). Fish consumption and bioindicators of inorganic mercury exposure. Sci Total Environ. 2007 Feb 1;373(1):68-76.
Björnberg KA, Vahter M, Petersson-Grawé K, Glynn A, Cnattingius S, Darnerud PO, Atuma S, Aune M, Becker W, Berglund M. (2003). Methyl mercury and inorganic mercury in Swedish pregnant women and in cord blood: influence of fish consumption. Environ Health Perspect. 2003 Apr;111(4):637-41
Gosselin NH, Brunet RC, Carrier G, Bouchard M, Feeley M. (2006). Reconstruction of methylmercury intakes in indigenous populations from biomarker data. J Expo Sci Environ Epidemiol. 2006 Jan;16(1):19-29 ( quote : To monitor MeHg exposure in individuals, organic and inorganic mercury are often measured in blood samples or in hair strands, the latter being by far the best integrator of past exposure. With knowledge of the MeHg kinetics in humans, the levels of both biomarkers can be related to MeHg body burden and intakes. In the present study, we use the toxicokinetic model of Carrier et al. (2001) describing the distribution and excretion of MeHg in humans...) p. 19.
Holmes AS, Blaxill MF, Haley BE (2003). Reduced levels of mercury in first baby haircuts of autistic children Int J Toxicol. 2003 Jul-Aug;22(4):277-85
Gerstenberger SL, Cross CL, Divine DD, Gulmatico ML, Rothweiler AM. (2006). Assessment of mercury concentrations in small mammals collected near Las Vegas, Nevada. Environ Toxicol. 2006 Dec;21(6):583-9.
Maramba NP, Reyes JP, et al (2006). Environmental and human exposure assessment monitoring of communities near an abandoned mercury mine in the Philippines: A toxic legacy. J Environ Manage. 2006 Oct;81(2):135-45.
Hair levels are stated (on some blogsites) to only be affected by blood levels at the time the hair is made. Much is written about how it can't be affected by anything other than direct blood level. Although it seems that this statement may have been uncritically accepted by some as fact, this is not in keeping with the literature, and that the ratio is different for different individuals:
Gundacker C, Komarnicki G, Jagiello P, Gencikova A, Dahmen N, Wittmann KJ, Gencik M. (2007). Glutathione-S-transferase polymorphism, metallothionein expression, and mercury levels among students in Austria. Sci Total Environ. 2007 Oct 15;385(1-3):37-47. Epub 2007 Aug 22. Showing that there are individual differences in blood to hair correlations, and tracing them to a specific gene.
"In order to identify genetic factors underlying the inter-individual variance in detoxification capacity for the heavy metal mercury, 192 students were investigated. We focused on the relationship between polymorphisms in glutathione-S-transferase genes and mercury concentrations in blood, urine, and hair. The correlation between blood mercury levels, GSTT1 and GSTM1 polymorphism, and gene expression of certain metallothionein subgroups was evaluated in a further group of students (N=30)....The following was noted: a) hair mercury concentrations are significantly increased in persons with the double deleted genotype (GSTT1-/- and GSTM1-/-) as compared to persons with the intact genotype, " (p. 37).
Thomas DJ, Fisher HL, Sumler MR, Mushak P, Hall LL (1987). Sexual differences in the excretion of organic and inorganic mercury by methyl mercury-treated rats. Environ Res. 1987 Jun;43(1):203-16. (shows that sex affects the ratio of blood levels to hair excretion)
Bartell SM, Ponce RA, Sanga RN, Faustman EM. (2000). Human variability in mercury toxicokinetics and steady state biomarker ratios.Environ Res. 2000 Oct;84(2):127-32 full text if you have access to a university library computer. Here is a quote if not:
"...methylmercury and inorganic mercury are eventually excreted through the hair, urine, and feces (Smith et al., 1994; Gray, 1995)." p. 127. and "...Frequently cited ratios include 250 ppm in hair per mg/L in blood and 0.95 lg/L in blood per lg/day of mercury intake (WHO, 1990; Boischio and Henshel, 1996; EPA 1997). These ratios are often applied uniformly across a population to estimate the biomarker concentration that would result from a given exposure rate or vice versa. However, experimental evidence indicates that significant variability among individuals may exist in the biokinetics of mercury..." (p. 129).
Q. What is wrong with bloggers posting anything they want?
A. Nothing if everyone reading it took the time to verify everything they read on their own. Not all websites are authored by well-rounded authorities, and certainly many have their own agendas (unlike journals, where there is some attempt at random and outside expert judgment before publishing). Also, as a reputable scientist, my real name, my place of work, contact information and my credentials are all published with my articles. My reputation is on the line with what I publish. When one publishes anonymously there is no real accountability--and some of what is written is REALLY misleading. For some examples....
Ip acknowleded their error
Autism Street: "DeSoto & Hitlan revisited the data from Ip, Wong, Ho, Lee, & Wong (2004) and found that the Ip et al. made an error in their analysis. Ip et al. acknowledged the error and an erratum report was published. The new data still did not yield a statistically significant score using a two-tailed test, although near-statistical significance was found. DeSoto & Hitlan (2007) chronicles this, but goes on to offer new criticism and opinions on Ip et al. (2004). These will be discussed below..."
This is probably misleading to a reader who sees this first.
1. This sounds like an erratum was published and then we further critiqued. Not so.
We found the error and the journal published a correction along with our report. The erratum was requested of Ip and Wong by the editor. This is to give the original authors a chance to address the correction as they choose as well as publicly acknowledge an error was made.
2. This confuses the issue in other ways as well: the 2004 numbers yield a statistically significant difference by any standard (using a one- or two-tailed test, etc). Ip et al incorrectly reported that it was far from statistical significance-- that is -- they incorrectly said that the data showed no connection at all between mercury blood levels and autism. Someone wanting to clarify would want to make this part clear and not confusing.
3. It also obscures the fact that the new data set does yield a statistically significant difference, it does not have to be a one-tailed test.
4. As a whole, the paragraph picks at one argument and hides the main findings. Honestly – if you read the above paragraph on a blogsite, would you not be surprised that 1, 2 and 3 are true? Of course someone who read this would be surprised 1 , 2 and 3 are true, but they are. Our original article has been posted on line, so I hear. Please read it as well as the accompanying editorial by the journal editor and the retraction (there are three relevant articles all published together, (see http://jcn.sagepub.com/content/vol22/issue11/).
Here are some additional troubling quotes (my points are in CAPS):
Autism Street: "What this means is that the authors took out the data that was much different from the main body of data. The authors note that such outliers can unduly bias the results. Such a practice certainly has advocates in both descriptive and inferential statistics, but there are also many scientists and statisticians who don’t agree with such a practice and/or don’t use it."
HITLAN AND I HAVE BOTH TAUGHT STATISTICS AT THE UNIVERSITY LEVEL, HITLAN HAS TAUGHT FROM AND/OR READ SEVERAL DOZEN TEXTBOOKS. NEITHER OF US HAVE EVER SEEN A TEXT THAT RECOMMENDS SIMPLY IGNORING EXTREME OUTLIERS AS SOUND STATISTICAL PRACTICE.
Autism Street goes on…
“The authors are careful to note that removing the outliers in this case, did not cause them to find statistical significance. In fact it reduced the difference between the groups. Although the removal of the outliers does not change whether statistical significance was achieved, DeSoto & Hitlan list it as a flaw in the original analysis."
THIS IS NOT CORRECT. REMOVING THE OUTLIERS DOES ALTER THE STATISTICAL SIGNIFICANCE. ALSO WE DID NOT LIST IT AS A FLAW IN THE ORIGINAL ANALYSIS; ACTUALLY THIS STATEMENT BY AUTISM STREET COULD REPRESENT AN IMPORTANT MISUNDERSTANDING. THE NUMBERS AS REPORTED IN THE 2004 ARTICLE HAD MUCH SMALLER STANDARD DEVIATIONS, WHICH USUALLY MEANS NO OUTLIERS EXIST. OUR CRITICISM OF THE ORIGINAL ARTICLE IS VERY VERY SIMPLE: THE MEANS REPORTED RESULT IN A STATISTICALLY SIGNIFICANT DIFFERENCE, BUT IP ET AL WROTE THAT IS WAS NOT EVEN CLOSE TO STATISTICAL SIGNIFICANCE.
Autism Street: "The authors also take issue with the dismissal by Ip et al. of the near statistical significance for blood mercury level. DeSoto & Hitlan propose that the finding here was so close, that is should be called “marginally significant” at least in 'this particular case'. DeSoto & Hitlan justify such an interpretation by noting how important it is that researchers understand what is 'behind the rise in autism'. We take issue with this. "
We take issue with the twisting of our words and taking quotes out of context. Read the article and read a stats book. Learned persons who are objective have evaluated our arguments and found them persuasive. JCN is peer reviewed (the editor is not an idiot though Autism Street contributor calls him such - see autism street quote below), see also FACULTY 1000 biology.
Autism Street question posed by Schawrtz:
"If the outliers are kept in, and a one tailed test used, does the result end up being significant, or do they even do this?"
Yes, we do this.
Mr. Cubbine writes:
For the life of me I cannot figure out why DeSoto didn’t simply do multiple analyses (1 and 2 tail) on both the original typo-laden dataset and the corrected data and compare the results. I’m left to conclude that they’re either ignorant or had an agenda. I’ve never heard of someone submitting an article critiquing a paper without showing, using multiple strategies, that there’s a change in the outcome. Another idiot here is the editor.
WHY DO YOU ASSUME THIS WAS NOT DONE? (WE KNOW WHY-- BUT PERHAPS YOU SHOULD CONSIDER THE SOURCE OF YOUR INFORMATION.)
This is with the outliers in and the two-tailed on the new data set:
“Based on their corrected analysis, the authors report the revised P value for their t test to actually be P = .056.” (DeSoto and Hitlan, p. 1309).
This is in regards to the original data published in 2004:
“The means and standard deviations reported in the 2004 article yielded an easily significant t value (autism mean = 19.53 nmol/L, SD = 5.6, n = 82; control mean = 17.68 nmol/L, SD = 2.48, n = 55 gives a t = 2.283, two-tailed P =.024 or one-tailed P = .012).” (p. 1309)
This is the 2007 data set with the outliers removed:
“Logistic regression was performed using blood mercury level as the predictor and the autistic/control group as the criterion….r = .20, r2 = .04, F(1,133) = 5.76, P = .017. This finding indicates that there is a statistically significant relationship between mercury levels in the blood and Diagnosis…” (p.1309).
**For clarity, this is not a "one-tailed test". This would be inferred because when a one-tailed test is used, it is customary to so state it.
We see why one might think we did not do any of this: Autism Street says we didn’t. But we did.
Autism Street: "They didn’t calculate it. There were two outlier data points in this study. The bigger of the two was an unusually high mercury level for someone in the autistics group. The authors felt that by removing this, the two groups were brought closer together, but no stats were run."
Also, we did not “feel” that removing the outlier the means were brought closer together… they were closer together when an extreme number was removed. This is not rocket science. It works like this:
4,5,6,7,50 has a mean of 14.4 ; 2,3,4,5 has a mean of 3.5. If you take out the outlier (50), the means become 5.5 and 3.5 --- What stats would one run on something so straightforward?
It is no wonder that readers of Autism Street might say this “shows her very bad grasp on toxicology, physiology and autism. it’s amazing that someone with a PhD would make these kinds of mistakes.”
ouch
Q. One reader of a misinforming blogsite laments that I would not be accepted as “serious scientist by the scientific community. If she ever came up for tenure and this article was reviewed by any of the well-respected scientists in the autism community…well, she would not fare very well.” Are you a laughed at among scientists?
A. I don't think so, no. Forgive my defensiveness, but my academic credentials are actually quite good. I graduated from a major university in four years while working and raising a small child as a single mother, and I graduated at the top (as in number one) in my graduating class (SIU, 1989). I have published in the very top journals in my field (I have published in the number one journal of experimental psychology and my dissertation results were picked up by Science magazine). Prior research results have actually been internationally recognized. Here is my vita. Here is some other information. (a bit out dated I warn you). My logic and reasoning skills are not poor as some have written, I scored above the 90th percentile of the GRE logic subtest (although in the spirit of full disclosure, this was my lowest subtest, if I recall). I am already tenured, I got tenure early. But more to the point--other respected scientists have actually not found this article to be of low quality, as of this writing (Dec, 2007) it is the number two "hidden Jewel" in all of biology by http://www.f1000biology.com/top10/jewels
What is F 1000 Biology?
|
However, to be fair to the writer of the above quote, If I read Autism Street about my paper, I would think the same thing. I hope anyone who wants to get all sides but stumbled onto some confusing information first now will take the time to note that many other blogsite writers have been far more balanced and a bit less error-prone.
A. Well, strike one up for misinformation and confusing the issue. This is the question I get asked the most, so I am addressing it once more to be super clear. The answer is …… No.
Although Autism Street blogsite goes on and on about this, it is a side issue; and it is only serving to draw attention away from the main point: that the widely cited conclusion published by Ip that blood levels of mercury and autism diagnosis are not related was wrong. The numbers they published in their paper yield a significant t statistic. One tailed, two tailed, it does not matter. I think this can be understood by objective readers from our article (DeSoto & Hitlan, 2007), but it is possible to confuse the issue by emphasizing certain things in the article and ignoring others.
Here is the original Ip et al (2004) numbers:
17.68 (SD = 2.48) with a sample size of 55 for the control group and 19.53 (SD = 5.65) with a sample size of 82 for the autistic group, with the samples size of 137. (this means the degrees of freedom is 135 if one needs this to calculate the t statistic).
Here is a t-test calculator, find your own on line (anyone can do this).
Select 95% confidence level – this is what is normally used unless there is a reason not to (it is the same things as p < .05 level)
http://www.dimensionresearch.com/resources/calculators/ttest.html
see? It does not matter if it is one-tailed or two-tailed.
Q. How can I check the data set published in the 2007 issue of JCN myself?
You have to get the numbers from the JCN website (recommended if you have had at least some stats training). You can do a simple t-test here:
http://www.changbioscience.com/stat/ttest.html and get the .056 p value that is reported by Wong in 2007.
Q. Why aren’t these two p values the same?
A. The numbers given by Wong in 2007 suggest that multiple mistakes were made in the 2004 publication. At first, when we contacted the editor, we thought the only mistake was the reported p value (the level of statistical significance). But based on the numbers that Wong provided, it seems that the mean for the autistic group was wrong, the standard deviation for the autistic group was wrong, and the standard deviation for the control group was wrong. The increase in the standard deviation marks the presence of outliers. Using the data set provided by Wong, and the exact same test Ip et al used in 2004, the numbers just miss conventional statistical significance. However, the presence of extreme outliers are not to be simply ignored in a data set. The choice of HOW to deal with outliers is nuanced. On the other hand, there is little question that they should be dealt with in some manner. I hate to appeal to education and authority, but here I must because the fact is that a discussion of this topic is beyond the scope of a general reader website. I have had many courses in math and higher statistics, and I am still learning. If you are reading this and hoping to be able to make an informed judgment about how to deal with outliers, this may not be realistic goal unless you have already taken some advanced stats classes. Any advanced statistics book would discuss the topic, but one that has "regression" in the title would be best.
Q. Why did you use Medical Hypotheses as a source in your article?
A. Because it published the article we wished to refer to. Here, we would agree with Autism Street and others who say that the peer review process is valuable and that many non-peer reviewed journals are not reputable. We also think this standard applies to blogsites. Being peer-reviewed gives some assurance of the quality of the information. Sometimes poor "researchers" repeatedly publish their "science" in non-peer reviewed journal outlets. When this happens, it is a red-flag. The question for readers is if a researcher or a topic is ONLY published in non-peer reviewed journals AND if the peer reviewed journal body of knowledge ever quotes from the journal in question. In both cases, Medical Hypothesis appears to be a fairly reputable, though non peer-reviewed, journal.
Am I the only one who ever cites from this journal?
Certainly not-- check out Google Scholar by typing in "Medical Hypotheses" into the search line and see numerous articles... For example, Sialorrhea in amyotrophic lateral sclerosis: a hypothesis of a new treatment, Bushara (1997) has been cited 60 times. You can click on the list of citations to see that the article is cited in numerous respected peer reviewed outlets. The exact article we were blasted for citing has been cited by 115 other authors in journals ranging from Pediatrics (probably the most respected and widely cited pediatric journal) to Molecular Psychiatry (ISI impact factor 11.8). Medical Hypotheses actually serves a unique niche, publishing theoretical papers without formal peer review giving the authors more leeway and a more rapid publication time. Unlike many non peer-reviewed journal outlets, it is indexed by all major scientific indices. Again, peer reviewed is preferred, but Medical Hypotheses is unique in that most of their contributors normally also publish in peer-reviewed outlets. To prove this to yourself, type in KO Bushera (or other authors you select) into Google Scholar to see the long list of peer-reviewed publications by Bushera.
Q. Why are you taking the time to write all of this?/ Who cares what a blog says?
Unfortunately, the main bloggers of Autism Street have taken the time to respond to almost all of the other blogs about this article (there are a LOT, dozens at least). The Autism Street "critique" has been quoted all over (today I saw it on a regular news website). People who read it will certainly be misled, and I care. Furthermore, I did not want my silence to be taken as any sort of concession that what is being said about our article is rational or correct. Finally, a graduate student at Mt. Sinai, Mr. Young, read the original article, and like the F1000 independent body of biological science experts, noted the importance of the paper and wrote about it on his website, Pure Pedantry. But then he was contacted by one of the blog sites that is full of misinformation. I must admit, that were I to read these blog sites, I would also think I was a junk scientist. I truly sympathize with those of you who read that Hitlan and I did not bother to run certain stats or read that the original Ip et al (2004) paper was fine except maybe they should have used a one-tailed test. However, this is not the way science works-- not how academic journals work. If the only problem with the 2004 paper was limited to a question of whether a one or two-tailed test should have been used no retraction would ever be issued and we would not have had our article published. Journals have higher standards. Furthermore, such inane criticism (limited to an argument that Ip et al should have used a one-tailed test) would not have lead to the scientific bodies that exist to evaluate the merit of new articles listing it as a must read and a top three Hidden Jewel were it a junk article. It was not so listed because it is an example of shoddy research, to the contrary, it was reviewed and ranked because it is important that a previously widely cited journal article (Ip 2004) was found to be in error. I realize this is defensive, but my scholarship, writing ability, intelligence and home decor has been harshly (and falsely) attacked by persons who apparently hate the idea that mercury might be linked to autism in any way. And worse, some people who are sincerely curious have not realized that postings such as Autism Street have clear falsehoods and agendas. I believe that Autism Street regularly refers to those it disagrees with as Idiots. The DeSoto Hitlan paper was correctly done and is important. The Ip et al 2004 was based on mistakes and has now been retracted. How can I sit by when I can see that inquiring minds are being led to believe the opposite? I can't. To wit: Pure Pedantry's doctor in training, referring to the recalled Ip et al 2004 paper after reading the false information presented on Autism Street stated he now thinks the "original study got a fair conclusion and performed fair statistical analysis," Mr. Young went on to characterize our supposed thinking about our 2007 article as "If we did a one-tailed t-test it would be significant, so this is a result worth following up..." *Sigh* This was the straw man that I had to respond to and the straw that broke the camel's back. I teach graduate students; I cannot read false hoods about my own study knowing they are misleading future scientists and not at least try to shed some light on the matter. There is much more I could respond to. Do not hesitate to write to me, I would prefer this to reading false information and not asking. I do not ignore emails and believe I have responded to the about 100 I have received thus far. In sum, if you have a question don't ask others -- ask us and don't assume everything you read is true. I will respond personally, or if several people ask the same question, I will post a response here (so check back).
Thank you.
If you have found this web site helpful, please make a small donation to Autism Speaks: www.autismspeaks.org
We will post more comments as time goes on, be sure and write if you have any questions, I do not plan to respond to individual blogsite postings. We want this all to be clear.
*************************************
"To use one's god given cortex to think about Nature is the sincerest form of praise."
-Anonymous
"He who finds even one thought that take him closer to the eternal mystery of Nature has been granted great grace."
-Albert Einstein
WEBpages and BLOGS referring to our article (note that posting the link does not indicate we agree or disagree with the statements found within, judge for yourselves).
http://www.translatingautism.com/2008/04/micro-history-of-recent-mercury-autism.html
http://www.americanchronicle.com/articles/47182
http://autisminnb.blogspot.com/2008/01/autism-and-mercury-data-reanalyzed.html
http://www.ageofautism.com/2007/12/the-ip-blip-and.html
http://www.ageofautism.com/2007/11/when-smart-scie.html
http://www.autismstreet.org/weblog/?p=158
http://combatingautismfromwithin.blogspot.com/2007/12/reply-for-anb.html
http://groups.google.com/group/uk.people.health/msg/dd6156eaed6b1954
http://photoninthedarkness.com/?m=200712
http://www.congress.org/a-champ/issues/alert/?alertid=10666286
http://teachers.net/mentors/special_education/topic17170/12.28.07.12.33.57.html
http://thetruthaboutmcs.blogspot.com/2007/12/autism-correlated-with-mercury-mcs.html
http://www.wikio.com/health/diseases/mental_health/autism/search=desoto
http://scienceblogs.com/purepedantry/2007/11/mercury_autism_and_a_note_on_s.php
http://scienceblogs.com/purepedantry/2007/11/mercuryautism_redux_what_is_a.php
http://www.mailtribuneforum.com/forums/index.php?showtopic=8485&mode=threaded
http://www2.ljworld.com/news/2007/dec/16/childrens_health/
http://groups.google.de/group/freepage-news/msg/75559697772dc865
http://doctorjp.blogspot.com/2007/12/mercury-causes-autism-new-research.html
http://forums.c-n.com/viewtopic.php?t=21362
http://myreader.co.uk/msg/11884644.aspx
http://www.autismspeaks.org/community/forums/showthread.php?t=3199
http://www.topix.com/forum/health/autism/TKJVRQK338JK30P78
http://www.medworm.com/rss/search.php?qu=Autism+Street&t=Autism+Street&s=Search&f=source
http://www.websitetoolbox.com/tool/post/tisdale/printadd?id=2365635
http://www.autismspeaks.org/community/forums/showthread.php?t=3695
http://www.dienviro.com/index1.aspx?BD=18676
What Faculty 1000 Biology said:
|
||||||||
|
|||||||
Mary Catherine
DeSoto
