Experimental Wise Error Correction
Contents |
Descriptive Statistics Hypothesis Testing General Properties of Distributions Distributions Normal Distribution Sampling Distributions Binomial and Related Distributions Student's t Distribution Chi-square and F Distributions Other Key Distributions Testing for Normality and Symmetry ANOVA One-way ANOVA family wise error correction Factorial ANOVA ANOVA with Random or Nested Factors Design of Experiments ANOVA with
Family Wise Error Rate Correction
Repeated Measures Analysis of Covariance (ANCOVA) Miscellaneous Correlation Reliability Non-parametric Tests Time Series Analysis Survival Analysis Handling Missing Data Regression experiment wise error anova Linear Regression Multiple Regression Logistic Regression Multinomial and Ordinal Logistic Regression Log-linear Regression Multivariate Descriptive Multivariate Statistics Multivariate Normal Distribution Hotelling’s T-square MANOVA Repeated Measures Tests Box’s Test Factor Analysis Cluster Analysis Appendix Mathematical
Experiment Wise Error Definition
Notation Excel Capabilities Matrices and Iterative Procedures Linear Algebra and Advanced Matrix Topics Other Mathematical Topics Statistics Tables Bibliography Author Citation Blogs Tools Real Statistics Functions Multivariate Functions Time Series Analysis Functions Missing Data Functions Data Analysis Tools Contact Us Experiment-wise error rate We could have conducted the analysis for Example 1 of Basic Concepts for ANOVA by conducting multiple two sample tests. E.g. to decide whether experiment wise type 1 error or not to reject the following null hypothesis H0: μ1 = μ2 = μ3 We can use the following three separate null hypotheses: H0: μ1 = μ2 H0: μ2 = μ3 H0: μ1 = μ3 If any of these null hypotheses is rejected then the original null hypothesis is rejected. Note however that if you set α = .05 for each of the three sub-analyses then the overall alpha value is .14 since 1 – (1 – α)3 = 1 – (1 – .05)3 = 0.142525 (see Example 6 of Basic Probability Concepts). This means that the probability of rejecting the null hypothesis even when it is true (type I error) is 14.2525%. For k groups, you would need to run m = COMBIN(k, 2) such tests and so the resulting overall alpha would be 1 – (1 – α)m, a value which would get progressively higher as the number of samples increases. For example, if k = 6, then m = 15 and the probability of finding at least one significant t-test, purely by chance, even when the null hypothesis is true is over 50%. In fact, one of the reasons for performing ANOVA instead of separate t-tests is to reduce the type I error. The only problem is that once you have p
the experimentwise error rate is: where αew http://www.real-statistics.com/one-way-analysis-of-variance-anova/experiment-wise-error-rate/ is experimentwise error rate αpc is the per-comparison error rate, and c is the number of comparisons. For example, if 5 independent comparisons http://davidmlane.com/hyperstat/A43646.html were each to be done at the .05 level, then the probability that at least one of them would result in a Type I error is: 1 - (1 - .05)5 = 0.226. If the comparisons are not independent then the experimentwise error rate is less than . Finally, regardless of whether the comparisons are independent, αew ≤ (c)(αpc) For this example, .226 < (5)(.05) = 0.25.
Tour Start here for a quick overview of the site Help Center Detailed answers to any questions you might have Meta Discuss the workings and policies of this site http://stats.stackexchange.com/questions/156007/correcting-for-family-wise-error-rate-with-series-of-repeated-measures-anova About Us Learn more about Stack Overflow the company Business Learn more about hiring developers or posting ads with us Cross Validated Questions Tags Users Badges Unanswered Ask Question _ Cross Validated is a question and answer site for people interested in statistics, machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute: Sign up Here's how it works: Anybody wise error can ask a question Anybody can answer The best answers are voted up and rise to the top Correcting for family-wise error rate with series of repeated measures ANOVA? up vote 1 down vote favorite I am trying to make requested revisions to an accepted manuscript, and I am baffled by the following comment from a reviewer: "Eight hypothesis tests are reported in the final paragraph wise error rate of the results section but the authors do not say if they corrected for their Type I familywise error rate. What correction was done or should be done?" To give background, I had a pre- and post-test design, and change was assessed on eight different (theoretically distinct) variables. So, basically, I conducted eight separate repeated measures ANOVAs. I can understand that Type I error rate would be inflated from doing multiple tests, but I have not come across any correction for use with a series of RM ANOVA. Is there a standard procedure or recommended correction? repeated-measures type-i-errors share|improve this question asked Jun 8 '15 at 16:39 Sara Sohr-Preston 61 Because they are "(theoretically distinct) variables", you could simply state that the eight tests represent different families of tests and therefore each test is conducted with an alpha of .05. Otherwise, if you were in fact to adjust your alpha level, you could use a Bonferroni correction and use an alpha of .05/8. –Patrick Coulombe Jun 8 '15 at 17:06 add a comment| active oldest votes Know someone who can answer? Share a link to this question via email, Google+, Twitter, or Facebook. Your
be down. Please try the request again. Your cache administrator is webmaster. Generated Thu, 13 Oct 2016 22:35:39 GMT by s_ac5 (squid/3.5.20)