Comparison-wise Type 1 Error
Contents |
Descriptive Statistics Hypothesis Testing General Properties of Distributions Distributions Normal Distribution Sampling Distributions Binomial and Related Distributions Student's t Distribution Chi-square and F Distributions Other Key Distributions comparison wise error rate Testing for Normality and Symmetry ANOVA One-way ANOVA Factorial ANOVA ANOVA with family wise type 1 error Random or Nested Factors Design of Experiments ANOVA with Repeated Measures Analysis of Covariance (ANCOVA) Miscellaneous Correlation Reliability type 1 error multiple comparisons Non-parametric Tests Time Series Analysis Survival Analysis Handling Missing Data Regression Linear Regression Multiple Regression Logistic Regression Multinomial and Ordinal Logistic Regression Log-linear Regression Multivariate Descriptive Multivariate Statistics Multivariate
Experiment Wise Error
Normal Distribution Hotelling’s T-square MANOVA Repeated Measures Tests Box’s Test Factor Analysis Cluster Analysis Appendix Mathematical Notation Excel Capabilities Matrices and Iterative Procedures Linear Algebra and Advanced Matrix Topics Other Mathematical Topics Statistics Tables Bibliography Author Citation Blogs Tools Real Statistics Functions Multivariate Functions Time Series Analysis Functions Missing Data Functions Data Analysis Tools Contact Us Experiment-wise error rate experiment wise error anova We could have conducted the analysis for Example 1 of Basic Concepts for ANOVA by conducting multiple two sample tests. E.g. to decide whether or not to reject the following null hypothesis H0: μ1 = μ2 = μ3 We can use the following three separate null hypotheses: H0: μ1 = μ2 H0: μ2 = μ3 H0: μ1 = μ3 If any of these null hypotheses is rejected then the original null hypothesis is rejected. Note however that if you set α = .05 for each of the three sub-analyses then the overall alpha value is .14 since 1 – (1 – α)3 = 1 – (1 – .05)3 = 0.142525 (see Example 6 of Basic Probability Concepts). This means that the probability of rejecting the null hypothesis even when it is true (type I error) is 14.2525%. For k groups, you would need to run m = COMBIN(k, 2) such tests and so the resulting overall alpha would be 1 – (1 – α)m, a value which would get progressively higher as the number of samples increases. For example, if k = 6, then m = 15 and the probability of fi
the experimentwise error rate is: where αew http://www.real-statistics.com/one-way-analysis-of-variance-anova/experiment-wise-error-rate/ is experimentwise error rate αpc is the per-comparison error rate, and c is the number of comparisons. For example, if 5 independent comparisons http://davidmlane.com/hyperstat/A43646.html were each to be done at the .05 level, then the probability that at least one of them would result in a Type I error is: 1 - (1 - .05)5 = 0.226. If the comparisons are not independent then the experimentwise error rate is less than . Finally, regardless of whether the comparisons are independent, αew ≤ (c)(αpc) For this example, .226 < (5)(.05) = 0.25.
specific comparison There are many occasions on which the comparisons among means are more complicated than simply comparing one mean with another. This section shows how to test these more complex comparisons. The methods in this section assume that the comparison among means was decided on before looking at the data. Therefore these comparisons are http://onlinestatbook.com/chapter10/specific_comparisons.html called planned comparisons. A different procedure is necessary for unplanned comparisons. Let's begin with the made-up data from a hypothetical experiment shown in Table 1. Twelve subjects were selected from a population of high-self-esteem subjects (esteem = 1) and an additional 12 subjects were selected from a population of low-self-esteem subjects (esteem = 2). Subjects then performed on a task and (independent of how well they really did) half were told they succeeded (outcome = 1) and the other half were told they wise error failed (outcome = 2). Therefore there were six subjects in each esteem/success combination and 24 subjects altogether. After the task, subjects were asked to rate (on a 10-point scale) how much of their outcome (success or failure) they attributed to themselves as opposed to being due to the nature of the task. Table 1. Data from Hypothetical Experiment outcome esteem attrib 1 1 7 1 1 8 1 1 7 1 1 8 1 1 9 1 1 5 1 2 6 1 2 type 1 error 5 1 2 7 1 2 4 1 2 5 1 2 6 2 1 4 2 1 6 2 1 5 2 1 4 2 1 7 2 1 3 2 2 9 2 2 8 2 2 9 2 2 8 2 2 7 2 2 6 The means of the four conditions are shown in Table 2. Table 2. Mean ratings of self-attributions of success or failure. Success High Self Esteem 7.333 Low Self Esteem 5.500 Failure High Self Esteem 4.833 Low Self Esteem 7.833 There are several questions we can ask about the data. We begin by asking whether, on average, subjects who were told they succeeded differed significantly from subjects who were told they failed. The means for subjects in the success condition are 7.333 for the high-self-esteem subjects and 5.500 for the low-self-esteem subjects. Therefore, the mean of all subjects in the success condition is (7.333 + 5.500)/2 = 6.417. Similarly, the mean for all subjects in the failure condition is (4.833 + 7.833)/2 = 6.333. The question is, how do we do a significance test for this difference of 6.417-6.333 = 0.083? The first step is to express this difference in terms of a linear combination of a set of coefficients and the means. This may sound complex, but it is really pretty easy. We can compute the mean of the success conditions by multiplying each success mean by 0.5 and then adding the result. In other words, we compute (
be down. Please try the request again. Your cache administrator is webmaster. Generated Thu, 06 Oct 2016 00:45:32 GMT by s_hv902 (squid/3.5.20)