Controlling For Family Wise Error Rate
Contents |
may be challenged and removed. (June 2016) (Learn how and when to remove this template message) In statistics, family-wise error rate (FWER) is family wise error rate post hoc the probability of making one or more false discoveries, or type
Family Wise Error Rate R
I errors, among all the hypotheses when performing multiple hypotheses tests. Contents 1 History 2 Background 2.1 how to calculate family wise error rate Classification of multiple hypothesis tests 3 Definition 4 Controlling procedures 4.1 The Bonferroni procedure 4.2 The Šidák procedure 4.3 Tukey's procedure 4.4 Holm's step-down procedure (1979) 4.5 family wise error rate formula Hochberg's step-up procedure 4.6 Dunnett's correction 4.7 Scheffé's method 4.8 Resampling procedures 5 Alternative approaches 6 References History[edit] Tukey coined the terms experimentwise error rate and "error rate per-experiment" to indicate error rates that the researcher could use as a control level in a multiple hypothesis experiment.[citation needed] Background[edit] Within the statistical framework, there
Family Wise Error Rate Definition
are several definitions for the term "family": Hochberg & Tamhane defined "family" in 1987 as "any collection of inferences for which it is meaningful to take into account some combined measure of error".[1][pageneeded] According to Cox in 1982, a set of inferences should be regarded a family:[citation needed] To take into account the selection effect due to data dredging To ensure simultaneous correctness of a set of inferences as to guarantee a correct overall decision To summarize, a family could best be defined by the potential selective inference that is being faced: A family is the smallest set of items of inference in an analysis, interchangeable about their meaning for the goal of research, from which selection of results for action, presentation or highlighting could be made (Yoav Benjamini).[citation needed] Classification of multiple hypothesis tests[edit] Main article: Classification of multiple hypothesis tests The following table defines various errors committed when testing multiple null hypotheses. Suppose we have a number m of
Alerts Search this journal Advanced Journal Search » Impact Factor:4.634 | Ranking:Medical Informatics 1 out of 20 | Statistics & Probability 1 out of 123 | Mathematical & Computational Biology 4 family wise error rate correction out of 56 | Health Care Sciences & Services 4 out of 87 | familywise error rate 5-Year Impact Factor:4.247 | 5-Year Ranking:Medical Informatics 2 out of 20 | Statistics & Probability 5 out of 123 | Mathematical
Per Comparison Error Rate
& Computational Biology 6 out of 56 | Health Care Sciences & Services 9 out of 87 Source:2016 Release of Journal Citation Reports, Source: 2015 Web of Science Data Controlling the familywise error rate in https://en.wikipedia.org/wiki/Family-wise_error_rate functional neuroimaging: a comparative review Thomas Nichols Department of Biostatistics, University of Michigan, Ann Arbor, MI, USA, nichols{at}umich.edu Satoru Hayasaka Department of Biostatistics, University of Michigan, Ann Arbor, MI, USA Abstract Functional neuroimaging data embodies a massive multiple testing problem, where 100 000 correlated test statistics must be assessed. The familywise error rate, the chance of any false positives is the standard measure of Type I errors in multiple testing. In http://smm.sagepub.com/content/12/5/419.short?rss=1&ssource=mfc this paper we review and evaluate three approaches to thresholding images of test statistics: Bonferroni, random field and the permutation test. Owing to recent developments, improved Bonferroni procedures, such as Hochberg’s methods, are now applicable to dependent data. Continuous random field methods use the smoothness of the image to adapt to the severity of the multiple testing problem. Also, increased computing power has made both permutation and bootstrap methods applicable to functional neuroimaging. We evaluate these approaches on t images using simulations and a collection of real datasets. We find that Bonferroni-related tests offer little improvement over Bonferroni, while the permutation method offers substantial improvement over the random field method for low smoothness and low degrees of freedom. We also show the limitations of trying to find an equivalent number of independent tests for an image of correlated test statistics. CiteULike Connotea Delicious Digg Facebook Google+ LinkedIn Mendeley Reddit StumbleUpon Twitter What's this? « Previous | Next Article » Table of Contents This Article doi: 10.1191/0962280203sm341ra Stat Methods Med Res October 2003 vol. 12 no. 5 419-446 » Abstract Full Text (PDF) References Services Email this article to a colleague Alert me when this article is cited Alert me if a correction is posted Similar articles in this journal Similar articles
describe a number of different ways of testing which means are different Before describing the tests, it is necessary to consider two different ways of thinking about http://www.psych.utoronto.ca/courses/c1/chap12/chap12.html error and how they are relevant to doing multiple comparisons Error Rate per Comparison (PC) This is simply the Type I error that we have talked about all along. So far, we have been simply setting its value at .05, a 5% chance of making an error Familywise Error Rate (FW) Often, after an ANOVA, we want to do a number of comparisons, not just one The collection of error rate comparisons we do is described as the "family" The familywise error rate is the probability that at least one of these comparisons will include a type I error Assuming that a ¢ is the per comparison error rate, then: The per comparison error: a = a ¢ but, the familywise error: a = 1 - (1-a ¢ )c Thus, if we do two comparisons, but keep a ¢ at 0.05, wise error rate the FWerror will really be: a = 1 - (1 - 0.05)2 =1 - (0.95)2 = 1 - 0.9025 = 0.0975 Thus, there is almost a 10% chance of one of the comparisons being significant when we do two comparisons, even when the nulls are true. The basic problem then, is that if we are doing many comparisons, we want to somehow control our familywise error so that we don’t end up concluding that differences are there, when they really are not The various tests we will talk about differ in terms of how they do this They will also be categorized as being either "A priori" or "post hoc" A priori: A priori tests are comparisons that the experimenter clearly intended to test before collecting any data Post hoc: Post hoc tests are comparisons the experimenter has decided to test after collecting the data, looking at the means, and noting which means "seem" different. The probability of making a type I error is smaller for A priori tests because, when doing post hoc tests, you are essentially doing all possible comparisons before deciding which to test in a formal statistical manner Steve: Significant F issue An example for context See page 351 for