Naive Bayes Error Rate
Contents |
categories) and is analogous to the irreducible error.[1][2] A number of approaches to the estimation of the Bayes error rate exist. One method seeks to obtain analytical bounds which are bayes error rate example inherently dependent on distribution parameters, and hence difficult to estimate. Another approach focuses bayes error rate in r on class densities, while yet another method combines and compares various classifiers.[2] The Bayes error rate finds important use in error rate definition the study of patterns and machine learning techniques.[3] Error determination[edit] In terms of machine learning and pattern classification, the labels of a set of random observations can be divided into 2 or more
Bayes Error Example
classes. Each observation is called an instance and the class it belongs to is the label. The Bayes error rate of the data distribution is the probability an instance is misclassified by a classifier that knows the true class probabilities given the predictors. For a multiclass classifier, the Bayes error rate may be calculated as follows:[citation needed] p = ∫ x ∈ H i ∑ C i error rate classification ≠ C max,x P ( C i | x ) p ( x ) d x {\displaystyle p=\textstyle \int \limits _{x\in H_{i}}\sum _{C_{i}\neq C_{\text{max,x}}}P(C_{i}|x)p(x)\,dx} where x is an instance, Ci is a class into which an instance is classified, Hi is the area/region that a classifier function h classifies as Ci.[clarification needed] The Bayes error is non-zero if the classification labels are not deterministic, i.e., there is a non-zero probability of a given instance belonging to more than one class.[citation needed] See also[edit] Naive Bayes classifier References[edit] ^ Fukunaga, Keinosuke (1990) Introduction to Statistical Pattern Recognition by ISBN 0122698517 pages 3 and 97 ^ a b K. Tumer, K. (1996) "Estimating the Bayes error rate through classifier combining" in Proceedings of the 13th International Conference on Pattern Recognition, Volume 2, 695–699 ^ Hastie, Trevor. The Elements of Statistical Learning (2nd ed.). http://statweb.stanford.edu/~tibs/ElemStatLearn/: Springer. p.17. ISBN978-0387848570. This statistics-related article is a stub. You can help Wikipedia by expanding it. v t e Retrieved from "https://en.wikipedia.org/w/index.php?title=Bayes_error_rate&oldid=743880528" Categories: Statistical classificationBayesian statisticsStatistics stubsHidden categories: All articles with unsourced statementsArticles with unsourced statements from February 2013Wikipedia articles needing clarification from February 2013All stub articles Navigation menu Personal tools Not logged inTalkContributionsCreate
Tour Start here for a quick overview of the site Help Center Detailed answers to any questions you might have Meta Discuss the workings and policies of this site About Us Learn more about Stack Overflow bayes error rate explained the company Business Learn more about hiring developers or posting ads with us Cross
Estimating The Bayes Error Rate Through Classifier Combining
Validated Questions Tags Users Badges Unanswered Ask Question _ Cross Validated is a question and answer site for people interested in statistics,
Classification Error Rate In R
machine learning, data analysis, data mining, and data visualization. Join them; it only takes a minute: Sign up Here's how it works: Anybody can ask a question Anybody can answer The best answers are voted up and https://en.wikipedia.org/wiki/Bayes_error_rate rise to the top Calculating the error of Bayes classifier analytically up vote 5 down vote favorite 3 If two classes $w_1$ and $w_2$ have normal distribution with known parameters ($M_1$, $M_2$ as their means and $\Sigma_1$,$\Sigma_2$ are their covariances) how we can calculate error of the Bayes classifier for them theorically? Also suppose the variables are in N-dimensional space. Note: A copy of this question is also available at http://math.stackexchange.com/q/11891/4051 that is http://stats.stackexchange.com/questions/4949/calculating-the-error-of-bayes-classifier-analytically still unanswered. If any of these question get answered, the other one will be deleted. probability self-study normality naive-bayes bayes-optimal-classifier share|improve this question edited May 25 at 5:26 Tim 23.5k454102 asked Nov 26 '10 at 19:36 Isaac 490615 1 Is this question the same as stats.stackexchange.com/q/4942/919 ? –whuber♦ Nov 26 '10 at 20:40 @whuber Your answer suggests it is the case indeed. –chl♦ Nov 26 '10 at 20:47 @whuber: Yes. i don't know this question suited to which one. I am waiting for a response for one to remove the other one. Is it against the rules? –Isaac Nov 26 '10 at 20:49 It might be easier, and surely would be cleaner, to edit the original question. However, sometimes a question is restarted as a new one when the earlier version collects too many comments that are made irrelevant by the edits, so it's a judgment call. In any event it's helpful to place cross-references between closely related questions to help people connect them easily. –whuber♦ Nov 26 '10 at 20:52 add a comment| 3 Answers 3 active oldest votes up vote 14 down vote accepted There's no closed form, but you could do it numerically. As a concrete example, consider two Gaussians with following parameters $$\mu_1=\left(\begin{matrix} -1\\\\ -1 \end{matrix}
here for a quick overview of the site Help Center Detailed answers to any questions you might have Meta Discuss the workings and policies of this site About Us Learn more about Stack Overflow the company Business http://stackoverflow.com/questions/19129141/naive-bayes-and-logistic-regression-error-rate Learn more about hiring developers or posting ads with us Stack Overflow Questions Jobs Documentation Tags Users Badges Ask Question x Dismiss Join the Stack Overflow Community Stack Overflow is a community of 6.2 million programmers, just like you, helping each other. Join them; it only takes a minute: Sign up Naive Bayes and Logistic Regression Error Rate up vote 3 down vote favorite 2 I have been trying to figure out the correlation between error rate the error rate and the number of features in both of these models. I watched some videos, and the creator of the video said that a simple model can be better than a complicated model. So I figured that the more features I had the greater the error rate would be. This did not prove to be true in my work, and when I had less features the error rate went up. I'm not sure bayes error rate if I'm doing this incorrectly, or if the guy in the video made a mistake. Can someone care to explain? I also am curious how features relate to Logistic Regression's error rate as well. machine-learning share|improve this question asked Oct 2 '13 at 2:34 Taztingo 5351520 2 This isn't a programming question; stats.stackexchange.com is more appropriate. –Dougal Oct 2 '13 at 2:43 1 That said, "a simple model can be better than a complicated model" doesn't mean a simple model is always better than a complicated model; there's a tradeoff. Otherwise a constant predictor would be the best possible model and there would be no such field as machine learning. –Dougal Oct 2 '13 at 2:44 Thank you, I will ask my questions there from now on. –Taztingo Oct 2 '13 at 3:12 The complexity of a logistic regression classifier is identical to the Naive Bayes classifier if the event space is the same---they form a generative/discriminative pair, and have identical forms of classification rule. See ai.stanford.edu/~ang/papers/nips01-discriminativegenerative.pdf –Ben Allison Oct 2 '13 at 9:32 add a comment| 1 Answer 1 active oldest votes up vote 20 down vote accepted Naive Bayes and Logistic Regression are a "generative-discriminative pair," meaning they have the same model form (a linear classifier), but they estimate parameters in different ways. For fea
be down. Please try the request again. Your cache administrator is webmaster. Generated Thu, 20 Oct 2016 23:09:54 GMT by s_nt6 (squid/3.5.20)