I am trying to understand the difference between discriminative models and generative models. One of the helpful answers at stack overflow is here: What is the difference between a generative and a discriminative algorithm?
In the top answer (see the above link), there was a simple example where there are only four data points of the form (x,y)
. The author of the answer said the following: The distribution p(y|x)
is the natural distribution for classifying a given example x
into a class y
, which is why algorithms that model this directly are called discriminative algorithms. Generative algorithms model p(x,y)
, which can be transformed into p(y|x)
by applying Bayes rule and then used for classification. However, the distribution p(x,y)
can also be used for other purposes. For example, you could use p(x,y)
to generate likely (x,y)
pairs.
I don’t quite understand how one could use p(x,y)
to generate likely (x,y)
pairs. Could someone help me by giving a specific example of (x,y)
pair that is generated by using the joint probability distribution p(x,y)
? Also, could someone also tell me why the conditional probability distribution p(y|x)
CANNOT be used to generate new pairs? Thank you so much.
I am quite confused about this. Please help me.
Oatmeals is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.