Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- 1. Latent variable is a hidden variable unobserved neither in training or testing phases
- 2. Why probabilistic model
- 1. Quantify the uncertainty of the prediction
- 2. Missing values
- 3. Introduce latent variable may simplify models (less edges)
- 1. Fewer parameters
- 2. sometimes meaningful
- 3. can be harder to work with
- 4. Probabilistic clustering
- 1. Hard clustering
- 2. soft clustering (probability p(cluster idx|x) instead of cluster idx=f(x))
- 3. can be used in hyperparameter tuning in determining the number of clusters
- 4. Generate model of the data
- 5. Gaussian Mixture Model
- 1. Weighted multiple Gaussian distributions
- 2. Train GMM
- 1. MLE
- 2. hard to fit with stochastic optimizer
- 1. hard to comply some constraints
- 2. Expectation maximization algorithm is much faster and more efficient
- 6. training GMM
- 1. latent variable t, p(t) = weight
- 2. p(x|t) = Gaussian (x)
- 3. EM algorithm
- 1. Start with 2 randomly placed Gaussian parameters theta
- 2. Unitl convergence repeat to update Gaussian parameter
- 4. Global Optimum NP-hard
- 5. EM wonβt give global optimal (heuristics), suffer from local optimal
- 6. Choose the best run among several training attempts with different random initializations
- 7. Choose the one with the highest training log-likelihood or with highest validation log-likelihood
- 8. EM can train GMM faster than SGD and also handles complicated constraint, suffers from local maxima
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement