Gaussian mixture model sometimes seems to fit very badly

Question

In the following code, I fit a gaussian mixture model (GMM) to some randomly sampled data. I do this twice. Each time, the data represent two well separated gaussians, the only difference being the seed I use for the random number generator.   N = 100000; EFFECT_SIZE = 5; seedList = [1 6]; for s = seedList rng(s) X = [randn(N,1); randn(N,1)+EFFECT_SIZE]; figure hist(X,101) GMModel = fitgmdist(X,2) end If you run that code -- you will need the Statistics Toolbox -- you will see that the first distribution is fit very well, and the second one terribly. I am trying to understand why. I would expect such well separated peaks to be fit well essentially every time.   This is not a fluke. I ran 1,000 different seeds, and got the bad fit about 18% of the time. Also, those bad fits tend to cluster relatively close the same parameter values.   Any thoughts? I am a novice at using GMM, so maybe I am just naive about how well this should do. I am running R2014b on Mac OS X Yosemite.

Kshitij Singh · Accepted Answer

Hi cyclist, 
the default starting values are selected from the data at random, and you have discovered that this sometimes does not work well. There's an argument 'replicates' that you can use to have it try this multiple times and deliver the bone fit. However, in more recent releases including R2014b there is a new argument 'Start','plus' that uses a better starting method, one based on the kmeans++ algorithm for clustering. Here's a variant of your function that shows the relative performance of the default random start and the 'plus' start:
 
N = 100000;
EFFECT_SIZE = 5;
seedList = 1:20;
means1 = zeros(length(seedList),2);
means2 = zeros(length(seedList),2);
for s = seedList
    s
    rng(s)
    X = [randn(N,1); randn(N,1)+EFFECT_SIZE];
    rng(s)  % randomness is used in the fit also
    GMModel = fitgmdist(X,2);
    means1(s,:) = GMModel.mu';
    rng(s)
    GMModel = fitgmdist(X,2,'start','plus');
    means2(s,:) = GMModel.mu';
end
plot(means1(:,1),means1(:,2),'bx',means2(:,1),means2(:,2),'ro')

Gaussian mixture model sometimes seems to fit very badly

Need a Custom Version or Complete Simulation for This Problem?

How to I use get param to get a value from an output of a bus?

How do I uninstall and reinstall MathWorks Service Host?

Why am I receiving error messages about singularities in my Simulink model?

Simulink turn off data logging

MatLab App Design - how to use default value for Edit Field function?

When is subsystem reference a better choice than model reference?

Modeling and Simulation of an Induction Motor Using Sliding Mode Control in MATLAB

Solar Inverter with BMS in MATLAB Simulink Complete Simulation Project

Simulation and Kinematic Analysis of a 5-DOF RRRRRP Robotic Manipulator Using MATLAB

Get a Free Consultation or a Sample Assignment Review!