Downhill Simplex (Nelder-Mead)

[altenbach]

As I said, my current implementation is a near literal translation of the NI code and maintains all of the shortcomings (one of the reasons I did no publish it in the past :D). A +1 step for each parameter to generate the initial simplex is really bad for e.g. the "Sum of 3 Gaussians with offset" example, we have 3 Gaussians with dramatically diffferent amplitudes, each ~0.01 wide and spaced about ~0.1 apart. This means that the function after the increment of each parameter might land in a completely different valley and will never find it's way back to the global minimum. My own code requires an array of increments ("simplex step") in the data variant, which makes it no longer drop-in compatible, but makes it more flexible.

Another thing you discussed is constraining the parameters to a certain range. Yes, as Bob mentioned, you can just give a large penalty if you step outside (after you define what "large" should mean). Another option would be to just coerce the parameters to the valid range at each step. I have not tried, but that's what "Constrained nonlinear curve fit" basically does. Starting with my code above, it would be trivial to create a "constrained Nelder-Mead fit" that would be input compatible with the constrained LM version. All we need is add the "parameter bounds" input and change the code accordingly. You could even recycle the "CNR project parameters" subVI found inside constrained nonlinear curve fit. 😄

[blessedk]

Bob's idea of introducing a "heavy penalty" ( multiplying by say 1000) is great. It appears this may be the only approach to constrain the parameters. I have tried forcing the parameters to not go beyond a certain range ( so for example if the realistic range is from 2 to 10, and the parameter for a given iteration goes to 15, I will overwrite that to 10), but this didn't work. So I think the "heavy penalty" approach is the way to go.

[Bob_Schor]

I cannot take credit for constraining parameters for the Simplex Method to a region by "inflating" the value computed for the Simplex point.  I learned of this method from the literature, including this procedure for parameter constraint.   My first time to implement this algorithm was in Pascal, where I was fitting transfer functions to data for my thesis work (I don't think LabVIEW had been invented yet ...).  My most recent implementation was, of course, in LabVIEW, where I used it to auto-calibrate a triaxial Accelerometer by implementing a 10-second procedure that involved holding the accelerometer motionless for a tenth of a second in six (approximate) orientations (flat, upside-down, right-side-down, left-side-down, nose-down, and nose-up) and taking (raw) readings from the three axes.  If the accelerometer channels are linear, this gives you six non-linear equations in six unknowns, and the solution takes milliseconds.

Bob Schor