User:Jarle Pahr/SciPy: Difference between revisions
Jarle Pahr (talk | contribs) |
Jarle Pahr (talk | contribs) |
||
(22 intermediate revisions by the same user not shown) | |||
Line 25: | Line 25: | ||
=Optimization= | =Optimization= | ||
http:// | http://scicomp.stackexchange.com/questions/83/is-there-a-high-quality-nonlinear-programming-solver-for-python | ||
http://docs.scipy.org/doc/scipy/reference/ | Tutorial: http://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html#tutorial | ||
http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_slsqp.html | http://stackoverflow.com/questions/49926/open-source-alternative-to-matlabs-fmincon-function | ||
For comparison see http://www.mathworks.se/help/optim/ug/fmincon.html | |||
See also http://scicomp.stackexchange.com/questions/83/is-there-a-high-quality-nonlinear-programming-solver-for-python and http://openopt.org/SciPy | |||
See also: http://scipy-lectures.github.io/advanced/mathematical_optimization/ | |||
Constrained minimization of multivariate scalar functions (minimize): http://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html#tutorial-sqlsp | |||
==Functions== | |||
'''minimize:''' http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.minimize.html#scipy.optimize.minimize | |||
*General interface to several methods for minimization of multi-variate scalar function. | |||
'''fmin:''' http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin.html#scipy.optimize.fmin | |||
*Unconstrained,non-linear optimization | |||
*Uses the heuristic Nelder-Mead simplex method: http://en.wikipedia.org/wiki/Nelder%E2%80%93Mead_method | |||
*(Don't confuse with Danzig's simplex method for linear programming) | |||
*Equivalent to MATLABs fminsearch: http://stackoverflow.com/questions/19070943/numpy-scipy-analog-of-matlabs-fminsearch | |||
'''scipy.optimize.fmin_powell:''' | |||
*Unconstrained, non-linear optimization using Powell's conjugate direction method | |||
*http://en.wikipedia.org/wiki/Powell%27s_method | |||
fmin_cg: http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_cg.html#scipy.optimize.fmin_cg | |||
*Unconstrained, non-linear optimization using a conjugate gradient algorithm. | |||
'''Constrained Optimization BY Linear Approximation (COBYLA):''' http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_cobyla.html#scipy.optimize.fmin_cobyla | |||
*Constrained non-linear optimization with inequality constraints | |||
*Variable bounds and equality constraints not explicitly supported (must be implemented as inequality constraints). | |||
*Reference: | |||
Advances in Optimization and Numerical Analysis | |||
Mathematics and Its Applications Volume 275, 1994, pp 51-67 | |||
A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation. M. J. D. Powell. | |||
'''Sequential Least Squares Programming (SLSQP):''' | |||
*Nonlinear optimization method for problems where the objective function and constraints are twice differentiable. | |||
*Allows variable bounds and equality/inequality constraints. | |||
*fmin_slsqp: http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_slsqp.html | |||
*Seems more similar to MATLABs fmincon than optimize.fmin | *Seems more similar to MATLABs fmincon than optimize.fmin | ||
*Tutorial: docs.scipy.org/doc/scipy/reference/tutorial/optimize.html#tutorial-sqlsp | *Tutorial: docs.scipy.org/doc/scipy/reference/tutorial/optimize.html#tutorial-sqlsp | ||
*See also http://www.pyopt.org/reference/optimizers.slsqp.html | |||
*See also: http://scipy-user.10969.n7.nabble.com/using-scipy-optimize-fmin-slsqp-and-setting-bounds-None-None-td14134.html | |||
'''scipy.optimize.fmin_ncg:''' | |||
*Unconstrained optimization by Newton-Conjugate Gradient(NCG) method. | |||
'''scipy.optimize.fmin_tnc:''' | |||
*Truncated Newton-CG method. Allows variable bounds. Does not support equality/inequality constraints. | |||
'''scipy.optimize.fmin_bfgs:''' http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_bfgs.html#scipy.optimize.fmin_bfgs | |||
*Unconstrained, non-linear optimization using the Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm. | |||
http://en.wikipedia.org/wiki/Broyden%E2%80%93Fletcher%E2%80%93Goldfarb%E2%80%93Shanno_algorithm | |||
'''scipy.optimize.fmin_l_bfgs_b:''' http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_l_bfgs_b.html#scipy.optimize.fmin_l_bfgs_b | |||
*Non-linear optimization with bound constraints using the limited-memory BFGS with boundaries (L-BFGS-B) algorithm. | |||
*http://en.wikipedia.org/wiki/Limited-memory_BFGS | |||
'''Global solvers:''' | |||
Brute: http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.brute.html#scipy.optimize.brute | |||
'''scipy.optimize.anneal:''' http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.anneal.html#scipy.optimize.anneal | |||
*Simulated annealing. Non-linear optimization. Supports bound constraints. |
Latest revision as of 05:58, 14 January 2014
Notes on the SciPy Python library:
http://scipy-lectures.github.io/
http://oneau.wordpress.com/2011/02/28/simple-statistics-with-scipy/
Testing:
import scipy as sci sci.test()
Installation files
http://www.lfd.uci.edu/~gohlke/pythonlibs/#scipy-stack
Linalg
http://docs.scipy.org/doc/scipy/reference/linalg.html
Optimization
Tutorial: http://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html#tutorial
http://stackoverflow.com/questions/49926/open-source-alternative-to-matlabs-fmincon-function
For comparison see http://www.mathworks.se/help/optim/ug/fmincon.html
See also http://scicomp.stackexchange.com/questions/83/is-there-a-high-quality-nonlinear-programming-solver-for-python and http://openopt.org/SciPy
See also: http://scipy-lectures.github.io/advanced/mathematical_optimization/
Constrained minimization of multivariate scalar functions (minimize): http://docs.scipy.org/doc/scipy/reference/tutorial/optimize.html#tutorial-sqlsp
Functions
- General interface to several methods for minimization of multi-variate scalar function.
fmin: http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin.html#scipy.optimize.fmin
- Unconstrained,non-linear optimization
- Uses the heuristic Nelder-Mead simplex method: http://en.wikipedia.org/wiki/Nelder%E2%80%93Mead_method
- (Don't confuse with Danzig's simplex method for linear programming)
- Equivalent to MATLABs fminsearch: http://stackoverflow.com/questions/19070943/numpy-scipy-analog-of-matlabs-fminsearch
scipy.optimize.fmin_powell:
- Unconstrained, non-linear optimization using Powell's conjugate direction method
- http://en.wikipedia.org/wiki/Powell%27s_method
- Unconstrained, non-linear optimization using a conjugate gradient algorithm.
Constrained Optimization BY Linear Approximation (COBYLA): http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_cobyla.html#scipy.optimize.fmin_cobyla
- Constrained non-linear optimization with inequality constraints
- Variable bounds and equality constraints not explicitly supported (must be implemented as inequality constraints).
- Reference:
Advances in Optimization and Numerical Analysis Mathematics and Its Applications Volume 275, 1994, pp 51-67 A Direct Search Optimization Method That Models the Objective and Constraint Functions by Linear Interpolation. M. J. D. Powell.
Sequential Least Squares Programming (SLSQP):
- Nonlinear optimization method for problems where the objective function and constraints are twice differentiable.
- Allows variable bounds and equality/inequality constraints.
- fmin_slsqp: http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_slsqp.html
- Seems more similar to MATLABs fmincon than optimize.fmin
- Tutorial: docs.scipy.org/doc/scipy/reference/tutorial/optimize.html#tutorial-sqlsp
- See also http://www.pyopt.org/reference/optimizers.slsqp.html
- See also: http://scipy-user.10969.n7.nabble.com/using-scipy-optimize-fmin-slsqp-and-setting-bounds-None-None-td14134.html
scipy.optimize.fmin_ncg:
- Unconstrained optimization by Newton-Conjugate Gradient(NCG) method.
scipy.optimize.fmin_tnc:
- Truncated Newton-CG method. Allows variable bounds. Does not support equality/inequality constraints.
scipy.optimize.fmin_bfgs: http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_bfgs.html#scipy.optimize.fmin_bfgs
- Unconstrained, non-linear optimization using the Broyden–Fletcher–Goldfarb–Shanno (BFGS) algorithm.
http://en.wikipedia.org/wiki/Broyden%E2%80%93Fletcher%E2%80%93Goldfarb%E2%80%93Shanno_algorithm
scipy.optimize.fmin_l_bfgs_b: http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.fmin_l_bfgs_b.html#scipy.optimize.fmin_l_bfgs_b
- Non-linear optimization with bound constraints using the limited-memory BFGS with boundaries (L-BFGS-B) algorithm.
- http://en.wikipedia.org/wiki/Limited-memory_BFGS
Global solvers:
Brute: http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.brute.html#scipy.optimize.brute
scipy.optimize.anneal: http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.anneal.html#scipy.optimize.anneal
- Simulated annealing. Non-linear optimization. Supports bound constraints.