解无约束优化问题的新的两点步长梯度方法(英文).docx

解无约束优化问题的新的两点步长梯度方法(英文)
Introduction
Unconstrainedoptimizationproblemsariseinvariousdisciplinessuchasengineering,economics,andscience.Thegoalofunconstrainedoptimizationistofindthevaluesofthedecisionvariablesthatminimizeormaximizeaspecificobjectivefunction.Thegradientdescentmethodisoneofthemostpopularoptimizationmethodsusedtosolveunconstrainedoptimizationproblems.Inthispaper,weproposeanewtwo-pointstepsizegradientdescentmethodtosolvetheunconstrainedoptimizationproblem.Thispaperwilldiscussthealgorithmofthismethod,itsconvergenceproperties,anditsadvantagesovertheexistinggradientdescentmethods.
Algorithmfortwo-pointstepsizegradientdescentmethod
Thestandardgradientdescentmethodupdatesthedecisionvariablesiterativelyasfollows:
x_k+1=x_k-t_k*∇f(x_k)
wherex_kisthevalueofthedecisionvariableatiterationk,t_kisthestepsizeatiterationk,and∇f(x_k)isthegradientoftheobjectivefunctionatx_k.
Ourproposedtwo-pointstepsizegradientdescentmethodisanextensionofthestandardgradientdescentmethod.Insteadofusingafixedstepsizeineachiteration,weuseatwo-pointstepsizethatiscomputedbasedonthecurvatureoftheobjectivefunction.Thealgorithmforthetwo-pointstepsizegradientdescentmethodisasfollows:
Step1:Initializex_0andsetk=0.
Step2:Computethegradientoftheobjectivefunctionatx_k,i.e.,∇f(x_k).
Step3:Computethefunctionvalueatthetwocandidatepointsx_k-t_k/2*∇f(x_k)andx_k-t_k*∇f(x_k),wheret_kisthestepsizeatiterationk.
Step4:Choosethecandidatepointthatproducesthesmallestfunctionvalueandsetx_k+1=x_k-t_k*∇f(x_k).
Step5:Updatethestepsizeusingtheformulat_k+1=α*t_k,whereαisaconstantbetween0and1.
Step6:Ifthestoppingcriterionissatisfied,stop.Otherwise,setk=k+1andgotostep2.
Convergenceproperties
Wenowprovetheconvergenceofthetwo-pointstepsizegradientdescentmethod.Weassumethattheobjectivefunctionf(x)iscontinuouslydifferentiableandhasauniqueglobalminimum.
Theorem:Let{x_k}bethesequencegeneratedbythetwo-pointstepsizegradientdescentmethod.Supposethatthestepsizesequence{t_k}satisfies:
1)∑(k=0)^∞t_k=∞
2)∑(k=0)^∞t_k^2<∞
Then{x_k}conv