Home | Current | Past volumes | About | Login | Notify | Contact | Search
 Probability Surveys > Vol. 2 (2005) open journal systems 


On some recent aspects of stochastic control and their applications

Huyên Xuan Pham, Laboratoire de Probabilités et Modèles Aléatoires, University Paris 6-Paris 7


Abstract
This paper is a survey on some recent aspects and developments in stochastic control. We discuss the two main historical approaches, Bellman's optimality principle and Pontryagin's maximum principle, and their modern exposition with viscosity solutions and backward stochastic differential equations. Some original proofs are presented in a unifying context including degenerate singular control problems. We emphasize key results on characterization of optimal control for diffusion processes, with a view towards applications. Some examples in finance are detailed with their explicit solutions. We also discuss numerical issues and open questions.

AMS 2000 subject classifications: 93E20, 49J20, 49L20, 60H30.

Keywords: Controlled diffusions, dynamic programming, maximum principle, viscosity solutions, backward stochastic differential equations, finance.

Creative Common LOGO

Full Text: PDF


Pham, Huyên Xuan, On some recent aspects of stochastic control and their applications, Probability Surveys, 2, (2005), 506-549 (electronic).

References

[1]   Akian M. (1990) : “Analyse de l’algorithme multigrille FMGH de résolution d’équations d’Hamilton-Jacobi-Bellman”, Analysis and Optimization of systems, Lect. Notes in Contr. and Inf. Sciences, 144, Springer-Verlag, pp. 113-122.

[2]   Antonelli F. (1993) : “Backward-forward stochastic differential equations”, Annals of Appl. Prob., 3, 777-793.

[3]   Artzner P., Delbaen F., Eber J.M. and D. Heath (1999) : “Coherent measures of risk”, Mathematical Finance, 9, 203-228.

[4]   Bally V. and G. Pagès (2003) : “Error analysis of the optimal quantization algorithm for obstacle problems”, Stochastic Processes and their Applications, 106, 1-40.

[5]   Baras J., Elliott R. and M. Kohlmann (1989) : “The partially obsered stochastic minimum principle”, SIAM J. Cont. Optim., 27, 1279-1292.

[6]   Barles G. and E. Jakobsen (2004) : “Error bounds for monotone approximations schemes for Hamilton-Jacobi-Bellman equations”, to appear in SIAM J. Num. Anal..

[7]   Barles G. and P. Souganidis (1991) : : “Convergence of approximation schemes for fully non linear second-order equations”, Asymptotics Analysis, 4, pp.271-283.

[8]   Bellman R. (1957) : Dynamic programming, Princeton university press.

[9]   Bensoussan A. and J.L. Lions (1982) : Contrôle impulsionnel et inéquations variationnelles, Dunod.

[10]   Bensoussan A. (1992) : Stochastic control of partially observable systems, Cambridge University Press.

[11]   Bensoussan A. and H. Nagai (1991) : “An ergodic control problem arising from the principal eigenfunction of an elliptic operator”, J. Math. Soc. Japan, 43, 49-65.

[12]   Bielecki T. and S. Pliska (1999) : “Risk-sensitive dynamic asset management”, Applied Math. Optim., 39, 337-360. MR1675114

[13]   Blanchet-Scalliet C., El Karoui N., Jeanblanc M. and L. Martellini (2002) : “Optimal investment and consumption when time-horizon is uncertain”, Preprint.

[14]   Bobrovnytska O. and M. Schweizer (2004) : “Mean-variance hedging and stochastic control : beyond the Brownian setting”, IEE Trans. on Aut. Cont., 49, 1-14.

[15]   Borkar V. (1989) : Optimal control of diffusion processes, Pitman Research Notes in Math., 203. MR1005532

[16]   Borkar V. (2005) : “Controlled diffusion processes”, Probability surveys, 2, 213-244.

[17]   Bouchard B. and H. Pham (2004) : “Wealth-path dependent utility maximization in incomplete markets”, Finance and Stochastics, 8, 579-603.

[18]   Bouchard B. and N. Touzi (2004) : “Discrete time approximation and Monte Carlo simulation of backward stochastic differential equations”, Stochastic Processes and their Applications, 111, 175-206.

[19]   Brekke K. and B. Oksendal (1994) : “Optimal switching in an economic activity under uncertainty”, SIAM J. Cont. Optim., 32, 1021-1036.

[20]   Broadie M., Cvitanic J. and M. Soner (1998) : “Optimal replication of contingent claims under portfolio constraints”, Review of Fin. Studies, 11, 59-79.

[21]   Carmona R. and N. Touzi (2004) : “Optimal multiple stopping and the valuation of swing options”, to appear in Mathematical Finance.

[22]   Çetin U., Jarrow R. and P. Protter (2004) : “Liquidity risk and arbitrage pricing theory”, Finance and Stochastics, 8, 311-341.

[23]   Crandall M., Ishii. H and P.L. Lions (1992) : “User’s Guide to Viscosity Solutions of Second Order Partial Differential Equations”, Bull. Amer. Math. Soc., 27, 1-67.

[24]   Cvitanic J., Pham H. and N. Touzi (1999a) : “Superreplication in stochastic volatility models under portfolio constraints”, Journal of Appplied Probability, 36, 523-545. MR1724796

[25]   Cvitanic J., Pham H. and N. Touzi (1999b) : “A closed form solution for the super-replication problem under transaction costs”, Finance and Stochastics, 3, 35-54.

[26]   Davis M. and P. Varaiya (1973) : “Dynamic programming conditions for partially observable systems”, SIAM J. Cont., 11, 226-261.

[27]   Delarue, F. (2002) : “On the existence and uniqueness of solutions to FBSDEs in a non-degenerate case”. Stoch. Process. Appl., 99, 209-286.

[28]   Delarue F. and S. Menozzi (2004) : “A forward-backward stochastic algorithm for quasi-linear PDEs”, to appear in Annals of Applied Probability.

[29]   Douglas J., Ma J. and P. Protter (1996) : “Numerical methods for forward-backward stochastic differential equations”, Annals of Applied Probability, 6, 940-968.

[30]   Duckworth K. and M. Zervos (2001) : “A model for investment decisions with switching costs”, Annals of Applied Probability, 11, 239-250.

[31]   El Karoui N. (1981) : Les Aspects Probabilistes du Contrôle Stochastique, Lect. Notes in Math., 816, Springer Verlag.

[32]   El Karoui N. and L. Mazliak (editors) (1997) : Backward stochastic differential equations, Pitman research notes in mathematics series.

[33]   El Karoui N., S. Peng and M.C. Quenez (1997) : “Backward stochastic differential equations in finance”, Mathematical Finance, 7, 1-71. MR1434407

[34]   El Karoui N. and M.C. Quenez (1995) : “Dynamic programming and pricing contingent claims in incomplete markets”, SIAM J. Cont. Optim., 33, 29-66. MR1311659

[35]   Fitzpatrick B. and W. Fleming (1991) : “Numerical methods for an optimal Investment-Consumption Model”, Mathematics of Operation Research, 16, pp.823-841.

[36]   Fleming W. (1968) : “Optimal control of partially observable systems”, SIAM J. Cont. Optim., 6, 194-214.

[37]   Fleming, W. and D. Hernandez-Hernandez (2003) : “An optimal consumption model with stochastic volatility”, Finance Stoch., 7, 245-262.

[38]   Fleming W. and W. McEneaney (1995) : “Risk-sensitive control on an infinite horizon”, SIAM J. Cont. and Optim., 33, 1881-1915.

[39]   Fleming W. and R. Rishel (1975) : Deterministic and stochastic optimal control, Springer Verlag.

[40]   Fleming W. and S. Sheu (2000) : “Risk sensitive control and an optimal investment model”, Math. Finance, 10, 197-213. MR1802598

[41]   Fleming W. and M. Soner (1993) : Controlled Markov processes and viscosity solutions, Springer Verlag.

[42]   Fuhrman M., Hu Y. and G. Tessitore (2005) : “On a class of stochastic optimal control problems related to BSDEs with quadratic growth”, Preprint.

[43]   Gobet E., Lemor J.P. and X. Warin (2004) : “A regression-based Monte-Carlo method for backward stochastic differential equations”, to appear in Annals of Applied Probability.

[44]   Gundel A. (2004) : “Robust utility maximization for complete and incomplete market models”, Finance and Stochastics, 9, 151-176.

[45]   Guo X. (2001) : “An explicit solution to an optimal stopping problem with regime switching”, Journal of Applied Probability, 38, 464-481.

[46]   Guo X. and H. Pham (2005) : “Optimal partially reversible investment with entry decision and general production function”, Stoc. Proc. Appli., 115, 705-736. MR2132595

[47]   Hata H. and J. Sekine (2005) : “Solving a large deviations control problem with a nonlinear factor model”, Preprint.

[48]   Hu Y. and S. Peng (1995) : “Solution of forward-backward stochastic differential equations”, Prob. Theo. Rel. Fields, 103, 273-283.

[49]   Jeanblanc M. and A. Shiryaev (1995) : “Optimization of the flow of dividends”, Russian Math Surveys, 50, 257-277.

[50]   Kabanov, Yu. and C. Kluppelberg (2004) : “A geometric approach to portfolio optimization in models with transaction costs”, Finance and Stochastics, 8, 207-227.

[51]   Karatzas I. (1980) : “On a stochastic representation for the principal eigenvalue of a second order differential equation”, Stochastics and Stochastics Reports, 3, 305-321.

[52]   Kobylanski M. (2000) : “Backward stochastic differential equations and partial differential equations with quadratic growth”, Annals of Probability, 28, 558-602.

[53]   Kohlmann M. and S. Tang (2002) : “Global adapted solution of one-dimensional backward stochastic differential Riccati equations, with application to the mean-variance hedging”, Stochastic Process. Appl., 97, 255–288.

[54]   Kohlmann M. and X.Y. Zhou (2000) : “Relationship between backward stochastic differential equations and stochastic controls : a linear-quadratic approach”, SIAM Journal on Control and Optimization, 38, 1392-1407.

[55]   Korn R. (1998) : “Portfolio optimization with strictly positive transaction costs and impulse control”, Finance and Stochastics, 2, 85-114.

[56]   Krylov N. (1980) : Controlled Diffusion Processes, Springer Verlag.

[57]   Krylov N. (2000) : “On the rate of convergence of finite difference approximations for Bellman’s equations with variable coefficients”, Probab. Theory Relat. Fields, 117, 1-16.

[58]   Kushner H.J. (1977) : “Approximation and weak convergence methods for random processes, with applications to stochastic systems theory”, MIT Press Series in Signal Processing, Optimization, and Control, 6, MIT Press, Cambridge, MA, 1984, 269 pp.

[59]   Kushner H.J. and P. Dupuis (2001) : Numerical methods for stochastic control problems in continuous time, 2nd edition, Applications of Mathematics, 24, Stochastic Modelling and Applied Probability, Springer-Verlag, New York, 475 pp.

[60]   Ladyzhenskaya O., Solonnikov V. and N. Uralseva (1968) : Linear and quasilinear equations of parabolic type, American Mathematical Society, Providence.

[61]   Lions P.L. (1983) : “Optimal control of diffusion processes and Hamilton-Jacobi-Bellman equations””, Comm. P.D.E, 8, Part I, 1101-1134, Part II, 1229-1276

[62]   Lions P.L. (1988) : “Viscosity solutions and optimal stochastic control in infinite dimension”, Part I Acta Math., 161, 243-278.

[63]   Ly Vath V., Mnif M. and H. Pham (2005) : “A model of optimal portfolio selection under liquidity risk and price impact”, preprint PMA, University Paris 6-Paris 7.

[64]   Ma J., Protter P., San Martin J. and S. Torres (2002) : “Numerical method for backward stochastic differential equations”, Ann. Appl. Prob., 12, 302-316.

[65]   Ma J., Protter P. and J. Yong (1994) : “Solving forward-backward stochastic differential equations explicitly : a four step scheme”, Prob. Theo. Rel. Fields, 98, 339-359.

[66]   Ma J. and J. Yong (2000) : Forward-backward stochastic differential equations and their applications, Lect. Notes in Math., 1702.

[67]   Mania M. and R. Tevzadze (2003) : “Backward stochastic PDE and imperfect hedging”, Int. J. Theo. App. Fin., 6, 663-692.

[68]   Nisio M. (1981) : Lectures on Stochastic Control Theory, ISI Lect. Notes, 9, Kaigai Publ. Osaka.

[69]   Oksendal B. and A. Sulem (2002) : “Optimal consumption and portfolio with both fixed and proportional transaction costs”, SIAM J. Cont. Optim., 40, 1765-1790.

[70]   Oksendal B. and A. Sulem (2004) : Applied stochastic control of jump diffusion, Springer Verlag. MR2109687

[71]   Pagès G., Pham H. and J. Printems (2004a) : “An optimal Markovian quantization algorithm for multi-dimensional stochastic control problems”, Stochastics and Dynamics, 4, 501-545.

[72]   Pagès G., Pham H. and J. Printems (2004b) : “Optimal quantization methods and applications to numerical problems in finance”, in Handbook of computational and numerical methods in finance, ed. S. Rachev, Birkhäuser. MR2083048

[73]   Pardoux E. and S. Peng (1990) : “Adapted solutions of a backward stochastic differential equation”, Systems and Control Letters, 14, 55-61.

[74]   Pardoux E. and S. Tang (1999) : “Forward-backward stochastic differential equations and quasilinear parabolic pdes ”, Prob. Theo. Rel. Fields., 114, 123-150.

[75]   Peng S. (1990) : “A general stochastic maximum principle for optimal control diffusions”, SIAM J. Cont. Optim., 28, 966-979.

[76]   Pham H. (2000) : “On quadratic hedging in continuous time”, Math. Meth. Oper. Res., 51, 315-339.

[77]   Pham H. (2002) : “Smooth solutions to optimal investment models with stochastic volatilities and portfolio constraints”, Appl. Math. Optim., 46, 55-78.

[78]   Pham H. (2003a) : “A large deviations approach to optimal long term investment‘”, Finance and Stochastics, 7, 169-195.

[79]   Pham H. (2003b) : “A risk-sensitive control dual approach to a large deviations control problem”, Systems and Control Letters, 49, 295-309.

[80]   Pham H. (2005a) : “On the smooth-fit property for one-dimensional optimal switching problem”, to appear in Séminaire de Probabilités, Vol. XL.

[81]   Pham H. (2005b) : Optimisation et contrôle stochastique appliqués à la finance, to appear, Series Mathématiques et Applications, Springer Verlag.

[82]   Schied A. (2003) : “Optimal investment for robust utility functionals in complete markets”, to appear in Mathematics of Operations Research.

[83]   Schweizer M. (2001) : “A guided tour through quadratic hedging approaches”, in Option pricing, Interest rates and risk management, J. Cvitanic, E. Jouini, and m. Musiela eds., Cambridge Univ. Press.

[84]   Shreve S. and M. Soner (1994) : “Optimal investment and consumption with transaction costs”, Annals of Applied Probability, 4, 609-692.

[85]   Soner M. and N. Touzi (2000) : “Super replication under gamma constraints”, SIAM Journal on Control and Optimization, 39, 73-96.

[86]   Soner M. and N. Touzi (2002) : “Stochastic target problems, dynamic programming and viscosity solutions”, SIAM Journal on Control and Optimization, 41, 404-424.

[87]   Tang S. and J. Yong (1993) : “Finite horizon stochastic optimal switching and impulse controls with a viscosity solution approach”, Stoch. and Stoch. Reports, 45, 145-176. MR1306930

[88]   Tourin A. and T. Zariphopoulou (1994) : “Numerical schemes for investment models with singular transactions”, Computational Economics, 7, pp.287-307.

[89]   Yong J. and X.Y. Zhou (2000) : Stochastic controls, Hamiltonian systems and HJB equations, Springer Verlag. MR1696772

[90]   Zariphopoulou T. (1988) : Optimal investment-consumption models with constraints, Brown University, Phd.

[91]   Zariphopoulou T. (2001) : “A solution approach to valuation with unhedgeable risks”, Finance and Stochastics, 5, 61-82. MR1807876

[92]   Zitkovic G. (2005) : “Utility Maximization with a Stochastic Clock and an Unbounded Random Endowment”, Annals of Applied Probability, 15, 748-777.

[93]   Zhou X.Y. (1993) : “On the necessary conditions of optimal controls for stochastic partial differential equations”, SIAM J. Cont. Optim., 31, 1462-1478.




Home | Current | Past volumes | About | Login | Notify | Contact | Search

Probability Surveys. ISSN: 1549-5787