Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-6bb9c88b65-9c7xm Total loading time: 0 Render date: 2025-07-21T13:14:44.349Z Has data issue: false hasContentIssue false

References

Published online by Cambridge University Press:  17 July 2025

Thomas J Sargent  and
John Stachurski
Thomas J Sargent
Affiliation:
Hoover Institute
John Stachurski
Affiliation:
Australian National University, Canberra
Get access

Summary

A summary is not available for this content so a preview has been provided. Please use the Get access link above for information on how to access this content.
Image of the first page of this content. For PDF version, please use the ‘Save PDF’ preceeding this image.'

Information

Type
Chapter
Information
Dynamic Programming
Finite States
 , pp. 348 - 364
Publisher: Cambridge University Press
Print publication year: 2025

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Achdou, Y., Han, J., Lasry, J.-M., Lions, P.-L., and Moll, B. (2022). “Income and wealth distribution in macroeconomics: A continuous-time approach”. In: The Review of Economic Studies 89.1, pp. 45–86.10.1093/restud/rdab002CrossRefGoogle Scholar
Açıkgöz, Ö. T. (2018). “On the existence and uniqueness of stationary equilibrium in Bewley economies with production”. In: Journal of Economic Theory 173, pp. 18–55.CrossRefGoogle Scholar
Aiyagari, S. R. (1994). “Uninsured idiosyncratic risk and aggregate saving”. In: The Quarterly Journal of Economics 109.3, pp. 659–684.10.2307/2118417CrossRefGoogle Scholar
Albuquerque, R., Eichenbaum, M., Luo, V. X., and Rebelo, S. (2016). “Valuation risk and asset pricing”. In: The Journal of Finance 71.6, pp. 2861–2904.10.1111/jofi.12437CrossRefGoogle Scholar
Algan, Y., Allais, O., den Haan, W. J., and Rendahl, P. (2014). “Solving and simulating models with heterogeneous agents and aggregate uncertainty”. In: Handbook of Computational Economics. Vol. 3. North Holland, pp. 277–324.10.1016/B978-0-444-52980-0.00006-2CrossRefGoogle Scholar
Aliprantis, C. D. and Burkinshaw, O. (1998). Principles of Real Analysis. 3rd ed. Academic Press.Google Scholar
Amador, M., Werning, I., and Angeletos, G.-M. (2006). “Commitment vs. flexibility”. In: Econometrica 74.2, pp. 365–396.10.1111/j.1468-0262.2006.00666.xCrossRefGoogle Scholar
Andreoni, J. and Sprenger, C. (2012). “Risk preferences are not time preferences”. In: American Economic Review 102.7, pp. 3357–3376.10.1257/aer.102.7.3357CrossRefGoogle Scholar
Antràs, P. and de Gortari, A. (2020). “On the geography of global value chains”. In: Econometrica 88.4, pp. 1553–1598.CrossRefGoogle Scholar
Applebaum, D. (2019). Semigroups of Linear Operators. Vol. 93. Cambridge University Press.CrossRefGoogle Scholar
Arellano, C. (2008). “Default risk and income fluctuations in emerging economies”. In: American Economic Review 98.3, pp. 690–712.10.1257/aer.98.3.690CrossRefGoogle Scholar
Arellano, C. and Ramanarayanan, A. (2012). “Default and the maturity structure in sovereign bonds”. In: Journal of Political Economy 120.2, pp. 187–232.10.1086/666589CrossRefGoogle Scholar
Arrow, K. J., Harris, T., and Marschak, J. (1951). “Optimal inventory policy”. In: Econometrica 19.3, pp. 250–272.10.2307/1906813CrossRefGoogle Scholar
Asienkiewicz, H. and Jaśkiewicz, A. (2017). “A note on a new class of recursive utilities in Markov decision processes”. In: Applicationes Mathematicae 44, pp. 149–161. Atkinson, K. and W. Han (2005). Theoretical Numerical Analysis. Vol. 39. Springer.10.4064/am2317-1-2017CrossRefGoogle Scholar
Augeraud-Véron, E., Fabbri, G., and Schubert, K. (2019). “The value of biodiversity as an insurance device”. In: American Journal of Agricultural Economics 101.4, pp. 1068–1081.10.1093/ajae/aaz002CrossRefGoogle Scholar
Azinovic, M., Gaegauf, L., and Scheidegger, S. (2022). “Deep equilibrium nets”. In: International Economic Review 63.4, pp. 1471–1525.10.1111/iere.12575CrossRefGoogle Scholar
Backus, D. K., Routledge, B. R., and Zin, S. E. (2004). “Exotic preferences for macroeconomists”. In: NBER Macroeconomics Annual 19, pp. 319–390.10.1086/ma.19.3585343CrossRefGoogle Scholar
Bagliano, F.-C. and Bertola, G. (2004). Models for Dynamic Macroeconomics. Oxford University Press.10.1093/0199266824.001.0001CrossRefGoogle Scholar
Balbus, Ł. (2020). “On recursive utilities with non-affine aggregator and conditional certainty equivalent”. In: Economic Theory 70.2, pp. 551–577.CrossRefGoogle Scholar
Balbus, Ł., Reffett, K., and Woźny, Ł. (2014). “A constructive study of Markov equilibria in stochastic games with strategic complementarities”. In: Journal of Economic Theory 150, pp. 815–840.10.1016/j.jet.2013.09.005CrossRefGoogle Scholar
Balbus, Ł., Reffett, K., and Woźny, Ł. (2018). “On uniqueness of time-consistent Markov policies for quasi-hyperbolic consumers under uncertainty”. In: Journal of Economic Theory 176, pp. 293–310.10.1016/j.jet.2018.04.003CrossRefGoogle Scholar
Balbus, Ł., Reffett, K., and Woźny, Ł. (2022). “Time-consistent equilibria in dynamic models with recursive payoffs and behavioral discounting”. In: Journal of Economic Theory 204, pp. 1–39.10.1016/j.jet.2022.105493CrossRefGoogle Scholar
Bansal, R., Kiku, D., and Yaron, A. (2012). “An empirical evaluation of the long-run risks model for asset prices”. In: Critical Finance Review 1.1, pp. 183–221.10.1561/104.00000005CrossRefGoogle Scholar
Bansal, R. and Yaron, A. (2004). “Risks for the long run: A potential resolution of asset pricing puzzles”. In: The Journal of Finance 59.4, pp. 1481–1509.10.1111/j.1540-6261.2004.00670.xCrossRefGoogle Scholar
Barbu, V. and Precupanu, T. (2012). Convexity and Optimization in Banach Spaces. Springer Science & Business Media.10.1007/978-94-007-2247-7CrossRefGoogle Scholar
Barillas, F., Hansen, L. P., and Sargent, T. (2009). “Doubts or variability?” In: Journal of Economic Theory 144.6, pp. 2388–2418.Google Scholar
Bartle, R. G. and Sherbert, D. R. (2011). Introduction to Real Analysis. 4th ed. Wiley.Google Scholar
Bastianello, L. and Faro, J. H. (2023). “Choquet expected discounted utility”. In: Economic Theory 75, pp. 1071–1098.10.1007/s00199-022-01438-0CrossRefGoogle Scholar
Basu, S. and Bundick, B. (2017). “Uncertainty shocks in a model of effective demand”. In: Econometrica 85.3, pp. 937–958.10.3982/ECTA13960CrossRefGoogle Scholar
Bäuerle, N. and Glauner, A. (2022). “Markov decision processes with recursive risk measures”. In: European Journal of Operational Research 296.3, pp. 953–966.CrossRefGoogle Scholar
Bäuerle, N. and Jaśkiewicz, A. (2017). “Optimal dividend payout model with risk sensitive preferences”. In: Insurance: Mathematics and Economics 73, pp. 82–93.Google Scholar
Bäuerle, N. and Jaśkiewicz, A. (2018). “Stochastic optimal growth model with risk sensitive preferences”. In: Journal of Economic Theory 173, pp. 181–200.10.1016/j.jet.2017.11.005CrossRefGoogle Scholar
Bäuerle, N. and Jaśkiewicz, A. (2024). “Markov decision processes with risk-sensitive criteria: an overview”. In: Mathematical Methods of Operations Research 99.1, pp. 141–178.10.1007/s00186-024-00857-0CrossRefGoogle Scholar
Bäuerle, N. and Rieder, U. (2011). Markov Decision Processes with Applications to Finance. Springer Science & Business Media.10.1007/978-3-642-18324-9CrossRefGoogle Scholar
Becker, R. A., Boyd, J. H., and Sung, B. Y. (1989). “Recursive utility and optimal capital accumulation. I. Existence”. In: Journal of Economic Theory 47.1, pp. 76–100.10.1016/0022-0531(89)90104-XCrossRefGoogle Scholar
Becker, R. A. and Rincon-Zapatero, J. P. (2023). Recursive Utility for Thompson Aggregators: Least Fixed Point, Uniqueness, and Approximation Theories. Tech. rep. 4456037, Social Science Research Network.10.2139/ssrn.4456037CrossRefGoogle Scholar
Bellman, R. (1957). Dynamic Programming. American Association for the Advancement of Science.Google ScholarPubMed
Bellman, R. (1984). Eye of the Hurricane. World Scientific.10.1142/0076CrossRefGoogle Scholar
Benhabib, J., Bisin, A., and Zhu, S. (2015). “The wealth distribution in Bewley economies with capital income risk”. In: Journal of Economic Theory 159, pp. 489–515.10.1016/j.jet.2015.07.013CrossRefGoogle Scholar
Benzion, U., Rapoport, A., and Yagil, J. (1989). “Discount rates inferred from decisions: An experimental study”. In: Management Science 35.3, pp. 270–284.10.1287/mnsc.35.3.270CrossRefGoogle Scholar
Bertsekas, D. (2012). Dynamic Programming and Optimal Control. Vol. 1. Athena Scientific.Google Scholar
Bertsekas, D. (2021). Rollout, Policy Iteration, and Distributed Reinforcement Learning. Athena Scientific.Google Scholar
Bertsekas, D. (2022a). Abstract Dynamic Programming. 3rd ed. Athena Scientific.Google Scholar
Bertsekas, D. (2022b). “Newton’s method for reinforcement learning and model predictive control”. In: Results in Control and Optimization 7, p. 100121.10.1016/j.rico.2022.100121CrossRefGoogle Scholar
Bertsekas, D. and Tsitsiklis, J. N. (1996). Neuro-Dynamic Programming. Athena Scientific.Google Scholar
Bhandari, J. and Russo, D. (2022). “Global optimality guarantees for policy gradient methods”. In: arXiv, 1906.01786.Google Scholar
Bianchi, J. (2011). “Overborrowing and systemic externalities in the business cycle”. In: American Economic Review 101.7, pp. 3400–3426.10.1257/aer.101.7.3400CrossRefGoogle Scholar
Bloise, G., Le Van, C., and Vailakis, Y. (2024). “Do not blame Bellman: It is Koopmans’ fault”. In: Econometrica 92.1, pp. 111–140.10.3982/ECTA20386CrossRefGoogle Scholar
Bloise, G. and Vailakis, Y. (2018). “Convex dynamic programming with (bounded) recursive utility”. In: Journal of Economic Theory 173, pp. 118–141.10.1016/j.jet.2017.10.008CrossRefGoogle Scholar
Bloise, G. and Vailakis, Y. (2022). “On sovereign default with time-varying interest rates”. In: Review of Economic Dynamics 44, pp. 211–224.10.1016/j.red.2021.03.001CrossRefGoogle Scholar
Bloom, N. (2009). “The impact of uncertainty shocks”. In: Econometrica 77.3, pp. 623–685.Google Scholar
Bloom, N., Bond, S., and Van Reenen, J. (2007). “Uncertainty and investment dynamics”. In: The Review of Economic Studies 74.2, pp. 391–415.10.1111/j.1467-937X.2007.00426.xCrossRefGoogle Scholar
Blundell, R., Graber, M., and Mogstad, M. (2015). “Labor income dynamics and the insurance from taxes, transfers, and the family”. In: Journal of Public Economics 127, pp. 58–73.10.1016/j.jpubeco.2014.04.011CrossRefGoogle Scholar
Bocola, L., Bornstein, G., and Dovis, A. (2019). “Quantitative sovereign default models and the European debt crisis”. In: Journal of International Economics 118, pp. 20–30.10.1016/j.jinteco.2019.01.011CrossRefGoogle Scholar
Bollobás, B. (1999). Linear Analysis: An Introductory Course. Cambridge University Press.10.1017/CBO9781139168472CrossRefGoogle Scholar
Bommier, A., Kochov, A., and Le Grand, F. (2017). “On monotone recursive preferences”. In: Econometrica 85.5, pp. 1433–1466.10.3982/ECTA11898CrossRefGoogle Scholar
Bommier, A., Kochov, A., and Le Grand, F. (2019). “Ambiguity and endogenous discounting”. In: Journal of Mathematical Economics 83, pp. 48–62.10.1016/j.jmateco.2019.04.001CrossRefGoogle Scholar
Bommier, A. and Villeneuve, B. (2012). “Risk aversion and the value of risk to life”. In: Journal of Risk and Insurance 79.1, pp. 77–104.10.1111/j.1539-6975.2010.01390.xCrossRefGoogle Scholar
Bond, S. and Van Reenen, J. (2007). “Microeconometric models of investment and employment”. In: Handbook of Econometrics. Ed. by Heckman, J. and Leamer, E.. Vol. 6. Elsevier, pp. 4417–4498.10.1016/S1573-4412(07)06065-5CrossRefGoogle Scholar
Borovička, J. and Stachurski, J. (2020). “Necessary and sufficient conditions for existence and uniqueness of recursive utilities”. In: The Journal of Finance 75.3, pp. 1457–1493.Google Scholar
Boyd, J. H. (1990). “Recursive utility and the Ramsey problem”. In: Journal of Economic Theory 50.2, pp. 326–345.10.1016/0022-0531(90)90006-6CrossRefGoogle Scholar
Brémaud, P. (2020). Markov Chains: Gibbs Fields, Monte Carlo Simulation and Queues. Vol. 31. Springer Nature.10.1007/978-3-030-45982-6CrossRefGoogle Scholar
Brock, W. A. and Mirman, L. J. (1972). “Optimal economic growth and uncertainty: The discounted case”. In: Journal of Economic Theory 4.3, pp. 479–513.10.1016/0022-0531(72)90135-4CrossRefGoogle Scholar
Bullen, P. S. (2003). Handbook of Means and Their Inequalities. Vol. 560. Springer Science & Business Media.10.1007/978-94-017-0399-4CrossRefGoogle Scholar
Burdett, K. (1978). “A theory of employee job search and quit rates”. In: American Economic Review 68.1, pp. 212–220.Google Scholar
Cagetti, M., Hansen, L. P., Sargent, T., and Williams, N. (2002). “Robustness and pricing with uncertain growth”. In: The Review of Financial Studies 15.2, pp. 363–404.10.1093/rfs/15.2.363CrossRefGoogle Scholar
Calsamiglia, C., Fu, C., and Güell, M. (2020). “Structural estimation of a model of school choices: The Boston mechanism versus its alternatives”. In: Journal of Political Economy 128.2, pp. 642–680.10.1086/704573CrossRefGoogle Scholar
Campbell, J. Y. (2017). Financial Decisions and Markets: A Course in Asset Pricing. Princeton University Press.Google Scholar
Cao, D. (2020). “Recursive equilibrium in Krusell and Smith (1998)”. In: Journal of Economic Theory 186: 104978, pp. 1–71.10.1016/j.jet.2019.104978CrossRefGoogle Scholar
Cao, D. and Werning, I. (2018). “Saving and dissaving with hyperbolic discounting”. In: Econometrica 86.3, pp. 805–857.10.3982/ECTA15112CrossRefGoogle Scholar
Carroll, C. D. (1997). “Buffer-stock saving and the life cycle/permanent income hypothesis”. In: Quarterly Journal of Economics 112.1, pp. 1–55.10.1162/003355397555109CrossRefGoogle Scholar
Carroll, C. D. (2009). “Precautionary saving and the marginal propensity to consume out of permanent income”. In: Journal of Monetary Economics 56.6, pp. 780–790.10.1016/j.jmoneco.2009.06.016CrossRefGoogle Scholar
Carruth, A., Dickerson, A., and Henley, A. (2000). “What do we know about investment under uncertainty?” In: Journal of Economic Surveys 14.2, pp. 119–154.Google Scholar
Carvalho, V. M. and Grassi, B. (2019). “Large firm dynamics and the business cycle”. In: American Economic Review 109.4, pp. 1375–1425.10.1257/aer.20151317CrossRefGoogle Scholar
Chatterjee, S. and Eyigungor, B. (2012). “Maturity, indebtedness, and default risk”. In: American Economic Review 102.6, pp. 2674–99.10.1257/aer.102.6.2674CrossRefGoogle Scholar
Cheney, W. (2013). Analysis for Applied Mathematics. Vol. 208. Springer Science & Business Media.Google Scholar
Chetty, R. (2008). “Moral hazard versus liquidity and optimal unemployment insurance”. In: Journal of Political Economy 116.2, pp. 173–234.10.1086/588585CrossRefGoogle Scholar
Christensen, T. M. (2022). “Existence and uniqueness of recursive utilities without boundedness”. In: Journal of Economic Theory 200: 105413, pp. 1–37.10.1016/j.jet.2022.105413CrossRefGoogle Scholar
Christiano, L. J., Motto, R., and Rostagno, M. (2014). “Risk shocks”. In: American Economic Review 104.1, pp. 27–65.10.1257/aer.104.1.27CrossRefGoogle Scholar
Çınlar, E. and Vanderbei, R. J. (2013). Real and Convex Analysis. Springer Science & Business Media.10.1007/978-1-4614-5257-7CrossRefGoogle Scholar
Cochrane, J. H. (2009). Asset Pricing: Revised Edition. Princeton University Press.Google Scholar
Colacito, R., Croce, M., Ho, S., and Howard, P. (2018). “BKK the EZ way: International long-run growth news and capital flows”. In: American Economic Review 108.11, pp. 3416–3449.10.1257/aer.20141123CrossRefGoogle Scholar
Cruces, J. J. and Trebesch, C. (2013). “Sovereign defaults: The price of haircuts”. In: American Economic Journal: Macroeconomics 5.3, pp. 85–117.Google Scholar
Daley, D. (1968). “Stochastically monotone Markov Chains”. In: Probability Theory and Related Fields 10.4, pp. 305–317.Google Scholar
Dasgupta, P. and Maskin, E. (2005). “Uncertainty and hyperbolic discounting”. In: American Economic Review 95.4, pp. 1290–1299.10.1257/0002828054825637CrossRefGoogle Scholar
Davey, B. A. and Priestley, H. A. (2002). Introduction to Lattices and Order. Cambridge University Press.10.1017/CBO9780511809088CrossRefGoogle Scholar
De Castro, L. and Galvao, A. (2019). “Dynamic quantile models of rational behavior”. In: Econometrica 87.6, pp. 1893–1939.10.3982/ECTA15146CrossRefGoogle Scholar
De Castro, L. and Galvao, A. (2022). “Static and dynamic quantile preferences”. In: Economic Theory 73.2–3, pp. 747–779.10.1007/s00199-021-01355-8CrossRefGoogle Scholar
De Castro, L., Galvao, A., and Nunes, D. (2022). Dynamic Economics with Quantile Preferences. Tech. rep. SSRN, 4108230.10.2139/ssrn.4108230CrossRefGoogle Scholar
De Groot, O., Richter, A. W., and Throckmorton, N. A. (2022). “Valuation risk revalued”. In: Quantitative Economics 13.2, pp. 723–759.10.3982/QE1779CrossRefGoogle Scholar
De Groot, O., Richter, A. W., and Throckmorton, N. A. (2018). “Uncertainty shocks in a model of effective demand: Comment”. In: Econometrica 86.4, pp. 1513–1526.10.3982/ECTA15405CrossRefGoogle Scholar
De Nardi, M., French, E., and Jones, J. B. (2010). “Why do the elderly save? The role of medical expenses”. In: Journal of Political Economy 118.1, pp. 39–75.10.1086/651674CrossRefGoogle Scholar
Deaton, A. and Laroque, G. (1992). “On the behaviour of commodity prices”. In: Review of Economic Studies 59.1, pp. 1–23.10.2307/2297923CrossRefGoogle Scholar
DeJarnette, P., Dillenberger, D., Gottlieb, D., and Ortoleva, P. (2020). “Time lotteries and stochastic impatience”. In: Econometrica 88.2, pp. 619–656.10.3982/ECTA16427CrossRefGoogle Scholar
Denardo, E. V. (1967). “Contraction mappings in the theory underlying dynamic programming”. In: Siam Review 9.2, pp. 165–177.10.1137/1009030CrossRefGoogle Scholar
Denardo, E. V. (1981). Dynamic Programming: Models and Applications. Prentice Hall PTR.Google Scholar
Diamond, P. and Köszegi, B. (2003). “Quasi-hyperbolic discounting and retirement”. In: Journal of Public Economics 87.9–10, pp. 1839–1872.10.1016/S0047-2727(02)00041-5CrossRefGoogle Scholar
Dixit, R. K. and Pindyck, R. S. (2012). Investment under Uncertainty. Princeton University Press.10.2307/j.ctt7sncvCrossRefGoogle Scholar
Drugeon, J.-P. and Wigniolle, B. (2021). “On Markovian collective choice with heterogeneous quasi-hyperbolic discounting”. In: Economic Theory 72.4, pp. 1257–1296.10.1007/s00199-020-01291-zCrossRefGoogle Scholar
Du, Y. (1990). “Fixed points of increasing operators in ordered Banach spaces and applications”. In: Applicable Analysis 38.01-02, pp. 1–20.10.1080/00036819008839957CrossRefGoogle Scholar
Duarte, V. (2018). Machine learning for continuous-time economics. Tech. rep. SSRN, 3012602.Google Scholar
Dudley, R. M. (2002). Real Analysis and Probability. Vol. 74. Cambridge University Press.10.1017/CBO9780511755347CrossRefGoogle Scholar
Duffie, D. (2010). Dynamic Asset Pricing Theory. Princeton University Press.Google Scholar
Duffie, D. and Garman, M. B. (1986). Intertemporal Arbitrage and the Markov Valuation of Securities. Citeseer.Google Scholar
Dupuis, P. and Ellis, R. S. (2011). A Weak Convergence Approach to the Theory of Large Deviations. John Wiley & Sons.Google Scholar
Dvoretzky, A., Kiefer, J., and Wolfowitz, J. (1952). “The inventory problem: I. Case of known distributions of demand”. In: Econometrica 20.2, pp. 187–222.10.2307/1907847CrossRefGoogle Scholar
Engel, K.-J. and Nagel, R. (2006). A Short Course on Operator Semigroups. Springer Science & Business Media.Google Scholar
Epstein, L. G. and Zin, S. E. (1989). “Risk aversion and the temporal behavior of consumption and asset returns: A theoretical framework”. In: Econometrica 57.4, pp. 937–969.10.2307/1913778CrossRefGoogle Scholar
Epstein, L. G. and Zin, S. E. (1991). “Substitution, risk aversion, and the temporal behavior of consumption and asset returns: An empirical analysis”. In: Journal of Political Economy 99.2, pp. 263–286.10.1086/261750CrossRefGoogle Scholar
Ericson, K. M. and Laibson, D. (2019). “Intertemporal choice”. In: Handbook of Behavioral Economics: Applications and Foundations 1. Vol. 2. Elsevier, pp. 1–67.Google Scholar
Ericson, R. and Pakes, A. (1995). “Markov-perfect industry dynamics: A framework for empirical work”. In: The Review of Economic Studies 62.1, pp. 53–82.10.2307/2297841CrossRefGoogle Scholar
Erosa, A. and González, B. (2019). “Taxation and the life cycle of firms”. In: Journal of Monetary Economics 105, pp. 114–130.10.1016/j.jmoneco.2019.04.006CrossRefGoogle Scholar
Eslami, K. and Phelan, T. (2023). The Art of Temporal Approximation: An Investigation into Numerical Solutions to Discrete and Continuous-Time Problems in Economics. Tech. rep. FRB of Cleveland Working Paper.Google Scholar
Esponda, I. and Pouzo, D. (2021). “Equilibrium in misspecified Markov decision processes”. In: Theoretical Economics 16.2, pp. 717–757.10.3982/TE3843CrossRefGoogle Scholar
Fagereng, A., Holm, M. B., Moll, B., and Natvik, G. (2019). Saving behavior across the wealth distribution: The importance of capital gains. Tech. rep. National Bureau of Economic Research.10.3386/w26588CrossRefGoogle Scholar
Fajgelbaum, P. D., Schaal, E., and Taschereau-Dumouchel, M. (2017). “Uncertainty traps”. In: The Quarterly Journal of Economics 132.4, pp. 1641–1692.10.1093/qje/qjx021CrossRefGoogle Scholar
Farmer, J. D., Geanakoplos, J., Richiardi, M. G., Montero, M., Perelló, J., and Masoliver, J. (2023). Discounting the distant future: What do historical bond prices imply about the long term discount rate? Tech. rep. arXiv, 2312.17157.Google Scholar
Farmer, L. E. and Toda, A. A. (2017). “Discretizing nonlinear, non-Gaussian Markov processes with exact conditional moments”. In: Quantitative Economics 8.2, pp. 651–683.10.3982/QE737CrossRefGoogle Scholar
Fedus, W., Gelada, C., Bengio, Y., Bellemare, M. G., and Larochelle, H. (2019). “Hyperbolic discounting and learning over multiple horizons”. In: arXiv, 1902.06865.Google Scholar
Fei, Y., Yang, Z., Chen, Y., and Wang, Z. (2021). “Exponential Bellman equation and improved regret bounds for risk-sensitive reinforcement learning”. In: Advances in Neural Information Processing Systems 34, pp. 20436–20446.Google Scholar
Fernández-Villaverde, J., Hurtado, S., and Nuno, G. (2023). “Financial frictions and the wealth distribution”. In: Econometrica 91.3, pp. 869–901.10.3982/ECTA18180CrossRefGoogle Scholar
Fisher, I. (1930). “The theory of interest as determined by impatience to spend income and opportunity to invest it”. In: Bulletin of the American Mathematical Society 36, pp. 783–784.Google Scholar
Föllmer, H. and Knispel, T. (2011). “Entropic risk measures: Coherence vs. convexity, model ambiguity and robust large deviations”. In: Stochastics and Dynamics 11.02n03, pp. 333–351.10.1142/S0219493711003334CrossRefGoogle Scholar
Foss, S., Shneer, V., Thomas, J. P., and Worrall, T. (2018). “Stochastic stability of monotone economies in regenerative environments”. In: Journal of Economic Theory 173, pp. 334–360.10.1016/j.jet.2017.11.004CrossRefGoogle Scholar
Frederick, S., Loewenstein, G., and O’donoghue, T. (2002). “Time discounting and time preference: A critical review”. In: Journal of Economic Literature 40.2, pp. 351–401.10.1257/jel.40.2.351CrossRefGoogle Scholar
Friedman, M. (1956). Theory of the Consumption Function. Princeton University Press.Google Scholar
Gao, Y., Lui, K. Y. C., and Hernandez-Leal, P. (2021). “Robust risk-sensitive reinforcement learning agents for trading markets”. In: arXiv, 2107.08083.Google Scholar
Garman, M. B. (1985). “Towards a semigroup pricing theory”. In: The Journal of Finance 40.3, pp. 847–861.10.1111/j.1540-6261.1985.tb05010.xCrossRefGoogle Scholar
Geist, M. and Scherrer, B. (2018). Anderson acceleration for reinforcement learning. Tech. rep.Google Scholar
Gennaioli, N., Martin, A., and Rossi, S. (2014). “Sovereign default, domestic banks, and financial institutions”. In: The Journal of Finance 69.2, pp. 819–866.10.1111/jofi.12124CrossRefGoogle Scholar
Gentry, M. L., Hubbard, T. P., Nekipelov, D., and Paarsch, H. J. (2018). “Structural econometrics of auctions: A review”. In: Foundations and Trends in Econometrics 9.2–4, pp. 79–302.10.1561/0800000031CrossRefGoogle Scholar
Ghosh, A. R., Kim, J. I., Mendoza, E. G., Ostry, J. D., and Qureshi, M. S. (2013). “Fiscal fatigue, fiscal space and debt sustainability in advanced economies”. In: The Economic Journal 123.566, F4–F30.10.1111/ecoj.12010CrossRefGoogle Scholar
Gillingham, K., Iskhakov, F., Munk-Nielsen, A., Rust, J., and Schjerning, B. (2022). “Equilibrium trade in automobiles”. In: Journal of Political Economy 130.10, pp. 2534–2593.10.1086/720463CrossRefGoogle Scholar
Giovannetti, B. C. (2013). “Asset pricing under quantile utility maximization”. In: Review of Financial Economics 22.4, pp. 169–179.10.1016/j.rfe.2013.05.008CrossRefGoogle Scholar
Gomez-Cram, R. and Yaron, A. (May 2020). “How important are inflation expectations for the nominal yield curve?” In: The Review of Financial Studies 34.2, pp. 985–1045.10.1093/rfs/hhaa039CrossRefGoogle Scholar
Goyal, V. and Grand-Clement, J. (2023). “A first-order approach to accelerated value iteration”. In: Operations Research 71.2, pp. 517–535.10.1287/opre.2022.2269CrossRefGoogle Scholar
Guan, G. and Wang, X. (2020). “Time-consistent reinsurance and investment strategies for an AAI under smooth ambiguity utility”. In: Scandinavian Actuarial Journal 2020.8, pp. 677–699.10.1080/03461238.2020.1719880CrossRefGoogle Scholar
Guo, D., Cho, Y. J., and Zhu, J. (2004). Partial Ordering Methods in Nonlinear Problems. Nova Publishers.Google Scholar
Guvenen, F. (2007). “Learning your earning: Are labor income shocks really very persistent?” In: American Economic Review 97.3, pp. 687–712.Google Scholar
Guvenen, F. (2009). “An empirical investigation of labor income processes”. In: Review of Economic Dynamics 12.1, pp. 58–79.10.1016/j.red.2008.06.004CrossRefGoogle Scholar
Häggström, O. et al. (2002). Finite Markov Chains and Algorithmic Applications. Cambridge University Press.10.1017/CBO9780511613586CrossRefGoogle Scholar
Han, J., Yang, Y., et al. (2021). “Deepham: A global solution method for heterogeneous agent models with aggregate shocks”. In: arXiv, 2112.14377.Google Scholar
Hansen, L. P., Heaton, J. C., and Li, N. (2008). “Consumption strikes back? Measuring long-run risk”. In: Journal of Political Economy 116.2, pp. 260–302.10.1086/588200CrossRefGoogle Scholar
Hansen, L. P. and Miao, J. (2018). “Aversion to ambiguity and model misspecification in dynamic stochastic environments”. In: Proceedings of the National Academy of Sciences 115.37, pp. 9163–9168.10.1073/pnas.1811243115CrossRefGoogle ScholarPubMed
Hansen, L. P. and Renault, E. (2010). “Pricing kernels”. In: Encyclopedia of Quantitative Finance.10.1002/9780470061602.eqf19009CrossRefGoogle Scholar
Hansen, L. P. and Sargent, T. (1980). “Formulating and estimating dynamic linear rational expectations models”. In: Journal of Economic Dynamics and Control 2, pp. 7–46.10.1016/0165-1889(80)90049-4CrossRefGoogle Scholar
Hansen, L. P. and Sargent, T. (1990). Rational Expectations Econometrics. CRC Press.Google Scholar
Hansen, L. P. and Sargent, T. (1995). “Discounted linear exponential quadratic Gaussian control”. In: IEEE Transactions on Automatic Control 40.5, pp. 968–971.10.1109/9.384242CrossRefGoogle Scholar
Hansen, L. P. and Sargent, T. (2007). “Recursive robust estimation and control without commitment”. In: Journal of Economic Theory 136.1, pp. 1–27.10.1016/j.jet.2006.06.010CrossRefGoogle Scholar
Hansen, L. P. and Sargent, T. (2011). Robustness. Princeton University Press.Google ScholarPubMed
Hansen, L. P. and Sargent, T. (2020). “Structured ambiguity and model misspecification”. In: Journal of Economic Theory, p. 105165.Google Scholar
Hansen, L. P. and Scheinkman, J. A. (2009). “Long-term risk: An operator approach”. In: Econometrica 77.1, pp. 177–234.Google Scholar
Harrison, J. M. and Kreps, D. M. (1978). “Speculative investor behavior in a stock market with heterogeneous expectations”. In: The Quarterly Journal of Economics 92.2, pp. 323–336.10.2307/1884166CrossRefGoogle Scholar
Hassett, K. A. and Hubbard, G. R. (2002). “Tax policy and business investment”. In: Handbook of Public Economics. Ed. by Auerbach, A. and Feldstein, M.. Vol. 3. Elsevier, pp. 1293–1343.Google Scholar
Havranek, T., Horvath, R., Irsova, Z., and Rusnak, M. (2015). “Cross-country heterogeneity in intertemporal substitution”. In: Journal of International Economics 96.1, pp. 100–118.10.1016/j.jinteco.2015.01.012CrossRefGoogle Scholar
Hayashi, F. (1982). “Tobin’s marginal q and average q: A neoclassical interpretation”. In: Econometrica 50.1, pp. 213–224.10.2307/1912538CrossRefGoogle Scholar
Hayashi, T. and Miao, J. (2011). “Intertemporal substitution and recursive smooth ambiguity preferences”. In: Theoretical Economics 6.3, pp. 423–472.10.3982/TE843CrossRefGoogle Scholar
Heathcote, J. and Perri, F. (2018). “Wealth and volatility”. In: The Review of Economic Studies 85.4, pp. 2173–2213.10.1093/restud/rdx074CrossRefGoogle Scholar
Hens, T. and Schindler, N. (2020). “Value and patience: The value premium in a dividend-growth model with hyperbolic discounting”. In: Journal of Economic Behavior & Organization 172, pp. 161–179.10.1016/j.jebo.2020.01.028CrossRefGoogle Scholar
Hernández-Lerma, O. and Lasserre, J. B. (2012a). Discrete-Time Markov Control Processes: Basic Optimality Criteria. Vol. 30. Springer Science & Business Media.Google Scholar
Hernández-Lerma, O. and Lasserre, J. B. (2012b). Further Topics on Discrete-Time Markov Control Processes. Vol. 42. Springer Science & Business Media.Google Scholar
Herrendorf, B., Rogerson, R., and Valentinyi, A. (2021). “Structural change in investment and consumption: A unified analysis”. In: The Review of Economic Studies 88.3, pp. 1311–1346.10.1093/restud/rdaa013CrossRefGoogle Scholar
Hill, E., Bardoscia, M., and Turrell, A. (2021). “Solving heterogeneous general equilibrium economic models with deep reinforcement learning”. In: arXiv, 2103.16977.Google Scholar
Hills, T. S., Nakata, T., and Schmidt, S. (2019). “Effective lower bound risk”. In: European Economic Review 120, p. 103321.10.1016/j.euroecorev.2019.103321CrossRefGoogle Scholar
Hirsh, M. and Smale, S. (1974). Differential Equations, Dynamical Systems and Linear Algebra. Academic Press.Google Scholar
Hopenhayn, H. A. (1992). “Entry, exit, and firm dynamics in long run equilibrium”. In: Econometrica 60, pp. 1127–1150.10.2307/2951541CrossRefGoogle Scholar
Hopenhayn, H. A. and Prescott, E. C. (1992). “Stochastic monotonicity and stationary distributions for dynamic economies”. In: Econometrica 60.6, pp. 1387–1406.10.2307/2951526CrossRefGoogle Scholar
Horn, R. A. and Johnson, C. R. (2012). Matrix Analysis. Cambridge University Press.10.1017/CBO9781139020411CrossRefGoogle Scholar
Howard, R. A. (1960). Dynamic Programming and Markov Processes. John Wiley & Sons.Google Scholar
Hsu, W.-T. (2012). “Central place theory and city size distribution”. In: The Economic Journal 122.563, pp. 903–932.10.1111/j.1468-0297.2012.02518.xCrossRefGoogle Scholar
Hsu, W.-T., Holmes, T. J., and Morgan, F. (2014). “Optimal city hierarchy: A dynamic programming approach to central place theory”. In: Journal of Economic Theory 154, pp. 245–273.10.1016/j.jet.2014.09.018CrossRefGoogle Scholar
Hu, T.-W. and Shmaya, E. (2019). “Unique monetary equilibrium with inflation in a stationary Bewley–Aiyagari model”. In: Journal of Economic Theory 180, pp. 368–382.10.1016/j.jet.2019.01.003CrossRefGoogle Scholar
Hubmer, J., Krusell, P., and A. A. Smith Jr (2020). “Sources of US wealth inequality: Past, present, and future”. In: NBER Macroeconomics Annual 2020, volume 35.Google Scholar
Huggett, M. (1993). “The risk-free rate in heterogeneous-agent incomplete-insurance economies”. In: Journal of Economic Dynamics and Control 17.5–6, pp. 953–969.10.1016/0165-1889(93)90024-MCrossRefGoogle Scholar
Huijben, I. A., Kool, W., Paulus, M. B., and Van Sloun, R. J. (2022). “A review of the gumbel-max trick and its extensions for discrete stochasticity in machine learning”. In: IEEE Transactions on Pattern Analysis and Machine Intelligence.Google Scholar
Iskhakov, F., Rust, J., and Schjerning, B. (2020). “Machine learning and structural econometrics: Contrasts and synergies”. In: The Econometrics Journal 23.3, S81– S124.10.1093/ectj/utaa019CrossRefGoogle Scholar
Jacobson, D. H. (1973). “Optimal Stochastic Linear Systems with Exponential Performance Criteria and Their Relation to Deterministic Differential Games”. In: IEEE Transactions for Automatic Control AC-18, pp. 1124–1131.Google Scholar
Jang, E., Gu, S., and Poole, B. (2016). “Categorical reparameterization with gumbel-softmax”. In: arXiv, 1611.01144.Google Scholar
Jaśkiewicz, A. and Nowak, A. S. (2014). “Stationary Markov perfect equilibria in risk sensitive stochastic overlapping generations models”. In: Journal of Economic Theory 151, pp. 411–447.10.1016/j.jet.2014.01.005CrossRefGoogle Scholar
Jaśkiewicz, A. and Nowak, A. S. (2021). “Markov decision processes with quasi-hyperbolic discounting”. In: Finance and Stochastics 25.2, pp. 189–229.10.1007/s00780-020-00443-2CrossRefGoogle Scholar
Jasso-Fuentes, H., Menaldi, J.-L., and Prieto-Rumeau, T. (2020). “Discrete-time control with non-constant discount factor”. In: Mathematical Methods of Operations Research, pp. 1–23.10.1007/s00186-020-00716-8CrossRefGoogle Scholar
Jensen, N. R. (2019). “Life insurance decisions under recursive utility”. In: Scandinavian Actuarial Journal 2019.3, pp. 204–227.10.1080/03461238.2018.1541025CrossRefGoogle Scholar
Jovanovic, B. (1979). “Firm-specific capital and turnover”. In: Journal of Political Economy 87.6, pp. 1246–1260.10.1086/260834CrossRefGoogle Scholar
Jovanovic, B. (1982). “Selection and the Evolution of Industry”. In: Econometrica 50.3, pp. 649–670.10.2307/1912606CrossRefGoogle Scholar
Jovanovic, B. (1984). “Matching, turnover, and unemployment”. In: Journal of Political Economy 92.1, pp. 108–122.10.1086/261210CrossRefGoogle Scholar
Ju, N. and Miao, J. (2012). “Ambiguity, learning, and asset returns”. In: Econometrica 80.2, pp. 559–591.Google Scholar
Kahou, M. E., Fernández-Villaverde, J., Perla, J., and Sood, A. (2021). Exploiting symmetry in high-dimensional dynamic programming. Tech. rep. National Bureau of Economic Research.10.3386/w28981CrossRefGoogle Scholar
Kamihigashi, T. (2014). “Elementary results on solutions to the Bellman equation of dynamic programming: Existence, uniqueness, and convergence”. In: Economic Theory 56, pp. 251–273.10.1007/s00199-013-0789-4CrossRefGoogle Scholar
Kamihigashi, T. and Stachurski, J. (2014). “Stochastic stability in monotone economies”. In: Theoretical Economics 9.2, pp. 383–407.10.3982/TE1367CrossRefGoogle Scholar
Kaplan, G., Moll, B., and Violante, G. L. (2018). “Monetary policy according to HANK”. In: American Economic Review 108.3, pp. 697–743.10.1257/aer.20160042CrossRefGoogle Scholar
Karp, L. (2005). “Global warming and hyperbolic discounting”. In: Journal of Public Economics 89.2–3, pp. 261–282.10.1016/j.jpubeco.2004.02.005CrossRefGoogle Scholar
Kase, H., Melosi, L., and Rottner, M. (2022). Estimating Nonlinear Heterogeneous Agents Models with Neural Networks. Tech. rep. CEPR Discussion Paper No. DP17391.10.21033/wp-2022-26CrossRefGoogle Scholar
Keane, M. P., Todd, P. E., and Wolpin, K. I. (2011). “The structural estimation of behavioral models: Discrete choice dynamic programming methods and applications”. In: Handbook of Labor Economics. Ed. by Ashenfelter, O. and Card, D.. Vol. 4. Elsevier, pp. 331–461.Google Scholar
Keane, M. P. and Wolpin, K. I. (1997). “The career decisions of young men”. In: Journal of Political Economy 105.3, pp. 473–522.10.1086/262080CrossRefGoogle Scholar
Kelle, P. and Milne, A. (1999). “The effect of (s, S) ordering policy on the supply chain”. In: International Journal of Production Economics 59.1–3, pp. 113–122.10.1016/S0925-5273(98)00232-1CrossRefGoogle Scholar
Kikuchi, T., Nishimura, K., Stachurski, J., and Zhang, J. (2021). “Coase meets Bellman: Dynamic programming for production networks”. In: Journal of Economic Theory 196, p. 105287.10.1016/j.jet.2021.105287CrossRefGoogle Scholar
Kleinman, B., Liu, E., and Redding, S. J. (2023). “Dynamic spatial general equilibrium”. In: Econometrica 91.2, pp. 385–424.10.3982/ECTA20273CrossRefGoogle Scholar
Klibanoff, P., Marinacci, M., and Mukerji, S. (2009). “Recursive smooth ambiguity preferences”. In: Journal of Economic Theory 144.3, pp. 930–976.10.1016/j.jet.2008.10.007CrossRefGoogle Scholar
Knight, F. H. (1921). Risk, Uncertainty and Profit. Vol. 31. Houghton Mifflin.Google Scholar
Kochenderfer, M. J., Wheeler, T. A., and Wray, K. H. (2022). Algorithms for Decision Making. MIT Press.Google Scholar
Kohler, M., Krzyżak, A., and Todorovic, N. (2010). “Pricing of high-dimensional American options by neural networks”. In: Mathematical Finance 20.3, pp. 383–410.10.1111/j.1467-9965.2010.00404.xCrossRefGoogle Scholar
Koopmans, T. C. (1960). “Stationary ordinal utility and impatience”. In: Econometrica 28.2, pp. 287–309.10.2307/1907722CrossRefGoogle Scholar
Koopmans, T. C., Diamond, P. A., and Williamson, R. E. (1964). “Stationary utility and time perspective”. In: Econometrica 32.1/2, pp. 82–100.10.2307/1913736CrossRefGoogle Scholar
Krasnosel’skii, M. A., Vainikko, G. M., Zabreiko, P. P., Rutitskii, Y. B., and Stetsenko, V. Y. (1972). Approximate Solution of Operator Equations. Springer Netherlands.10.1007/978-94-010-2715-1CrossRefGoogle Scholar
Kreps, D. M. and Porteus, E. L. (1978). “Temporal resolution of uncertainty and dynamic choice theory”. In: Econometrica 46.1, pp. 185–200.10.2307/1913656CrossRefGoogle Scholar
Kreyszig, E. (1978). Introductory Functional Analysis with Applications. Vol. 1. Wiley New York.Google Scholar
Kristensen, D., Mogensen, P. K., Moon, J. M., and Schjerning, B. (2021). “Solving dynamic discrete choice models using smoothing and sieve methods”. In: Journal of Econometrics 223.2, pp. 328–360.10.1016/j.jeconom.2020.02.007CrossRefGoogle Scholar
Krueger, D., Mitman, K., and Perri, F. (2016). “Macroeconomics and household heterogeneity”. In: Handbook of Macroeconomics. Ed. by Taylor, J. and Uhlig, H.. Vol. 2. Elsevier, pp. 843–921.Google Scholar
Krusell, P. and A. A. Smith Jr (1998). “Income and wealth heterogeneity in the macroeconomy”. In: Journal of Political Economy 106.5, pp. 867–896.10.1086/250034CrossRefGoogle Scholar
Lasota, A. and Mackey, M. C. (1994). Chaos, Fractals, and Noise: Stochastic Aspects of Dynamics. 2nd ed. Vol. 97. Springer Science & Business Media.10.1007/978-1-4612-4286-4CrossRefGoogle Scholar
Lee, J. and Shin, K. (2000). “The role of a variable input in the relationship between investment and uncertainty”. In: American Economic Review 90.3, pp. 667–680.10.1257/aer.90.3.667CrossRefGoogle Scholar
Legrand, N. (2019). “The empirical merit of structural explanations of commodity price volatility: Review and perspectives”. In: Journal of Economic Surveys 33.2, pp. 639–664.10.1111/joes.12291CrossRefGoogle Scholar
Lehrer, E. and Light, B. (2018). “The effect of interest rates on consumption in an income fluctuation problem”. In: Journal of Economic Dynamics and Control 94, pp. 63–71.10.1016/j.jedc.2018.07.004CrossRefGoogle Scholar
Lettau, M. and Ludvigson, S. C. (2014). “Shocks and crashes”. In: NBER Macroeconomics Annual 28.1, pp. 293–354.10.1086/674605CrossRefGoogle Scholar
Li, H. and Stachurski, J. (2014). “Solving the income fluctuation problem with unbounded rewards”. In: Journal of Economic Dynamics and Control 45, pp. 353–365.10.1016/j.jedc.2014.06.003CrossRefGoogle Scholar
Liao, J. and Berg, A. (2018). “Sharpening Jensen’s inequality”. In: The American Statistician.Google Scholar
Liggett, T. M. (2010). Continuous Time Markov Processes: An Introduction. Vol. 113. American Mathematical Society.Google Scholar
Light, B. (2018). “Precautionary saving in a Markovian earnings environment”. In: Review of Economic Dynamics 29, pp. 138–147.10.1016/j.red.2017.12.004CrossRefGoogle Scholar
Light, B. (2020). “Uniqueness of equilibrium in a Bewley–Aiyagari model”. In: Economic Theory 69.2, pp. 435–450.10.1007/s00199-018-1167-zCrossRefGoogle Scholar
Ljungqvist, L. (2002). “How do lay-off costs affect employment?” In: The Economic Journal 112.482, pp. 829–853.Google Scholar
Ljungqvist, L. and Sargent, T. (2018). Recursive Macroeconomic Theory. 4th ed. MIT Press.Google Scholar
Loewenstein, G. and Prelec, D. (1991). “Negative time preference”. In: American Economic Review 81, pp. 347–352.Google Scholar
Loewenstein, G. and Sicherman, N. (1991). “Do workers prefer increasing wage profiles?” In: Journal of Labor Economics 9, pp. 67–84.10.1086/298259CrossRefGoogle Scholar
Longstaff, F. A. and Schwartz, E. S. (2001). “Valuing American options by simulation: A simple least-squares approach”. In: The Review of Financial Studies 14.1, pp. 113–147.10.1093/rfs/14.1.113CrossRefGoogle Scholar
Lucas, R. and Prescott, E. (1971). “Investment under uncertainty”. In: Econometrica 39.5, pp. 659–81.10.2307/1909571CrossRefGoogle Scholar
Lucas, R. and Sargent, T. (1981). Rational Expectations and Econometric Practice. Vol. 2. University of Minnesota Press.Google Scholar
Lucas, R. E. (1976). “Econometric policy evaluation: A critique”. In: Carnegie-Rochester Conference Series on Public Policy. Vol. 1. North-Holland, pp. 19–46.10.1016/S0167-2231(76)80003-6CrossRefGoogle Scholar
Lucas, R. E. (1978a). “Asset prices in an exchange economy”. In: Econometrica 46.6, pp. 1429–1445.10.2307/1913837CrossRefGoogle Scholar
Lucas, R. E. (1978b). “Unemployment policy”. In: American Economic Review 68.2, pp. 353–357.Google Scholar
Lucas, R. E. and Stokey, N. L. (1984). “Optimal growth with many consumers”. In: Journal of Economic Theory 32.1, pp. 139–171.10.1016/0022-0531(84)90079-6CrossRefGoogle Scholar
Luo, Y. and Sang, P. (2022). “Penalized sieve estimation of structural models”. In: arXiv, 2204.13488.Google Scholar
Ma, Q. and Stachurski, J. (2021). “Dynamic programming deconstructed: Transformations of the Bellman equation and computational efficiency”. In: Operations Research 69.5, pp. 1591–1607.10.1287/opre.2020.2006CrossRefGoogle Scholar
Ma, Q., Stachurski, J., and Toda, A. A. (2020). “The income fluctuation problem and the evolution of wealth”. In: Journal of Economic Theory 187, p. 105003.10.1016/j.jet.2020.105003CrossRefGoogle Scholar
Ma, Q. and Toda, A. A. (2021). “A theory of the saving rate of the rich”. In: Journal of Economic Theory 192, p. 105193.10.1016/j.jet.2021.105193CrossRefGoogle Scholar
Majumdar, A., Singh, S., Mandlekar, A., and Pavone, M. (2017). “Risk-sensitive inverse reinforcement learning via coherent risk models”. In: Robotics: Science and Systems. Vol. 16, p. 117.Google Scholar
Maliar, L., Maliar, S., and Winant, P. (2021). “Deep learning for solving dynamic economic models.” In: Journal of Monetary Economics 122, pp. 76–101.10.1016/j.jmoneco.2021.07.004CrossRefGoogle Scholar
Marcet, A., Obiols-Homs, F., and Weil, P. (2007). “Incomplete markets, labor supply and capital accumulation”. In: Journal of Monetary Economics 54.8, pp. 2621–2635.10.1016/j.jmoneco.2006.12.011CrossRefGoogle Scholar
Marimon, R. (Aug. 1984). “General Equilibrium and Growth under Uncertainty: The Turnpike Property”. Northwestern University Economics Department Discussion Paper 624.Google Scholar
Marimon, R. (1989). “Stochastic turnpike property and stationary equilibrium”. In: Journal of Economic Theory 47.2, pp. 282–306.10.1016/0022-0531(89)90021-5CrossRefGoogle Scholar
Marinacci, M. and Montrucchio, L. (2010). “Unique solutions for stochastic recursive utilities”. In: Journal of Economic Theory 145.5, pp. 1776–1804.10.1016/j.jet.2010.02.005CrossRefGoogle Scholar
Marinacci, M. and Montrucchio, L. (2019). “Unique Tarski fixed points”. In: Mathematics of Operations Research 44.4, pp. 1174–1191.10.1287/moor.2018.0959CrossRefGoogle Scholar
Marinacci, M., Principi, G., and Stanca, L. (2023). “Recursive preferences and ambiguity attitudes”. In: arXiv, 2304.06830.Google Scholar
Martins-da-Rocha, V. F. and Vailakis, Y. (2010). “Existence and uniqueness of a fixed point for local contractions”. In: Econometrica 78.3, pp. 1127–1141.Google Scholar
McCall, J. J. (1970). “Economics of information and job search”. In: The Quarterly Journal of Economics 84.1, pp. 113–126.10.2307/1879403CrossRefGoogle Scholar
McFadden, D. (1974). “The measurement of urban travel demand”. In: Journal of Public Economics 3.4, pp. 303–328.10.1016/0047-2727(74)90003-6CrossRefGoogle Scholar
Mei, J., Xiao, C., Szepesvari, C., and Schuurmans, D. (2020). “On the global convergence rates of softmax policy gradient methods”. In: International Conference on Machine Learning. PMLR, pp. 6820–6829.Google Scholar
Meissner, T. and Pfeiffer, P. (2022). “Measuring preferences over the temporal resolution of consumption uncertainty”. In: Journal of Economic Theory 200, p. 105379.10.1016/j.jet.2021.105379CrossRefGoogle Scholar
Melo, F. S. (2001). Convergence of Q-learning: A simple proof. Tech. rep. Institute of Systems and Robotics.Google Scholar
Meyer, C. D. (2000). Matrix Analysis and Applied Linear Algebra. Vol. 71. Siam.10.1137/1.9780898719512CrossRefGoogle Scholar
Miao, J. (2006). “Competitive equilibria of economies with a continuum of consumers and aggregate shocks”. In: Journal of Economic Theory 128.1, pp. 274–298.10.1016/j.jet.2005.01.005CrossRefGoogle Scholar
Michelacci, C., Paciello, L., and Pozzi, A. (2022). “The extensive margin of aggregate consumption demand”. In: The Review of Economic Studies 89.2, pp. 909–947.10.1093/restud/rdab041CrossRefGoogle Scholar
Mirman, L. J. and Zilcha, I. (1975). “On optimal growth under uncertainty”. In: Journal of Economic Theory 11.3, pp. 329–339.10.1016/0022-0531(75)90022-8CrossRefGoogle Scholar
Mitten, L. (1964). “Composition principles for synthesis of optimal multistage processes”. In: Operations Research 12.4, pp. 610–619.10.1287/opre.12.4.610CrossRefGoogle Scholar
Mogensen, P. K. (2018). “Solving dynamic discrete choice models: Integrated or expected value function?” In: arXiv, 1801.03978.Google Scholar
Mordecki, E. (2002). “Optimal stopping and perpetual options for Lévy processes”. In: Finance and Stochastics 6.4, pp. 473–493.10.1007/s007800200070CrossRefGoogle Scholar
Mortensen, D. T. (1986). “Job search and labor market analysis”. In: Handbook of Labor Economics 2, pp. 849–919.10.1016/S1573-4463(86)02005-9CrossRefGoogle Scholar
Al-Najjar, N. I. and Shmaya, E. (2019). “Recursive utility and parameter uncertainty”. In: Journal of Economic Theory 181, pp. 274–288.10.1016/j.jet.2019.02.005CrossRefGoogle Scholar
Newhouse, D. (2005). “The persistence of income shocks: Evidence from rural Indonesia”. In: Review of Development Economics 9.3, pp. 415–433.10.1111/j.1467-9361.2005.00285.xCrossRefGoogle Scholar
Nirei, M. (2006). “Threshold behavior and aggregate fluctuation”. In: Journal of Economic Theory 127.1, pp. 309–322.10.1016/j.jet.2004.08.006CrossRefGoogle Scholar
Noor, J. and Takeoka, N. (2022). “Optimal discounting”. In: Econometrica 90.2, pp. 585–623.10.3982/ECTA16050CrossRefGoogle Scholar
Norets, A. (2010). “Continuity and differentiability of expected value functions in dynamic discrete choice models”. In: Quantitative Economics 1.2, pp. 305–322.10.3982/QE41CrossRefGoogle Scholar
Norris, J. R. (1998). Markov Chains. Cambridge University Press.Google Scholar
Nota, C. and Thomas, P. S. (2019). “Is the policy gradient a gradient?” In: arXiv, 1906.07073.Google Scholar
Nuño, G. and Moll, B. (2018). “Social optima in economies with heterogeneous agents”. In: Review of Economic Dynamics 28, pp. 150–180.10.1016/j.red.2017.08.003CrossRefGoogle Scholar
Ok, E. A. (2007). Real Analysis with Economic Applications. Vol. 10. Princeton University Press.10.1515/9781400840892CrossRefGoogle Scholar
Paciello, L. and Wiederholt, M. (2014). “Exogenous information, endogenous information, and optimal monetary policy”. In: Review of Economic Studies 81.1, pp. 356–388.10.1093/restud/rdt024CrossRefGoogle Scholar
Paroussos, L., Mandel, A., Fragkiadakis, K., Fragkos, P., Hinkel, J., and Vrontisi, Z. (2019). “Climate clubs and the macro-economic benefits of international cooperation on climate policy”. In: Nature Climate Change 9.7, pp. 542–546.10.1038/s41558-019-0501-1CrossRefGoogle Scholar
Perla, J. and Tonetti, C. (2014). “Equilibrium imitation and growth”. In: Journal of Political Economy 122.1, pp. 52–76.10.1086/674362CrossRefGoogle Scholar
Peskir, G. and Shiryaev, A. (2006). Optimal Stopping and Free-Boundary Problems. Springer Verlag.Google Scholar
Pissarides, C. A. (1979). “Job matchings with state employment agencies and random search”. In: The Economic Journal 89.356, pp. 818–833.10.2307/2231501CrossRefGoogle Scholar
Pohl, W., Schmedders, K., and Wilms, O. (2019). “Relative existence for recursive utility”. In: SSRN, 3432469.10.2139/ssrn.3432469CrossRefGoogle Scholar
Pohl, W., Schmedders, K., and Wilms, O. (2018). “Higher order effects in asset pricing models with long-run risks”. In: The Journal of Finance 73.3, pp. 1061–1111.10.1111/jofi.12615CrossRefGoogle Scholar
Powell, W. B. (2007). Approximate Dynamic Programming: Solving the Curses of Dimensionality. Vol. 703. John Wiley & Sons.10.1002/9780470182963CrossRefGoogle Scholar
Privault, N. (2013). Understanding Markov Chains: Examples and Applications. Springer-Verlag Singapore.Google Scholar
Puterman, M. L. (2005). Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley Interscience.Google Scholar
Quah, D. (1990). “Permanent and transitory movements in labor income: An explanation for excess smoothness in consumption”. In: Journal of Political Economy 98.3, pp. 449–475.10.1086/261690CrossRefGoogle Scholar
Ráfales, J. and Vázquez, C. (2021). “Equilibrium models with heterogeneous agents under rational expectations and its numerical solution”. In: Communications in Nonlinear Science and Numerical Simulation 96, p. 105673.10.1016/j.cnsns.2020.105673CrossRefGoogle Scholar
Ramsey, F. P. (1928). “A mathematical theory of saving”. In: The Economic Journal 38, pp. 543–559.10.2307/2224098CrossRefGoogle Scholar
Ren, G. and Stachurski, J. (2021). “Dynamic programming with value convexity”. In: Automatica 130, p. 109641.10.1016/j.automatica.2021.109641CrossRefGoogle Scholar
Rendahl, P. (2016). “Fiscal policy in an unemployment crisis”. In: The Review of Economic Studies 83.3, pp. 1189–1224.10.1093/restud/rdv058CrossRefGoogle Scholar
Rendahl, P. (2022). “Continuous vs. discrete time: Some computational insights”. In: Journal of Economic Dynamics and Control 144, p. 104522.10.1016/j.jedc.2022.104522CrossRefGoogle Scholar
Riedel, F. (2009). “Optimal stopping with multiple priors”. In: Econometrica 77.3, pp. 857–908.Google Scholar
Roberts, M. J. and Tybout, J. R. (1997). “The decision to export in Colombia: An empirical model of entry with sunk costs”. In: American Economic Review 87.4, pp. 545–564.Google Scholar
Rogers, L. C. (2002). “Monte Carlo valuation of American options”. In: Mathematical Finance 12.3, pp. 271–286.10.1111/1467-9965.02010CrossRefGoogle Scholar
Rogerson, R., Shimer, R., and Wright, R. (2005). “Search-theoretic models of the labor market: A survey”. In: Journal of Economic Literature 43.4, pp. 959–988.10.1257/002205105775362014CrossRefGoogle Scholar
Ross, S. A. (2009). Neoclassical Finance. Princeton University Press.10.2307/j.ctvcm4gxtCrossRefGoogle Scholar
Rubinstein, A. (2003). “Economics and psychology? The case of hyperbolic discounting”. In: International Economic Review 44.4, pp. 1207–1216.Google Scholar
Rust, J. (1987). “Optimal replacement of GMC bus engines: An empirical model of Harold Zurcher”. In: Econometrica 55.5, pp. 999–1033.10.2307/1911259CrossRefGoogle Scholar
Rust, J. (1994). “Structural estimation of Markov decision processes”. In: Handbook of Econometrics 4, pp. 3081–3143.10.1016/S1573-4412(05)80020-0CrossRefGoogle Scholar
Rust, J. (1997). “Using randomization to break the curse of dimensionality”. In: Econometrica 65.3, pp. 487–516.10.2307/2171751CrossRefGoogle Scholar
Ruszczyński, A. (2010). “Risk-averse dynamic programming for Markov decision processes”. In: Mathematical Programming 125.2, pp. 235–261.10.1007/s10107-010-0393-3CrossRefGoogle Scholar
Saijo, H. (2017). “The uncertainty multiplier and business cycles”. In: Journal of Economic Dynamics and Control 78, pp. 1–25.10.1016/j.jedc.2017.02.008CrossRefGoogle Scholar
Samuelson, P. A. (1939). “Interactions between the multiplier analysis and the principle of acceleration”. In: The Review of Economics and Statistics 21.2, pp. 75–78.10.2307/1927758CrossRefGoogle Scholar
Sargent, T. (1980). “Tobin’s q and the rate of investment in general equilibrium”. In: Carnegie-Rochester Conference Series on Public Policy. Vol. 12. Elsevier, pp. 107–154.10.1016/0167-2231(80)90024-XCrossRefGoogle Scholar
Sargent, T. (1987). Dynamic Macroeconomic Theory. Harvard University Press.Google Scholar
Sargent, T. and Stachurski, J. (2023a). “Completely abstract dynamic programming”. In: arXiv, 2308.02148.Google Scholar
Sargent, T. and Stachurski, J. (2023b). Economic Networks: Theory and Computation. Cambridge University Press.Google Scholar
Savage, L. J. (1951). “The theory of statistical decisions”. In: Journal of the American Statistical Association 46.253, pp. 55–67.10.1080/01621459.1951.10500768CrossRefGoogle Scholar
Scarf, H. (1960). “The optimality of (S, s) policies in the dynamic inventory problem”. In: Mathematical Methods in the Social Sciences, pp. 196–202.Google Scholar
Schechtman, J. (1976). “An income fluctuation problem”. In: Journal of Economic Theory 12.2, pp. 218–241.10.1016/0022-0531(76)90075-2CrossRefGoogle Scholar
Schorfheide, F., Song, D., and Yaron, A. (2018). “Identifying long-run risks: A Bayesian mixed-frequency approach”. In: Econometrica 86.2, pp. 617–654.10.3982/ECTA14308CrossRefGoogle Scholar
Shapley, L. S. (1953). “Stochastic games”. In: Proceedings of the National Academy of Sciences 39.10, pp. 1095–1100.10.1073/pnas.39.10.1095CrossRefGoogle ScholarPubMed
Shen, Y., Tobia, M. J., Sommer, T., and Obermayer, K. (2014). “Risk-sensitive reinforcement learning”. In: Neural Computation 26.7, pp. 1298–1328.10.1162/NECO_a_00600CrossRefGoogle ScholarPubMed
Shiryaev, A. N. (2007). Optimal Stopping Rules. Vol. 8. Springer Science & Business Media.Google Scholar
Sidford, A., Wang, M., Wu, X., and Ye, Y. (2023). “Variance reduced value iteration and faster algorithms for solving Markov decision processes”. In: Naval Research Logistics (NRL) 70.5, pp. 423–442.10.1002/nav.21992CrossRefGoogle Scholar
Stachurski, J. (2022). Economic Dynamics: Theory and Computation. 2nd ed. MIT Press.Google Scholar
Stachurski, J. and Toda, A. A. (2019). “An impossibility theorem for wealth in heterogeneous-agent models with limited heterogeneity”. In: Journal of Economic Theory 182, pp. 1–24.10.1016/j.jet.2019.04.001CrossRefGoogle Scholar
Stachurski, J., Wilms, O., and Zhang, J. (2022). “Unique solutions to power-transformed affine systems”. In: arXiv, 2212.00275.Google Scholar
Stachurski, J. and Zhang, J. (2021). “Dynamic programming with state-dependent discounting”. In: Journal of Economic Theory 192, p. 105190.10.1016/j.jet.2021.105190CrossRefGoogle Scholar
Stanca, L. (2023). Recursive preferences, correlation aversion, and the remporal resolution of uncertainty. Working papers. University of Torino.Google Scholar
Stokey, N. and Lucas, R. (1989). Recursive Methods in Dynamic Economics. Harvard University Press.10.2307/j.ctvjnrt76CrossRefGoogle Scholar
TallariniJr, T. D. (2000). “Risk-sensitive real business cycles”. In: Journal of Monetary Economics 45.3, pp. 507–532.10.1016/S0304-3932(00)00012-XCrossRefGoogle Scholar
Tauchen, G. (1986). “Finite state Markov-chain approximations to univariate and vector autoregressions”. In: Economics letters 20.2, pp. 177–181.10.1016/0165-1765(86)90168-0CrossRefGoogle Scholar
Thaler, R. (1981). “Some empirical evidence on dynamic inconsistency”. In: Economics Letters 8.3, pp. 201–207.10.1016/0165-1765(81)90067-7CrossRefGoogle Scholar
Toda, A. A. (2014). “Incomplete market dynamics and cross-sectional distributions”. In: Journal of Economic Theory 154, pp. 310–348.10.1016/j.jet.2014.09.015CrossRefGoogle Scholar
Toda, A. A. (2019). “Wealth distribution with random discount factors”. In: Journal of Monetary Economics 104, pp. 101–113.10.1016/j.jmoneco.2018.09.006CrossRefGoogle Scholar
Toda, A. A. (2021). “Perov’s contraction principle and dynamic programming with stochastic discounting”. In: Operations Research Letters 49.5, pp. 815–819.10.1016/j.orl.2021.09.001CrossRefGoogle Scholar
Tsitsiklis, J. N. (1994). “Asynchronous stochastic approximation and Q-learning”. In: Machine Learning 16, pp. 185–202.10.1007/BF00993306CrossRefGoogle Scholar
Tyazhelnikov, V. (2022). “Production clustering and offshoring”. In: American Economic Journal: Microeconomics 14.3, pp. 700–732.Google Scholar
Van, C. and Dana, R.-A. (2003). Dynamic Programming in Economics. Springer.Google Scholar
Von Neumann, J. and Morgenstern, O. (1944). Theory of Games and Economic Behavior. Princeton University Press.Google Scholar
Walker, H. F. and Ni, P. (2011). “Anderson acceleration for fixed-point iterations”. In: SIAM Journal on Numerical Analysis 49.4, pp. 1715–1735.10.1137/10078356XCrossRefGoogle Scholar
Wang, P. and Wen, Y. (2012). “Hayashi meets Kiyotaki and Moore: A theory of capital adjustment costs”. In: Review of Economic Dynamics 15.2, pp. 207–225.10.1016/j.red.2011.09.004CrossRefGoogle Scholar
Watkins, C. J. C. H. (1989). “Learning from delayed rewards”. In: PhD Thesis, Cambridge University.Google Scholar
Weil, P. (1990). “Nonexpected utility in macroeconomics”. In: The Quarterly Journal of Economics 105.1, pp. 29–42.10.2307/2937817CrossRefGoogle Scholar
Whittle, P. (1981). “Risk-sensitive linear/quadratic/Gaussian control”. In: Advances in Applied Probability 13.4, pp. 764–777.10.2307/1426972CrossRefGoogle Scholar
Woodford, M. (2011). “Simple analytics of the government expenditure multiplier”. In: American Economic Journal: Macroeconomics 3.1, pp. 1–35. issn: 1945-7715.Google Scholar
Ying, L. and Zhu, Y. (2020). A note on optimization formulations of Markov decision processes. Tech. rep. arXiv, 2012.09417.Google Scholar
Yu, L., Lin, L., Guan, G., and Liu, J. (2023). “Time-consistent lifetime portfolio selection under smooth ambiguity”. In: Mathematical Control and Related Fields 13.3, pp. 967–987.10.3934/mcrf.2022023CrossRefGoogle Scholar
Yue, V. Z. (2010). “Sovereign default and debt renegotiation”. In: Journal of International Economics 80.2, pp. 176–187.10.1016/j.jinteco.2009.11.004CrossRefGoogle Scholar
Zaanen, A. C. (2012). Introduction to Operator Theory in Riesz Spaces. Springer.Google Scholar
Zhang, Z. (2012). Variational, Topological, and Partial Order Methods with Their Applications. Vol. 29. Springer.Google Scholar
Zhao, G. (2020). “Ambiguity, nominal bond yields, and real bond yields”. In: American Economic Review: Insights 2.2, pp. 177–192.Google Scholar
Zhu, S. (2020). “Existence of stationary equilibrium in an incomplete-market model with endogenous labor supply”. In: International Economic Review 61.3, pp. 1115–1138.10.1111/iere.12451CrossRefGoogle Scholar

Accessibility standard: Unknown

Accessibility compliance for the PDF of this book is currently unknown and may be updated in the future.

Save book to Kindle

To save this book to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

  • References
  • Thomas J Sargent, Hoover Institute , John Stachurski, Australian National University, Canberra
  • Book: Dynamic Programming
  • Online publication: 17 July 2025
  • Chapter DOI: https://doi.org/10.1017/9781009540780.015
Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

  • References
  • Thomas J Sargent, Hoover Institute , John Stachurski, Australian National University, Canberra
  • Book: Dynamic Programming
  • Online publication: 17 July 2025
  • Chapter DOI: https://doi.org/10.1017/9781009540780.015
Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

  • References
  • Thomas J Sargent, Hoover Institute , John Stachurski, Australian National University, Canberra
  • Book: Dynamic Programming
  • Online publication: 17 July 2025
  • Chapter DOI: https://doi.org/10.1017/9781009540780.015
Available formats
×