A study of model-based average reward reinforcement learning Public Deposited

http://ir.library.oregonstate.edu/concern/graduate_thesis_or_dissertations/n296x195z

Descriptions

Attribute NameValues
Creator
Abstract or Summary
  • Reinforcement Learning (RL) is the study of learning agents that improve their performance from rewards and punishments. Most reinforcement learning methods optimize the discounted total reward received by an agent, while, in many domains, the natural criterion is to optimize the average reward per time step. In this thesis, we introduce a model-based average reward reinforcement learning method called "H-learning" and show that it performs better than other average reward and discounted RL methods in the domain of scheduling a simulated Automatic Guided Vehicle (AGV). We also introduce a version of H-learning which automatically explores the unexplored parts of the state space, while always choosing an apparently best action with respect to the current value function. We show that this "Auto-exploratory H-Learning" performs much better than the original H-learning under many previously studied exploration strategies. To scale H-learning to large state spaces, we extend it to learn action models and reward functions in the form of Bayesian networks, and approximate its value function using local linear regression. We show that both of these extensions are very effective in significantly reducing the space requirement of H-learning, and in making it converge much faster in the AGV scheduling task. Further, Auto-exploratory H-learning synergistically combines with Bayesian network model learning and value function approximation by local linear regression, yielding a highly effective average reward RL algorithm. We believe that the algorithms presented here have the potential to scale to large applications in the context of average reward optimization.
Resource Type
Date Available
Date Copyright
Date Issued
Degree Level
Degree Name
Degree Field
Degree Grantor
Commencement Year
Advisor
Committee Member
Academic Affiliation
Non-Academic Affiliation
Subject
Rights Statement
Peer Reviewed
Language
Digitization Specifications
  • File scanned at 300 ppi (Monochrome, 8-bit Grayscale) using ScandAll PRO 1.8.1 on a Fi-6670 in PDF format. CVista PdfCompressor 4.0 was used for pdf compression and textual OCR.
Replaces
Additional Information
  • description.provenance : Approved for entry into archive by Patricia Black(patricia.black@oregonstate.edu) on 2012-10-25T19:10:44Z (GMT) No. of bitstreams: 1 OkDoKyeong1996.pdf: 5956811 bytes, checksum: 19b4efd39a58eae2bfc84b5e1afdb2f0 (MD5)
  • description.provenance : Made available in DSpace on 2012-10-25T19:10:44Z (GMT). No. of bitstreams: 1 OkDoKyeong1996.pdf: 5956811 bytes, checksum: 19b4efd39a58eae2bfc84b5e1afdb2f0 (MD5) Previous issue date: 1996-05-09
  • description.provenance : Submitted by John Valentino (valentjo@onid.orst.edu) on 2012-10-19T19:25:32Z No. of bitstreams: 1 OkDoKyeong1996.pdf: 5956811 bytes, checksum: 19b4efd39a58eae2bfc84b5e1afdb2f0 (MD5)
  • description.provenance : Approved for entry into archive by Patricia Black(patricia.black@oregonstate.edu) on 2012-10-22T16:50:54Z (GMT) No. of bitstreams: 1 OkDoKyeong1996.pdf: 5956811 bytes, checksum: 19b4efd39a58eae2bfc84b5e1afdb2f0 (MD5)

Relationships

Parents:

This work has no parents.

Last modified

Downloadable Content

Download PDF

Items