Hierarchical structure discovery and transfer in sequential decision problems Public Deposited

http://ir.library.oregonstate.edu/concern/graduate_thesis_or_dissertations/4j03d2489

Descriptions

Attribute NameValues
Creator
Abstract or Summary
  • Acting intelligently to efficiently solve sequential decision problems requires the ability to extract hierarchical structure from the underlying domain dynamics, exploit it for optimal or near-optimal decision-making, and transfer it to related problems instead of solving every problem in isolation. This dissertation makes three contributions toward this goal. The first contribution is the introduction of two frameworks for the transfer of hierarchical structure in sequential decision problems. The MASH framework facilitates transfer among multiple agents coordinating within a domain. The VRHRL framework allows an agent to transfer its knowledge across a family of domains that share the same transition dynamics but have differing reward dynamics. Both MASH and VRHRL are validated empirically in large domains and the results demonstrate significant speedup in the solutions due to transfer. The second contribution is a new approach to the discovery of hierarchical structure in sequential decision problems. HI-MAT leverages action models to analyze the relevant dependencies in a hierarchically-generated trajectory and it discovers hierarchical structure that transfers to all problems whose actions share the same relevant dependencies as the single source problem. HierGen advances HI-MAT by learning simple action models, leveraging these models to analyze non-hierarchically-generated trajectories from multiple source problems in a robust causal fashion, and discovering hierarchical structure that transfers to all problems whose actions share the same causal dependencies as those in the source problems. Empirical evaluations in multiple domains demonstrate that the discovered hierarchical structures are comparable to manually-designed structures in quality and performance. Action models are essential to hierarchical structure discovery and other aspects of intelligent behavior. The third contribution of this dissertation is the introduction of two general frameworks for learning action models in sequential decision problems. In the MBP framework, learning is user-driven; in the PLEX framework, the learner generates its own problems. The frameworks are formally analyzed and reduced to concept learning with one-sided error. A general action-modeling language is shown to be efficiently learnable in both frameworks.
Resource Type
Date Available
Date Copyright
Date Issued
Degree Level
Degree Name
Degree Field
Degree Grantor
Commencement Year
Advisor
Committee Member
Academic Affiliation
Non-Academic Affiliation
Keyword
Subject
Rights Statement
Peer Reviewed
Language
Replaces
Additional Information
  • description.provenance : Approved for entry into archive by Julie Kurtz(julie.kurtz@oregonstate.edu) on 2011-11-03T15:23:53Z (GMT) No. of bitstreams: 1 MehtaNevilleC2011.pdf: 2587651 bytes, checksum: b568708bdca60cf31c91f02cf895ee8c (MD5)
  • description.provenance : Approved for entry into archive by Laura Wilson(laura.wilson@oregonstate.edu) on 2011-11-04T16:06:23Z (GMT) No. of bitstreams: 1 MehtaNevilleC2011.pdf: 2587651 bytes, checksum: b568708bdca60cf31c91f02cf895ee8c (MD5)
  • description.provenance : Submitted by Neville Mehta (mehtane@onid.orst.edu) on 2011-11-03T08:10:07Z No. of bitstreams: 1 MehtaNevilleC2011.pdf: 2587651 bytes, checksum: b568708bdca60cf31c91f02cf895ee8c (MD5)
  • description.provenance : Made available in DSpace on 2011-11-04T16:06:23Z (GMT). No. of bitstreams: 1 MehtaNevilleC2011.pdf: 2587651 bytes, checksum: b568708bdca60cf31c91f02cf895ee8c (MD5) Previous issue date: 2011-09-23

Relationships

In Administrative Set:
Last modified: 08/15/2017

Downloadable Content

Download PDF
Citations:

EndNote | Zotero | Mendeley

Items