Shrunken learning rates do not improve AdaBoost on benchmark datasets Public Deposited

http://ir.library.oregonstate.edu/concern/graduate_thesis_or_dissertations/ks65hg77k

Descriptions

Attribute NameValues
Creator
Abstract or Summary
  • Recent work has shown that AdaBoost can be viewed as an algorithm that maximizes the margin on the training data via functional gradient descent. Under this interpretation, the weight computed by AdaBoost, for each hypothesis generated, can be viewed as a step size parameter in a gradient descent search. Friedman has suggested that shrinking these step sizes could produce improved generalization and reduce overfitting. In a series of experiments, he showed that very small step sizes did indeed reduce overfitting and improve generalization for three variants of Gradient_Boost, his generic functional gradient descent algorithm. For this report, we tested whether reduced learning rates can also improve generalization in AdaBoost. We tested AdaBoost (applied to C4.5 decision trees) with reduced learning rates on 28 benchmark datasets. The results show that reduced learning rates provide no statistically significant improvement on these datasets. We conclude that reduced learning rates cannot be recommended for use with boosted decision trees on datasets similar to these benchmark datasets.
Resource Type
Date Available
Date Copyright
Date Issued
Degree Level
Degree Name
Degree Field
Degree Grantor
Commencement Year
Advisor
Committee Member
Academic Affiliation
Non-Academic Affiliation
Subject
Rights Statement
Peer Reviewed
Language
Digitization Specifications
  • File scanned at 300 ppi (Monochrome) using ScandAll PRO 1.8.1 on a Fi-6670 in PDF format. CVista PdfCompressor 4.0 was used for pdf compression and textual OCR.
Replaces
Additional Information
  • description.provenance : Approved for entry into archive by Patricia Black(patricia.black@oregonstate.edu) on 2012-07-12T19:29:49Z (GMT) No. of bitstreams: 1 ForrestDanielLK2002.pdf: 234059 bytes, checksum: 7b46db1ea826e7a80bd93207ad7ee00e (MD5)
  • description.provenance : Made available in DSpace on 2012-07-12T19:29:49Z (GMT). No. of bitstreams: 1 ForrestDanielLK2002.pdf: 234059 bytes, checksum: 7b46db1ea826e7a80bd93207ad7ee00e (MD5) Previous issue date: 2001-11-30
  • description.provenance : Submitted by Kaylee Patterson (kdpscanner@gmail.com) on 2012-06-26T20:14:11Z No. of bitstreams: 1 ForrestDanielLK2002.pdf: 234059 bytes, checksum: 7b46db1ea826e7a80bd93207ad7ee00e (MD5)
  • description.provenance : Approved for entry into archive by Patricia Black(patricia.black@oregonstate.edu) on 2012-07-12T19:26:53Z (GMT) No. of bitstreams: 1 ForrestDanielLK2002.pdf: 234059 bytes, checksum: 7b46db1ea826e7a80bd93207ad7ee00e (MD5)

Relationships

Parents:

This work has no parents.

Last modified

Downloadable Content

Download PDF

Items