Graduate Thesis Or Dissertation
 

Convergence Analysis Framework for Fixed-Point Algorithms in Machine Learning and Signal Processing

Public Deposited

Downloadable Content

Download PDF
https://ir.library.oregonstate.edu/concern/graduate_thesis_or_dissertations/dv140236k

Descriptions

Attribute NameValues
Creator
Abstract
  • Iterative algorithms are simple yet efficient in solving large-scale optimization problems in practice. With a surge in the amount of data in past decades, these methods have become increasingly important in many application areas including matrix/tensor recovery, deep learning, data mining, and reinforcement learning. To optimize or improve iterative algorithms, it is crucial to understand how to characterize their performance. Existing works in the literature offer bounds (including global) on the convergence rate of such algorithms. In most cases, a general global convergence analysis tends to produce conservative convergence rate estimates. In contrast, exact rate analysis predicts accurately the behavior of iterative algorithms in practice. In this dissertation, the goal is to develop a unified framework, with theoretical foundations, to aid the derivation of sharp convergence results for iterative algorithms in machine learning and signal processing (MLSP) problems. By viewing iterative methods as fixed-point iterations, the existing powerful tools in fixed-point theory are utilized to study their asymptotic convergence. Via the linear approximation of the fixed-point operator around the solution, the proposed approach provides the following key results in convergence analysis: sufficient conditions for local linear convergence, the exact linear rate of convergence, and the number of iterations required to reach certain accuracy. A collection of fundamental MLSP problems are examined to demonstrate the applicability of the proposed framework. In certain problems, such as matrix completion, the novel insight into the local convergence behavior furthers our understanding of the problem and establishes intriguing connections with existing convergence results in the literature. Finally, the dissertation discusses practical methods to obtain the optimal rate of convergence and acceleration techniques that exploit the closed-form expressions of the convergence rate.
License
Resource Type
Date Issued
Degree Level
Degree Name
Degree Field
Degree Grantor
Commencement Year
Advisor
Committee Member
Academic Affiliation
Rights Statement
Publisher
Peer Reviewed
Language

Relationships

Parents:

This work has no parents.

In Collection:

Items

Thumbnail Title Date Uploaded Visibility Actions
file details VuTrungV2023.pdf 2022-09-14 Public Download