This thesis addresses the problem of temporal action segmentation in videos, where the goal is to label every video frame with the appropriate action class present. We focus on the domain of NFL football videos, where action classes represent common football play types. For action segmentation, we use a temporal...
Alignment of genomic sequences from different species is becoming an increasingly powerful method in biology, and is being used for many purposes. The result of sequence alignments is a list of pairs of matched locations between the pattern string and the text string. However, without any proper visualization tools to...
In this dissertation, we address action segmentation in videos under limited supervision. The goal of action segmentation is to predict an action class for each frame of a video. The limited supervision means ground truth labels of video frames are not available in training. We focus on three types of...
Significance: Movement intent decoding algorithms can interpret human bioelectrical signals to control prosthetic limbs with many degrees of freedom (DOFs). This work involves decoding volitional movement intent from surface electromyogram (sEMG) signals to control prosthetic arms. To train these algorithms, patients flex their muscles to “follow” a movement prompt, and...
Humans are remarkably efficient in learning by interacting with other people and observing their behavior. Children learn by watching their parents’ actions and mimic their behavior. When they are not sure about their parents demonstration, they communicate with them, ask questions, and learn from their feedback. On the other hand,...
Knowledge workers are struggling in the information flood. There is a growing interest in intelligent desktop environments that help knowledge workers organize their daily life. Intelligent desktop environments allow the desktop user to define a set of “activities” that characterize the user’s desktop work. These environments then attempt to identify...
Partial programming is a field of study where users specify an outline or skeleton of a program, but leave various parts undefined. The undefined parts are then completed by an external mechanism to form a complete program. Adaptation-Based Programming (ABP) is a method of partial programming that utilizes techniques from...
Movement intent decoders, which interpret volitional movement intent from human bioelectric signals, can be incorporated into modern neuroprostheses to offer people living with limb loss or paralysis the potential to regain their lost motor control. Machine learning methods have become the research standard for continuous decoders with high degrees of...
Robotic Bipedal locomotion holds the potential for efficient, robust traversal of difficult terrain. The difficulty lies in the dynamics of locomotion which complicate control and motion planning. Bipedal locomotion dynamics are dimensionally large problems, extremely nonlinear, and operate on the limits of actuator capabilities, which limit the performance of generic...
As bipedal robots move ever closer to being integrated into all manner of real world envi-ronments there is a necessity to push their dynamic capabilities to meet or exceed those of humans and animals. Advancements must be made to address ordinary challenges that arise everyday in the same environments that...
A fundamental problem in computer vision is to partition an image into meaningful segments. While image segmentation is required by many applications, the thesis focuses on segmentation of computed tomography (CT) images for analysis and quality control of composite materials. The key research contribution of this thesis is a novel...
Advances in sensor technology are greatly expanding the range of quantities that can be measured while simultaneously reducing the cost. However, deployed sensors drift out of calibration and fail, so every sensor network requires quality control procedures to promptly detect these failures. To address these problems, we propose a two-level...
Coordinating multiple robots to achieve a complex task requires solving two distinct control problems: the high-level control problem of ensuring that each robot aims to perform a useful task (e.g., coordination) and the low-level control problem of ensuring that each robot actually performs the correct actions to achieve its task...
This dissertation explores the idea of applying machine learning technologies to help computer users find information and better organize electronic resources, by presenting the research work conducted in the following three applications: FolderPredictor, Stacking Recommendation Engines, and Integrating Learning and Reasoning.
FolderPredictor is an intelligent desktop software tool that helps...
In an increasingly computation-driven world, algorithms and mathematical models significantly impact decision making across various fields. To foster trust and understanding, it is crucial to provide users with clear and concise explanations of the reasoning behind the results produced by computational tools, especially when recommendations appear counterintuitive. Legal frameworks in...
Bayesian Optimization (BO) methods are often used to optimize an unknown function f(•) that is costly to evaluate. They typically work in an iterative manner. In each iteration, given a set of observation points, BO algorithms select k ≥ 1 points to be evaluated. The results of those points are...
In this dissertation, we present a user-in-the-loop method for the design of an interactive motion data structure that benefits from the advantages of both motion graphs and blend-based techniques. Our novel approach automatically analyzes a traditional motion graph built from labeled motion clips. The result is a more condensed, coarser...
This dissertation delves into understanding, characterizing, and addressing dataset shift in deep learning, a pervasive issue for deployed machine learning systems. Integral aspects of the problem are examined: We start with the use of counterfactual explanations in order to characterize the behavior of deep reinforcement learning agents in visual input...
Relational binary operators, such as join, are arguably the most costly and frequently used operations in relational data systems. In many join algorithms, the majority of the process time is spent on scanning and attempting to join the parts of the relations that do not satisfy the join condition and...
For a certain class of Z²-actions, we provide a proof of a conjecture that the ratio of the Perron eigenvalues of the transfer matrices of the free boundary restrictions converge to the entropy of that action. Also, a novel method for computing the entropy of Z²-actions is conjectured.