http://hdl.handle.net/1957/8305 Search the Channel search http://ir.library.oregonstate.edu/jspui/simple-search http://hdl.handle.net/1957/13225 Title: End-User Feature Engineering in the Presence of Class Imbalance<br/><br/>Authors: Oberst, Ian; Moore, Travis; Wong, Weng-Keen; Kulesza, Todd; Stumpf, Simone; Riche, Yann; Burnett, Margaret<br/><br/>Abstract: Intelligent user interfaces, such as recommender systems and email classifiers, use machine learning algorithms to customize their behavior to the preferences of an end user. Although these learning systems are somewhat reliable, they are not perfectly accurate. Traditionally, end users who need to correct these learning systems can only provide more labeled training data. In this paper, we focus on incorporating new features suggested by the end user into machine learning systems. To investigate the effects of user-generated features on accuracy we developed an auto- coding application that enables end users to assist a machine-learned program in coding a transcript by adding custom features. Our results show that adding user-generated features to the machine learning algorithm can result in modest improvements to its F1 score. Further improvements are possible if the algorithm accounts for class imbalance in the training data and deals with low-quality user-generated features that add noise to the learning algorithm. We show that addressing class imbalance improves performance to an extent but improving the quality of features brings about the most beneficial change. Finally, we discuss changes to the user interface that can help end users avoid the creation of low-quality features. http://hdl.handle.net/1957/12770 Title: A Strategy-Centric Approach to the Design of End-User Debugging Tools<br/><br/>Authors: Grigoreanu, Valentina; Burnett, Margaret; Robertson, George<br/><br/>Abstract: End-user programmers’ code is notoriously buggy. This problem is amplified by the increasing complexity of end users’ programs. To help end users catch errors early and reliably, we employ a novel approach for the design of end-user debugging tools: a focus on supporting end users’ effective debugging strategies. This paper has two core contributions. We first demonstrate the potential of a strategy-centric approach to tool design by presenting StratCel, a strategy-based tool for Excel. Second, we show the benefits of this design approach: participants using StratCel found twice as many bugs as participants using standard Excel, they fixed four times as many bugs, and all this in only a small fraction of the time. Furthermore, this strategy-based approach helped the participants who needed it the most: boosting novices’ debugging performance near experienced participants’ improved levels. Finally, we reveal several opportunities for future research about strategy-based debugging tools. http://hdl.handle.net/1957/12706 Title: End-User Debugging of Machine-Learned Programs: Toward Principles for Baring the Logic<br/><br/>Authors: Kulesza, Todd; Stumpf, Simone; Riche, Yann; Burnett, Margaret; Wong, Weng-Keen; Oberst, Ian; Moor, Travis; McIntosh, Kevin; Bice, Forrest<br/><br/>Abstract: Many applications include machine learning algorithms intended to learn “programs” (rules of behavior) from an end user’s actions. When these learned programs are wrong, their users receive little explanation as to why, and even less freedom of expression to help the machine learn from its mistakes. In this paper, we develop and explore a set of candidate principles for providing salient debugging information to end users who would like to correct these programs. We informed the candidate principles through a formative study, built a prototype that instantiates them, and conducted a user study of the prototype to collect empirical evidence to inform future variants. Our results suggest the value of exposing the machine’s reasoning process, supporting a flexible debugging vocabulary, and illustrating the effects of user changes to the learned program’s logic. http://hdl.handle.net/1957/12443 Title: Design implications for end-user debugging tools : a strategy-based view<br/><br/>Authors: Grigoreanu, Valentina; Burnett, Margaret; Robertson, George<br/><br/>Abstract: End-user programmers’ code (e.g., accountants’ spreadsheet formulas) is fraught with errors. To help mitigate this problem, end-user software engineering research is becoming established. However, most of this work has focused on feature usage, rather than debugging strategies. If a debugging tool were to support end-user programmers’ specific debugging strategy needs, what should it take into account and how? To consider the design of such tools, this work contributes a comprehensive overview of end-user debugging strategies at four strategy levels. An example empirical study in Microsoft Excel demonstrates that this view of debugging provides useful insights, and we argue that many of these insights generalize to other environments. Our results include end-user debugging tactics and the effective and ineffective moves employed to achieve them, ten end-user debugging strategems applied to a new environment, and how these strategems were used within three contexts: by strategy used, by sensemaking step, and by debugging phase. These findings coalesce into a comprehensive overview of end-user debugging strategies and detailed implications for the design of strategy-based end-user debugging tools.