Graduate Thesis Or Dissertation
 

Local and Global Explanations for Deep Image Classification via Structured Attention Graphs

Public Deposited

Downloadable Content

Download PDF
https://ir.library.oregonstate.edu/concern/graduate_thesis_or_dissertations/jw827k54g

Descriptions

Attribute NameValues
Creator
Abstract
  • Attention maps are popular tools of explaining the decisions of convolutional neural net-works (CNNs) for image classification. Typically, for each image of interest, a single attention map is produced, which assigns weights to pixels based on their importance to the classification. We argue that a single attention map provides an incomplete under-standing since there are often many other maps that explain a classification equally well. In this thesis, we show that there are indeed multiple relatively localized explanations for many images which can be systematically enumerated by search methods such as beam search. Based on this finding, we introduce structured attention graphs (SAGs), which compactly represent sets of attention maps for an image by capturing how different combinations of image regions impact the confidence of a classifier. We propose an approach to compute SAGs and a visualization for SAGs so that deeper insight can be gained into the classifier’s decisions. We conduct a user study comparing the use of SAGs to traditional attention maps for answering counterfactual questions about image classifications. Our results show that the users answer comparative counterfactual questions better when presented with SAGs compared to attention map baselines. Further, we extend SAGs from providing local explanations for image instances to provide global explanations for class instances that hold across sets of images. We build an interpretable nearest-neighbour classifier by agglomeratively grouping important image patches obtained from SAGs into clusters that are coherent in semantics and separable by class labels.
License
Resource Type
Date Issued
Degree Level
Degree Name
Degree Field
Degree Grantor
Commencement Year
Advisor
Academic Affiliation
Rights Statement
Publisher
Peer Reviewed
Language
Embargo reason
  • Pending Publication
Embargo date range
  • 2021-07-10 to 2022-08-11

Relationships

Parents:

This work has no parents.

In Collection:

Items

Thumbnail Title Date Uploaded Visibility Actions
file details ShitoleVivswan2022.pdf 2021-07-10 Public Download