Workshop: FTXS: Workshop for Faults, Trustworthiness, and eXplainability for AI Systems at Scale
Authors: Kathryn Knight (Oak Ridge National Laboratory (ORNL)) and Patrick Widener and Heather Bort (Oak Ridge National Laboratory)
Abstract: We address an incipient type of data-based faults, driven by increasing data reuse in AI-enhanced computational science workflows. A dataset is created under specific conditions; its categories, measures, and labels depend on that context, but that context is usually unavailable downstream and abandoned when datasets are used to train models. We propose encoding this context, called domain bounds, with the data to which it applies. A domain bound flags apparently valid data proposed for reuse in an incompatible context, even when every value falls within its expected range. We present cases from computational science and biomedicine where missing domain bounds allow silent reuse errors, and show why description and provenance approaches miss them. Our detector uses standard data container technologies and catches faults from a taxonomy of published cases, at a cost independent of dataset size. It caught 17 of 18 injected domain-bound reuses that lineage and descriptive records accepted.
Back to FTXS: Workshop for Faults, Trustworthiness, and eXplainability for AI Systems at Scale Archive Listing
Back to Full Workshop Archive Listing