Artificial Intelligence (AI) systems are increasingly outclassing human performances in many different fields and applications. Unfortunately, due to the usage of opaque Machine Learning (ML) algorithms, the logic behind their predictions often is difficult, if not impossible, to understand from a human standpoint and, for this reason, they are often called “black-box” models. As a consequence, in recent years, there has been an ever growing interest, from a technical, social, and legal point of view, in defining eXplainable AI (XAI) methods to describe the behavior of black-box models used by AI decision systems.

There are different approaches for explaining ML models depending on the input data, the black-box and the kind of explanation required. However, while interpretability is widely studied for tabular data [1], images [2] and only partially for texts [3], there is a surprising lack of research on black-box models for tasks regarding sequential data. Furthermore, the increasing availability of data stored in the form of time series such as electrocardiogram records, motion sensors data, climate measurements, stock indices, and so on, contributed to the diffusion of a wide range of time series classifiers employed in high-stakes decision making, where the explanation aspect becomes the crucial building brick for a trustworthy interaction between the human expert and the AI system.

For these reasons we tackled the problem of interpretability for opaque time series classifiers, i.e. black-box models that take as input a time series and predict its label, by proposing a Local Agnostic Subsequence-based Time Series explainer (LASTS)[7], whose objective is to return an explanation that is easily understandable from a human standpoint. The human mind usually tends to reason in terms of counterfactuals, i.e. modifications in the input that change the prediction outcome [4]. In fact, while “direct” explanations such as decision rules are crucial for understanding the reasons for a certain outcome, a counterfactual reveals what should be different in the classified instance in order to have a different black-box outcome. Our goal is to provide an explanation that offers the most possible complete way to understand the decision of a time series black-box classifier. Therefore, the output explanation of LASTS is composed both by a factual and a counterfactual subsequence-based rule that show the reasons for the classification in terms of logic conditions, indicating the subsequences of the time series that must, and must not, be contained in order to have a specific label returned by the black-box. In addition, the explanation contains also a set of exemplars and counterexemplars time series. Exemplars are instances classified with the same label as the time series being explained, i.e. prototypes highlighting similarities and common parts responsible for the classification. On the other hand, counterexemplars are instances similar to the one being explained but with a different label, and provide evidence of how the time series could be “morphed” to be classified with a different label, giving an intuitive idea of the decision boundary of the black-box, as pictured in Fig. 1 for the dataset Cylinder-Bell-Funnel.

LASTS uses an autoencoder, i.e. a neural network trained in order to compress and reconstruct instances as well as possible, to first encode a time series into a latent, simpler, representation. Then, in this latent space, through a genetic algorithm [5], it generates a neighborhood of new instances similar to the input instance and learns a local decision tree classifier to find factual and counterfactual rules. Latent instances respecting these rules are then decoded into exemplar and counterexemplar time series. These extracted time series are used to learn an interpretable shapelet-based decision tree that imitates the black-box, explaining its prediction in terms of subsequences that must, and must not, be contained in the time series. In Fig. 2 we show an example of the explanation for an instance of the dataset Cylinder-Bell-Funnel.

We tested LASTS on four datasets and three black-boxes and showed that it effectively challenges existing state of the art explainers, like SHAP [6], in terms of fidelity and stability, providing also easily interpretable explanations.

Figure 1. A morphing matrix for the dataset Cylinder-Bell-Funnel generated by modifying horizontally and vertically the two latent features obtained with the encoder compression. In green are depicted instances of the class bell, slowly morphing into instances having a different label, depicted in red.

Figure 2. An example of the explanation for an instance of the dataset Cylinder-Bell-Funnel. The instance x is correctly classified by the black-box as belonging to the class bell. In the second and third rows we can see, respectively, the exemplars (green) and counterexemplars (red) and the shapelet factual and counterfactual rules. The rules indicate that the black-box is “looking” at the slope of the central part of the time series.

Written by Francesco Spinnato

REFERENCES

[1] R. Guidotti, A. Monreale, S. Ruggieri, F. Turini, D. Pedreschi, and F. Giannotti.  A survey of methods for explaining black box models, 2018.

[2] D.  V.  Carvalho,  E.  M.  Pereira,  and  J.  S.  Cardoso.   Machine  learning interpretability:  A survey on methods and metrics. Electronics, 8(8), 2019.

[3] H. Liu, Q. Yin, and W. Y. Wang.  Towards explainable NLP: A generative explanation framework for text classification.  InProceedings  of  the 57th  Annual  Meeting  of  the  Association  for  Computational  Linguistics, pages 5570–5581, Florence, Italy, July 2019. Association for Computational Linguistics.

[4] R. M. Byrne.  Counterfactuals in explainable artificial intelligence (xai): evidence  from  human  reasoning.   In Proceedings  of  the  Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19, pages 6276–6282, 2019.

[5] R. Guidotti, A. Monreale, S. Ruggieri, D. Pedreschi, F. Turini, and F. Giannotti. Local rule-based explanations of black box decision systems.CoRR,abs/1805.10820, 2018.

[6] S.  Lundberg  and  S.-I.  Lee.   A  unified  approach  to  interpreting  model predictions, 2017.

[7] R. Guidotti, Anna Monreale, F. Spinnato, D. Pedreschi and F. Giannotti, 2020. Explaining Any Time Series Classifier. IEEE Second International Conference on Cognitive Machine Intelligence (CogMI), 2020.