softXplain

Contact persons:

• Prof. Kurt Schneider
• Hannah Deters

Software systems are becoming increasingly complex and at the same time we are dependent on these systems in more and more areas. It is therefore becoming increasingly difficult to understand these systems, while at the same time becoming more critical. Since there is usually no contact person available for questions and problems when using software systems, the software systems must be able to answer possible questions themselves. This property of a software system to explain its own behaviour is called explainability. Similar to usability, security or maintainability, explainability is a non-functional requirement whose exact meaning and degree of implementation must be specified in a software project. If these properties are implemented well, confusion during use can be reduced and acceptance of the system can be increased.

Since different quality attributes may contradict each other, it is important to determine which explainability requirements are truly required. The focus is on which features are useful, what is realistically implementable and how a balance between competing quality requirements can be achieved. To make this possible, we seek to answer the following key questions:

• How can the software know what behaviour is expected of it?
• How can we tell if people really need explanations?
• What is the correct form and the appropriate timing for these explanations?

In support of this, we want to develop demonstrators and prototypes that enable empirical studies in the context of our research.

The focus of this project is not to make black box models such as neural networks explainable. Indeed, we do not focus on areas such as artificial intelligence or machine learning in particular. Rather, complicated, self-explanatory software systems are to be investigated in general, revealing when explanations are truly needed and to what extent they can be provided.

One possible cause of confusion is a discrepancy between the users' mental model and the actual behaviour of a system. A mental model is a construct of ideas that users have built up about the system. Depending on this, users form expectations about the system's behaviour and adapt their own actions towards it. If the users' predictions do not match the actual behaviour, confusion arises. This confusion can be counteracted with the help of explanations. For this purpose, explicit mental models can be created that predict the deviation of the expected behaviour from the actual behaviour. The figure below illustrates this process. If there is a deviation, explanations are provided and thus the confusion is prevented.

• Deters, H., Droste, J., & Schneider, K. (2023, June). A means to what end? evaluating the explainability of software systems using goal-oriented heuristics. In Proceedings of the 27th International Conference on Evaluation and Assessment in Software Engineering (pp. 329-338). https://doi.org/10.1145/3593434.3593444
• Droste, J., Deters, H., Obaidi, M., & Schneider, K. (2024). Explanations in Everyday Software Systems: Towards a Taxonomy for Explainability Needs. arXiv preprint arXiv:2404.16644. https://doi.org/10.48550/arXiv.2404.16644
• Deters, H., Droste, J., Obaidi, M., & Schneider, K. (2024, March). How Explainable Is Your System? Towards a Quality Model for Explainability. In International Working Conference on Requirements Engineering: Foundation for Software Quality (pp. 3-19). https://doi.org/10.1007/978-3-031-57327-9_1
• Droste, J., Fuchs, R., Deters, H., Klünder, J., Schneider, K. (2024). Explainability Requirements for Time Series Forecasts: A Study in the Energy Domain.
• Chazette, L., & Schneider, K. (2020). Explainability as a non-functional requirement: challenges and recommendations. Requirements Engineering, 25(4), 493-514. https://doi.org/10.1007/s00766-020-00333-1