Abstract

From its birth to current development, artificial intelligence (AI) has had a long history: from basic problem-solving algorithms to modern chatbots, it is a multi-disciplinary model that will continue to evolve. However, its consistent involvement in important industries and our daily-life appliances leads us to many important questions, including: “how does AI make its decisions?”. Recently, this has evolved into individuals questioning the approach of AI and its criteria for formulating decisions, particularly in industries where a substantial amount of decision-making is involved, creating doubts about the credibility and explainability of AI when it comes to handling major decisions in a sector. For this purpose, the more advanced variant of AI, explainable AI (XAI), is now being deployed. From chatbots to machine learning (ML) models, this new technology seeks to eradicate and answer the doubts created by previous “opaque” models of AI, which oftentimes did not state the reasoning behind a particular decision. This paper seeks to explore the history of AI and XAI, including XAI’s recent role in a multitude of disciplines, such as healthcare, finance and law. The paper seeks to explore both the advantages and disadvantages of deploying XAI in the concerned fields, and to also visualise its use in the future, where it may be developed as a significantly more advanced version of what it is today.

1. Introduction

1.1 Artificial Intelligence

Artificial intelligence (AI) has inadvertently, over the years, accumulated multiple definitions. The term “artificial intelligence” was invented back in 1955 by American computer scientist John McCarthy (Wilks, 2019). However, there has not been a fixed (or standard) definition of AI, despite multiple attempts at describing the function of AI, particularly by various state and international institutions and by a number of case studies (European Union, 2020). This may be due to a lack of a fixed defining statement, as AI has a multi-disciplinary nature because of its demand in a variety of industrial sectors. A number of dictionaries, such as the Merriam-Webster’s Dictionary, describe AI as: “a program or set of programs developed using tools (such as machine learning and neural networks) and used to generate content, analyse complex patterns (as in speech of digital images) or automate complex tasks” (Merriam-Webster, 2026). However, for the sake of this research paper, we will assume (as operational) the European Union-established High Level Expert Group’s (HLEG) definition of AI, which states: “artificial intelligence (AI) systems are software (and possibly also hardware) systems designed by humans that, given a complex goal, act in the physical or digital dimension by perceiving their environment through data acquisition, interpreting the collected structured or unstructured data, reasoning on the knowledge, or processing the information, derived from this data and deciding the best action(s) to take to achieve the given goal. AI systems can either use symbolic rules or learn a numeric model, and they can also adapt their behaviour by analysing how the environment is affected by their previous actions” (HLEG, 2019, cited in European Commission, 2020). This definition states the major key functions of AI, and is regarded by the European Union’s AI Watch to be a basis for an operational definition to be constructed (ibid.).

As it is stated in the selected operational definition, AI can adapt to changing behaviour in accordance with the change in environment; this function of AI uses a process of fetching a collection of data and using suitable algorithms to train itself, otherwise known as machine learning (ML). ML is the branch of computer science which characterises a collection of widely used methods to reach a suitable solution to a practical problem, aided by specialised computer systems, without being specifically programmed to do so; instead, computer systems (that are pre-programmed and fetched with relevant code) and methods are used to identify a pattern, which is broken down into solvable classifications (Koza et al., 1996, cited in Kühl et al., 2022). Essentially, constructed ML models can construct mathematical models without manually-written code involved in the process (Jiao et al., 2020). This mechanism proves to be categorical, currently for the globe, as much of the automated systems that we can identify mostly utilise ML to adapt to the changing needs of humans. Likewise, deep learning (DL) is a sub-part of ML which makes use of neural networks to construct a solution (Sharifani & Amini, 2023). To put it as a linear perspective, neural networks are a collection of tiny-categorised nodes that exhibit behaviour similar to that of neurons in the human brain (Gurney, 1997). This is also supported by Kühl et al. (2022), who illustrate that ML models can imitate human cognitive behaviour. However, it should be made clear that this does not suggest that neural networks “think” in the same way as a human-brain neuron, as for the construction of ML algorithms, there is rigorous coding and training involved, rather than a straightforward process.

As determined above, AI retrieves, processes and thinks upon data to reach a suitable conclusion to a problem, while ML and DL are the concepts which are fundamentally the processes behind those actions.

Figure 1: Comparison of AI performance with human performance from 2000 to 2020 onwards (Kiela et al., 2023, cited in Roser, 2026).

Figure 1 demonstrates ML benchmarks in contrast to human processing capabilities between recent years and more than a decade ago (Kiera et. al., 2023, cited in Roser, 2026). As Roser (2026) evaluates, over a decade ago, there was no reliable algorithm or system that could precisely provide accurate natural language and image recognition equivalent to that of a human. However, as figure 1 illustrates, AI has now become consistently accurate, to the point that it has surpassed humans in terms of processing power and speed (Roser, 2026). This suggests improving trends in the training and deployment of AI, which may be the consequence of the introduction of adaptive ML models, and their deployment in imagery and videography-related tasks over the years. This information showcases the adaptability and development in the field of AI enhancement, to the point where it now rivals human capabilities in some of the regular work fulfilled day-to-day by humans. However, this does not mean that AI has surpassed humans completely. As the above figure illustrates, there have been only a handful of AI models that have categorically surpassed humans in the benchmark, which was only the result of years of training. The majority of the models still could not meet up with the standards of the baseline human performance, which indicates impending development and space for improvement.

However, with these rapid developments, it is crucial to understand how AI has evolved from something as plain simple to almost matching human performance in a number of day-to-day objectives. For this, we need to carefully study decisions and processes involved in such AI models. However, many AI models, instead of explaining a process, only process its inputs and outputs. To deal with this issue, an explainable version of AI, which details how and why a specific decision was taken (or the reasoning behind why and how an output was given), also known as explainable AI (XAI) has been developed, which is further discussed in the following sections.

1.2 Explainable Artificial Intelligence

The rapid development of modern artificial intelligence such as deep learning has enabled machine learning models to achieve a high level of predictive performance across a range of complex tasks. However, as these models have become increasingly sophisticated and more prevalent in sectors such as healthcare and finance, understanding how they generate individual predictions has become considerably difficult (Dwivedi et al., 2023; Phillips et al., 2021). As a result, there has been a shift in research towards developing AI systems that retain the high accuracy rates and are also capable of providing understandable explanations for their decisions. This has led to the emergence of explainable artificial intelligence. 

The National Institute of Standards and Technology (NIST) describes XAI as AI systems that provide evidence, support or reasoning for their outputs while ensuring that these explanations are meaningful and appropriately reflect the system’s behaviour (Phillips et al., 2021). Likewise, Dwivedi et al. (2023) define XAI as a collection of machine learning techniques that allows a user to understand AI models, develop appropriate trust in their predictions and improve the interpretability of the model. Therefore, XAI can be summarily defined as a collection of approaches that make AI decision-making more understandable to human users without changing the model’s intended task 

The increasing demand for XAI is linked to the large-scale adoption of AI systems in high-stakes sectors where automated decisions can have significant consequences for both individuals and organisations (Arrieta et al., 2020). Although many advanced AI systems achieve excellent predictive performance, their internal reasoning is difficult to interpret because of the complexity of their algorithms (Dwivedi et al., 2023; Ortigossa et al., 2024). These systems are commonly known as “black-box” models because; although their inputs and outputs can be observed, the reasoning behind their predictions is not easily understood (Dwivedi et al., 2023). Modern deep learning models contain millions or even billions of parameters that are organised across multiple hidden layers that learn very complex internal representations of data (Samek et al., 2021). While this enables them to achieve incredible predictive accuracy, it also makes it difficult for humans to trace how individual features contribute to a particular prediction (Arrieta et al., 2020; Rudin, 2019). XAI aims to address this limitation by improving the transparency and interpretability of AI systems. This can be achieved either through inherently interpretable models or through post-hoc explanation techniques that identify factors contributing to a model’s predictions (Dwivedi et al., 2023). The specific techniques used to achieve explainability are discussed in section 2 of this paper.

It can therefore be argued that explainability has become an important consideration in AI development to provide understandable justifications for their outputs. However, in some cases, increasing model interpretability may require the use of simpler models, which can reduce predictive performance compared with modern deep learning models (Molnar, 2022). Also, explanations that are technically accurate may still be difficult for non-expert users to interpret, and researchers continue to debate which characteristics make an explanation trustworthy and useful for different users (Miller, 2019). This is why Phillips et al. (2021) highlight that effective explanations should accurately represent a model’s behaviour, provide meaningful information to users and communicate limitations where applicable. 

Consequently, XAI represents a shift in the area of AI, beyond achieving high predictive performance and towards considering whether AI decisions can be understood, examined and trusted by humans (Arrieta et al., 2020; Phillips et al., 2021). As AI systems are becoming increasingly used in high-stakes applications such as healthcare, finance and autonomous vehicles, the ability to provide reliable and interpretable explanations is becoming a necessity for XAI (Arrieta et al., 2020).

1.3 Advantages and Key Features of XAI

XAI has become an increasingly important field of AI because it addresses one of the major limitations of modern machine learning, which is the lack of transparency in complex models. Many high-performing algorithms achieve remarkable predictive accuracy, but often operate as black boxes, which means that users cannot easily understand the process and the rationale behind a particular decision or prediction it makes (Kalasampath, 2025). This lack of reasoning creates challenges in  applications where decisions have significant consequences, such as healthcare, finance and criminal justice (Kalasampath, 2025). 

On the other hand, XAI seeks to overcome this limitation by providing explanations that make the behaviour of AI systems understandable to humans while maintaining high predictive performance. One of the key features of XAI is transparency, which enables users to identify the key factors which contribute to a model’s prediction (Adadi, 2018). Depending on the complexity of the model, transparency can be achieved through inherently interpretable models, such as decision trees and linear regression, of which decision-making processes can be understood directly by humans (Doshi & Kim, 2017). Another important feature is the distinction between local and global explanations: local explanations focus on why a model produced a specific prediction for an individual case, whereas global explanations describe the overall behaviour of the model across an entire dataset. Ribeiro Singh and Guestrin introduced the Local Interpretable Model-agnostic Explanations (LIME) method, which approximates the behaviour of a complex model around an individual prediction using a simpler interpretable model, thereby allowing users to understand the reasoning behind specific decisions. Similarly, Lundberg and Lee (Kalasampath, 2025) developed the SHapley Additive exPlanations (SHAP) approach, which applies concepts from cooperative game theory to assign each feature a contribution value, providing consistent and theoretically grounded explanations for machine learning predictions. (Kalasampath, 2025).

Beyond these technical features, XAI offers several important advantages. Firstly, it improves trust by enabling users to determine whether an AI system has based its decision on reasonable evidence rather than accepting the predictions blindly. Secondly, XAI assists developers in identifying biases and errors within machine learning models, which allows these problems to be corrected. Thirdly, XAI promotes fairness and accountability, particularly in high-stakes domains. By revealing the main factors which affect the decision, XAI ensures organisations investigate potential discrimination and produce more justified results which affect other individuals (Arrieta, 2020). Furthermore, XAI supports collaboration between humans and AI by allowing professionals to combine algorithmic predictions with their own expertise, instead of solely relying on automated results. Overall, these advantages demonstrate that explainability has become a fundamental component of responsible AI, which ensures that AI systems are transparent, trustworthy and fair across a wide range of applications.

1.4 Challenges in Explainable AI

Multiple sources explore the challenges within XAI (Mienye et al., 2024; Rudin, 2019). These challenges largely revolve around the quality of the explanations given (Le et al., 2023), the technical problems in developing interpretability without losing accuracy (Marcinkevičs & Vogt, 2023) and finally the legislation around this developing technology (Brkan & Bonnet, 2020).

A current challenge in XAI is the difficulty in evaluating explainability quality, meaning developers struggle to identify a low-quality or high-quality explanation (Le et al., 2023). The cause of this challenge can be assigned to a few key components. First, there is a lack of standardised benchmarks by which to evaluate explanation quality due to different toolkits utilising individual evaluation metrics, so results are not comparable (ibid). These evaluation metrics may include fidelity, interpretability, completeness and stability (Mienye et al., 2024). Secondly, there are very few uses of ground-truth explanations, meaning that it is difficult to assess the accuracy and fidelity of the explanations (Micró-Nicolau et al., 2024). Finally, black-box AI inevitably leads to a struggle in identifying if issues come from the model or the explanation (ibid). The ability to evaluate the explanations provided by XAI may be vital to the progress of this technology.

Micró-Nicolau et al. (2024) argue the significance of fidelity to XAI’s trustworthiness because explanations should accurately portray the underlying process by which the model reached the result. The continued practice of using XAI with low fidelity may pose a challenge in the growth of the technology as lower levels of trust may develop. Therefore, fidelity as an evaluation metric (Mienye et al., 2024) should be used to identify the most trustworthy XAI.

In the field of XAI, there is a common belief in a consistent trade-off between accuracy and interpretability with an increase in interpretability costing accuracy (Marcinkevičs & Vogt, 2023). The loss of accuracy would be damaging in the context of XAI’s real-world applications, such as its use in remote diagnosis (Mienye et al., 2024), as incorrect and potentially dangerous results could be given. However, these assumptions are challenged by Rudin (2019), who argues that there is no notable difference in accuracy between complex, black-box models such as deep neural networks and simpler, interpretable models such as decision lists. This could mean that by carefully balancing explainability with accuracy, this challenge can be avoided.

There are numerous legal complications regarding XAI (Brkan & Bonnet, 2020); for example, the EU has legal requirements for explanations to be provided for AI’s decisions under specific, significant circumstances (ibid). This is complicated due to the fact that these may originate from black-box models which fundamentally means that results cannot be verified and understood with absolute accuracy (Rudin, 2019). The law regulating XAI is generally vague and does not directly align with the realities of the technology (Brkan & Bonnet, 2020).

To summarise, the challenges of XAI can be both technical and user-based. For the future development of XAI, there are a few recommendations that can be given in order to face the challenges mentioned previously. Firstly, standardised benchmarks should be provided to help assess explanations’ quality, allowing progress within the technology to be provided. Secondly, the fidelity of explanations should be prioritised. Thirdly, interpretability and accuracy should each be valued, while measures must be taken to stop the loss of either. Finally, the reality of the technology should be considered when laws are written regarding XAI.

2. How Does XAI Work?

XAI uses methods that show the relationship between an AI model’s inputs, internal processes and outputs. Within the machine learning lifecycle, Dwivedi et al. (2023) identify two additional phases that support explainability: the understanding phase and the explaining phase. The understanding phase happens before deployment of the model and involves activities such as model training, validation and quality assurance whereas the explaining phase occurs after development by allowing users to interpret predictions made by AI systems (Dwivedi et al., 2023). This distinction is important because explainability has different purposes throughout the AI lifecycle. During development, the understanding phase allows developers to evaluate model behaviour and identify errors to improve model reliability. After deployment, the explaining phase supports the end-user, domain experts and regulators by allowing them to assess AI-made decisions (Phillips et al., 2021; Arrieta et al., 2020). This shows that explainability is a process that should be considered throughout the process of designing, evaluating and using AI systems, not just an additional feature that is applied after the model is created. The importance of these phases can be seen in figure 2 which illustrates the range of applications it supports. Although explainability can be considered throughout the AI lifecycle, the methods needed to achieve it depend on the characteristics of the underlying AI model. This leads to a distinction between white-box and black-box models.

Figure 2: Use cases of XAI during the understanding and explaining phases (Dwivedi et al., 2023).

White-box models are naturally interpretable because their internal decision-making processes can be directly seen by the users. Some examples include linear models, decision trees and rule-based models, where the relationship between input features and outputs is quite transparent (Arrieta et al., 2020; Guidotti et al., 2019). However, although these models provide greater interpretability, they may struggle to represent highly complex relationships within large datasets (Arrieta et al., 2020). In contrast, black-box models, including deep neural networks and complex ensemble methods, can identify complicated patterns within data but contain internal structures that are difficult for humans to understand (Arrieta et al., 2020; Ortigossa et al., 2024). So, XAI frequently focuses on developing methods that explain black-box models rather than replacing them, which allows the continued use of complex high-performing models while improving their interpretability and transparency (Arrieta et al., 2020; Guidotti et al., 2019). This distinction between transparent and opaque models naturally leads to different explainability strategies, particularly intrinsic and post-hoc approaches.

XAI methods can also be classified according to when explanations are generated and the level of information they provide. Intrinsic explainability refers to models that are designed to be interpretable from the beginning, meaning that their decision-making process can be directly understood without needing additional explanation methods. Examples include decision trees, linear models and rule-based systems, where users can see how input variables contributed towards the final decision (Arrieta et al., 2020; Dwivedi et al., 2023). However, these models are less effective when representing highly complex relationships compared with more advanced black-box models (Dwivedi et al., 2023). In contrast, post-hoc explainability generates explanations after a model has already been trained and is useful for interpreting complex black-box models such as deep neural networks (Arrieta et al., 2020; Guidotti et al., 2019). 

Beyond the timing of the explanation generation, XAI methods can also be classified according to the scope of the information that they provide. This creates another distinction between global and local explanations. Global explanations describe the overall behaviour of a model by identifying general patterns and influential features across a dataset, meanwhile local explanations focus on explaining the reasoning behind one specific and individual prediction (Dwivedi et al., 2023). This classification is important because understanding the general behaviour of a model needs a different type of explanation compared to investigating a specific decision. However, global explanations may overlook important differences between individual predictions, whereas local explanations provide detailed information about specific decisions but may not accurately represent the overall behaviour of the model (Molnar, 2022; Guidotti et al., 2019). 

Because post-hoc explanations are particularly important for interpreting complex black-box models, multiple post-hoc techniques have been developed to generate explanations for machine learning models such as the widely used LIME approach, which explains individual predictions by creating a simpler interpretable model around a specific prediction. LIME does this by modifying input data and analysing how these changes influence the original model’s output (Ribeiro et al., 2016). An important advantage of LIME is that it is model-agnostic which means it can generate explanations without requiring knowledge of the internal structure of the original model (Ribeiro et al., 2016). However, LIME has been criticised because its explanations depend on the randomly generated samples used to approximate the model locally. As a result, small changes in the selected perturbations can produce different explanations, reducing reliability (Molnar, 2022).

Another widely adopted post-hoc technique is SHAP, which applies concepts from cooperative game theory to assign each feature a contribution value, showing how strongly individual features influence a model’s prediction (Lundberg & Lee, 2017). For instance, SHAP can indicate whether a particular feature increases or decreases the likelihood of a certain outcome. Compared with LIME, SHAP has a stronger theoretical foundation; however, calculating the exact Shapley values can be expensive for models with large numbers of features, limiting its use in some large-scale applications (Lundberg & Lee, 2017; Molnar, 2022).

However, no single XAI technique is universally optimal because each approach involves trade-offs between interpretability, computational cost and the quality of explanation (Dwivedi et al., 2023; Tahir et al., 2024). Therefore, selecting an appropriate technique depends on the requirements of the application domain. For example, healthcare applications may prioritise explanations that accurately represent clinical reasoning and decision-making, while financial systems may require transparency for regulatory compliance, whereas autonomous vehicles may instead require rapid explanations that support real-time safety decisions (Arrieta et al., 2020; Phillips et al., 2021). This displays the importance of choosing appropriate XAI techniques.

3. Applications of Explainable AI

3.1 Applications of Explainable AI in Education

XAI is becoming increasingly important in education as AI is integrated into learning platforms such as intelligent tutoring systems and educational analytics (Khosravi et al., 2022). Unlike traditional black-box AI models, XAI provides transparent and interpretable explanations for its predictions and recommendations, which enables teachers, students and parents to understand why an AI system has reached a particular decision (Khosravi et al., 2022). In educational settings, this transparency is essential because AI systems often influence high-stake decisions, such as identifying students who are at risk, recommending personalised learning materials and evaluating student performance (Khosravi et al., 2022). Consequently, XAI improves trust, accountability and the effective collaboration between educators and AI systems (Khosravi et al., 2022).

One major application of XAI in education is personalised learning through intelligent tutoring systems (ITSs). Modern ITSs adapt educational content according to a student’s knowledge level and previous performance. However if students do not understand why specific exercises or recommendations are provided, they may lose confidence in the system. XAI addresses this issue by presenting explanations with recommendations, which helps learners to understand which skills require improvement and why particular learning activities have been selected. Conaiti et al. (2021) demonstrate that personalised explanations can significantly improve learners’ understanding of AI recommendations while increasing engagement and trust. Similarly, Karpouzis (2024) describe the iRead intelligent tutoring system, where explainable recommendations allow both teachers and students to understand how adaptive learning paths are generated. This transparency enables educators to intervene more effectively and support students in becoming more active participants in their own learning process.

3.2 Applications of Explainable AI in Finance

One important use of XAI is in the field of finance, where it is being increasingly used by financial institutions to automate complicated decision-making processes such as credit assessment, fraud detections and risk analysis (Zhang, 2025). However, because these decisions have very high legal and financial consequences, they need to be both accurate and understandable. Therefore, XAI is being used in financial systems by enabling shareholders to interpret and justify AI-generated decisions rather than treating them as opaque doubts (Weber et al., 2024).

One of the most important applications of XAI is credit scoring and loan approval. Banks increasingly rely on machine learning models such as XGBoost and random forests (RFs) to predict the probability that a borrower will default (Kalasampath et al., 2025; Weber et al., 2024). Although these models generally outperform traditional statistical methods, financial regulations such as the General Data Protection Regulation (GDPR) require institutions to provide meaningful explanations for automated decisions. In particular, Articles 13, 14 and 15 require data subjects to receive meaningful information about the logic decisions based solely on automated processing that produce legal or similarly significant effects (European Union, 2016). XAI enables lenders to identify the factors influencing a customer’s credit decision, such as income, debt ratio, repayment history or credit utilisation. This transparency not only helps customers to understand why a loan was accepted or rejected, but also assists financial institutions in identifying potential biases and ensuring fair lending practices (Kalasampath, 2025).

Another important application of XAI is in fraud detection as this is where AI systems analyse large volumes of financial transactions to detect suspicious behavioural patterns that could indicate fraudulent activity. Deep learning models have improved detection accuracy, however this still does not provide an explanation as to why a certain transaction was flagged as fraudulent. Instead, XAI can be used here to identify the transaction characteristics that contributed most strongly to a fraud prediction, which will allow financial investigators to distinguish genuine fraudulent behaviour from false positives (Dwivedi et al., 2023; Kalasampath et al., 2025). Weber et al. (2024) additionally observe that fraud detection is still the principal application of XAI within finance, which shows that XAI is highly important in increasing the accuracy and efficiency within financial investigations.

There is another emerging application of XAI in finance, which is dynamic pricing, where AI models are continuously adjusting the prices of products and services based on factors such as market conditions, customer behaviour and risk profiles. These models can optimise revenue and improve operational efficiency, however their pricing decisions are often difficult for both regulators and customers to interpret (Weber et al., 2024). XAI responds to this challenge by identifying the factors that contributed the most significantly to a pricing decision, allowing businesses to justify why prices changed and verify that pricing strategies are consistent and free from unintended bias (Černevičienė & Audrius, 2024). Examples include insurance premiums, airline ticket pricing and personalised online retail pricing, where AI systems continuously adjust prices according to customer behaviour, market demand and estimated risk (Černevičienė & Audrius, 2024). Without explainability, customers may perceive these pricing decisions as unfair or discriminatory. XAI helps organisations demonstrate that pricing decisions are based on objective financial indicators rather than unintended bias, improving transparency and supporting regulatory oversight (Weber et al., 2024). This may benefit businesses by increasing transparency which will strengthen customer trust and confidence by showing that pricing decisions are supported by objective financial indicators instead of an arbitrary algorithm. Additionally, it will enable institutions to understand an AI’s reasoning and therefore decide whether or not it is an appropriate and logical decision. Despite these benefits, implementing XAI within financial services may provide explanations that are difficult for customers without technical knowledge to understand, while organisations must balance transparency with protecting commercially sensitive algorithms and proprietary business models (Molnar, 2022; Weber et al., 2024).

Overall, XAI is becoming an increasingly important part of modern financial services by improving the transparency and fairness in AI-driven decision making. Explainability supports credit assessment, fraud detection and dynamic pricing, enabling financial institutions to justify automated decisions while maintaining regulatory compliance and increasing stakeholder trust. As AI continues to become more widely adopted across the financial sector, selecting appropriate explainability techniques will remain essential for balancing predictive performance with transparency and fairness.

3.3 Applications of Explainable AI in Criminal Justice

According to the National Institute of Justice (NIJ), the major segments in which AI can play its role in criminal justice can be distributed among four segments: video and image recognition; DNA identification; detection of gunshots; and the prediction of future criminal (suspected) activity (Rigano, 2020). The use of AI in criminal justice is perhaps both questioned and supported. However, it is important to discuss the impacts XAI could have on criminal justice, as court or institutional decisions oftentimes leave a colossal impact on an individual or organisation, making transparency, explainability and fairness vital.

Image and video recognition is an essential tool for the judicial system as it aids relevant authorities in visually identifying individuals, places and objects in order to support criminal investigations. However, visual identification, in most cases, necessitates the use of a manual workforce, which costs a colossal amount of capital for the deployment of personnel (ibid.). It is also pertinent to note that the analysis of photos and videos can be directly affected by human error, owing to the substantial amount of data that may be involved in only one set of an instance, let alone multiple incidents or criminal activities. XAI enables a pathway to resolve these problems; while conventional software approaches are limited to functions such as identification of eye colour and body-part distance, modern AI models use ML to learn repetitive patterns, while also adapting to changing parameters by establishing independent rules and procedures (Rigano, 2020). This approach enables swift identification of illegally possessed items, such as guns or other controlled-objects, while requiring minimal time and effort.

DNA analysis is no different, with the exception that it involves biological samples such as saliva or semen. This segment is concerning the sensitivity of biological samples in criminal investigations, as there may be a significant delay in the submission of evidence and the actual criminal event taking place (Rigano, 2020). This creates difficulty when manually reviewing the samples as an incorrect identification may be done in error. Unlike humans, XAI, or AI in general, can eradicate this defect by recognising repeated patterns at a micro-level, especially considering the advent of neural networks (ibid.). Furthermore, according to Benfatto et al., who assessed the use of XAI in relation to medical diagnosis, XAI has enabled the ability to classify various tumors based on their DNA methylation profiles using AI, which showcases a positive step towards AI and DNA research (Benfatto et al., 2025). Benfatto et al. use RF models in their framework, which is essentially a collection of binary trees, which are multiple different simple ML algorithms that exhibit a behaviour of decision making similar to that of humans. This illustrates the rapid development in the field of DNA analysis using XAI, showing that the tools we have currently are also capable of swiftly tracing down most DNA linked to a crime, something which can ultimately lead to less criminal activity, as perpetrators are likely to be caught quickly and brought to justice.

COMPAS is an AI model currently used by the US justice system in multiple states to inform judges pre-trial of the defendant’s likelihood of recidivism (Engels et al., 2025). This model has been criticised for its bias against both young defendants and Black defendants, especially because it lacks the transparency to inform the judge of when these biases are being perpetuated (ibid). Rudin (2019) also criticises the use of non-transparent and non-accountable AI in vital situations such as criminal justice. Therefore, it could be debated that XAI should be used to identify bias and assess accuracy of predictions when models like COMPAS are used for finding the chance of recidivism. However, Rudin (2019) drives this argument even further and challenges the use of black-box AI in criminal justice entirely and advises the use of “inherently interpretable” models instead.

XAI can also be used to forecast crime, to find when and where crime will take place through various techniques utilising deep learning, statistics and mapping hotspots (Ersöz et al., 2025). Forecasting crime is crucial to the efficient use of police resources and the development of proactive crime prevention (ibid). Khalfa et al. (2025) suggest that crime-forecasting XAI must be further developed as providing trustworthy explanations may “bridge the gap between predictive analytics and actionable crime prevention strategies”. Improving explainability in crime forecasting may also aid in avoiding the bias mentioned by Ersöz et al. (2025) which is often caused by the training data.

4. Ethical Considerations in XAI

With the use of XAI in a number of fields, as comprehensively discussed hereinabove, it is important to consider a number of factors related to the safety, security and transparency of the use of such technology, all of which come under the ethical considerations of using XAI. Notably, the major ethical considerations consist of privacy and data protection, human trust and human-AI collaboration, and fairness and bias.

Within XAI, privacy is both an obligation and a challenge, as increasing the transparency of models risks the privacy of training data (Nguyen et al., 2025). For example, Nguyen et al. (2025) explore how case-based models can explain data points used to reach the result, which could lead to revealing sensitive data to the user. Ethically, developers should seek to prioritise data protection, especially as XAI is consistently utilised in crucial fields such as healthcare and criminal justice (Rudin, 2019). This prioritisation must exist on both a technical and legal basis for ethical principles of privacy to be reinforced (Brkan & Bonnet, 2020).

Rong et al. (2024) portray both trust and transparency as essential to the ethical deployment of AI. This may be because these metrics allow users to make decisions based on a full understanding of the model’s results, allowing for a higher level of autonomy. Trust in XAI can be achieved through the fidelity of explanations (Micró-Nicolau et al., 2024) as users require explanations that accurately represent the inner workings of the model to build trust in XAI. Effective human-AI collaboration requires a foundation of humans understanding the AI’s decisions through meaningful explanations so that users can trust the decisions to an appropriate extent to better utilise the results (Bansal et al., 2021).

Moreover, despite the advancements in the development of XAI, it is still limited in terms of ensuring complete, or at least nearly complete, fairness (Papaniku, 2025). As Deck et al. (2024) state, many claims related to XAI are ambiguous and facile. For example, claims like “XAI can help achieve fairness” treat fairness as a uniform object rather than explaining how or what aspect XAI establishes fairness in (ibid). Likewise, XAI is sometimes also titled as “sufficient”, implying that it ensures complete accountability and that the mechanisms behind it ensure full fairness (Deck et al., 2024). On the contrary, it is pertinent to note that “fairness” is not limited to one object, topic or discipline; rather it covers a multitude of issues, which negates the “complete fairness” claim (ibid). Additionally, even an intelligible model, which can be scrutinised by any widely known scrutinising method, may still be considered biased by some (ibid). This comes to show that fairness and bias are not two individual “things”, rather they are complete multi-dimensional mechanisms that have to be considered when scrutinising a compartment of XAI. It also shows that fairness and bias have some criteria, which may defer stakeholder to stakeholder; while one may consider the aspects to be “fair”, the other may disagree. However, this does not imply that XAI cannot ensure fairness. It depends on what type of fairness an individual or organisation is looking for and its criteria for performing a particular action via the use of XAI. 

On the contrary, with the increase in demand for transparency from multiple organisations and stakeholders, XAI has become increasingly relevant, which is also a consequence of consistent deployment of black-box ML models, as concerns have been raised about decisions being taken that are unjustifiable, incorrect or, oftentimes, having no context or stepwise refinement being presented as to why that decision was taken by an AI model (Arrieta et al., 2019). Multiple researchers have agreed that trustworthiness is the primary aim of XAI models, however, it must be understood that every “trusted” model is not explainable (ibid.). This implies that even though trust is a major factor when factoring out fairness and bias in a model, it does not mean that every model would be “explainable”. Arrieta et al. (2019) argue that explainability can be considered as a major factor that establishes a pathway to ensure fairness in ML models, further iterating that XAI has the connected objective of identifying potential bias that a data model was exposed to. This indicates that, even though, as noted above, XAI may not be able to ensure “complete” fairness, it still plays a major role in ensuring that bias is filtered out from any type of model, particularly those which are used in regular decision making.

5. Conclusion

In this paper, we have described the current field of XAI as well as assessing its future potential. The paper aimed to develop an understanding of XAI techniques, weigh the advantages against the challenges of the technology and explore the numerous applications of XAI. The examined applications were focused within education, finance and criminal justice.

This paper found that XAI are AI systems which provide evidence, support and reasoning for the given outputs which accurately reflect the model’s process in the form of understandable explanations (Phillips et al., 2021). Explaining the process of AI can be advantageous to users as it allows them to understand how and why a model reaches a particular decision, which increases the reliability and transparency. It also enables users to evaluate whether the prediction is reasonable. In this way, users can identify potential biases and errors and support greater fairness and accountability in AI systems, particularly in high-stake application fields (Kalasampath et al., 2025). The applications examined within these fields revealed just how crucial explainability is.

However, XAI faces numerous challenges. Firstly, there is a lack of standard benchmarks by which to judge explanation quality, which leads to difficulty in acknowledging and continuing progress in XAI (Le et al., 2023). Secondly, many believe in a trade-off between interpretability and accuracy, meaning that there is a challenge in developing interpretability without losing the accuracy of the model (Marcinkevičs & Vogt, 2023). Finally, laws are being developed that do not align with the realities of the technology, complicating the social and technical landscape surrounding XAI (Brkan & Bonnet, 2020). 

XAI has vast potential to grow in the future, if the challenges of XAI are correctly avoided. Development of XAI would allow for greater transparency and accuracy when AI is used to assist in major decisions, such as its use in medical diagnostics (Mienye et al., 2024). In the future ethics, law and technology must all be considered and refined to promote the future of XAI.

Adadi, A. and Berrada, M. (2018) ‘Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)’, IEEE Access, 6, pp. 52138-52160.

Amini, M. and Sharifani, K. (2023) ‘Machine Learning and Deep Learning: A Review of Methods and Applications’, World Information Technology and Engineering Journal, 10(7), pp. 3897-3904.

Arrieta, A.B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., Chatila, R. and Herrera, F. (2020) ‘Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI’, Information Fusion, 58, pp. 82-115.

Bansal, G., Wu, T., Zhou, J., Fok, R. and Weld, D.S. (2021) ‘Does the Whole Exceed Its Parts? The Effect of AI Explanations on Complementary Team Performance’, Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 81, pp. 1-16.

Benfatto, S., Sill, M., Jones, D.T.W., Pfister, S.M., Sahm, F., von Deimling, A., Capper, D. and Hovestadt, V. (2025) ‘Explainable artificial intelligence of DNA methylation-based brain tumor diagnostics’, Nat Commun, 16(1787).

Brkan, M. and Bonnet, G. (2020) ‘Legal and Technical Feasibility of the GDPR’s Quest for Explanation of Algorithmic Decisions: Of Black Boxes, White Boxes and Fata Morganas’, European Journal of Risk Regulation, 11(2), pp. 270-295.

Černevičienė, J. and Audrius, K. (2024) ‘Explainable artificial intelligence (XAI) in finance: A systematic literature review’, Artificial Intelligence Review, 57, pp. 1-45.

Conati, C., Barral, O., Putnam, V. and Rieger, L. (2021) ‘Toward personalized explainable AI for intelligent tutoring systems’, Artificial Intelligence, 298, 103503.

Deck, L., Schoeffer, J., Arteaga, M.D. and Kühl, N.A. (2024) ‘Critical Survey on Fairness Benefits of Explainable AI’, Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency (Association for Computing Machinery: New York, United States).

Doshi-Velez, F. and Kim, B. (2017) ‘Towards a Rigorous Science of Interpretable Machine Learning’, arXiv, 1702.08608.

Dwivedi, R., Dave, D., Naik, H., Singhal, S., Rana, O., Patel, P., Qian, B., Wen, Z., Shah, T., Morgan, G. and Ranjan, R. (2023) ‘Explainable AI (XAI): Core Ideas, Techniques, and Solutions’, ACM Computing Surveys, 55(9), pp. 1-33.

Engels, J., Linhardt, L. and Schubert, C. (2025) ‘Code is Law: How COMPAS Affects the Way the Judiciary Handles the Risk of Recidivism’, Artificial Intelligence and Law, 33, pp. 383-404.

Ersöz, F., Ersöz, T., Marcelloni, F. and Ruffini, F. (2025) ‘Artificial Intelligence in Crime Prediction: A Survey With a Focus on Explainability’, IEEE Access, 13, pp. 59646-59674.

European Union (2016) Regulation (EU) 2016/679 of the European Parliament and of the Council (General Data Protection Regulation). <https://eur-lex.europa.eu/eli/reg/2016/679/oj>

Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F. and Pedreschi, D. (2019) ‘A survey of methods for explaining black box models’, ACM Computing Surveys, 51(5), pp. 1-42.

Gurney, K. (1997). ‘Neural networks—an overview’, An Introduction To Neural Networks. (UCL Press: London, United Kingdom), pp. 1-4.

Jiao, Z., Hu, P., Xu, H., and Wang, Q. (2020) ‘Machine Learning and Deep Learning in Chemical Health and Safety: A Systematic Review of Techniques and Applications’, ACS Chem. Health Saf, 27(6), pp. 316-334.

Kalasampath, K., Spoorthi, K.N., Sajeev, S., Kuppa, S.S., Ajay, K. and Angulakshmi, M. (2025) ‘A Literature Review on Applications of Explainable Artificial Intelligence (XAI)’, IEEE Access, 13, pp. 41111-41140.

Karpouzis, K. (2024) Explainable AI for Intelligent Tutoring Systems. In: Frontiers of Artificial Intelligence, Ethics, and Multidisciplinary Applications (Springer: Singapore).

Khalfa, R., Theinert, N. and Hardyns, W. (2025) ‘Comparing XAI Techniques for Interpreting Short-Term Burglary Predictions at Micro-Places’, Computational Urban Science, 5(1).

Khosravi, H., Shum, S.B., Chen, G., Conati, C., Tsai, Y.S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S. and Gašević, D. (2022) ‘Explainable Artificial Intelligence in Education’, Computers and Education: Artificial Intelligence, 3, 100074.

Kiela, D., Thrush, T., Ethayarajh, K. and Singh, A. (2023) ‘Plotting Progress in AI’, Contextual AI [online]. <https://contextual.ai/blog/plotting-progress-in-ai>

Kühl, N., Schemmer, M., Goutier, M. and Satzger, G. (2022) ‘Artificial intelligence and machine learning’, Electronic Markets, 32, pp. 2235-2244.

Le, P.Q., Nauta, M., Nguyen, V.B., Pathak, S., Schlötterer, J. and Seifert, C. (2023) ‘Benchmarking eXplainable AI: A Survey on Available Toolkits and Open Challenges’, Proceedings of the Thirty-Second International Joint Conference on Artificial Intelligence, pp. 6725-6733.

Lundberg, S.M. and Lee, S.-I. (2017) ‘A unified approach to interpreting model predictions’, Advances in Neural Information Processing Systems, 30, pp. 4765-4774.

Marcinkevičs, R. and Vogt, J.E. (2023) ‘Interpretable and Explainable Machine Learning: A Methods-Centric Overview with Concrete Examples’, WIREs Data Mining and Knowledge Discovery, 13(6), e1493.

Merriam-Webster (2026) ‘AI definition & meaning’, Merrian-Webster Dictionary [online]. <https://www.merriam-webster.com/dictionary/ai>

Mienye, I.D., Obaido, G., Jere, N., Mienye, E., Aruleba, K., Emmanuel, I.D. and Ogbuokiri, B. (2024) ‘A survey of explainable artificial intelligence in healthcare: Concepts, applications, and challenges’, Informatics in Medicine Unlocked, 51, 101587.

Miller, T. (2019). ‘Explanation in Artificial Intelligence: Insights from the Social Sciences’, Artificial Intelligence, 267, pp. 1-38.

Miró-Nicolau, M., Jaume-i-Capó, A. and Moyà-Alcover, G. (2024) ‘Assessing Fidelity in XAI Post-hoc Techniques: A Comparative Study with Ground Truth Explanations Datasets’, Artificial Intelligence, 335, 104179.

Molnar, C. (2022) Interpretable Machine Learning (Christopher Molnar: Munich).

Nguyen, T.T., Huynh, T.T., Ren, Z., Nguyen, T.T., Nguyen, P.L., Yin, H. and Nguyen, Q.V.H. (2025) ‘Privacy-preserving Explainable AI: A Survey’, Science China Information Sciences, 68(1), 111101. 

Ortigossa, E.S., Gonçalves, T. and Nonato, L.G. (2024) ‘Explainable Artificial Intelligence (XAI)—From Theory to Methods and Applications’, IEEE Access, 12, pp. 80799-80846.

Papanikou, V., Karidi, D.P., Pitoura, E., Panagiotou, E., and Ntoutsi, E. (2025) ‘Explanations as Bias Detectors: A Critical Study of Local Post-hoc XAI Methods for Fairness Exploration’, arXiv, 2505.00802.

Phillips, P.J., Hahn, C.A., Fontana, P.C., Yates, A.N. and Greene, K.K. (2021) ‘Four Principles of Explainable Artificial Intelligence’, NIST Interagency/Internal Report, 8312, pp. 1-17.

Ribeiro, M.T., Singh, S. and Guestrin, C. (2016). ‘‘Why should I trust you?’: Explaining the predictions of any classifier’, Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135-1144.

Rigano, C. (2019) ‘Using Artificial Intelligence to Address Criminal Justice Needs’, NIJ Journal, 280.

Rong, Y., Leemann, T., Nguyen, T.-T., Fiedler, L., Qian, P., Unhelkar, V., Seidel, T., Kasneci, G. and Kasneci, E. (2024) ‘Towards Human-Centered Explainable AI: A Survey of User Studies for Model Explanations’, IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(4), pp. 2104-2122.

Roser, M. (2022) The brief history of artificial intelligence: the world has changed fast — what might be next?’, Our World in Data [online]. <https://archive.ourworldindata.org/20260727-131016/brief-history-of-ai.html>

Rudin, C. (2019) ‘Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead’, Nature Machine Intelligence, 1(5), pp. 206-215.

Samek, W., Montavon, G., Lapuschkin, S., Anders, C.J. and Müller, K.-R. (2021) ‘Explaining Deep Neural Networks and Beyond: A Review of Methods and Applications’, Proceedings of the IEEE, 109(3), pp. 247-278.

Samoili, S., Lopez Cobo, M., Gomez Gutierrez, E., De Prato, G., Martinez-Plumed, F. and Delipetrev, B. (2020) ‘AI Watch: Defining Artificial Intelligence’, EUR 30117 EN (Publications Office of the European Union: Luxembourg).

Tahir, H.A., Alayed, W., Hassan, W.U. and Haider, A. (2024) ‘A Novel Hybrid XAI Solution for Autonomous Vehicles: Real-Time Interpretability Through LIME–SHAP Integration’, Sensors, 24(21), 6776.

Weber, P., Carl, K.V. and Hinz, O. (2024) ‘Applications of explainable artificial intelligence in finance: A systematic review of finance-specific XAI applications’, Management Review Quarterly, 74, pp. 867-907.

Wilks, Y. (2019) ‘What is Artificial Intelligence?’, Artificial Intelligence: Modern Magic or Dangerous Future? (Icon Books: London), p. 11.

Zhang, L. (2025) ‘Using Explainable Machine Learning to Predict Loan Risk in Consumer Finance’, CSAIDE ’25: Proceedings of the 2025 4th International Conference on Cyber Security, Artificial Intelligence and the Digital Economy, pp. 235-238.