  Machine learning (ML) and Artificial Intelligence (AI) are increasingly used
in energy and engineering systems, but these models must be fair, unbiased, and
explainable. It is critical to have confidence in AI's trustworthiness. ML
techniques have been useful in predicting important parameters and improving
model performance. However, for these AI techniques to be useful for making
decisions, they need to be audited, accounted for, and easy to understand.
Therefore, the use of Explainable AI (XAI) and interpretable machine learning
(IML) is crucial for the accurate prediction of prognostics, such as remaining
useful life (RUL) in a digital twin system to make it intelligent while
ensuring that the AI model is transparent in its decision-making processes and
that the predictions it generates can be understood and trusted by users. By
using AI that is explainable, interpretable, and trustworthy, intelligent
digital twin systems can make more accurate predictions of RUL, leading to
better maintenance and repair planning and, ultimately, improved system
performance. The objective of this paper is to understand the idea of XAI and
IML and justify the important role of ML/AI in the Digital Twin framework and
components, which requires XAI to understand the prediction better. This paper
explains the importance of XAI and IML in both local and global aspects to
ensure the use of trustworthy ML/AI applications for RUL prediction. This paper
used the RUL prediction for the XAI and IML studies and leveraged the
integrated python toolbox for interpretable machine learning (PiML).
