Explainable Software Defect Prediction using Word Embeddings and Deep Learning

Abstract

Defects in the source code discovered during or after deployments require resources and effort to resolve. Researchers have developed techniques to tackle the issue of detecting source code defects at an earlier stage. In this study, we examined several convolutional neural network architectures to build a bug detection classifier for Java source code. We utilized Word2Vec, GloVe, and FastText to train embedding models that assist in embedding Java source code in a suitable format for our trained networks. We also explored a few explainable AI methods to provide visual heatmap explanations of the decisions made by our neural networks. With all our models trained under the same conditions and for up to 300 epochs, we identified two top-performing models: ResNet18, with an average accuracy of 77.22%, average precision of 79.17%, average recall of 76.50%, and average F1 score of 77.39% across all embeddings; and MobileNet, with an average accuracy of 84.59%, average precision of 86.04% average recall of 83.69%, and average F1 score of 84.84%. In this study, we used saliency maps, Grad-CAM, Score-CAM, and FIMF score-CAM to generate and analyze heatmaps of Java source code methods, allowing us to analyze and compare the features that the models focus on when making predictions.