An empirical comparison of decision trees and decision graphs on supervised learning problems

Abstract

Recentlyinterpretablemachinelearningalgorithmshavegainedinterestduetothetransparency of their models. Some machine learning domains like healthcare and medical diagnosis expect the rationality and interpretability of decision models used for predictions. These domains need clear understanding of the decision rules, as it has a direct impact on human life. Currently, there are a few established algorithms like decision trees and decision graphs which support such explainable decision models. This study explores one of the efficient, but less known decision graph algorithms - Attribute-based Decision Graph (AbDG). In-built flexibility of this algorithm has led toits two variants, based on the edge creationapproach, called asp-Partite and c-Partite. It also supports multiple interval methods to split an attribute. Recommended interval methods intheoriginalalgorithmareMDLPC,ESADandEDADB.Thisstudycomparestheperformanceof all such variants with commonly used decision tree CART (Classification And Regression Tree). In addition, it also investigates a few enhancements to AbDG such as optimization of finding interval range for values which were not seen during the testing phase and using additional KB insinterval method. These empirical analysis results reveal that, decision tree models are more concise in certain scenarios, but are less accurate when compared to AbDG’s highest performing variants. Also, it shows that, there are variants like MDLPC which produce more concise but less accurate models when compared to decision tree. Thus, AbDG and its variants can be further explored as an alternative to decision tree, when more accurate and interpretable models are required