Testing machine learning algorithms for classifying authority in a hybrid institutional complex

Abstract

The growing diversity of institutions that make up Hybrid Institutional Complexes involved in global governance has meant growing masses of raw data. Although these forms of institutions are some of the most important contemporary governance bodies, that have not yet been adequately analysed in the literature. Annual Reports, meeting minutes, policy documents and Codes are constantly being produced and published by these institutions, but this data is not in a form useful for statistical analysis. The use of hand-coding techniques for textual data is extraordinarily time consuming, a problem that is exacerbated in a swiftly changing field where data collection and classification could easily fall behind the ongoing shifts in institutional collaboration. In order to keep up with the increasing complexity of these global governance bodies, research methodology needs to evolve accordingly, and develop new ways of capturing information about these institutions. By harnessing machine learning algorithms and especially deep learning networks for classifying textual-data, social scientists are able to deepen their research, particularly by creating new, usable datasets from the output documents of the institutions they research. This report demonstrates how the output documents released by the institutions in the global private security governance institutional complex can be successfully classified by machine learning algorithms. This research report focuses on developing, and then assessing the effectiveness of an automated text classification approach. It demonstrates how a deep neural network algorithm can classify textual data from the global private security governance complex with up to 90% accuracy compared to expert labelling of the texts. It further compares traditional machine learning models to deep learning models and finds that traditional models like the random forest algorithm can classify these texts with over 85% accuracy.