Dynamic causal modelling of MVC-normalised isometric wrist extension and wrist flexion using high-resolution EEG Abdul-Khaaliq Mohamed September 2022 A thesis submitted to the Faculty of Engineering and the Built Environment, University of Witwatersrand, for the degree of Doctor of Philosophy. Declaration i Declaration I declare that this thesis is my own, unaided work, unless otherwise acknowledged. It is submitted for the degree of Doctor of Philosophy at the University of Witwatersrand, Johannesburg. It has not been submitted before for any degree or examination at any other university. Date: Signature: Abdul-Khaaliq Mohamed 13 September 2022 Abstract ii Abstract Motor control information extracted from electroencephalography (EEG) can be used by a brain-computer interface (BCI) to control a bionic hand. This could assist individuals who have lost hand functionality. Interpreting the neural control information associated with wrist extension (WE) and wrist flexion (WF) movements is particularly challenging and currently limited. Dynamic causal modelling (DCM) was thus used in this thesis to elucidate the underlying control mechanisms of WE and WF motor tasks. The neural control of the WE and WF, and the neural mechanism of their differentiation, were hypothesised to rely on linear and non-linear, cross-frequency, causal communication between seven brain regions, using the full EEG frequency spectrum. These regions included: the hand homunculi of the primary motor cortex (M1-H), the ventral premotor cortices (PMv), the prefrontal cortices (PFC) and the supplementary motor cortex. A 128-channel EEG dataset was recorded from 14 participants while they performed repetitions of isometric WE and WF, normalised by their maximum voluntary contractions. The EEG data was fitted to 12 DCM architectures, which were compared using Bayesian model selection. The best performing model suggested that the control of the wrist motor tasks, and their differentiation, involved cross-frequency, bilateral couplings, both linear and nonlinear, involving all seven regions and the full EEG spectrum. Furthermore, DCM revealed that bilateral PMv, bilateral PFC and the high-gamma band played a significant role in the control and differentiation of the wrist motor tasks. The results suggest that future BCI and neurophysiological studies, involving WE and WF, or other hand movements, include a broad range of brain regions and frequencies in their analysis. Acknowledgments iii Acknowledgments All praises are due to Allah, my Creator, Who has enabled me to complete this thesis. I would also like to acknowledge the following people and institutions: 1. my wife, Raheemah Boomberg; my son, Abdul-Haadee Mohamed; my parents; and my siblings for their encouragement, patience and support; 2. my supervisors, Prof. Vered Ahoronson (University of Witwatersrand), Prof. Brian Wigdorotwitz and the late Dr. Lester John (Elemedic Pty Ltd) for their guidance and support; 3. the National Research Foundation and the University of Witwatersrand their financial assistance; 4. Mrs. Cara Swanepoel and Dr. John Cockcroft, from the Central Analytical Facilities at the University of Stellenbosch, for their assistance with recording the necessary EEG data; 5. and Mr. Muhammed Aswat and Mr. Phillip Muzi Mabaso for their assistance in developing the force dynamometer. Contents iv Contents Declaration ................................................................................................................................................................ i Abstract .................................................................................................................................................................... ii Acknowledgments .................................................................................................................................................. iii Contents .................................................................................................................................................................. iv List of figures ........................................................................................................................................................ viii List of tables .......................................................................................................................................................... xii Glossary of terms and abbreviations ..................................................................................................................... xiv Chapter 1 Introduction ................................................................................................................................. 1 1.1 A loss of wrist movement control reduces quality of life ..................................................................... 1 1.2 A neurally-controlled bionic hand as a possible solution ..................................................................... 2 1.3 Importance of WE and WF ................................................................................................................... 2 1.4 Using a sensorimotor brain-computer interface to control a bionic hand ............................................ 3 1.5 Towards improved BCI performance gained from knowledge of the underlying hand motor control mechanisms ......................................................................................................................................................... 6 1.6 Elucidating the motor control of WE and WF ...................................................................................... 7 1.7 Summary ............................................................................................................................................... 8 Chapter 2 Hypothesis, aims and objectives ................................................................................................. 9 Chapter 3 Background information on the study components: EEG, BCIs and DCM ............................. 13 3.1 Introduction ........................................................................................................................................ 13 3.2 EEG for BCIs and movement analysis studies ................................................................................... 13 3.2.1 High-resolution EEG ..................................................................................................................... 16 3.2.2 Neural source imaging and localisation ......................................................................................... 18 3.2.3 EEG frequencies and sensorimotor rhythms ................................................................................. 19 3.3 Brain-computer interfaces .................................................................................................................. 21 Contents v 3.4 Methods for analysing neural connectivity ........................................................................................ 24 3.5 Dynamic causal modelling (DCM) .................................................................................................... 25 3.5.1 DCM for induced responses (DCM-IR) of EEG ........................................................................... 26 3.5.2 Structure specification, fitting and comparison of models ............................................................ 28 3.6 Summary ............................................................................................................................................. 31 Chapter 4 Review of WE and WF motor control connectivity analysis literature .................................... 32 4.1 Introduction ........................................................................................................................................ 32 4.2 Review of FC and EC literature of WE and WF control .................................................................... 32 4.3 Review of DCM studies of hand movements ..................................................................................... 34 4.4 Summary ............................................................................................................................................. 37 Chapter 5 Frequencies, regions of interest and causality in neural connectivity analysis of hand motor control ………………………………………………………………………………………………...38 5.1 Introduction ........................................................................................................................................ 38 5.2 Exploring frequencies used in wrist motor control EEG BCI studies ................................................ 39 5.3 Exploring frequencies involved in hand motor control EEG-based EC and FC studies .................... 42 5.4 Exploring ROI relationships for hand motor control ......................................................................... 43 5.5 Summary ............................................................................................................................................. 49 Chapter 6 Recording high-resolution EEG data ........................................................................................ 50 6.1 Introduction ........................................................................................................................................ 50 6.2 EEG equipment .................................................................................................................................. 50 6.3 Participants ......................................................................................................................................... 51 6.4 Overall procedure for data recording .................................................................................................. 52 6.5 Setup in the lab ................................................................................................................................... 53 6.6 Details on performing motor tasks ..................................................................................................... 54 6.7 Measuring the MVC ........................................................................................................................... 58 6.8 Summary ............................................................................................................................................. 59 Chapter 7 Splint dynamometer development ............................................................................................ 60 7.1 Introduction ........................................................................................................................................ 60 7.2 Reviewing literature of dynamometers used for WE and WF ............................................................ 60 7.3 Design specifications of the dyno ....................................................................................................... 62 7.4 Mechanical design and construction .................................................................................................. 65 Contents vi 7.5 Electronic measurement system design and construction .................................................................. 69 7.5.1 Selection of a force transducer ...................................................................................................... 70 7.5.2 Force signal conditioning .............................................................................................................. 71 7.5.3 Force signal processing .................................................................................................................. 72 7.5.4 Calibration ..................................................................................................................................... 75 7.6 Testing methods and results ............................................................................................................... 76 7.7 Summary and conclusions .................................................................................................................. 81 Chapter 8 EEG data pre-processing and validation ................................................................................... 82 8.1 Introduction ........................................................................................................................................ 82 8.2 Data pre-processing ............................................................................................................................ 85 8.2.1 Removing artifacts ......................................................................................................................... 87 8.3 Frequency filtering ............................................................................................................................. 88 8.4 Spatial filtering, feature extraction, selection and classification ........................................................ 89 8.4.1 Feature extraction .......................................................................................................................... 90 8.4.2 Feature selection ............................................................................................................................ 91 8.4.3 Mahalanobis distance clustering classification .............................................................................. 92 8.4.4 Determining the best-performing IC ............................................................................................. 95 8.5 Evaluating classification accuracies ................................................................................................... 96 8.6 Summary and conclusions ................................................................................................................ 101 Chapter 9 DCM implementation ............................................................................................................. 102 9.1 Introduction ...................................................................................................................................... 102 9.2 Method .............................................................................................................................................. 102 9.2.1 Source localisation ....................................................................................................................... 104 9.2.2 Time-frequency representation of signals from ROIs ................................................................. 106 9.2.3 Definition of model space ............................................................................................................ 107 9.2.4 Estimation of coupling parameters .............................................................................................. 110 9.2.5 Bayesian model selection (BMS) and evaluation of model fit .................................................... 111 9.2.6 Inference on coupling parameters ................................................................................................ 113 9.2.7 Determining the exogenous input function .................................................................................. 115 9.3 Summary ........................................................................................................................................... 119 Chapter 10 DCM results ............................................................................................................................ 120 10.1 Introduction ...................................................................................................................................... 120 10.2 Bayesian model selection ................................................................................................................. 120 10.3 Spectrogram analysis and model fit ................................................................................................. 123 Contents vii 10.4 Inference on coupling parameters of motor tasks ............................................................................. 125 10.5 Summary ........................................................................................................................................... 133 Chapter 11 Discussion ............................................................................................................................... 134 11.1 Introduction ...................................................................................................................................... 134 11.2 Discussion and conclusion of DCM results ...................................................................................... 134 11.2.1 Model verification to answer the research question ............................................................... 134 11.2.2 INEN structure of the winning model ..................................................................................... 135 11.2.3 The role of high-gamma frequencies ...................................................................................... 135 11.2.4 Inference on the role of PMv in control of wrist motor tasks ................................................. 136 11.2.5 Inference on the role of PFC in control of wrist motor tasks .................................................. 137 11.2.6 Differences between WE and WF motor tasks ....................................................................... 138 11.2.7 Summary of insights ............................................................................................................... 139 11.3 Limitations and suggested future works ........................................................................................... 140 11.4 Summary and conclusions ................................................................................................................ 142 Chapter 12 Summary and conclusions ...................................................................................................... 143 References ............................................................................................................................................................ 146 Appendices .......................................................................................................................................................... 162 Appendix A: Ethics approval and participant handedness questionnaires – prior to EEG data recording ..... 162 Appendix B: Additional details of the dyno ................................................................................................... 175 Appendix C: Best-performing ICs .................................................................................................................. 177 Appendix D: Supplementary DCM results ..................................................................................................... 184 List of figures viii List of figures Figure 1.3.1: Performance of concentric WE and WF movements. ........................................................................ 3 Figure 1.4.1: Overall structure of a sensorimotor BCI used for control of a bionic hand. ...................................... 4 Figure 1.4.2: a) some ROIs and b) homunculi of M1 .............................................................................................. 5 Figure 2.1: Organisation of chapters in this thesis to meet the objectives and test the hypothesis. ...................... 10 Figure 3.2.1: EEG scalp electrodes recording a mixture of voltage signals from multiple cortical regions. ........ 14 Figure 3.2.2: 10-5 electrode placement system. .................................................................................................... 17 Figure 3.5.1: Description of dynamic causal modelling involving spectrally induced responses. ........................ 27 Figure 4.2.1: Method used to search for literature aimed at exploring the FC and EC of motor tasks involving WE and WF.. ................................................................................................................................................ 33 Figure 4.3.1: Method used to search for DCM literature aimed at exploring hand motor control. ....................... 35 Figure 5.2.1: Method used to search for EEG BCI literature involving WE and WF. .......................................... 39 Figure 5.4.1: ROIs and their interconnections from the winning models of DCM studies of hand motor control ...................................................................................................................................................................... 45 Figure 5.4.2: The 12 plausible DCM model structures involving seven ROIs evaluated in this thesis. Model structures 7–12 have more bilateral connectivity than model structures 1–6. The ROIs receiving the input function change moving from left to right.. ................................................................................................. 47 Figure 6.5.1: Depiction of how a participant will be seated in the lab with all surrounding equipment. a) shows the view from the side, b) shows the view from the front, c) shows view from the top. ............................. 54 Figure 6.6.1: Photograph of a RH hand in the correct position in the dyno. ......................................................... 55 Figure 6.6.2: Timing diagram of a single trial and the related changes in MVC-normalised wrist force.. ........... 56 Figure 6.6.3: Graphical user interface of the of dyno software shown on the computer screen, along with the instruction for S3, when a participant performed isometric WF at 15% of MVC. ...................................... 57 Figure 6.7.1: Diagram showing the timing and force co-ordination for measuring the MVC for WF. ................ 59 Figure 7.3.1: a) Anatomy of hand and forearm. b) typical dimensions of the human hand. ................................. 62 Figure 7.3.2: a) Front view of seated participant, showing the position of the shoulders and upper arm as well as the parameters used for Equation (7.1). b) side view of participant showing the angle and position of the forearm. c) top view of the hand strapped to the base of the dyno. ............................................................. 64 Figure 7.4.1: Major components and main measurements of the dyno shown from different views. a) shows the top view, b) shows the view of the left side and c) shows the view from the underside/bottom of the dyno. ...................................................................................................................................................................... 66 List of figures ix Figure 7.4.2: Depiction of hand and arm fitted into the dyno. .............................................................................. 68 Figure 7.5.1: Block diagram of force measurement system .................................................................................. 69 Figure 7.5.2: Demonstrating the mechanism of force measurement using RH WE as an example. ..................... 70 Figure 7.5.3: Circuit diagram of dyno electronics. ................................................................................................ 71 Figure 7.5.4: Third and main software routine to capture, calculate and display MVC-normalised real-time force data. .............................................................................................................................................................. 73 Figure 7.5.5: Second software routine to calculate the zero-force-offset values for each participant’s hand. ...... 74 Figure 7.5.6: First software routine to calculate the WE and WF MVCs for each participant’s hand. ................. 75 Figure 7.6.1: Experimental setup for Test 1 and Test 2 for the right cylindrical rod.. .......................................... 76 Figure 7.6.2: Plots of AF against F for a) right cylindrical force rod and b) the left cylindrical force rod. .......... 78 Figure 7.6.3: Testing method applied to determine the degree of force normalisation.. ....................................... 79 Figure 8.1.1: Overall method to pre-process the recorded EEG data and organise the data for the validation process. ......................................................................................................................................................... 84 Figure 8.1.2: Overall method to validate the pre-processed EEG data.. ............................................................... 84 Figure 8.2.1: An ERP of single trials for channel C3 for participant 1. ................................................................ 85 Figure 8.2.2: IC number 1, from participant 1, was identified as an EOG artifact. a) shows the scalp plots, b) shows the ERP and c) shows the power spectrum. ...................................................................................... 87 Figure 8.2.3: IC number 18, from participant 14 was identified as an EMG artifact. a) shows the scalp plots, b) shows the ERP and c) shows the power spectrum. ...................................................................................... 88 Figure 8.4.1: The process to determine the best-performing IC and its associated accuracy of classification. .... 89 Figure 8.4.2: TF feature extraction algorithm. ...................................................................................................... 90 Figure 8.4.3: Computing the BC using the overlap of two probability density distributions for a given feature.. 92 Figure 8.4.4: An illustration of Mahalanobis distance clustering between two classes (RH and LH) using two features as an example.. ............................................................................................................................... 93 Figure 8.5.1: Examples of 2D plots of best-performing ICs. ................................................................................ 96 Figure 8.5.2: Comparing the AOCs achieved in this thesis (green) to classification accuracies found in literature (orange) for a) the RLI and b) the EFI. ........................................................................................................ 98 Figure 8.5.3: Analysis of frequency bands producing the highest participant-specific AOCs. ............................. 99 Figure 9.2.1: Overview of DCM process for analysing EEG data for each hand of each participant. ................ 103 Figure 9.2.2: Overview of DCM analysis of all RH models across all participants.. .......................................... 104 Figure 9.2.3: a) The spherical head model consisted of the brain cortex (blue mesh), the inner skull (red mesh) and outer scalp surface (light brown mesh). The slices were linked to the MRI space. b) After co- registration, the positions of EEG electrodes were defined and are shown here as red and black dots ..... 105 Figure 9.2.4: TF transformation to compute the observed induced spectral responses at each ROI. .................. 107 Figure 9.2.5: The 12 plausible DCM model structures involving seven ROIs tested in this thesis. ................... 108 Figure 9.2.6: a) Typical anatomical locations of ROIs (Mayka et al., 2006). b) Graphical position of the seven ROIs plotted on the spherical head model, which is described in Section 9.2.1. ...................................... 109 Figure 9.2.7: Exogenous input function u(t) in time. .......................................................................................... 111 Figure 9.2.8: Examples of participant 1’s RH frequency-to-frequency couplings (from SMA to l-M1-H) shown in the reconstituted spectrograms from a) the A-matrix, b) the A+B-matrix and c) the B-matrix. ........... 114 List of figures x Figure 9.2.9: Time plots of exogenous input functions u(t) for a) 1-input models, b) 2-input models and c) 3- input models. .............................................................................................................................................. 117 Figure 9.2.10: Participant-specific results of BMS using FFX and family-level analysis to determine the exogenous input u(t).. ................................................................................................................................. 118 Figure 9.2.11: Group-level results of BMS using RFX and family-level analysis to determine the exogenous input. Models are labelled as explained in the caption of Figure 9.2.10. ................................................... 119 Figure 10.2.1: Group-level BMS RFX results to determine the most likely model structure for a) RH and b) LH. .................................................................................................................................................................... 121 Figure 10.2.2: RH BMS family-level inference comparing a) Bi-1 and Bi-2 families, b) INEN and ILEN families and c) input ROI families. ............................................................................................................ 122 Figure 10.2.3: LH BMS family-level inference comparing a) Bi-1 and Bi-2 families, b) INEN and ILEN families and c) input ROI families. ............................................................................................................ 122 Figure 10.3.1: Comparing the observed spectrograms to those predicted by model 11 for the RH of participant 1. . ................................................................................................................................................................... 124 Figure 10.4.1: Visual depiction of the most significant (p < 0.002) frequency-to-frequency A-matrix (WE) and A+B-matrix (WF) couplings between ROIs. . ........................................................................................... 126 Figure 10.4.2: Visual depiction of the most significant (P < 0.002) frequency-to-frequency B-matrix couplings between ROIs showing the difference in connectivity between WE and WF. . ........................................ 127 Figure B-0.1: Detailed dimensions (in mm) of dyno. .......................................................................................... 175 Figure B-0.2: Photographs of dyno. .................................................................................................................... 176 Figure C-0.1: 2D plots of best-performing ICs for participant 1 ......................................................................... 177 Figure C-0.2: 2D plots of best-performing ICs for participant 2 ......................................................................... 178 Figure C-0.3: 2D plots of best-performing ICs for participant 3 ......................................................................... 178 Figure C-0.4: 2D plots of best-performing ICs for participant 4 ......................................................................... 179 Figure C-0.5: 2D plots of best-performing ICs for participant 5 ......................................................................... 179 Figure C-0.6: 2D plots of best-performing ICs for participant 6 ......................................................................... 180 Figure C-0.7: 2D plots of best-performing ICs for participant 7 ......................................................................... 180 Figure C-0.8: 2D plots of best-performing ICs for participant 8 ......................................................................... 181 Figure C-0.9: 2D plots of best-performing ICs for participant 9 ......................................................................... 181 Figure C-0.10: 2D plots of best-performing ICs for participant 10 ..................................................................... 182 Figure C-0.11: 2D plots of best-performing ICs for participant 11 ..................................................................... 182 Figure C-0.12: 2D plots of best-performing ICs for participant 12 ..................................................................... 183 Figure C-0.13: 2D plots of best-performing ICs for participant 13 ..................................................................... 183 Figure C-0.14: 2D plots of best-performing ICs for participant 14 ..................................................................... 183 Figure D-0.1: Spectrograms for RH DCM 11 for participant 1. ......................................................................... 184 Figure D-0.2: Spectrograms for RH DCM 11 for participant 2. ......................................................................... 185 Figure D-0.3: Spectrograms for RH DCM 11 for participant 3. ......................................................................... 186 Figure D-0.4: Spectrograms for RH DCM 11 for participant 4. ......................................................................... 187 Figure D-0.5: Spectrograms for RH DCM 11 for participant 5. ......................................................................... 188 Figure D-0.6: Spectrograms for RH DCM 11 for participant 6. ......................................................................... 189 List of figures xi Figure D-0.7: Spectrograms for RH DCM 11 for participant 7. ......................................................................... 190 Figure D-0.8: Spectrograms for RH DCM 11 for participant 8. ......................................................................... 191 Figure D-0.9: Spectrograms for RH DCM 11 for participant 9. ......................................................................... 192 Figure D-0.10: Spectrograms for RH DCM 11 for participant 10. ..................................................................... 193 Figure D-0.11: Spectrograms for RH DCM 11 for participant 11. ..................................................................... 194 Figure D-0.12: Spectrograms for RH DCM 11 for participant 12. ..................................................................... 195 Figure D-0.13: Spectrograms for RH DCM 11 for participant 13. ..................................................................... 196 Figure D-0.14: Spectrograms for RH DCM 11 for participant 14. ..................................................................... 197 Figure D-0.15: Spectrograms for LH DCM 11 for participant 1. ........................................................................ 198 Figure D-0.16: Spectrograms for LH DCM 11 for participant 2. ........................................................................ 199 Figure D-0.17: Spectrograms for LH DCM 11 for participant 3. ........................................................................ 200 Figure D-0.18: Spectrograms for LH DCM 11 for participant 4. ........................................................................ 201 Figure D-0.19: Spectrograms for LH DCM 11 for participant 5. ........................................................................ 202 Figure D-0.20: Spectrograms for LH DCM 11 for participant 6. ........................................................................ 203 Figure D-0.21: Spectrograms for LH DCM 11 for participant 7. ........................................................................ 204 Figure D-0.22: Spectrograms for LH DCM 11 for participant 8. ........................................................................ 205 Figure D-0.23: Spectrograms for LH DCM 11 for participant 9. ........................................................................ 206 Figure D-0.24: Spectrograms for LH DCM 11 for participant 10. ...................................................................... 207 Figure D-0.25: Spectrograms for LH DCM 11 for participant 11. ...................................................................... 208 Figure D-0.26: Spectrograms for LH DCM 11 for participant 12. ...................................................................... 209 Figure D-0.27: Participant-specific BMS FFX results for the RH of participants 1 – 6. .................................... 210 Figure D-0.28: Participant-specific BMS FFX results for the RH of participants 7 – 12. .................................. 211 Figure D-0.29: Participant-specific BMS FFX results for the RH of participants 13 – 14. ................................ 212 Figure D-0.30: Participant-specific BMS FFX results for the LH of participants 1 – 6. .................................... 213 Figure D-0.31: Participant-specific BMS FFX results for the LH of participants 7 – 12. .................................. 214 Figure D-0.32: Statistically significant frequency-to-frequency couplings for RH WE (A-matrix spectrogram). .................................................................................................................................................................... 215 Figure D-0.33: Statistically significant frequency-to-frequency couplings for RH WF (A+B-matrix spectrogram). .............................................................................................................................................. 216 Figure D-0.34: Statistically significant frequency-to-frequency couplings showing differences between WE and WF for the RH (B-matrix spectrogram). .................................................................................................... 217 Figure D-0.35: Statistically significant frequency-to-frequency couplings for LH WE (A-matrix spectrogram). .................................................................................................................................................................... 218 Figure D-0.36: Statistically significant frequency-to-frequency couplings for LH WF (A+B-matrix spectrogram). .............................................................................................................................................. 219 Figure D-0.37: Statistically significant frequency-to-frequency couplings showing differences between WE and WF for the LH (B-matrix spectrogram). .................................................................................................... 220 List of tables xii List of tables Table 1.1: Estimated statistics of people suffering from a hand-motor impairment.. ............................................. 1 Table 3.1: Advantages and challenges of EEG for neural studies. ........................................................................ 15 Table 3.2: Typical frequency bands associated with EEG. ................................................................................... 20 Table 3.3: Comparing electrophysiological instrumentation methods for BCIs (adapted from Nicolas-Alonso and Gomez-Gil (2012) and Pistohl et al. (2012)). ........................................................................................ 22 Table 3.4: Summary of background information provided. .................................................................................. 31 Table 4.1: Summary of EC and FC articles involving WE and WF motor tasks. ................................................. 34 Table 4.2: Summary of DCM articles investigating hand movement control. ...................................................... 36 Table 5.1: Results of the review of EEG frequencies used for feature extraction in BCI studies involving WE and WF motor tasks. .................................................................................................................................... 40 Table 5.2: Results of the review of EEG frequencies used in studies involving EC and FC analyses of WE and WF motor tasks. ........................................................................................................................................... 42 Table 5.3: Results of the review of EEG frequencies used in DCM-IR studies involving hand motor control. ... 43 Table 6.1: Overall procedure and duration breakdown of experiment for each participant. ................................. 53 Table 7.1: Summary of dynamometers used in literature to measure wrist force or torque. ................................. 61 Table 7.2: Summary of the largest and smallest recorded wrist MVCs forces from literature. ............................ 63 Table 7.3: Summary of design specifications of the dyno and its support table. .................................................. 65 Table 7.4: Descriptions of physical components of the dyno. ............................................................................... 67 Table 7.5: Test 1 and Test 2 results for the left cylindrical rod. ............................................................................ 77 Table 7.6: Test 1 and Test 2 results for the right cylindrical rod.. ......................................................................... 78 Table 7.7: Test 3 results for wrist movement repetitions with the RH. ................................................................. 80 Table 7.8: Test 3 results for wrist movement repetitions with the LH. ................................................................. 81 Table 8.1: Review of classification accuracies used in studies involving WE and WF motor tasks.. .................. 83 Table 8.2: Nett number of usable trials for each participant. ................................................................................ 86 Table 8.3: Summary of frequency limits applied during featured extraction and the resulting number of features extracted per trial. ......................................................................................................................................... 91 Table 8.4: AOC results for the best performing IC for the RLI (%). .................................................................... 97 Table 8.5: AOC results for the best performing IC for the EFI for RH movements (%). ..................................... 97 Table 8.6: AOC results for best performing IC for the EFI for LH movements (%). ........................................... 98 Table 9.1: Details of ROIs abbreviations, anatomical names and derivation of MNI co-ordinates.. .................. 109 List of tables xiii Table 10.1: Summary of results of participant-specific BMS FFX. .................................................................... 120 Table 10.2: Quantitative results for the accuracy of fit of the best-performing model i.e., model 11. ............... 123 Table 10.3: Most significant (P < 0.002) frequency-to-frequency A-matrix couplings (for WE) for the RH.. .. 130 Table 10.4: Most significant (P < 0.002) frequency-to-frequency A+B-matrix couplings (for WF) for the RH.. .................................................................................................................................................................... 131 Table 10.5: Most significant (P < 0.002) frequency-to-frequency B-matrix couplings showing the difference in connectivity between RH WE and WF. ..................................................................................................... 131 Table 10.6: Most significant (P < 0.002) frequency-to-frequency A+B-matrix couplings (for WF) for the LH.. .................................................................................................................................................................... 132 Table 10.7: Most significant (P < 0.002) frequency-to-frequency B-matrix couplings showing the difference in connectivity between LH WE and WF. ..................................................................................................... 132 Table 10.8: Most significant (P < 0.002) frequency-to-frequency A-matrix couplings (for WE) for the LH.. .. 133 Glossary of terms and abbreviations xiv Glossary of terms and abbreviations Term Description AAR automatic artifact removal ADC analogue-to-digital converter ADL activity of daily living AOC accuracy of classification BC Bhattacharya coefficient BCI brain-computer interface BD Bhattacharya distance BEM boundary element method bionic hand prosthetic, robotic and orthotic hands that can be neurally-controlled BMS Bayesian model selection CMC cortico-muscular coherence CSV comma-separated-value CV coefficient of variation DCM dynamic causal modelling DCM-CSD dynamic causal modelling of cross-spectral densities DCM-ERP dynamic causal modelling of evoked responses DCM-IR dynamic causal modelling for induced responses DCMs dynamic causal models dyno force dynamometer for measuring the forces of isometric wrist extension and wrist flexion EC effective connectivity ECoG electrocorticography EEG electroencephalogram EFI extension vs. flexion investigation EMG electromyogram EOG electrooculogram ERD event-related desynchronisation ERP event-related potential ERS event-related synchronisation FC functional connectivity FFT fast Fourier transform FFX fixed effects fMRI functional magnetic resonance imaging IC independent component ICA independent component analysis ILEN intrinsic linear and extrinsic non-linear Glossary of terms and abbreviations xv INEN intrinsic non-linear and extrinsic non-linear LFP local field potentials LH left hand lPM lateral premotor cortex M1 primary motor cortex M1-H hand/wrist homunculus of primary motor cortex MD Mahalanobis distance MEG magnetoencephalography MNI Montreal Neurological Institute MRIs magnetic resonance images MSP multiple sparse priors MVC maximum voluntary contraction MVIC maximum voluntary contraction of isometric movements NIRS near infrared spectroscopy PFC prefrontal cortex PMA premotor area PMC premotor cortex PMd dorsal premotor cortex PMv ventral premotor cortex pst peristimulus time RFX random effects RH right hand RLI right vs. left investigation ROI region of interest S1 primary somatosensory cortex SMA supplementary motor area SMR sensorimotor rhythms SNR signal-to-noise ratio SPM Statistical Parametric Mapping SVD singular value decomposition TF time-frequency WE wrist extension WF wrist flexion Introduction 1 Chapter 1 Introduction 1.1 A loss of wrist movement control reduces quality of life We use our hands constantly to perform activities of daily living (ADLs), for example, eating, dressing, personal grooming, going to the toilet and handling objects (Mlinac and Feng, 2016). A person’s ability to use and control their hand may be impaired by amputation of a hand or arm, a stroke, a spinal cord injury or a neuromuscular disease. Consequently, their ability to perform ADLs by themselves declines, along with their quality of life (Mlinac and Feng, 2016; Wolpaw et al., 2002). This may, in turn, lead to a dependence on other individuals, institutions and mechanical devices for the performance of ADLs (Edemekong et al., 2020; Mlinac and Feng, 2016). Table 1.1: Estimated statistics of people suffering from a hand-motor impairment. The population of the USA was estimated at 332 million and the global population was estimated 7.78 billion. * refers to extrapolations based on studies of the population in the USA. ** refers to extrapolations based on studies of New Zealand’s population. Type of motor impairment Number of people globally References Hand amputation (including partial hand, excluding finger amputations) 17.9 million (Ziegler-Graham et al., 2008) Survivor of strokes 80 million (“Global Stroke Fact Sheet,” 2019) Quadriplegia due to spinal cord injuries 4.1 million * (Lasfargues et al., 1995; National Spinal Cord Injury Statistical Center, 2020; “Population Clock: World,” 2021; “Spinal Cord Injury Facts & Statistics,” 2019) Neuromuscular disease e.g., Muscular Dystrophies 252 850 ** (Theadom et al., 2014) The estimated number of people, globally, who suffer from a hand-motor impairment is summarised in Table 1.1. These individuals could benefit greatly from regaining the ability to perform ADLs. This requires regaining some essential hand functionality, which includes wrist Introduction 2 extension (WE) and wrist flexion (WF). The importance of these movements is detailed in Section 1.3. 1.2 A neurally-controlled bionic hand as a possible solution A prosthetic hand could substitute for an amputated hand. An orthotic hand or a robotic standalone hand/arm could return hand functionality to people afflicted with strokes, paralysis or neuromuscular diseases (Barsotti et al., 2015; Fang et al., 2015; Pfurtscheller et al., 2000; Wolpaw et al., 2002). Prosthetic, robotic and orthotic hands can be neurally-controlled (Fang et al., 2015) and are henceforth referred to as bionic hands in this thesis. If available, residual muscle activity can be used to control the bionic hand. This can be done either by means of mechanical movements of the body (e.g. the elbow or shoulder) (Plettenburg, 1998) or by interpreting muscular bio-signals, using electromyography (EMG) for example (Fang et al., 2015). When the muscles are not active, as in the case of paralysed or ‘locked in’ patients (Gabriel T. Velloso, 2012)), bio-signals from the brain can be used to control the bionic hand using a sensorimotor brain-computer interface (BCI) (Fang et al., 2015; Wolpaw et al., 2002). Electroencephalography (EEG) is the most widely used bio-signal for BCI applications (Shih et al., 2012). More background information on EEG and BCIs is provided in Chapter 3. 1.3 Importance of WE and WF WF is the movement of the hand at the wrist joint such that the angle between the palm and the anterior part of the forearm decreases (Widodo et al., 2019). WE is the movement of the hand at the wrist joint such that the angle between the posterior part of the hand and the posterior part of the forearm decreases (Widodo et al., 2019). This is shown in Figure 1.3.1 (adapted from McConnell et al. (2017)). The main muscles involved in the actuation of WE include the extensor carpi radialis brevis, extensor carpi radialis longus and the extensor carpi ulnaris; while those involved in WF include the flexor carpi radialis and flexor carpi ulnaris (Yoshii et al., 2015). Introduction 3 Figure 1.3.1: Performance of concentric WE and WF movements. WE and WF (along with other wrist movements) allow a person to move their entire hand when performing everyday activities. WE and WF assist to stabilise, position and control the hand; thus, enabling the control of grasping strength and finer finger movement (Rybski, 2004; Trombly and Radomski, 2002). This in turn enables performing ADLs (activities of daily living). Studies have quantified that WE and WF are required to perform ADLs such as cutting with a knife, writing, turning a doorhandle or doorknob, opening a jar, donning pants, perineal cleansing and drinking from a cup (Gates et al., 2015; Nelson et al., 1994). Consequently, the loss of the ability to perform WE and WF impairs a person’s ability to perform ADLs. Patients undergoing post-stroke rehabilitation perform a key exercise involving WE and WF in order to regain hand full hand and wrist motion (McConnell et al., 2017). A study by Bertels et al. (2009) specifically suggests that bionic hands capable of performing WF will allow users to perform ADLs more naturally. Some studies have included WE and WF to improve the functionality of a bionic hand controlled by EMG (Roche et al., 2014). Upper limb amputees have expressed desires for improved wrist movement and control (Carey et al., 2015; Engdahl et al., 2015). Hence, the effective control of a bionic hand using EEG, necessitates studies exploring the neural control of WF and WE. 1.4 Using a sensorimotor brain-computer interface to control a bionic hand The overall functionality an EEG-based sensorimotor BCI is shown in Figure 1.4.1 (Wolpaw et al., 2002). Referring to this figure, let us consider a case in which an EEG-based BCI controls wrist extension (WE) wrist flexion (WF) Introduction 4 a bionic hand for a paralysed person who has no movement capability in either hand. This person wears a scalp cap fitted with 128 EEG electrodes. Movement-related information contained within the EEG data form continuous inputs to the BCI (Wolpaw et al., 2002). The bionic hand is substituting the right hand (RH) of the BCI user. The BCI is responsible for controlling the kinematic type, as well as the speed, force, range and reach of movements of the bionic hand (Gu et al., 2009; McFarland et al., 2010; Roy et al., 2018). Through training, recognisable signal patterns — known as features — are mapped to different types of movements of the user’s RH (McFarland et al., 2010). When the training is complete, the BCI is used to continuously interpret the EEG features on order to determine the user’s movement intentions. It then actuates the bionic hand according to the intentions it interprets. Figure 1.4.1: Overall structure of a sensorimotor BCI used for control of a bionic hand. Interpreting a person’s movement intentions from EEG is challenging. EEG has a low-spatial resolution, a low signal-to-noise ratio (SNR) and is susceptible to contamination by artifacts. (The challenges associated with EEG are explained further in Section 3.2.) Invasive recording techniques provide advantages over EEG in this regard — this is explained in Section 3.3. Despite the challenges of EEG interpretation, some studies have demonstrated real-time, or online, BCI control using movement-related features extracted from EEG (El-Madani et al., 2015; Han et al., 2019; McFarland et al., 2010). (The differences between online and offline EEG analysis are explained in Section 3.2.) These studies suggest that control of a bionic hand could be achieved using a BCI extracting information from non-invasive EEG electrodes. However, in El-Madani et al. (2015) and McFarland et al. (2010), the authors reported that the BCI performances were inconsistent and unstable — like many other BCI studies (Wierzgała et al., 2018). Hence, current EEG-based BCIs require improvements to allow the consistent and practical control of assistive devices — such as bionic hands — in everyday life (McFarland et al., 2010; Wierzgała et al., 2018). Clean EEG signals Translate signals from EEG channels to signals from movement-related brain regions Extract movement- related features Interpret movement- related features to determine user’s intention Actuate bionic hand according to user’s intention Record signals from EEG channels when user intends to move User sees the movement of the bionic hand Introduction 5 In the online BCI studies mentioned above, the motor tasks involved the imagined movements of the RH, left hand (LH), tongue and/or foot. These motor tasks are controlled by spatially distant homunculi of the primary motor cortex (M1), as shown in Figure 1.4.2. These distances cannot benefit the kinematic discrimination of unilateral WE and WF (Edelman et al., 2016; Vučković and Sepulveda, 2012). The control signals for WE and WF originate from the same wrist/hand control homunculi of M1 — referred to as M1-H and shown in Figure 1.4.2 b). Additional neural control information may thus be required to differentiate, accurately and consistently, between unilateral WE and WF. Figure 1.4.2: a) some ROIs and b) homunculi of M1 (Cieslik et al., 2011; de Klerk et al., 2015; Kim et al., 2018; Mayka et al., 2006; Rizzolatti et al., 2002). In all three studies referred to above, the EEG features associated with the different motor imagery tasks, were focused topographically on the scalp and spectrally within specific frequency bands. More specifically, features were extracted from electrodes located topographically over only two cortical regions namely, the RH and LH homunculi of both primary motor cortices (right and left M1-H). These cortical regions are shown in Figure 1.4.2 b). The features were also isolated to the mu and beta frequency bands. (Section 3.2.3 provides more information on EEG frequency bands). These features stem from sensorimotor rhythms (SMRs) linked to movement preparation, imagination, execution and observation. (More Primary motor cortex (M1) Supplementary motor area (SMA) ventral premotor cortex (PMv) Pre-frontal cortex (PFC) dorsal premotor cortex (PMd) premotor cortex (PMC) a) b) Left M1-H Anterior Rostral Posterior Caudal lateral medial C3 Cz C4 Left M1-H Right M1-H Introduction 6 information on SMRs, which are commonly used in BCI studies, is provided in Section 3.2.3). The BCI users were trained to amplify their SMRs (and improve the SNR), yet the extracted SMR features did not provide reliable BCI performances. Invasive neural recordings provide a higher spatial resolution and SNR when compared to EEG, as shown in Table 3.3, Section 3.3. Hence, the use of invasive neural recordings is one suggested way to gather more neural control information (Pistohl et al., 2012; Shih et al., 2012). However, the above-mentioned SMR EEG features, and features from invasive neural recordings, have provided comparable BCI performances for 2D cursor control (Hochberg et al., 2006; McFarland et al., 2010; Wolpaw and McFarland, 2004). This suggests that the low EEG spatial resolution is not the limiting factor for reliable and effective BCI use (McFarland et al., 2010). So, what, in fact, is the limiting factor? 1.5 Towards improved BCI performance gained from knowledge of the underlying hand motor control mechanisms A growing body of research indicates that hand movements are controlled by a combination of distributed brain regions (Aflalo and Graziano, 2006; Cieslik et al., 2011; Dum and Strick, 2005; Grefkes et al., 2008; Ledberg et al., 2007; Luu et al., 2017). In addition to M1-H, these regions of interest (ROIs) include the premotor cortex (PMC), the supplementary motor area (SMA) and the prefontal cortex (PFC) (shown in Figure 1.4.2). Studies have also provided evidence that the ROIs communicate using a range of synchronous oscillations to achieve hand motor control — described in Chapter 5. These oscillations are not confined to the mu and beta rhythms (Luu et al., 2017). Furthermore, the causal relationships between oscillations of different frequencies and between ROIs have been identified in 13 articles investigating hand motor control — described in Table 5.3. Overall, hand motor control has been shown to involve causal cross-frequency communication (across multiple frequency bands) between multiple ROIs. Hence BCI features extracted from only the mu and beta frequency bands and from M1- H only, are likely to only encapsulate a limited portion of the hand motor control information. Elucidating a more complete picture of the underlying motor control of essential hand movements — such as WE and WF — may provide a-prior knowledge that could aid and improve the development of subject-specific BCIs in the following ways (Lee et al., 2020): Introduction 7 1. BCI feature extraction approaches could be improved with the aim of encapsulating a more complete picture of hand motor control. This could be implemented by extracting features from more ROIs and from more frequency bands, and that capture the causal relationships between ROIs (Hamedi et al., 2016; McFarland et al., 2010). 2. Predictors of BCI performance could be developed to identify participants who struggle to voluntarily control SMRs (who are categorised as BCI-inefficient) (Lee et al., 2020). Subsequently the knowledge of their motor control mechanisms could suggest alternate solutions to improve their BCI performances (Lee et al., 2020). This could lead to the improved accuracy and response time of a BCI controlling essential hand movements, which are executed by a bionic hand. 1.6 Elucidating the motor control of WE and WF To understand the underlying mechanisms of neural motor control, researchers commonly use brain connectivity analysis techniques to uncover the functional connectivity (FC) and effective connectivity (EC) of ROIs related to movement control. FC techniques indicate statistical relationships between temporal neural signals from anatomically-distinct ROIs, without providing information on the causality of these relationships (Chen et al., 2008; Grefkes et al., 2008; Hamedi et al., 2016). In contrast, EC shows causality by modelling the influences that ROIs have on themselves and on each other (Chen et al., 2008; Grefkes et al., 2008; Hamedi et al., 2016). EC and FC techniques are promising approaches to gaining knowledge of the underlying neural control mechanisms, which could lead to improved sensorimotor BCI performance (Hamedi et al., 2016). (Section 3.4 explains FC and EC in more detail.) Dynamic causal modelling (DCM) was first used in 2003 and is currently a mainstream neuroimaging analysis approach to exploring EC (Friston et al., 2019, 2003; Stephan et al., 2009). A search of the PubMed database in August 2021 listed 669 articles on DCM. The DCM approach involves using a framework of techniques to model the causal connections between ROIs as well as the modulatory effects of external conditions on these connections (Friston et al., 2003). (Section 3.5 provides more information on DCM.) DCM has been used to explore the EC for hand motor control. Plausible dynamic causal models, built on the cross-frequency causal interactions of multiple ROIs, have been developed Introduction 8 for hand motor tasks, including full finger grasps, finger taps and button presses. (More details are provided in Section 4.3.) These models provide evidence that the control of some hand movements utilise multiple ROIs, including M1, SMA, PMC and PFC. (More details are provided in Section 5.4.) They also show that these ROIs utilise causal, cross-frequency communication spanning multiple frequency bands: including delta, mu, beta and gamma. (More details are provided in Section 5.2.) It is not yet known whether the above applies to WF and WF control, since there are currently no DCM, EC or FC studies exploring WE and WF control and their differences, as explained in Chapter 4. This thesis addresses this knowledge gap. 1.7 Summary A bionic hand capable of performing WE and WF, controlled efficiently by an EEG-based BCI, could aid motor-impaired individuals in performing ADLs. The performance of BCIs to interpret users’ motor intentions needs to be improved to make this possible. Prominent and commonly used BCI signal features emanating from M1-H and found within the mu and beta frequency ranges may not provide enough information to discriminate between unilateral WE and WF. DCM could elucidate the motor control of WE and WF, and their neural control differences, which could provide insights to improve BCI signal feature extraction and thus BCI performance. DCM studies have suggested that the control of some types of hand movements involve the causal co-operation of multiple brain regions and multiple EEG frequency bands. The problem is, however, that this control mechanism has not yet been shown for WE and WF, as explained in Chapter 4. The next chapter specifies the hypothesis, aims and objectives to address this problem. It also provides a guide to the rest of the thesis. Hypothesis, aims and objectives 9 Chapter 2 Hypothesis, aims and objectives Chapter 1 introduced the problem addressed by this research, namely that DCM (dynamic causal modelling) studies have not yet elucidated the neural control mechanisms of WE (wrist extension) and WF (wrist flexion). It also explains the significance of addressing this problem. This chapter specifies the hypothesis, aims and objectives of this research. According to the literature review explained in Chapter 4, an EEG-based DCM study elucidating the differences between the neural control of WE and WF has not previously been carried out. DCM studies have elucidated the control of other hand motor tasks, including certain finger and hand movements. These movements are specified in Section 4.3 and are similar to WE and WF in the following ways: • They activate a similar area of the M1-H homunculus, as shown in Figure 1.4.2; • Their control involves similar features extracted from the mu, beta, delta and gamma frequency bands — as explained in Section 5.2. For this thesis, it was hypothesised that the neural control of the WE and WF, and the neural mechanism of their differentiation, rely on linear and non-linear, cross-frequency, causal couplings between bilateral M1-H, bilateral PMC, bilateral PFC and the SMA. The frequencies involved were believed to span the full EEG spectrum (described in Section 3.2.3), which includes the delta, theta, mu, beta, low-gamma and high-gamma bands. The hypothesis is based on knowledge gained from DCM EEG studies of hand movements other than WE and WF as well as knowledge gained from EEG BCI studies of wrist movements. (This is explained in Chapter 5.) This led to the first EEG-based DCM study elucidating differences between WE and WF neural control. Hypothesis, aims and objectives 10 The aim of this thesis was to test this hypothesis by developing biophysically plausible EEG- based dynamic causal models (DCMs) that could explain the differences in the control signals of unilateral WE and WF control. To do this, the following objectives were undertaken: 1. High-resolution EEG data was recorded while 14 human healthy participants performed repetitive WE and WF motor tasks. 2. A dynamometer — dyno for short — was developed to normalise the forces of the WE and WF movement repetitions during EEG recording. 3. The recorded EEG data was verified using analyses common to BCI literature. 4. Using the hypothesis as a guide, a set of biophysically plausible model structures were developed by drawing on existing anatomical, physiological, neuroscience, BCI and DCM knowledge. 5. Statistical Parametric Mapping (SPM) software was used to fit the plausible model structures to the recorded EEG data and to optimise the model parameters. 6. Bayesian model selection was used to identify the best-performing fitted model, which was then analysed to address the hypothesis. The above-mentioned process and objectives are outlined in Figure 2.1. Figure 2.1: Organisation of chapters in this thesis to meet the objectives and test the hypothesis. Development and testing of the dyno (Chapter 7) EEG data recording (Chapter 6) Experiment design to record new EEG data (Chapter 6) Pre-process and clean EEG data (Chapter 8) Validate EEG data (Chapter 8) Validation results (Chapter 8) Fitting cleaned EEG data to model structures (Chapter 9) Determine and analyse best performing model (Chapter 9) DCM results (Chapter 10) and discussion to address hypothesis (Chapter 11) Develop biophysically plausible model structures (Chapter 5) Illustrate the novelty of DCM EEG modeling of WE and WF (Chapter 4) Hypothesis, aims and objectives 11 The developed plausible model structures involved seven ROIs (regions of interest). (Section 5.4 explains the choice of ROIs in more detail.) In general, the signals from the ROIs cannot be extracted directly from EEG electrodes, since EEG electrodes do not only record information from cortical regions directly below them (Hamedi et al., 2016; Hassan and Wendling, 2018). Instead, EEG electrodes on the scalp record a complex mixture of signals from the various ROIs on the brain’s cortex — as shown in Figure 3.2.1 in Section 3.2. It is thus required to estimate the signals originating from each ROI from the EEG data. (Suitable techniques for this are presented in Section 3.2.2.) Studies have shown that using high- resolution EEG — 128 EEG electrodes or more — provides better localisation of signals from the source ROIs (Hassan et al., 2014; Hassan and Wendling, 2018; Lopes da Silva, 2013; Song et al., 2015). Hence, 128-channel EEG was used for the research in this thesis. Typical forces and torques associated with WE differ from that of WF (Decostre et al., 2015). (Details are provided in Table 7.2, Section 7.3.) This is in part due to the higher physiologic cross-sectional area of the wrist flexor muscles when compared to the wrist extensor muscles (Forman et al., 2020). (The muscles involved in actuating WF and WE are listed in Section 1.3.) It is known that variations in the force of finger and arm movements alter EEG signal patterns (Roy et al., 2018; Wang et al., 2017). It is therefore possible that any extracted EEG signal differences arising from the performance of WE and WF could be due to the inherent differences in forces between the two movements. This will be an unwanted result. Instead, the aim of this research is to find the EEG signal differences arising from the differences in kinematic control of the two movements. To remove any possible effects of force differences on the EEG analysis, the participants performed normalised, controlled versions of WE and WF. (More details are provided in Section 6.6.) To achieve this, a custom dyno was designed, built and tested. The dyno normalised the forces of WE and WF relative to their respective maximum voluntary contractions (MVCs) (Divekar and John, 2013). Details of the dyno are provided in Chapter 7. A database for 128-channel EEG data recorded for MVC-normalised WE and WF could not be found from online EEG database resources (“Database - BNCI Horizon 2020,” 2020; “EEG/ERP data available for free public download,” 2020). Hence, new EEG data were recorded. Hypothesis, aims and objectives 12 The experimental setup and choice of methods restricted the hypothesis to: • 14 healthy volunteer participants — more details are provided in Chapter 6; • the use of a 128-channel EEG machine in a controlled laboratory environment; • visually-cued MVC-normalised isometric versions of WE and WF, which are explained in Chapter 6. In summary: the EEG data that was recorded with aid from the dyno, was cleaned, validated and fitted to plausible DCM structures that were developed from literature. The resulting DCMs were evaluated to determine the best-performing model, which was then analysed to address the hypothesis. This chapter defines the novelty of this research and its hypothesis. The purposes and arrangement of Chapter 4 - Chapter 11, to address the hypothesis, are explained. Chapter 3 provides some necessary background information to contextualise the research. Chapter 12 concludes the thesis. Background information on the study components: EEG, BCIs and DCM 13 Chapter 3 Background information on the study components: EEG, BCIs and DCM 3.1 Introduction The research in this thesis involved DCM analysis of EEG for WE and WF motor tasks. The insights gained from this analysis could be used in future to improve the performance of EEG- based BCIs (brain-computer interfaces). Hence, this chapter provides background information for EEG, BCIs and DCM. 3.2 EEG for BCIs and movement analysis studies Activated neurons in the brain create post-synaptic electrical potentials in the neurons that they synapse with. This process results in a change in extracellular ionic charge, resulting in extra- neuronal volume currents. The resultant changes in electrical potentials, which may be read off the scalp via non-invasive electrodes, constitutes EEG (Niedermeyer and Lopes da Silva, 2005). EEG can therefore be used to extract electrical signals of interest from the brain. These signals may be used to derive information on actual and intended movements (Ungureanu et al., 2004). However, interpreting a person’s movement intentions from EEG is challenging. The brain produces simultaneously active neural processes (from approximately 120 billion neurons) (Herculano-Houzel, 2009; Wolpaw et al., 2002). The extracellular electrical potentials from these processes combine to form a superposition of electric fields on the scalp. EEG is the measurement of these fields using a limited number of scalp electrodes (Lopes da Silva, 2013). Hence each electrode records a mixture of neural information from all over the head, as shown in Figure 3.2.1 (Blankertz et al., 2008; Wolpaw et al., 2002). The electrical potentials are Background information on the study components: EEG, BCIs and DCM 14 further blurred as they pass through the high electrical resistance of the scalp and skull (Babiloni et al., 1997). Consequently, EEG has a low-spatial resolution and a low SNR (Blankertz et al., 2008; Wolpaw et al., 2002). Figure 3.2.1: EEG scalp electrodes recording a mixture of voltage signals from multiple cortical regions (shown in blue) (Onton and Makeig, 2009). Additionally, EEG signals are small (in the μV range) and are easily contaminated by artifacts. Artifacts originate from large muscle movements, head movements and facial movements (EMG artifacts); eye movements and eye blinks; cable movements; and noise from the mains power supply (Delorme and Makeig, 2004; Wolpaw et al., 2002). This makes the detection of signals of interest, such as SMRs (sensorimotor rhythms), challenging, since they are weaker in comparison to the artifacts (10 µV vs. 30–100 µV) (Wolpaw et al., 2002). Furthermore, EEG varies with time and according to circumstance, as well as between individuals (Wolpaw et al., 2002). Despite the challenges of EEG, accumulated data from clinical research have provided insights into the relationship between EEG signals and motor tasks (Aslanyan et al., 2015; Bradberry et al., 2010; Edelman et al., 2016; Vučković and Sepulveda, 2012; Wolpaw et al., 2002). Recent studies have used EEG to differentiate between the neural control of different unilateral hand movement intentions, which include WE and WF (Mohamed et al., 2011a; Mohamed and John, 2014; Vuckovic and Sepulveda, 2008; Vučković and Sepulveda, 2012). The advantages of EEG — shown in Table 3.1 — make it well suited for studying the neural mechanisms of motor control (Höller et al., 2013; Nicolas-Alonso and Gomez-Gil, 2012). EEG electrodes Skin of scalp Skull bone Cortical surface of brain Cortical ROI Background information on the study components: EEG, BCIs and DCM 15 Table 3.1: Advantages and challenges of EEG for neural studies. *Whole head analysis is enabled due to EEG electrodes spread across the scalp. This allows for signals from multiple ROIs to be investigated (Höller et al., 2013). This advantage could prove crucial for motor-impaired individuals, whose neural connectivity is subjected to neuroplastic changes (Höller et al., 2013). Advantages Challenges Non-invasive Low SNR, Portable Low spatial resolution Allows whole head analysis* Contamination by artifacts High temporal resolution Direct relationship to electrophysiological neural activity Low cost EEG analysis can be done online (in real-time) or offline (Wolpaw et al., 2002). In the case of offline analysis, data is recorded from several participants and analysis is done after recording and not in real-time (Wolpaw et al., 2002). Offline analysis was used for this thesis. Methods and parameters that show promise offline should be validated by extensive online testing, where the user’s neural signals are analysed in real-time (Vuckovic, 2009; Wolpaw et al., 2002). Classifiers used in BCI online and offline studies have been reviewed by Lotte et al. (2018). Linear classifiers, such as support vector machines and linear discriminant analysis have been the most popular types of classifiers for EEG-based online BCIs. Since 2007, more advanced adaptive classification techniques have been explored for both offline and online studies. For online BCIs, successful results have been achieved using supervised and unsupervised adaptative classifiers. The successful supervised adaptive classifiers included linear discriminant analysis, quadratic discriminant analysis and an ensemble of support vector machines. Linear discriminant analysis has been used as an unsupervised adaptive technique in several studies. In general, classifiers and spatial filters that require higher computational complexity for adaptation to new incoming real-time EEG data, are not suited to online BCIs. Offline EEG analysis is based on either multi-trial or single-trial techniques (Hallett et al., 2007; D. M. Herz et al., 2014; Vinh T. Nguyen et al., 2014; V. T. Nguyen et al., 2014; Sugata et al., 2014). A trial is a recorded segment of an EEG time-sequence containing a single instance of the stimulus or task of interest, such as the sequential contraction and relaxation of the muscles associated with one repetition of a particular motor task (Kohlmorgen and Blankertz, 2004). (A detailed description of a trial for the research in this thesis is provided in Section 6.6.) Offline single-trial methods simulate real-time BCI operation and are thus more Background information on the study components: EEG, BCIs and DCM 16 suitable for BCI studies (Hallett et al., 2007; Vuckovic, 2009). However, the large inter-trial variability and a low SNR (signal-to-noise ratio) in EEG signals make single-trial analysis challenging (Kohlmorgen and Blankertz, 2004). Event-related potentials (ERPs) refer to the time-locked changes in neural activity induced by events, such as the performance of motor tasks (Pfurtscheller and Lopes da Silva, 1999). ERPs rely on the assumption that a signal pattern of interest is time-locked to the events of similar single trials and is buried within activity from other background neural processes, which is treated as noise (Pfurtscheller and Lopes da Silva, 1999). Averaging over multiple trials allows the ERP pattern of interest to emerge from the surrounding noise, thus enhancing the SNR (Pfurtscheller and Lopes da Silva, 1999). ERPs have been widely used to model neural behaviour related to stimuli or events (Pfurtscheller and Lopes da Silva, 1999). However, instead of analysing time-series ERPs, many studies have relied on analysing changes in the power of specific frequency bands that are linked to event-related phenomena (such as motor tasks) (Kalcher and Pfurtscheller, 1995; Pfurtscheller and Lopes da Silva, 1999). This approach is more suited to single-trial analysis, as explained in Section 3.2.3. The research in this thesis incorporated multi-trial ERP and single-trial analysis. There is a concern that BCIs developed for healthy participants may not work for motor- impaired individuals, since motor impairments may change the motor cortices of the brain and thus alter the neural activity associated with motor activity (Bajaj et al., 2015; Höller et al., 2013; Turner et al., 2001). However, there is evidence that activity in the motor areas of the brain is still present in motor-impaired individuals (Han et al., 2019; McFarland et al., 2010; Turner et al., 2001). It is thus possible to adapt a BCI developed for healthy participants to work for motor-impaired individuals. The research in this thesis therefore focussed on healthy participants only, with the intention of adapting the analysis in subsequent studies to the relevant end-users with motor disabilities (Allison et al., 2007). 3.2.1 High-resolution EEG Using high-resolution EEG (128 –256 electrodes) is a way of trying to overcome the low spatial resolution of EEG (He et al., 2013; Michel and Brunet, 2019; Wierzgała et al., 2018). As mentioned in Chapter 2, the use of 128-channel EEG is better suited to localising signals from Background information on the study components: EEG, BCIs and DCM 17 ROIs than lower-resolution EEG. This is particularly relevant to this research, since WE and WF activate the same ROIs (refer to Section 1.4, Figure 1.4.2). EEG systems can base their electrode placement on the 10-20 system. This allows for the positioning and numbering of 21 electrodes (Oostenveld and Praamstra, 2001). Higher- resolution EEG systems rely on variations of the 10-20 system, which fill the spaces between the electrodes of the 10-20 system (He et al., 2013). The 10-10 system allows for up to 74 electrodes, while the 10-5 system allows for up to 345 electrodes (Oostenveld and Praamstra, 2001). The 10-5 system, shown in Figure 3.2.2, was used for this thesis (Oostenveld and Praamstra, 2001). More information on the EEG system used for this research is provided in Section 6.2. Figure 3.2.2: 10-5 electrode placement system. !"#$"%&' (# $)* +,-+, '.'$*/0 1%$ 23'" 4"& $)* #*53. 6*7#*6 /*6(23832$*&23 !"#$"%&'9 :" ;(<* 2# *=2/>3*? $)* *3*!$&"6* 3"!2$("# 1*$5**# @A 2#6 @B 5"%36 1* !233*6 @B)0 $)* 3"!28 $("# 1*$5**# @CA 2#6 @CB 5"%36 1* @CB)9 :)* 3"!2$("# )23452. 1*$5**# @A) 2#6 @CA) 5"%36 1* !233*6 @@CA)9 :)(' '.'$*/ 4"& *3*!$&"6* >32!*/*#$ 2#6 $)* 2''"!(2$*6 #2/(#; '!)*/* ;(<* &('* $" 2 6"%13(#; "4 $)* #%/1*& "4 /*6(23832$*&23 *3*!$&"6* !"#$"%&'0 2#6 $" 2 6"%13(#; "4 $)* #%/1*& "4 *3*!$&"6*' 23"#; $)*'* !"#$"%&'9 :)*&*4"&*0 $)* $"$23 #%/1*& "4 *3*!$&"6* 3"!2$("#' 2<2(3213* (# $)* BD *3*!$&"6* '.'$*/ (' 2>>&"=(/2$*3. E $(/*' $)* #%/1*& 2<2(3213* (# $)* +,-+, '.'$*/9 :)* $"$23 #%/1*& "4 3"!28 $("#' (# $)(' '.'$*/ (' 2&"%#6 AEB0 6*>*#6(#; "# )"5 /2#. *3*!$&"6*' "# $)* /"'$ (#4*&("& &"5' FGHI 2#6 G+,IJ 2&* (#!3%6*69 K(;9 L ')"5' $)* 3"!2$("#' "4 $)* *3*!$&"6* '($*' 6*7#*6 2!!"&6(#; $" $)(' '.'$*/9 M32!N 733*6 !(&!3*' (#6(8 !2$* $)* *3*!$&"6* 3"!2$("#' "4 $)* +,-L, '.'$*/9 O3*!$&"6* '($*' "4 $)* +,-+, '.'$*/ 2&* (#6(!2$*6 1. ;&*. 733*6 !(&!3*'9 P#$*&/*6(2$* *3*!$&"6* '($*' 6*7#*6 1. $)* BD '.'$*/ 2&* /2&N*6 2' 6"$'9 P# $)(' 7;%&*0 2 '*3*!$("# "4 $)* BD *3*!$&"6* 3"!2$("#' (' (#6(!2$*6 1. ">*# !(&!3*' (#!3%6(#; $)*(& *3*!$&"6* #2/*'9 :)(' '*3*!$("# "4 *3*!$&"8 6*'0 !"/1(#*6 5($) $)* +,-+, '.'$*/0 3*26' $" 2 '%1'*$ "4 +EL *3*!$&"6*' 5($) 2 !"/>3*$* 2#6 )"/";*#"%' !"<*&2;* "4 $)* )*26Q '** K(;'9 A-B9 :)* !"/1(#2$("# "4 $)* >"'(8 $("#' "4 $)* +,-+, '.'$*/ 5($) $)*'* 266($("#23 BD >"'(8 $("#' 5"%36 >&"<(6* 2 ;""6 '$2&$(#; >"(#$ 4"& +LR !)2##*3 OOS /*2'%&*/*#$'0 5)*&* >"''(13. '"/* "4 $)* 3"5*'$ *3*!$&"6*' !2# 1* 'N(>>*69 T%* $" $)* #2$%&* "4 $)* 6*7#($("# "4 $)* +,-L, '.'$*/0 $)* *3*!$&"6* &"5' 2$ $)* /"'$ "!!(>($23 2#6 4&"#$23 >2&$ "4 $)* )*26 FCU *# VKJ 1*!"/* <*&. !&"56*69 C&"1213. 4"& $)(' &*2'"#0 $)* &*>&*'*#$2$("# "4 $)* +,-+, '.'$*/ (# $)* VOW S%(6*3(#* FV/*&(!2# O3*!$&"*#!*>)23";&2>)(! W"!(*$.0 +HHEJ >32!*' "#3. "#* *3*!$&"6* 1*$5**# CUX 2#6 CUY F'(/(32& 4"& CUX8CUR 2#6 $)* VK &"5J9 C32!(#; $)&** *3*!$&"6*' 1*$5**# CUX 2#6 CUY (' )2&63. 4*2'(13* (# >&2!8 $(!*9 :)* >&2;/2$(! '"3%$("# (' $)2$ $5" *3*!$&"6*' 2&* >32!*6 (# 1*$5**# CUX 2#6 CUY9 P# $)2$ !2'*0 $)* ">$(/23 6('$&(1%8 $("# "4 $)* *3*!$&"6*' F5($) 6('$2#!*' "4 AAD 1*$5**# *3*!8 $&"6*'J 6"*' #"$ /2$!) 5($) $)* 2<2(3213* 321*3' 4"& $)* $)&** 3"!2$("#' 5)(!) 2&* 6*7#*6 2$ LBD >"'($("#' 23"#; $)2$ &"5 FCU+0 CUA 2#6 CUBJ9 Z2/(#; $)*'* $5" *3*!$&"6*' 2!!"&68 (#; $" $5" "4 $)* $)&** 2<2(3213* 321*3' 4"& $)2$ &"5 (# $)* +,- L, '.'$*/ (' !3*2&3. #"$ 2>>&">&(2$*9 [* $)*&*4"&* 321*3 $)*'* *3*!$&"6*' 2!!"&6(#; $" $)* 3"!2$("#' (# $)* BD '.'$*/ !" #$%&'()'*+, -" -.//0%&./ 1 2*3(34/* 5'6.$789%3$*$:9 ;;< =<>>;? @;AB@;CY+\ K(;9 L9 O3*!$&"6* >"'($("#' (# $)* >&">"'*6 +,-B '.'$*/9 C"'($("#' 266($("#23 $" $)* +,-+, '.'$*/ 2&* (#6(!2$*6 5($) 6"$'Q 2 '*3*!$("# "4 266($("#23 >"'($("#' %'*4%3 4"& 2 +LR !)2##*3 OOS '.'$*/ 2&* (#6(!2$*6 5($) ">*# !(&!3*'9 Background information on the study components: EEG, BCIs and DCM 18 Commonly used electrode locations for the extraction of SMRs for hand (and sometimes foot) movements are C3, C4 and Cz according the 10-20 and 10-5 systems. These correspond to the hand and feet homunculi of M1 and are shown in Figure 1.4.2 (Oostenveld and Praamstra, 2001). 3.2.2 Neural source imaging and localisation As mentioned in Section 3.2, signals from ROIs propagate towards multiple scalp EEG electrodes and not just to those in close proximity to the ROIs (Hamedi et al., 2016; Hassan and Wendling, 2018). Source localisation techniques use the recorded scalp EEG signals to estimate the signals originating from ROIs. The ‘forward solution’ is a term used to describe how a given current-producing electrical source in the brain produces an electrical potential field on the scalp that is recorded by EEG electrodes (Michel and Brunet, 2019; Michel and He, 2019). The forward solution uses a spherical head model. The more realistic and complicated models consider the different conductivities of the tissues of the brain and head, the exact head shape and the non- homogenous thickness of the scalp across the head (Michel and Brunet, 2019; Michel and He, 2019). The individual MRIs (magnetic resonance images) of each participant can be used to incorporate the above considerations into the model more accurately (Michel and He, 2019). If this is not possible, a less precise template MRI can be used (Michel and Brunet, 2019). Methods used to find the EEG forward solution include the boundary element method (BEM), finite element method (FEM) and spherical head model with anatomical constraints (SMAC) (Michel and He, 2019). The inverse problem is the opposite of the forward problem/solution. The former tries to determine the neural sources inside the brain that are responsible for generating a given EEG measurement (Michel and He, 2019). There is no unique solution to the inverse problem. Incorporating a priori knowledge about the brain sources allows a solution to be found (Michel and He, 2019). The inverse solution has been commonly modelled using the MSP (multiple sparse priors), MN (minimum morm), LORETA (low resolution electromagnetic tomography) and LAURA (local autoregressive average) approaches. Background information on the study components: EEG, BCIs and DCM 19 The forward and inverse solutions mentioned above consider the spatial relationships between signals from neural sources and the electrical potential fields on the scalp at a given instant in time. When the signals are time-varying (as is the case with SMRs), the associated scalp potential fields also vary with time. Therefore, spatiotemporal source imaging techniques have been developed. Two examples of such techniques are independent component analysis (ICA) and beamformer analysis (Michel and He, 2019). These techniques are also referred to as spatial filters. They are used to enhance signals originating from local cortical ROIs and attenuate signals from distant sources (Bashashati et al., 2007; van Vliet et al., 2016). Spatial filtering techniques aim to improve the SNR of EEG (Liao et al., 2017). ICA was used as a spatial filter for the research in this thesis. ICA is a statistical technique that decomposes measured signals, consisting of a mixture of source signals, such as EEG, into their underlying independent components (ICs) which represent the original source signals (Makeig et al., 2004; Wang and James, 2007). The mixture of signals is assumed to be a linear mixture of statistically independent source signals (Makeig et al., 2004). Following its first application to EEG by Makeig et al. (1995), ICA is now widely used in the EEG and BCI research community (Delorme et al., 2012). It is commonly used to separate contaminating artifacts from EEG. Its use to separate and study activity from separable neural processes is increasing (Delorme et al., 2012). In principle, ICA can be used to isolate independent cortical activity from ROIs linked to motor control (Makeig et al., 2004; Wang and James, 2007). It has been used to filter signals from spatially adjacent ROIs, i.e., the hand and wrist homunculi of M1-H (as shown in Figure 1.4.2, Section 1.4) (Mohamed et al., 2011a; Vuckovic and Sepulveda, 2008). ICA can be implemented using several algorithms, such as infomax, JADE and FastICA (Delorme and Makeig, 2004). 3.2.3 EEG frequencies and sensorimotor rhythms EEG is typically divided into the frequency bands shown in Table 3.2 (Gull et al., 2017; Khan and Sepulveda, 2010; Niedermeyer and Lopes da Silva, 2005; Seeber et al., 2015; Yang et al., 2020). SMRs are prominent electrophysiological features associated with the brain’s normal motor output channels. They are typically associated with oscillatory activity in the mu and beta rhythms (Nicolas-Alonso and Gomez-Gil, 2012; Wierzgała et al., 2018). Wierzgała et al. (2018) reviewed 131 motor-imagery BCI articles between 1997 and 2017. Most of the studies in these articles relied on features extracted from the mu and beta frequency ranges and Background information on the study components: EEG, BCIs and DCM 20 associated with M1. Additionally, rhythms from the delta, theta and gamma bands have also been linked to movement control, as explained in Section 5.2. Table 3.2: Typical frequency bands associated with EEG. Frequency Band Frequency (Hz) delta 0–3.5 theta 4–7 alpha 8–13 mu 8–12 beta 14–34 gamma > 35 low-gamma 35–50 Hz high-gamma 51–90 Hz SMRs are synchronised when no sensory inputs or motor outputs are being processed. This is called event-related synchronisation (ERS). The rhythms desynchronise upon sensory stimuli, movement actuation or movement preparation. This is called event-related desynchronisation (ERD) (Bashashati et al., 2007; Wolpaw et al., 2002). ERD begins in the contralateral Rolandic region (M1) about 2 s prior to movement onset and becomes bilaterally symmetrical just before movement execution (Bashashati et al., 2007; Pfurtscheller and Lopes da Silva, 1999). ERS occurs after movement when the rhythms increase again (Bashashati et al., 2007; Wolpaw et al., 2002). Beta ERS occurs about 1 s post movement in contralateral M1 (Pfurtscheller and Lopes da Silva, 1999). ERD and ERS are robust oscillatory brain signals with relatively good SNRs and are prevalent in most research participants (Pfurtscheller and Lopes da Silva, 1999). ERD and ERS patterns also occur during imagined movements; hence a motor-impaired individual can imagine a particular movement to actuate a bionic hand (Wolpaw et al., 2002; Zecca et al., 2002). Features based on ERD/ERS have been used successfully to discriminate between various types of wrist and hand movements (Gu et al., 2009; Mohamed et al., 2011a, 2011b; Vuckovic and Sepulveda, 2008). The use of the terms ERD/ERS in this research refers to the use of mu and beta frequencies, unless otherwise specified. Traditional ERD/ERS analysis involves averaging filtered envelopes over many trials, allowing patterns to emerge (Başar et al., 1999; Kalcher and Pfurtscheller, 1995; Pfurtscheller and Lopes da Silva, 1999). However, for single-trial analysis, time-frequency (TF) techniques are often used to successfully capture the relative changes in power over time within mu and Background information on the study components: EEG, BCIs and DCM 21 beta frequency sub-bands (Åberg and Wessberg, 2007; Bashashati et al., 2007; Gu et al., 2009; Vuckovic and Sepulveda, 2008). SMRs are weak in comparison to other signals that occur in the same frequency range e.g. the α-rhythm from the visual cortex or EMG artifacts (Blankertz et al., 2008). Hence spatial filters and source localisation are used to enhance these features (Mohamed et al., 2011a; Mohamed and John, 2014). 3.3 Brain-computer interfaces A brain-computer interface (BCI) provides a communication channel from the brain to the external world, circumventing the natural neuro-muscular pathway (Lotte et al., 2007; Wolpaw et al., 2002). BCI systems aim to assist people who suffer from neuromuscular disorders or motor disabilities. Such people suffer from spinal cord injuries, brainstem strokes, multiple sclerosis, amyotrophic lateral sclerosis, Parkinson’s disease and limb amputations. BCIs aim to give these people control of assistive devices, such as simple word processors, speech synthesizers, wheelchairs, prosthetics and orthotics (Bashashati et al., 2007; Wolpaw et al., 2002). The effective control of these assistive devices could improve the lives of people with these particular conditions. BCIs are categorised according to the types of neural information they extract. Categories of BCIs include sensorimotor, P300, visually-evoked potentials, slow cortical potentials and hybrid BCIs (Abiri et al., 2019; Rohm et al., 2013). Sensorimotor BCIs rely on SMRs (described in detail in Section 3.2.3) or other neural information from the sensorimotor cortex (shown in Figure 1.4.2, Section 1.4) to control a bionic hand (Afshar and Matsuoka, 2004; C. W. Chen et al., 2009; Morash et al., 2008). For users of BCIs who have some movement of their hand, neural information related to real hand movements can be extracted (Abiri et al., 2019). In cases where BCI users are unable to move their hands, the neural activity from the imagination of a hand movement can be extracted (Abiri et al., 2019). The research in this thesis will focus on real wrist movements only, which enables the movements to be normalised against their respective MVCs. Subsequent studies can explore the adaption of the analysis to imagined movements. Background information on the study components: EEG, BCIs and DCM 22 BCIs can operate in an asynchronously or synchronously (Leeb et al., 2007). Synchronous BCIs rely on external cues to instruct the user when to perform a task. The cues in a synchronous BCI demarcate the periods of the recorded EEG that must be analysed (Leeb et al., 2007). (Refer to Figure 6.6.2, Section 6.6 for an example of external cues.) Hence, synchronous BCIs are suitable for laboratory investigations — like the one described in this research (Leeb et al., 2007). A successful synchronous BCI can be subsequently adapted to an asynchronous BCI, which will be more suitable for real world applications, such as the control of a bionic hand in everyday life (Leeb et al., 2007; Pfurtscheller et al., 2005). Using an asynchronous or self-paced BCI, where the user decides when to perform the task, is more complicated (Leeb et al., 2007). The BCI needs to differentiate between periods when the user intends to control the BCI and periods of non-control. During periods of non-control, the user is engaged in activities not related to the intention to control the BCI, such as thinking or daydreaming (Leeb et al., 2007). Furthermore, during periods of control, an asynchronous BCIs must distinguish between different types of control according to the user’s intention (Leeb et al., 2007). Neural information used by the BCI can be extracted using various neuroimaging methods, such as EEG, magnetoencephalography (MEG), electrocorticography (ECoG), functional magnetic resonance imaging (fMRI), near infrared spectroscopy (NIRS) and intracortical microelectrode recording (spike potentials) (Nicolas-Alonso and Gomez-Gil, 2012; Pistohl et al., 2012). Table 3.3 compares the use EEG to other electrophysiological instrumentation methods for use in a BCI. Table 3.3: Comparing electrophysiological instrumentation methods for BCIs (adapted from Nicolas-Alonso and Gomez-Gil (2012) and Pistohl et al. (2012)). Neuroimaging method Type of physiological activity measured Direct/ Indirect Temporal resolution Spatial resolution Risk Portability EEG Electrical Direct ~0.05 s ~10 mm Non- invasive Potentially portable ECOG Electrical Direct ~0.003 s ~1 mm Invasive Potentially portable intracortical microelectrode recording spike Electrical Direct ~0.003 s ~0.5 mm – 0.005 mm Invasive Potentially portable MEG Magnetic Direct ~0.05 s ~5 mm Non- invasive Non- portable fMRI Metabolic blood oxygen levels Indirect ~1 s ~1 mm Non- invasive Non- portable NIRS Metabolic blood oxygen levels Indirect ~1 s ~5 mm Non- invasive Potentially portable Background information on the study components: EEG, BCIs and DCM 23 Like EEG, MEG is non-invasive and records the activity of large neuronal populations (Waldert et al., 2009). It measures the magnetic fields produced outside the head as a result of intracellular currents flowing through dendrites (Waldert et al., 2009). Unfortunately, MEG is too expensive and bulky to be used in everyday life (Nicolas-Alonso and Gomez-Gil, 2012). ECoG involves the placement of an electrode grid under the scalp onto the cerebral cortex to measure electrical signal activity from the brain (Nicolas-Alonso and Gomez-Gil, 2012). Compared to EEG, it provides a higher spatial and temporal resolution, lower SNR, lower susceptibility to artifacts and higher signal amplitudes (Ball et al., 2009; Nicolas-Alonso and Gomez-Gil, 2012). However, ECoG requires a craniotomy (exposing the surface of the brain by surgically removing a piece of the scalp), which carries numerous health risks (Nicolas- Alonso and Gomez-Gil, 2012; Wang et al., 2016). Hence, ECoG is commonly recorded from humans who have had the electrode grid implanted in preparation for drug-resistant epilepsy treatment (Jerbi et al., 2011). Analysis is thus limited to cortical areas in proximity to this grid (Wang et al., 2016). Furthermore, the long-term reliability of the ECoG electrodes and the signals they acquire have not yet been fully established (Nicolas-Alonso and Gomez-Gil, 2012; Wang et al., 2016). Ung et al. (2017) reported that signal features from implanted electrodes declined in the first three months post-surgery. Intracortical neuron recordings involve placing microelectrodes into the cortex (grey matter) to capture spike potentials and local field potentials (LFP) from neurons close to the electrodes’ tips (Nicolas-Alonso