Dynamic Data-Driven Brain-Machine Interface (DDDBMI)

     Several Brain Machine Interface Researches have been conducted in order to assist the human beings with limited motion behavior. However none of them has provided a seamless integration and natural functionality between the human brain and the prosthetic device in the way the brain and the body in the healthy human behaves. This project focuses on cognitive brain modeling from experiments with live subjects and the design of brain-inspired assistive systems for human beings with severe motor behavior limitations (e.g. paraplegics) through Brain Machine Interface. For the BMI to work naturally with human, the DDDBMI system would need to provide good models of brain motor control and movement plannings for processing the brain signals along with necessary adaptive algorithms incorporating artificially-created desired signals instead of the actual subject's position which is not present in the case of paraplegics. This project will contribute to the design of BMI systems that assist paraplegics in gaining control of artificial limbs.

     The baseline DDDBMI architecture is inspired by the work of Kawato and others [] which strongly suggest that the brain selectively uses multiple kinematic and dynamic internal models for movement planning and control. This leads to the need for a dynamically data-driven computational system with Multiple Paired Forward-Inverse Model (MPFIM). The MPFIM consists of multiple pairs of models, each comprising a forward model (for movement planning in the premotor cortex) and an inverse model (for movement execution in the primary motor cortex).

     Forward models predict how the motor system’s state changes in response to a given motor command, whereas inverse models devises the commands needed to accomplish the desired action. Individual model-pairs or combinations of model-pairs are used to control motion on the basis of realtime feedback data provided by sensors (visual or proprioceptive).

     Our proposal is to use MPFIM as the basis of DDDBMIs, where, some of the signals of the figure are generated by the motor, premotor and parietal cortices, and other signals are generated by the robot controller and movement sensors, while all the computational components are implemented using Grid computing. The premotor and motor cortex electrodes will be supplying the efferent copy and the desired trajectory respectively, while the posterior parietal will be providing the contextual signals of the figure. Movement sensors (e.g. a camera and object recognition software) will supply the desired trajectory (sensory feedback signals) of the figure. The robot controller will receive the feedforward motor command and provide the feedback motor command from position sensors.

     DDDBMI systems for the envisioned applications will have computational demands that exceed by orders of magnitude the capacity of a single powerful workstation. Depending on the scope of the system, it may be necessary to use hundreds or even thousands of processors to provide the needed computational power. Grid-computing infrastructures can potentially deliver these necessary resources on demand. However, a DDDBMI system has stringent real-time requirements, as a result of the need for low latency between brain signaling and sensory feedback. Research is thus needed to develop middleware that can aggregate resources, create appropriate parallel execution environments and also guarantee the necessary Quality-of-Service (QoS) in computation and communication to meet response deadlines associated with the above-mentioned latencies. A unique proven Grid-middleware infrastructure developed and deployed by the PIs, called In-VIGO, will provide the framework within which we will develop the new middleware techniques needed for DDDBMIs.

     Two broad scenarios will be considered for the computational infrastructure needed by DDDBMIs. One is the online scenario extensively discussed in this proposal, where real-time computation is needed for in-vivo experimentation. The other is the offline scenario where data from past experiments is “replayed” and analyzed, e.g. to generate statistics or train models. In both scenarios, a key requirement is the computation of hundreds of modules as discussed above. We envision In-VIGO middleware being used to provide BMI researchers with a Web-based interface that would allow them to specify the models they wish to experiment with and possibly additional information (e.g. desired start time, explicit QoS requirements and data collection/storage). In the offline case they would also specify which experiment(s) to replay. Our research will focus on the challenge of having In-VIGO automatically set up the necessary resources. In the online case this includes setting up the necessary connectivity among resources and the data acquisition system, and guaranteeing QoS requirements. In the offline case, a virtual application will have to be created to re-create experiments from stored data, but data rates can be slowed down, thus removing hard deadlines on computation.

Top  

      Research on middleware will focus on the VAS, RM and VMS sub-systems of In-VIGO to support two key needs of DDDBMI:

     (1) resource discovery based on quality of service specifications and scheduling based on virtual machine reservations and

     (2) dynamic steering of applications to computing resources based on run-time feedback from application inputs.

     We aim to support large number of concurrent models to execute in a data parallel fashion, without requiring all models to be subject to a stringent QoS guarantee, by selecting, based on application inputs and expected errors locally computed by each model, a smaller number of models that is the most representative in terms of the weights provided to the responsibility predictor. Current progress in the middleware could be found here.

Top  

 

     Research on adaptive algorithms focuses on new data models and learning algorithms.

     Our present work performs a discrete-to-continuous transformation in the data by binning the spike train data in 50 ms segments, and then derives “black box” types of models to predict the trajectory of the hand positions in 3D space. We propose here to investigate two different implementations:

     (1) a continuous variable implementation and
     (2) spike train data implementation.
In the continuous-variables approach the goal of each model is to predict the input to be fed to the dynamical model (robot) such that it will reduce the difference between the artificial hand and the object. In the discrete-variable approach we work directly with spike trains (which are point processes), and derive optimal predictive models directly from them. Current progress in the adaptive algorithms could be found here.

Top  

 

     Research on motor system concentrates on cognitive models of motor control and advancing our understanding of the neurobiology of movement.

     BMIs give us a unique avenue to research these principles because motor control is the most approachable of the cognitive human abilities. We will exploit these conceptual connections to produce a framework where multiple interpretations of incomplete information can be maintained and multi-sensory Gestalts can be integrated and translated in inference-based models with the ultimate goal of predicting actions of robotic devices observed through sensors in the external world. Current progress in the motor system research could be found here.

 

 

 

 

     The integration of the proposed Grid-computing infrastructure with neural decoding models will be used (and tested) in vivo to implement an animal experimental BMI behavioral paradigm. The goal is to create a controlled environment that mimics the workspace of an individual with upper limb paralysis. For our animal experimental
paradigm we have developed a three-target, two-state behavioral paradigm. In this environment, the animal will have to choose to press (with its front paw) one of three levers as cued by an LED to obtain a water reward. For BMI experiments, we propose to modify the paradigm once the animal is trained by placing the levers outside the cage and introducing a 6 DOF robotic arm that will be neurally controlled to press the correct lever and achieve the reward. The robotic arm will be used as a neurallycontrolled extension of the animal’s body which will be used to press the lever instead of the forelimb.

     In this closed-loop brain machine interface paradigm, the animal must learn to control the robotic arm to achieve the lever reward with its neural activity alone.

     The computationally complex component of this paradigm is using arm position (sensory) information in conjunction with model selection in real-time. The success of the modeling will be measured using standard tools such as the confusion matrix. We are seeking to determine the optimal signal processing technique that will produce the fewest lever classification errors per unit time and the highest probability of producing the correct lever class. Current progress in the animal studies could be found here.

Top