Deep Learning Models to Predict Prosthetic Ankle Torque

Version	Summary	Created by	Modification	Content Size	Created at	Operation
1		Christopher Prasanna	--	2088	2023-09-22 20:43:23	\|
2	layout	Camila Xu	-225 word(s)	1863	2023-09-25 03:21:40	\|

This entry is adapted from the peer-reviewed paper 10.3390/s23187712

Inverse dynamics from motion capture is the most common technique for acquiring biomechanical kinetic data. However, this method is time-intensive, limited to a gait laboratory setting, and requires a large array of reflective markers to be attached to the body. A practical alternative must be developed to provide biomechanical information to high-bandwidth prosthesis control systems to enable predictive controllers.

biomechanics machine learning deep neural networks robotic ankle prosthesis

Introduction

Use of a practical model that maps from control commands and easily observed states to a controllable future state of a system, such as the torque about a powered ankle–foot prosthesis (PAFP), has the potential to improve the mobility of individuals with lower limb loss ^[1]. By simulating PAFP dynamics over a period of time for different control commands, a model can provide information to search for a sequence of control commands that achieve a desired prosthesis behavior, such as time-varying impedance parameters or terrain-specific angle-torque profiles. Real-time model predictive control (MPC) has been proposed to control assistive robotic devices during physical interaction with humans, allowing the robot to adapt its behavior ^[2]^[3]. Integrating MPC with real-time feedback from sensors into PAFP control schemes may enable the prosthesis to better adapt to the user’s movements, allowing for more natural and intuitive interactions between the user, device, and environment. A key benefit of this approach is the optimization of robot behavior over a finite time horizon by finding the optimal control inputs that minimize a cost function. This allows a controller to achieve a desired task, such as tracking a desired angle-torque profile while rejecting disturbances and minimizing energy consumption. Additionally, MPC can be designed to ensure the robot operates within the constraints of the system, such as joint limits and actuator saturation. For prosthetic applications, such constraints can be essential for safe and robust operation.

In terms of modeling PAFPs, control commands (e.g., motor current) can be mapped to motor torque for some devices including series elastic actuators. For parallel elastic actuators, the passive dynamics play a critical role in the total ankle torque (i.e., the total prosthetic ankle torque is the summation of the active and passive components). In addition to the passive elements of the prosthesis, there are other important factors, including variations in limb loading, shoe/keel mechanics, the dynamics of the other limbs and joints, and environmental conditions that can influence prosthetic ankle torque. Furthermore, even under steady-state walking conditions, stride-to-stride variations are inevitable, and it is important for prosthesis controllers to compensate for this variability ^[4].

Developing a method to predict prosthetic torques could better allow a PAFP controller (e.g., model predictive controller) to adapt to these factors that are unaccounted for in modern prosthetic control strategies ^[5].

2. The Role of System Models for Robotic Controllers and the Need for Better Modeling Approaches

A dynamical model of a robot prosthesis system would enable the use of model-based control methods, which use forward predictions to adjust control commands that drive the system toward a desired behavior (Figure 1). Model-based controllers that provide a distribution of future states (e.g., prosthetic ankle torque profiles) and their costs (e.g., the error between desired ankle torque profile and the predicted torque profile) based on candidate control commands are, when presented with limited environmental interactions, typically more sample-efficient and converge (i.e., minimize cost) more quickly than model-free techniques ^[6]^[7]. This is because a model-based controller is able to leverage the system model to focus the control command search space ^[8]. Model-free control strategies require more time to minimize cost (e.g., require more walking trials to train the controller and achieve a desired torque profile). In some cases, a model-based control law can be used as an initialization for a model-free learner (e.g., ^[9]). The model-free learner can then fine-tune the control law to overcome any model uncertainties. Despite their promise, model-based control systems have not been implemented in robotic prosthesis control because modeling human prosthesis dynamical behavior remains a challenge.

Figure 1. Block diagram of the architecture of a generic model-based prosthesis control system. A trajectory generator outputs a desired ankle torque

y_{d}

, which is then fed into an optimizer. The optimizer samples different possible control commands

u_{k}

and conducts forward simulations based on system model predictions. The optimizer uses the results to determine which control command would achieve the closest one-sample ahead ankle torque response

y_{k + 1}

to the desired behavior. This control command is then sent to the human–robot system and the measurements of the system

x_{k}

(e.g., loading and motion information from wearable sensors) are fed back to the optimizer for the next control sample.

Deep neural networks (DNNs) provide a promising modeling strategy for model-based control of prosthetic ankles and are state of the art in other challenging learning tasks, such as natural language processing ^[10] and computer vision ^[11] that also involve high-dimensional nonlinear relationships. Once a DNN is sufficiently trained to model the human–robot system, it can be deployed to a real-time prosthesis control system and map system state measurements to an output prediction (e.g., ankle torque response), all without explicit knowledge of the system’s physics.

The capability of DNNs to output long-term predictions presents another key advantage in the context of prosthetic control. In certain cases, modeling approaches based on physics or first principles can provide insight toward system input–output relationships. However, it is often challenging and time-consuming to characterize system behaviors using first principles for systems with nonstationary dynamics that are highly nonlinear and vary over time (e.g., a human-prosthesis system). When mathematical process models are not known, DNN architectures have demonstrated an ability to accurately predict long-term behaviors including robotic trajectory ^[12] and human behavior prediction ^[13]. If high-fidelity DNN models that predict human-prosthetic dynamics can be developed, then it would be possible to proactively adjust prosthesis control actions to achieve a desired long-term trajectory or prepare for gait transitions.

2. Current Data-Driven Approaches in Prosthetic Control

Most data-driven biomechatronics research up to this point has been used to build models that relate environmental, system, and user data to intent recognition ^[14]^[15]^[16] or gait phase estimation ^[17] rather than to build predictive regression models of human–robot dynamical systems (e.g., estimating joint kinetics that cannot be observed instantaneously). A predictive model that outputs these dynamics directly can inform controllers and improve their performance (e.g., adapt the control inputs to achieve a desired ankle torque response). In addition, a model-based prosthesis controller can utilize joint dynamics information even outside the laboratory where motion capture (MoCap) is not available and inverse dynamics computations are more challenging.

Some early studies used neural networks and electromyography signal inputs to predict ankle dynamics; however, their predictions were noisy and less accurate when compared to a muscle model ^[18]^[19]. One explanation is that their shallow neural network architecture did not take the dynamic spatial–temporal relationships of human–robot systems into account. More recently, studies have successfully used more advanced neural network classes such as recurrent neural networks (RNNs) ^[20] and attention-based long short-term memory networks (LSTMs) ^[21]^[22] to generate reference trajectories for prosthesis controllers based on able-bodied data. Others have used wearable sensors and machine learning models to predict joint moments for exoskeleton control, also using able-bodied data ^[23]. However, deep learning models have not been used to predict prosthesis joint dynamics directly for individuals with transtibial amputation.

3. Deep Learning Modeling Approach

The field of wearable robotics is increasingly incorporating deep learning to improve control systems and user adaptability ^[24]. Recent studies have applied deep learning for intuitive control of powered knee–ankle prostheses ^[25], estimating joint moments and ground reaction forces in various walking conditions ^[26], and predicting joint moments in real time for exoskeleton controllers ^[27]^[28]. This signifies a growing trend towards more responsive and adaptive assistive technologies, yet gaps remain in understanding the potential and limitations of deep learning methods for PAFP control.

The development of models that can predict ankle mechanics is a key milestone in implementing a model-based control strategy for prosthetic devices, and this research investigates different model architectures for predicting prosthetic ankle torques. The models are trained and evaluated based on how well they can predict prosthetic ankle torque values from previously collected data. These methods use only a small number of model input features and do not rely on a full-body MoCap suit or a large array of electromyography sensors. Experimental data in the form of total prosthetic ankle torques from optical MoCap (target outputs) and system state data (predictor inputs) from optical MoCap, force plate measurements, and on-board sensors are used to train the predictive models.

The output of the models was chosen to be the forward predictions of the prosthetic ankle torque. These output targets were computed using the full-body MoCap marker set (Vicon) with inverse dynamics principles to derive the total torque value at the ankle based on the movements of all anatomical structures that affect the joint. The reason for choosing the prosthetic ankle torque as the time series of interest is because many robotic prosthesis control methods aim to achieve a desired ankle torque or deploy a mathematical formula based on ankle impedance (i.e., prescribing an assistive ankle torque based on changes in ankle position).

Three DNN architectures were developed and trained: (i) a simple multilayer feedforward network (FFN), which passes information in one direction ^[28]; (ii) a gated recurrent unit network (GRU), which includes feedback connections and gates used to keep track of long-term dependencies in the input sequences ^[29]; and (iii) a dual-stage attention-based gated recurrent unit network (DA-GRU), which shares the capabilities of the GRU and adds attention mechanisms that assign weights to the different elements of the input sequence based on their relevance to learning the given task ^[30]. Each DNN was trained to predict ankle torques from inverse dynamics a short time period (one sample) into the future. The one-sample-ahead prediction performances of the DNNs and an analytical model of the PAFP system, which was derived based on first principles, are then directly compared to PAFP torque computed from MoCap and inverse dynamics. Additionally, researchers assess the long-term predictive capabilities of neural networks that could realistically be applied to prosthetic control. Each DNN was trained to predict twenty samples, approximately half a gait cycle, into the future to compare each DNN’s ability to predict prosthetic ankle torques. Half a gait cycle was chosen, as it could enable a prosthesis controller to proactively respond to stride-to-stride variations or gait transitions.

In summary, while most prosthetic applications have yet to incorporate predictive models, advanced feedback control systems stand to benefit significantly from them, especially for managing the complex human–robot interactions in prosthetic ankle–foot devices. One of the primary challenges in predicting prosthetic ankle torque stems from the fact that it is influenced by both the control action of the device and the variable kinematic pattern adopted by the user when using devices that incorporate parallel elastic elements. Traditional physics-based models struggle to account for these complexities, making them less effective for real-time control. The main contribution of this research is the development and comparison of deep learning models that are specifically designed to predict this shared human–robot state, thus paving the way for more adaptive and intuitive prosthetic control systems in the future.

References

Nguyen-Tuong, D.; Peters, J. Model learning for robot control: A survey. Cogn. Process. 2011, 12, 319–340.
Teramae, T.; Noda, T.; Morimoto, J. EMG-Based Model Predictive Control for Physical Human–Robot Interaction: Application for Assist-As-Needed Control. IEEE Robot. Autom. Lett. 2018, 3, 210–217.
Özen, O.; Traversa, F.; Gadi, S.; Buetler, K.A.; Nef, T.; Marchal-Crespo, L. Multi-purpose Robotic Training Strategies for Neurorehabilitation with Model Predictive Controllers. In Proceedings of the 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR), Toroto, ON, Canada, 24–28 June 2019; pp. 754–759.
Winter, D.A. Kinematic and kinetic patterns in human gait: Variability and compensating effects. Hum. Mov. Sci. 1984, 3, 51–76.
Tucker, M.R.; Olivier, J.; Pagel, A.; Bleuler, H.; Bouri, M.; Lambercy, O.; Del Millán, J.R.; Riener, R.; Vallery, H.; Gassert, R. Control strategies for active lower extremity prosthetics and orthotics: A review. J. Neuroeng. Rehabil. 2015, 12, 1–30.
Boyan, J.A.; Moore, A.W. Generalization in Reinforcement Learning: Safely Approximating the Value Function. In Proceedings of the 7th International Conference on Neural Information Processing Systems, Denver, CO, USA, 1 January 1994; MIT Press: Cambridge, MA, USA, 1994; Volume 7, pp. 369–376.
Atkeson, C. Using Local Trajectory Optimizers to Speed Up Global Optimization in Dynamic Programming. In Proceedings of the Advances in Neural Information Processing Systems, Denver, CO, USA, 30 November–3 December 1993; Volume 6, pp. 663–670.
Rawlings, J.B.; Mayne, D.Q.; Diehl, M.M. Model Predictive Control: Theory, Computation, and Design, 2nd ed.; Nob Hill Publishing: San Francisco, CA, USA, 2017.
Sutton, R.S.; Mcallester, D.; Singh, S.; Mansour, Y. Policy gradient methods for reinforcement learning with function approximation. Adv. Neural Inf. Process. Syst. 2000, 12, 1057–1063.
Goldberg, Y. A primer on neural network models for natural language processing. J. Artif. Intell. Res. 2016, 57, 345–420.
Redmon, J.; Divvala, S.; Girshick, R.; Farhadi, A. You only look once: Unified, real-time object detection. In Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 27–30 June 2016; IEEE Computer Society: Washington, DC, USA, 2016; pp. 779–788.
Gomez-Gonzalez, S.; Prokudin, S.; Scholkopf, B.; Peters, J. Real time trajectory prediction using deep conditional generative models. IEEE Robot. Autom. Lett. 2020, 5, 970–976.
Almeida, A.; Azkune, G. Predicting human behaviour with recurrent neural networks. Appl. Sci. 2018, 8, 305.
Varol, H.A.; Sup, F.; Goldfarb, M. Multiclass real-time intent recognition of a powered lower limb prosthesis. IEEE Trans. Biomed. Eng. 2010, 57, 542–551.
Young, A.J.; Simon, A.M.; Fey, N.P.; Hargrove, L.J. Classifying the intent of novel users during human locomotion using powered lower limb prostheses. In Proceedings of the International IEEE/EMBS Conference on Neural Engineering, NER, San Diego, CA, USA, 6–8 November 2013; pp. 311–314.
Huang, H.; Zhang, F.; Hargrove, L.J.; Dou, Z.; Rogers, D.R.; Englehart, K.B. Continuous locomotion-mode identification for prosthetic legs based on neuromuscular-mechanical fusion. IEEE Trans. Biomed. Eng. 2011, 58, 2867–2875.
Vu, H.T.T.; Dong, D.; Cao, H.L.; Verstraten, T.; Lefeber, D.; Vanderborght, B.; Geeroms, J. A review of gait phase detection algorithms for lower limb prostheses. Sensors 2020, 20, 3972.
Au, S.K.; Bonato, P.; Herr, H. An EMG-position controlled system for an active ankle-foot prosthesis: An initial experimental study. In Proceedings of the 9th International Conference on Rehabilitation Robotics, Chicago, IL, USA, 28 June–1July 2005; pp. 375–379.
Au, S.; Berniker, M.; Herr, H. Powered ankle-foot prosthesis to assist level-ground and stair-descent gaits. Neural Netw. 2008, 21, 654–666.
Dedić, R.; Dindo, H. SmartLeg: An intelligent active robotic prosthesis for lower-limb amputees. In Proceedings of the 2011 23rd International Symposium on Information, Communication and Automation Technologies, ICAT, Sarajevo, Bosnia and Herzegovina, 27–29 October 2011; pp. 1–7.
Rai, V.; Sharma, A.; Preechayasomboon, P.; Rombokas, E. Coordinated movement for prosthesis reference trajectory generation: Temporal factors and attention. In Proceedings of the IEEE RAS and EMBS International Conference on Biomedical Robotics and Biomechatronics, New York, NY, USA, 29 November–1 December 2020; pp. 939–945.
Sharma, A.; Rombokas, E. Improving IMU-Based Prediction of Lower Limb Kinematics in Natural Environments Using Egocentric Optical Flow. IEEE Trans. Neural Syst. Rehabil. Eng. 2022, 30, 699–708.
Camargo, J.; Molinaro, D.; Young, A. Predicting biological joint moment during multiple ambulation tasks. J. Biomech. 2022, 134, 111020.
Dillen, A.; Steckelmacher, D.; Efthymiadis, K.; Langlois, K.; De Beir, A.; Marusic, U.; Vanderborght, B.; Nowe, A.; Meeusen, R.; Ghaffari, F.; et al. Deep learning for biosignal control: Insights from basic to real-time methods with recommendations. J. Neural Eng. 2022, 19, 011003.
Kim, M.; Simon, A.M.; Hargrove, L.J. Seamless and intuitive control of a powered prosthetic leg using deep neural network for transfemoral amputees. Wearable Technol. 2022, 3, E24.
Hossain, M.S.B.; Guo, Z.; Choi, H. Estimation of Lower Extremity Joint Moments and 3D Ground Reaction Forces Using IMU Sensors in Multiple Walking Conditions: A Deep Learning Approach. IEEE J. Biomed. Health Inform. 2023, 27, 2829–2840.
Molinaro, D.D.; Kang, I.; Camargo, J.; Gombolay, M.C.; Young, A.J. Subject-Independent, Biological Hip Moment Estimation During Multimodal Overground Ambulation Using Deep Learning. IEEE Trans. Med. Robot. Bionics 2022, 4, 219–229.
Schmidhuber, J. Deep learning in neural networks: An overview. Neural Netw. 2015, 61, 85–117.
Cho, K.; Van Merriënboer, B.; Gulcehre, C.; Bahdanau, D.; Bougares, F.; Schwenk, H.; Bengio, Y.; van Merrienboer, B.; Gulcehre, C.; Bahdanau, D.; et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Proceedings of the EMNLP 2014—2014 Conference on Empirical Methods in Natural Language Processing, Association for Computational Linguistics (ACL), Doha, Qatar, 25–29 October 2014; pp. 1724–1734.
Qin, Y.; Song, D.; Chen, H.; Cheng, W.; Jiang, G.; Cottrell, G. A dual-stage attention-based recurrent neural network for time series prediction. In Proceedings of the IJCAI International Joint Conference on Artificial Intelligence, Melbourne, Australia, 19–25 August 2017; pp. 2627–2633.

© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.

Upload a video for this entry

Information

Subjects: Robotics; Computer Science, Artificial Intelligence; Automation & Control Systems

Contributors MDPI registered users' name will be linked to their SciProfiles pages. To register with us, please refer to https://encyclopedia.pub/register :

Christopher Prasanna

Jonathan Realmuto

Anthony Anderson

Eric Rombokas

Glenn Klute

View Times: 296

Update Date: 25 Sep 2023

Table of Contents

Video Upload Options

Confirm