Learning from Biological Systems:

Modeling Neural Control

T

he biological neural system that controls posture and movement is a complex, nonlinear system [1]-[3]. It is remarkably adaptive and can easily accommodate changes in both environment and task specifications. A thorough understanding of the structure and function of this complex system has challenged biologists and engineers alike. Yet, this challenge is worth embracing for both biomedical and engineering purposes. Better knowledge of the functional structure of the central nervous system (CNS) can help to effectively treat diseases or injuries affecting posture and CONTROL SYSTEMS movement control. IN BIOLOGY C o n v e r s e l y, learning how the CNS controls complex movements and adapts to changing tasks and environmental perturbations will help engineers design better control systems. Experimental investigation has revealed much about the structure and organization of the central nervous system [4], [5]. For example, we know the pathways for information processing in the visual system, extensive circuit structures in the cerebellum, and certain connection pathways among various cerebral areas for motor control and in the spinal cord [6]-[8]. Advances in imaging and multichannel, chronic recording techniques have allowed the investigation of partial

functional structures of local circuits in the CNS from awake, normally functioning subjects [9], [10]. However, a more comprehensive investigation of the functional structures of the CNS remains a technical challenge. A computational approach, based on simulation, signal processing, and estimation, can be effective in examining hypothesized fundamental principles of neural control of movement. Computational models have been proposed and implemented to this end. These model development efforts have contributed to the understanding of functional structures of the CNS for motor control and, at the same time, have contributed to advances in the design of new controllers for robotic applications [11]-[14]. The neural control system has a hierarchical structure that provides the flexibility necessary for achieving the desired adaptation and robust stability (Fig. 1). At the bottom of the hierarchy is a musculoskeletal apparatus with sensory organs that provide measurements regarding the status of the system. Next are the local feedback loops for basic reflex actions in the spinal cord. These reflex loops are modulated in the spinal cord by a network of interneurons considered to be the regulators and pattern generators [4], [13], [15], [16]. Several command and control centers in the brain form the upper levels of the hierarchy as the supraspinal controller that receives the intention for action (referred to as volition). They include the brain stem, cerebellum, motor cortex, sen-

He ([email protected]) and Wang are with the Bioengineering Department at Arizona State University, Tempe, AZ 85287, U.S.A. Maltenfort and Hamm are with Barrow Neurological Institute, Phoenix, AZ 85013, U.S.A.

August 2001

0272-1708/01/$10.00©2001IEEE IEEE Control Systems Magazine

55

©1995 PHOTODISC, INC.

By Jiping He, Mitchell G. Maltenfort, Qingjun Wang, and Thomas M. Hamm

sory cortex, and many other areas in the CNS. Several ascending, descending, and propriospinal tracts communicate information among different levels in the hierarchy. Lesions or injuries that sever some of the connections disrupt normal function; insight into the functions of certain components of the CNS may be provided by such disruptions. Under certain conditions or assumptions based on the observed lesions or injuries, this complex system can be segregated into modules of simpler structures with extensive interconnections among the different modules: reflex loops, spinal circuits, cortical connections, sensory integration, and the dynamics of the musculoskeletal system. We have been pursuing a modular approach to modeling and investigating the neural control of posture and movement. A modular approach is a practical and natural way of investigating the function and structure of the complex CNS. The modules can include skeletal dynamics, muscle dynamics with recruitment and force-generation characteristics, basic spinal cord neural circuits responsible for the local stretch reflex, interneuronal networks that provide modulation of various reflexes, various structures of brain circuits, and control strategy decision-making centers. Fig. 1(b) shows a block diagram for investigating spinal control of posture and movement when supraspinal involvement can be assumed minimal. The anatomical detail and complexity of each module can vary depending on the specific scientific question or motor task to be investigated and the extent of information available on the hierarchy. To demonstrate the modular approach, we present two examples that illustrate how biological motor control tasks can be investigated in varying degrees of complexity and detail. One example shows that a model with detailed anatomical structure and dynamic properties has to be developed to investigate the detailed interactions among several sensory

feedback loops in the spinal cord. The other example shows that when investigating whole-body behavior, a grosser structured model with lumped components is more tractable so issues such as the interaction of spinal cord regulation and upper CNS intelligent control can be investigated.

Interactions of Sensory Feedback Pathways in the Spinal Cord In the local control circuit in the spinal cord for a single muscle, the α motoneuron is the command center that activates the muscle (also called extrafusal fibers in Fig. 2(a)). The output of the a motoneuron is regulated by several feedback loops (called reflexes by neurophysiologists), in addition to the inputs from supraspinal centers and spinal pattern generators (see Figs. 1 and 2). These loops include negative feedback from Renshaw cells (R in Fig. 2), also called recurrent inhibition (RI); muscle length feedback from spindles (intrafusal and Ia in Fig. 2); force feedback from Golgi tendon organs (GTO); and feedback from skin sensors and joint sensors (not shown in Fig. 2). Among these feedback loops, the sensitivity and gain of the spindle feedback is modulated by both γ (not shown) and β motoneurons. Fig. 2(b) shows the reciprocal connections among a pair of opposing muscles (agonist and antagonist, biceps and triceps, for example) on a joint (load). One of the many interneurons is shown as IaIN. Other interneuron connections are omitted in the figure for clarity of major connections. Interested readers can refer to [15]. It is still unclear how these complex sensory feedback loops and descending control signals are coordinated. The effect of each feedback loop seems obvious, but the coordination and interaction among these loops in modulating muscle force production are not clear. While the activities of α and γ motoneurons are potentially independent, the activity of β motoneurons is proportional to that of α. A stretch of

Volition GTO Function

Regulator

Spinal Cord Neural Circuit

Supraspinal Control and Pattern Generator

Motoneuron Dynamics

Muscle Dynamics

Limb Dynamics

Renshaw Cell

Neuro MusculoSkeletal Dynamics

Interneuron Dynamics

Spindle Dynamics

Sensory Information

(a)

(b)

Figure 1. Schematic structures of the neuromusculoskeletal system to be modeled and simulated: (a) is a more abstract, lumped model; (b) is a more detailed and frequently used model structure for spinal control of movement when the higher centers in (a) are further lumped as a control command.

56

IEEE Control Systems Magazine

August 2001

a muscle excites the spindle, which in turn excites both the α motoneurons inner vating the muscle and the β motoneurons innervating the spindle and muscle. Renshaw feedback, on the other hand, inhibits the a motoneurons [17]. Therefore, Renshaw feedback and the b motoneuron seem to have the opposite effect on muscle regulation [18]. Why do they coexist? It has been extremely difficult to carry out experimental investigation on the interactions of these reflex loops, though information on coordination and interaction of all reflex loops is crucial to our understanding of neural control of posture and movement. The modeling approach may provide some insights because it is easy to change parameters and structures during simulation; however, the model must contain enough detail in both anatomical structure and dynamic properties to be useful. Here, we describe a model based on the experimental measurement of every detailed reflex pathway and neuronal dynamics and show that the model can simulate experimental observations and generate predictions of interacting modulation of muscle behavior when both or either Renshaw feedback and β motoneuron are removed.

Model Development for a Detailed Neuromuscular System

rates. The transfer function from the combined synaptic current i to the motoneuron output m is given as:

(

)

m( s ) km 1 + s / 33 + ( s / 33 ) , = 2 i( s ) 1 + 2( s / 58 ) + ( s / 58 ) 2

(1)

where km is the static gain (≈ 1.5 pulses per second (p/s) per nA of injected current [19]).

Muscle Activation Dynamics The development of force by a muscle held isometrically (in constant length) depends on two factors: the release of calcium at the neuromuscular junction and the attachment and detachment of cross-bridges. It has been demonstrated that a second-order, critically damped system can approximate the dynamics of muscle force generation reasonably well [22], [23]. This model was later modified into two coupled first-order systems: a low-pass filter representing calcium dynamics q (2a) and a nonlinear activation dynamics a (2c), representing cross-bridge dynamics [24], [25]. The statedependent parameter b (2d) reflects the observation of muscle hysteresis and slower relaxation from larger forces: q& = 18.8496( m − q )

(2a)

Dynamics of Motoneuron Based on experimental observation [19]-[21], it is reasonable to model the response of the motoneuron as a bandpass filter spanning the relevant range of motoneuron firing

u=

q2 k2 + q 2

(2b)

Drive Load

Agonist

Antagonist

Intrafusal

O GT

R

Extrafusal

α/β la

α/β

α/β

R

R

Load

laIN la (a)

(b)

Figure 2. Detailed biological model represented as the neuro-musculoskeletal dynamics in Fig. 1. (a) Detailed connection pathways for a single muscle (extrafusal); α/β indicates motoneurons with innervations to both extrafusal (skeletal) and intrafusal (spindle) muscles. γ-drive to spindles (intrafusal) is not shown. (b) Illustration for a pair of muscles working against each other. Agonist indicates the active muscles and antagonist indicates the opposing muscles. R indicates Renshaw cells. GTO is the population of Golgi tendon organs sensing muscle force. Ia indicates primary pathways from spindles to muscle motoneurons. IaIN represents Ia reciprocal inhibitory interneurons. Filled circles indicate inhibition (reducing the activity of receiving neurons). Arrows indicate excitation (increasing activity of receiving neurons).

August 2001

IEEE Control Systems Magazine

57

a& = b(u − a)

(2c)

b = 22(1 − 0.51a) . 2

(2d)

The parameters m and k and initial conditions for q and a are selected to provide poles at 3 Hz at an active muscle force of 10 N (or 10% of maximum force, where a = 0.1). Letting k = 30 p/s produces a sigmoidal curve at which half the steady-state isometric force is produced at a motoneuron firing rate of 30 p/s. Heckman and Binder [26] and Fuglevand et al. [27] both constructed distributed models of motor nuclei that incorporate sigmoidal relationships between motoneuron force and firing rate and found that the overall relationship between isometric force and tonic drive to the motoneuron was also sigmoidal.

Muscle Mechanical Properties The muscle model follows a general structure described by Zajac [28]. The specific implementation is a modification based on He et al. [13], [29]. Active muscle force is defined as the product of the peak isometric force Fmax ( N ) with dimensionless quantities: Fact = Fmax × a × fl (l ) × fv (v ).

(3)

Fmax is assumed to be 100 N based on experimental values [30]. fv (v ) describes how normalized active force is dependent on whether the muscle is lengthening or shortening. fv can take values between zero and 1.8 and is based on hyperbolic equations for lengthening and shortening of the muscle: v / 1 + 4.0625v . 1 + 73125 fv (v ) =  v 1 + / 1 − 225 . v 

ν > 0 (stretching ) ν < 0 (shortening ). (4)

In these equations, the velocity v is normalized by the maximum speed of shortening (0.250 m/s in the case considered here). The formulation (4) assumes that the force-velocity function has a continuous derivative at zero. The assumption may not be valid for whole muscle, but it is necessary to perform certain theoretical analyses on the model. fl (l ) describes how the normalized active force output of muscle is dependent on its length l relative to its optimal length. It is based on an equation provided by Otten [25] and fit to data from Stephens et al. [31]:   l / 0.05 2.8286 − 1  2.3680  )   (  fl (l ) = exp  −  .   . 0 6182      

(5)

The use of an optimum fiber length of 0.05 m is somewhat arbitrary, although compatible with Spector et al. [30].

58

What is important is that the percentage change in tetanic tension relative to the difference between the muscle length and its optimum length matches experimental data. The passive muscle force increases exponentially with muscle stretch up to 56 mm, a length of 6 mm beyond the optimum length of the length-tension curve. The passive force is ∼5% of the maximum tetanic force at a length of 4 mm above the optimum length. We assume that this nonlinear behavior extends up to 13 N of passive force and that there is no passive force at 14 mm below the optimum length. Beyond 13 N of passive force, we assume the passive stiffness becomes linear. Using stochastic perturbations, Kirsch et al. [32] found that the passive muscle stiffness and viscosity estimates depended on the statistical properties of the perturbations. Both viscosity and passive muscle stiffness varied linearly with background force. Based on their estimates, the passive muscle viscosity is about 1% of the passive stiffness. The equations governing passive muscle force are thus: Fp = fp (l ) + Bp (l ) × v

(6a)

. 01663 ( exp{208.2(l − 0.036 )} −1) 0.036 < l < 0.057  fp (l ) =  13.0076 + 2743(l − 0.057) l ≥ 0.057  0 l ≤ 0.036 (6b) 

0.34624 exp{208.2(l − 0.036 )} l < 0.057 Bp (l ) =  274 . l ≥ 0.057. (6c)  We confirmed that the gain and phase of the passive force produced in response to a 16-Hz sinusoidal stretch matched that reported by Rosenthal et al. [33]. This transfer function greatly attenuated “ringing” due to differences in stiffness between muscle and tendon. The tendon (l t ) is assumed to be a nonlinear elastic element in which the stiffness increases linearly with force up to 25% of maximum (at 1.3 mm stretch) and remains constant thereafter:  2(exp{2000l t } − 1) l t < 0.0013 ft (l t ) =  24 . 9275 + 53855 ( − 0. 0013 ) l l  t ≥ 0.0013. t

(7)

Dynamics of Segmental Feedback Pathways There are three identified segmental feedback pathways in this simple system: Renshaw feedback (RI), spindle feedback (Ia), and Golgi tendon organ feedback (Ib). Delays were placed in all segmental and spinal pathways based on known (or estimated) conduction distances, conduction velocities, and synaptic delays.

IEEE Control Systems Magazine

August 2001

Dynamics of Renshaw Cell Feedback The dynamics of the Renshaw cell are represented as a nonlinear integrator followed by a linear function. The parameters of the transfer function described by (8) were obtained empirically [34] and have no clear relation to the cellular mechanisms responsible for Renshaw cell activity. The linear component is based on the observed static poles and zeroes in the frequency response of the Renshaw cells [34]. The pole of the nonlinear function increases linearly with motoneuron firing rate m:

purposes of this study, it is assumed to be a static gain and a delay. K GTO is set to 0.01 nA/N, giving synaptic current of 1 nA at 100 N maximum force.

Learning how the CNS controls complex movements and adapts to changing tasks and environmental perturbations will help engineers design better control systems.

3.25 3. 25(1 + 0.071 m) + s (1 + s / 7)(1 + s / 0. 36 ) . × (1 + s / 0. 48 )(1 + s / 120 π )

HR ( s) = K R ×

Spindle Structure and Function

(8)

The second factor on the right-hand side of (8) produces values of the rate-dependent pole comparable to those given by Windhorst [34] for mean stimulation rates of 30 to 40 p/s. We assume that the nonlinear dynamics seen at higher stimulus rates were exaggerated by the synchronized activation of synapses on Renshaw cells from shocks to ventral root filaments or motor nerves. The term also produces a saturation nonlinearity that reaches 50% of maximum at an input rate of 14.1 p/s. K R is used to set the static gain of RI. The effective synaptic current on medial gastrocnemius (MG) motoneurons at resting membrane potentials from Renshaw cells receiving excitations from other muscles is 0.4 nA; on motoneurons near threshold, it may be two or three times this quantity [36]. The RI of MG motoneurons will be twice that of the inhibition from other muscles, so the total possible steady-state recurrent inhibition may be as high as 2.4 nA.

The stretch seen by the sensory region in the spindle is proportional to the tension. This tension will increase with stretch velocity via a force-velocity relationship as described in (3) and (4) for the muscle. In steady state, both the sensory region and the contractile region are considered as stiff springs. The stiffness of the contractile region may increase with input from β and γ motoneurons. We begin with two equations from Hasan [37]: K ⋅ z = E ( y − c ) × fvi ( y& )

where K is the stiffness of the sensory region, E is the stiffness of the contractile region, x is the length of the muscle, y is the length of the spindle, z is the length of the sensory region, and c is the slack length of the spindle. c is assumed fixed at 35 mm below the maximum muscle stretch. This value was selected to produce reasonable behavior for the spindle during ramp-and-hold stretches. Since the length-tension characteristics of spindle fibers are nonlinear, we have expressed E as a nonlinear function of fiber length, E = Efl ( y − c ). Use of this term prevents discontinuities in the model if the spindle goes slack. A nonlinear differential equation for z is

Golgi Tendon Organ Feedback The model of Houk and Simon [38] is used to describe the dynamics of the Golgi tendon organ response to muscle force. Prochazka and Gorassini [35] further confirmed the applicability of this model to tendon organ afferent behavior during normal motion. The transfer function for Golgi tendon organ is a band-pass filter H GTO ( s ) = K GTO ×

(1 + s / 015 . )(1 + s / 15 . )(1 + s / 16 ) . (1 + s / 0.2)(1 + s / 2)(1 + s / 37)

(9)

  bz z& = x& − fvi−1    fl ( x − z − c )

(10a)

0.0009197exp(400l ) if l < 0.0025 fl (l ) =  l l ≥ 0.0025 

(10b)

where b is the ratio K /E and l = x − z − c. The force-velocity function for lengthening (y& > 0) has the general form

Although the force feedback from Golgi tendon organ is received by motoneurons through a disynaptic pathway, the exact dynamic behavior of the interneuron between the Golgi tendon organ and motoneuron is still unknown. For

August 2001

x =y+z

IEEE Control Systems Magazine

1 + C 1 y& 1 + C 2 y&  fvi ( y& ) =  1 + C 3 y& 1 − C 4 y&

y& > 0 y& < 0 (10c)

59

where C 1 = 85.5 s/m and C 2 = 22.7 s/m for lengthening, and C 3 = 1 s/m and C 4 = 62.8 s/m for shortening. This function is easily invertible to find the function fvi−1 from (10a). The static gain of the spindle is determined by h / (b + 1), where h is the static gain of the sensory region. The model assumes that the stiffness of the sensory region will be substantially larger than the stiffness of the contractile regions of the spindle, i.e., b > 1. This assumption is based on estimates of the gain of the transducer/encoder processes of the sensory region of the spindle, which are on the order of 100 p/s/mm, relative to the spindle’s overall steady-state gain. The performance of the model is relatively insensitive to the precise values used for h when b is large. Spindle response would be proportional to the product of the length response and a velocity-dependent term. We assume h = 320

Ia Inhibitory Interneurons and Mutual Inhibition Between Interneurons When a pair or more muscles are included in a model, the interconnections among them through various kinds of interneurons have to be considered. Such interneuron networks have been investigated by sophisticated and lengthy experimental procedures in a few laboratories [15], [40]. Information regarding the firing behavior of interneurons is not yet available, though extensive interconnection patterns are known. An agonist-antagonist pair consists of two identical systems of the kind described in Fig. 2(a) connected to a common load. The stretching of the agonist muscle by the load will produce shortening of the antagonist muscle, and vice versa. The two muscle-reflex systems are further connected by mutual inhibition between their respective Renshaw cell populations, by inhibition to each motor nucleus from Ia inhibitory interneurons (IaIN) that are excited by spindle feedback and inhibited by Renshaw cells of the antagonist muscle/motor nucleus, by mutual inhibition between the two groups of IaINs, and by mutual inhibition between Ib inhibitory interneurons (IbIN, not shown) receiving input from Golgi tendon organ. The system is illustrated in Fig. 2(b).

The challenge is to find a reasonable compromise between a manageable level of biological detail and the objectives of the investigation. p/s/mm and b = 99 for no β and b = 55 when β is present at 10 p/s. The resulting static gains are 3.2 p/s/mm and 5.7 p/s/mm, respectively. The static gain of the spindle changes in a manner consistent with 1 / b increasing linearly as a function of β motoneuron firing. This is consistent with the definition of b as the ratio between the stiffness of the sensory region and the stiffness of the contractile region; with increasing β or γ drive, the tension in the spindle should increase linearly. Assuming that the firing rate of motoneurons is about 10 p/s, we define b as a function of motoneuron firing rate m: b( m) = 99 / (1 + 0.08 m).

(11)

Next we need to define the encoder dynamics and the spindle shortening with respect to changes in motoneuron activity. The most direct estimate of the relationship between spindle tension and receptor potential is the work of Hunt and Wilkinson [39]. They found that the tension in spindles was a power-law function of the sinusoidal stretch frequency, with an average exponent of 0.14. In primary afferents, the receptor potential also had a fractional power dependence. It follows that the encoder function would have a similar behavior, with an exponent of approximately 0.32. A power-law frequency response is the result of a continuum of linear elements; we approximated such a response with a linear filter: (1 + s / 723 . )(1 + s / 74.07) . H encoder ( s ) = (1 + s / 1246 . )(1 + s / 123.28 )(1 + s / 250 ) (12)

60

Simulation Results Since the model developed in the previous section contains enough detail about the circuit structure in the spinal cord, we can use it to examine hypothesized interactions of several feedback pathways affecting muscle force and stiffness properties. The simulated muscle is held at 45 mm below maximum physiological stretch and activated by tonic drive to the motoneuron (6.5 nA, to produce approx 10 p/s motoneuron firing). Once the isometric force and neuronal activity have reached a stable equilibrium, the muscle-reflex system receives a load perturbation. The load perturbation could be a rampand-hold stretch or an inertial loading. Because the spindle behavior is closest to experimental observations [22] at ramp stretches of 10 and 20 mm/s, the ramp stretch is performed first and then the inertial load selected to produce a force comparable to that observed during ramp stretch. The inertial loaded system then receives a step increase in drive sufficient to return the muscle to its starting length. This allows us to consider the effects of b and RI loops on the stiffness of the muscle-reflex system and on the damping of muscle movement.

Stiffness During Ramp Stretch We considered two measures of stiffness. Static stiffness is simply the net increment in force between the time when the ramp stretch begins and 1 s after it ends (long enough for the system

IEEE Control Systems Magazine

August 2001

to return to equilibrium), divided by the size of the ramp (10 mm). Dynamic stiffness is estimated during the ramp phase as the derivative of force, with respect to time, divided by the derivative of length, with respect to time. Fig. 3 shows the simulation results to investigate the effect of β or RI loops on the regulation of muscle stiffness when a muscle is stretched. At t = 0.9 s, the musculotendon structure received a 10 mm ramp-and-hold stretch with a constant rate of 20 mm/s [Fig. 3(c)]. The feedback from the spindle caused the motoneuron firing rate to increase while the Golgi tendon organ feedback decreased the firing rate, so the net effect was a smaller increase in motoneuron firing rate (Fig. 3(a)) than one would expect from the increase in spindle firing rate and the gain of spindle feedback (Fig. 3(b)). Comparison of the effects of removing RI and β motoneuron shows that these two components have opposite effects on force and motoneuron activity.

Stiffness During Inertial Loading When a perturbation in the effective loading is applied to the muscle, the control function of the β drive and RI loop on the system is shown in Fig. 4. Similar to what we observed in Fig. 3, the two loops have opposite effects on motoneuron firing and muscle stiffness. In this simulation, in which the perturbation is a load that demands a balancing muscle (tendon) force, the stiffness changes produced by removal of RI and β drive are evident in the smaller and greater excursions of load movement, respectively.

Agonist-Antagonist Response to Inertial Loading and Step Change in Command Simulation results from the single-muscle model described in the previous sections indicated that, as predicted, the RI and

Spindle Firing (p/s)

300

18

200 (b) 16

100

0 0

0.5

1

1.5

2

Ramp (mm)

(c)

45

Motoneuron Firing (p/s)

14 50

12

40 0

0.5

1

1.5

2

10

Tendon Force (N)

30

20

8 (d)

10

6

0 0

0.5

1

1.5

2

0

0.5

Time [s]

1

1.5

2

(a)

RI Loop Removed Normal—Both Loops Intact β Loop Removed

Figure 3. The responses of the single-muscle system when a ramp-and-hold experiment is simulated (c): (a) changes in motoneuron firing rates (pulses per second: p/s); (b) changes in spindle firing rates; (d) force changes. Responses are when both β and RI loops are intact and when either is removed are shown.

August 2001

IEEE Control Systems Magazine

61

β loops had opposite effects on muscle stiffness. We repeated the simulation on a two-muscle joint model. A stochastic noise input, ±0.4 mm position perturbation with Gaussian distribution (standard deviation (s.d.) of 10 mm/s velocity), was applied to the model. Both muscles received a constant input that produced an output of 10 p/s from their respective motoneurons. The simulation results are shown in Fig. 5. The two plots at the top show the dynamic stiffness of the system determined using the perturbation input. Removing the β loop had little effect on stiffness. Removing RI increased stiffness, with the exception of a dip at 10 Hz, and the phase was decreased through much of the low-frequency range.

The bottom two plots in Fig. 5 show the magnitude and phase of load movement produced by a step plus stochastic synaptic drive to the agonist motoneuron. Removing either RI or the β loop increased the amplitude of load movement over much of the physiological range. Thus, the effects of RI and the β loop do not oppose each other under these conditions, and, although the β loop increases stiffness during load or length perturbations in the single-muscle simulations, it does not appear to increase the speed or size of the response to the central drive. We also examined the response of the agonist-antagonist pair to a step synaptic input to the agonist motoneuron. The top two plots in Fig. 6 show discharge of the agonist and an-

Spindle Firing (p/s)

300

18

200 (b)

16

100

0

0

0.2

0.4

0.6

0.8

14

1

46 44

(c)

42 40 0

0.2

0.4

0.6

0.8

1

Motoneuron Firing (p/s)

Load Movement (mm)

48

10

8

15

Tendon Force (N)

12

10 (d)

6

5

0

4 0

0.2

0.4

0.6

0.8

1

0

0.2

Time [s]

0.4

0.6

0.8

1

(a)

RI Loop Removed Normal—Both Loops Intact β Loop Removed

Figure 4. The responses of the single-muscle system to a change in inertial load (at t = 0.4 s) are simulated: (a) motoneuron firing rate; (b) spindle’s firing rate in response to the inertial load change. Plots (c) and (d) are the resulting muscle length and force changes. Responses are when both β and RI loops are intact and when either is removed are shown.

62

IEEE Control Systems Magazine

August 2001

tagonist motoneurons with and without the coactivation of Ia inhibitory interneurons by the same synaptic drive applied to the agonist motoneurons. The lower plot shows load movement with and without this coactivation and, during IaIN coactivation, without either RI or the β loop. With the Ia inhibitory pathway coactivated, the oscillation in activity of the two motoneurons was significantly reduced. Removing the β loop had little effect on load movement, whereas removing RI increased it. The simulations described in this section demonstrate that a model, taking into consideration as much anatomical and structural detail as feasible, can provide a convenient tool for investigating some fundamental questions about the roles and functions of certain neuronal pathways. Such an approach makes it possible to examine hy-

potheses that are difficult to evaluate by either experiments or modeling alone. The model forms a basic building block for more complex models for simulating motor tasks and investigating the interaction of more complex neural circuits. On the other hand, such a detailed model requires extensive validation of parameters and assumed structures. These parameters may be difficult or impossible to get from limited experimental data on a larger-scale model, such as a postural control model or an arm movement model that may involve many muscles and extensive interconnections of sensory pathways. In this situation, a lumped model with less anatomical detail and more functional structures is a more practical and fruitful approach. An example is shown in the next section.

101

100

Stiffness (N/mm)

80

60 40 100

101

0 100

Phase (deg)

100

20

Load Movement (mm/nA)

100

101

500

0

–500

10–1

–1000

–1500 10–2 100

101

–2000 100

101

Frequency (Hz) RI Loop Removed Normal—Both Loops Intact β Loop Removed

Figure 5. Dynamic stiffness (top) and load movement (bottom) for the agonist-antagonist muscle system.

August 2001

IEEE Control Systems Magazine

63

Postural Control Using a Lumped Model

generates the data for testing different hypotheses on neural system structures, neuromusculoskeletal interactions, In the previous example, a model of a detailed neuronal cir- and coordination in postural balance maintenance. Results cuit was developed to investigate the interactions of various from these experiments indicate that control of postural stafeedback loops regulating a single muscle or a pair of mus- bility involves several supraspinal structures to process incles. In this section, we will show that to investigate the func- formation from multimodal sensors. The complexity of the neuromuscular system for postural tional control of a larger entity, a model with less neuronal detail may be better for examining advanced control control and the interpretation of the data from experimental schemes and investigating control strategies. The example is investigations have demanded a suitable computer model for simulation of control strategies and perturbation repostural stability with large external perturbations. sponses. Numerous models have been developed over the past few decades, as also indicated in Abbas and Gillette’s article in this issue (see pp. 80-90). He et al. [13] reported that a sensory feedback controller based on a linear quadratic regulator (LQR) can be designed to maintain the stability of both a three-segment cat limb and a four-segment human model when the foot is held flat on the ground and the perturbations are small. These models show Postural stability and its maintenance are fundamental that sensory feedback control can maintain the stability of to most motor tasks for the neuromuscular system. Exten- the system under very small perturbations. Although this approach produced reasonable results unsive research has been devoted to understanding the control mechanism of the posture system. The investigation der restricted conditions, the controllers designed with this approaches range from animal and human experiments to simplification cannot control models with more realistic feacomputer modeling [41], [42]. Experimental investigation tures, such as heel lift and interactions with the ground. When the perturbation to the body is large, the foot will tilt or even move. In the model described here, the foot was 6 AGONIST allowed to move, and two sets of nonlin4 ear springs were used to simulate the 2 foot-ground interaction. When the time delay in the sensory pathways was in0 cluded in the model, a compensation 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 based on the Smith predictor had to be 6 designed for stability [42]. The tuning of ANTAGONIST the LQR to achieve stability became ex4 tremely difficult, if not impossible, un2 der these conditions. A fuzzy logic controller was therefore implemented 0 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 to provide robust control for the stabil0 ity of the model. A lumped model with joint torque –2 generators was used in this investiga–4 tion for simulating an actual experimental condition with large external –6 1 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 perturbations. In this case, a detailed neuromuscular model became unrealisTime [s] RI Loop Removed tic, because of the undue burden of obNormal—laIN coact. taining and verifying parameters for a Normal—No laIN coact. large number of muscles and neural β Loop Removed connections, and unnecessary, because Figure 6. Discharge of the agonist (top) and antagonist motoneuron pools (middle) and we wanted to understand the general the load movement (bottom) following a step command to the agonist motoneuron pool. control strategy for the global system. Load Movement (mm)

Motoneuron Firing (p/s)

We present two examples that illustrate the strategy to model and investigate biological motor control tasks with varying degrees of complexity and detail.

64

IEEE Control Systems Magazine

August 2001

Model Development for Postural Control System

where fk is the spring force and b (= 2000) represents the damping coefficient.

The basic model structure is shown in Fig. 7 and described in Tian and He [42]. The model has six degrees of freedom: four joints and the position ( x , y ) of the foot. It has the following standard form:

(

)

(

M ϑ , ϑ& + N ( ϑ )g + Tp ϑ , ϑ& && = J −1( ϑ ) ϑ + Ta ϑ , ϑ& + E ( ϑ )Fe 

(

)

) ,  

(13)

where ϑ is the angular and positional vector, J(.) is the moment of inertia matrix, M (.) is the vector of moments created by the centrifugal and Coriolis forces, N(.) are the coefficients for the gravitational force, g is the gravitation constant, Tp (.) are the passive joint torques, Ta (.) are the active joint torques, E(.) are the coefficients for the moments created by external perturbation forces, and Fe is the external perturbation force applied at the waist level. A soft constraint, including a nonlinear spring and a damper, was implemented at both the toe and heel of the foot in the model, allowing the body to rise above the ground for future extension of the model to simulate stepping. The ground reaction force was assumed to saturate at k ⋅ Mg, where k is a scaling factor and Mg is the body weight. The experimental data showed that k = 2 was adequate since the ground reaction forces were smaller than 2 Mg in data recorded for all subjects in an experiment. The resting length of the spring was x 0 , beyond which the ground reaction force was set to zero. The force distribution was assumed to be 40/60 between the toe and heel springs. The ground reaction force was f = fk − bx&,

(14)

The Design of Combined LQR and FLC The central nervous system controls postural stability with a hierarchical structure: local feedback based on proprioceptive sensory information through spinal cord neural networks and global sensory feedback based on supraspinal neural networks. The spinal cord network has been proposed to behave LQR-like under small perturbations [13]. The LQR approach provides a convenient means of finding feedback gains that produce a smooth and stable trajectory when the perturbation is small. When a perturbation becomes large, or when the linearization process ignores too many nonlinearities, the LQR controller becomes less effective. Both situations arise in the current model. First, the linearization ignores the constraints imposed on each joint by the physical limitations of movement range, especially at the knee joint. In upright posture, the knee joint is close to the locked position. The constraint force to prevent hyperextension is very large. The time delay in the sensory feedback pathways demands a Smith predictor type of compensation in combination with LQR [42]. When the foot is allowed to rise above the ground and the joint constraints are included in the model, the tuning of LQR parameters becomes extremely difficult and cannot stabilize the model when the body posture is perturbed. The postural control is not entirely regulated by the spinal cord circuits, however. Supraspinal control action has been shown to play an important role in maintaining body posture. Time latency of significant muscle actions in responding to a perturbation is much longer than that in the spinal cord feedback loops. Further evidence for supraspinal control is the

Head-Arms-Trunk

y

Cable

Load Cell Release Hook

Pneumatic Cylinder Force Plates

b4 a4

c4

θHip

b3

θKnee

c3

θThigh

a3 θAnkle

Shanks

Release Cylinder

z

θTrunk

θShank

a2

Feet

Belt

Thighs

Pulley

x

θFoot

a1

b2 c2 b1 c1

Figure 7. The schematics of the experimental setup and the model structure.

August 2001

IEEE Control Systems Magazine

65

rate” at a value of one. This makes intuitive sense since at some point a human subject would just group all large values together in a linguistic description such as “poslarge” (for “postive large”) or “neglarge.” The membership functions at the outermost edges appropriately capture this phenomenon by using “greater than” for the right side and “less than” for the left side. For the output variables of the FLC, the control torques at each joint, Tankle, Tknee, and Thip, the membership functions at the outermost edges are not saturated. The reason is that in the decision-making process, an exact value for the process input needs to be specified, although for the knee joint torque, any value larger than 40 is acceptable to keep the legs from collapsing. The Fuzzy Control Rule Base: A very coarse rule base is used in this model because of the adoption of an LQR controller that provides the fine tuning of the control action. As a result, only 28 rules are implemented. These rules are easily implemented in IF-THEN statements based on biological reasoning and common sense insight into maintaining upright postural stability. Fuzzy Implication: The minimum operator is used to represent the implication “then.” The maximum operator is chosen to represent aggregation. Fuzzy Inference to Determine Control Rules: The premises of all the rules are compared to the controller inputs to determine which rules apply to the current situation. Determining the applicability of each rule is called “matching.” This matching process involves determining the certainty that each rule would apply. A rule is said to be “on at time t” if its premise membership function µpremise (com_x, com_vx, com_y, com_vy) > 0. There are actually several ways of quantifying the “and” and the “or” operations. This model uses minimum to represent the “and” operation and maximum to represent the “or” operation. For specified values of com_x, com_vx, com_y, and com_vy, the value of the premise certainty µpremise (com_x, com_vx, com_y, com_vy) represents the extent to which the rule is applicable for specifying torque inputs to the model. Defuzzification: Also thought of as “decoding,” defuzzification operates on the implied fuzzy sets produced by the inference mechanism and combines their efforts to provide the “most certain” controller outBiological put. The “centroid” defuzzification Control Noise External method is used for combining the recStrategy Disturbance ommendations represented by the im+ plied fuzzy sets from all the rules. Body + Actuator Integrating all elements together, the Mechanics Dynamics Fuzzy Logic model (Fig. 8) was implemented in Controller Sensory Simulink. A graphical interface was deDynamics veloped to access the model and controller parameters and to visualize Sensory simulation results in animated displays. Feedback A stick-figure animation on the fly, as Global Variables (CoM, CoP, etc.) well as scopes and autoscaled graphs, were used to visualize the simulation. Figure 8. The block diagram of a postural control model. apparent regulation of body posture based on the center of mass, a global variable calculated from all segments of the body. To model this regulation of the center of mass, a fuzzy logic controller (FLC) is designed to simulate the overall supraspinal control action. The FLC takes the global variable, the center of mass, as its input and generates the joint torque to maintain the center of mass location within the stability region. The structure of the combined LQR-FLC controller is shown in Fig. 8. The working principle of the controller is that FLC provides the global control and manipulates the center of mass within the stability region, and LQR provides fine tuning control based on full state feedback. The design of an efficient and robust fuzzy controller generally requires the tuning of a large number of parameters. This set of parameters includes the contents of the rule base, the distribution and shape of membership functions for every input, state and output variable, and the choice of methods for logical reasoning, fuzzification, and defuzzification. One way to alleviate the curse of dimensionality is to reduce the number of variables and use a minimum set of membership functions for every variable. However, this approach may compromise the performance of the resulting system in general. By combining the design of special fuzzy controllers with LQR, we have been able to avoid this problem. In our example, a Mamdani-type fuzzy controller was implemented where a small number of parameters and the simplest membership functions were used. Instead of every state variable (12 in the model), only the global variables specifying the location and velocity of the center of mass were used as input to the fuzzy controller. The membership functions were selected in the simplest form possible (triangular functions with saturation at both ends) because the purpose of the model and the simulation was to investigate the control principles that may likely be employed by the biological neural system. As inputs to the FLC, the crisp values of fuzzy variables were converted to fuzzy membership function values using the simple singular-value fuzzification. The membership function values can also be thought of as the “encoding” of the fuzzy controller numeric input values. The outermost membership functions of input variables are set to “satu-

66

IEEE Control Systems Magazine

August 2001

Simulation Results for a Postural Control Model with Large Perturbations

0.04

–8

CoM Displacement in Horizontal Direction (m)

–0.06 –0.08

0

0.5

1

2

2.5

0 –2 –4 –6 –8 –10

–10

–12

–12 0

0.5

1

1.5

2

2.5

0

0.5

1

0

0.5

1

Time [s] 4

4

2

2

Foot Joint Angle (deg)

Ankle Angle (deg)

1.5

Figure 9. The center of mass movement after the perturbation. The thin line is from the model simulation under the hybrid LQR and FLC control. The thick line is from experimental measurement. Notice that the human subject did not return to the original position.

2

–6

–0.04

Time [s]

2

–4

–0.02

–0.14

4

–2

0

–0.12

4

0

0.02

–0.1

Knee Joint Angles (deg)

Hip Joint Angle (deg)

In the human experiment, a short-duration, impulsive pulling force was applied at the pelvis level to perturb the upright stance. The force was approximately 25% of body weight. The subjects were instructed to maintain their upright posture with both arms folded in front. They were also instructed to achieve the best performance without stepping. Joint trajectories were measured, as well as ground reaction forces and muscle activities from major leg muscles. The actual perturbations recorded from the experiments were used in the simulation of the original nonlinear model. Trajectories of the center of mass were calculated from the measured data. The results from the model were compared with the recorded human responses. Fig. 9 shows the trajectory of the center of mass in response to a perturbation. Fig. 10 shows the joint angles before, during, and after the perturbation. In these figures, experimentally measured and model simulation results are compared. Both the center of mass and joint trajectories match well, though, at the end of the perturbation, human subjects may not always return to the exact original position before the perturbation. It should be

0.06

0 –2 –4 –6 –8

1.5 Time [s]

2

2.5

1.5

2

2.5

0 –2 –4 –6 –8 –10

–10

–12

–12 0

0.5

1

1.5

2

2.5

Time [s]

Time [s]

Figure 10. Comparison of joint angles obtained from model simulation and experiment (thicker line in each of the subplots). The trajectories from the model simulation are generated by the combined control of LQR and FLC, as shown in Fig. 7. The simulation under LQR and FLC returned the model to the original position.

August 2001

IEEE Control Systems Magazine

67

possible to reproduce this phenomenon by designing a pure fuzzy logic controller with the final position specified by a fuzzy set to represent the stability range of the upright posture. With the hybrid control combining LQR and FLC in our model, the final position always returned to the original position due to the action from the LQR. Based on the principle derived by Raibert [43] in controlling dynamically stable running robots, we designed a fuzzy logic rule base to control the location of the center of mass during standing posture, instead of controlling each individual state variable. We set the objective of the FLC to maintain the center of mass within the base of support, defined by the size of the foot and a range of heights above the ground, similar to the forward running speed (zero in standing posture) and the hopping height regulation control. However, this FLC, based on the regulation of the center of mass alone and with relatively few rules, was not adequate for stabilizing the system. By combining the LQR and FLC, and utilizing the fine-tuning capacity of the LQR and the robustness of the FLC in a smooth and reliable control scheme, the number of rules in the FLC became manageable and the tuning of LQR weighting matrices became less sensitive to system parameters.

Discussion We have presented two examples that represent two extremes of the modeling spectrum; the neuromuscular model included fine details of the organization and structure of the biological system, whereas the postural control model lumped the system structure into a few global variables. Using the detailed model, we could examine hypotheses on the functional significance of individual components or signal flow pathways. Without such details about the specific pathways from β drive to spindles, Renshaw cell recurrent inhibition to regulate motoneuron firing rates, spindle primary inhibitory interneuron (IaIN) pathways among muscles for mutual inhibition, and detailed dynamics of each component, we could not examine the interactions among these pathways. The issues of how β drive and Renshaw feedback interact and what each pathway would do in a functional system are impossible to evaluate experimentally at this time due to the difficulty in manipulating these pathways in an experiment. We have been able to shed light on the issues using a model that contains adequate detail about the neuromuscular system. On the other hand, when investigating the behavior of the whole mechanism, sacrifice of certain details becomes necessary. If we had to include every neural pathway and every muscle in the postural control model, the task of obtaining a reasonable set of parameters for the system would be extremely difficult, and some parameters may even be impossible to verify. The challenge is to find a reasonable compromise between a manageable level of biological detail and the objectives of the investigation when using a modeling and simulation approach.

68

The relatively simple structure of the model for postural control allowed us to examine feasible control strategy that simulates the interaction between the spinal cord regulation and the supraspinal control. The implementation of the FLC also provides us the freedom of incorporating control strategies derived from experimental investigations of human postural control. There are many other neuromuscular control tasks that can be addressed by model analysis and simulation, and these are future challenges for control theory and system science in the realm of biology.

Acknowledgments The work is supported in part by PHS grants (NS-22454 to T.M. Hamm, NS-37088 to J. He), an NIH NRSA fellowship (NS 10341 to M.G. Maltenfort), and the Whitaker Foundation Biomedical Engineering Research grant to J. He.

References [1] A.P. Georgopoulos, “Higher order motor control,” Annu. Rev. Neurosci., vol. 14, pp. 361-377, 1991. [2] Z. Hasan and D.G. Stuart, “Animal solutions to problems of movement control: The role of proprioceptors,” Annu. Rev. Neurosci., vol. 11, pp. 199-223, 1988. [3] A. Prochazka, “Proprioceptive feedback and movement regulation,” in Exercise: Regulation and Integration of Movement, L. Rowell and J.T. Sheperd, Eds. New York: American Physiological Society, 1996, pp. 89-127. [4] E.D. Schomburg, “Spinal functions in sensorimotor control of movements,” Neurosurg. Rev., vol. 13, no. 3, pp. 179-185, 1990. [5] Z. Hasan, “Role of proprioceptors in neural control,” Current Opinion in Neurobiology, vol. 2, pp. 824-829, 1992. [6] P.A. Fortier, A.M. Smith, and J.F. Kalaska, “Comparison of cerebellar and motor cortex activity during reaching: Directional tuning and response variability,” J. Neurophysiol., vol. 69, pp. 1136-1149, 1993. [7] J.F. Marsden et al., “Coherence between cerebellar thalamus, cortex and muscle in man: Cerebellar thalamus interactions,” Brain, vol. 123, pp. 1459-1470, 2000. [8] G.E. Loeb, W.S. Levine, and J. He, “Understanding sensorimotor feedback through optimal control,” in Proc. Cold Spring Harb. Symp. Quant. Biol., 1990, vol. 55, pp. 791-803. [9] M.A. Nicolelis et al., “Sensorimotor encoding by synchronous neural ensemble activity at multiple levels of the somatosensory system,” Science, vol. 268, pp. 1353-1358, 1995. [10] P.R. Kennedy, “Corticospinal, rubrospinal and rubro-olivary projections: A unifying hypothesis,” Trends in Neurosciences, vol. 13, pp. 474-479, 1990. [11] J.S. Albus,”A new approach to manipulator control: The cerebellar model articulation controller (CMAC),” J. Dyn. Syst., Meas. Contr., pp. 220-233, 1975. [12] H.J. Chizeck, “Adaptive and nonlinear control methods for neural prostheses,” in Neural Prostheses: Replacing Motor Function after Disease or Disability, R.B. Stein, P.H. Peckham, and D.B. Popovic, Eds. New York: Oxford Univ. Press. pp. 298-327, 1992. [13] J. He, W.S. Levine, and G.E. Loeb, “Feedback gains for correcting small perturbations to standing posture,” IEEE Trans. Automat. Contr., vol. 36, pp. 322-332, 1991. [14] M. Kawato and D. Wolpert, “Internal models for motor control,” in Novartis Found. Symp., 1998, vol. 218, pp. 291-304; discussion pp. 304-7. [15] D.A. McCrea, “Spinal cord circuitry and motor reflexes,” Exercise and Sport Sci. Rev., vol. 14, pp. 105-142, 1986. [16] G.E. Loeb, I.E. Brown, and E.J. Cheng, “A hierarchical foundation for models of sensorimotor control,” Exp. Brain Res., vol. 126, no. 1, pp. 1-18, 1999. [17] U. Windhorst, “On the role of recurrent inhibitory feedback in motor control,” Prog. Neurobiol., vol. 49, pp. 517-587, 1996. [18] M. Illert, H. Kümmel, and J.J.A. Scott, “Beta innervation and recurrent inhibition: A hypothesis for manipulatory and postural control,” Pflgüers Arch., vol. 432, pp. 61-67, 1996. [19] F. Baldissera, P. Campadelli, and L. Piccinelli, “The dynamic response of cat α motoneurons investigated by intracellular injection of sinusoidal currents,” Exp. Brain Res., 1984.

IEEE Control Systems Magazine

August 2001

[20] C. Christakos, “The mathematical basis of population rhythms in nervous and neuromuscular systems,” Int. J. Neurosci., vol. 29, pp. 103-107, 1986. [21] P. Matthews, “Spindle and motoneuronal contributions to the phase advance of the human stretch reflex and the reduction of tremor,” J. Physiol., vol. 498, pp. 249-275, 1997. [22] J. Houk, P. Crago, and W. Rymer, “Function of the spindle dynamic response in stiffness regulation—A predictive mechanism provided by nonlinear feedback,” in Muscle Receptors and Movement, A. Taylor and A. Prochazka, Eds. New York: Oxford Univ. Press, 1981, pp. 299-309. [23] R. Baratta and M. Solomonow, “The dynamic response of nine different skeletal muscles,” IEEE Trans. Biomed. Eng., vol. 37, pp. 243-251, 1990. [24] J. Bobet and R. Stein, “A simple model of force generation by skeletal muscle during dynamic isometric contractions,” IEEE Trans. Biomed. Eng., vol. 45, pp. 1010-1016, 1998. [25] E. Otten, “A myocybernetic model of the jaw system of the rat,” J. Neurosci. Meth., vol. 21, pp. 287-302, 1987. [26] C. Heckman and M. Binder, “Computer simulation of the steady-state input-output function of the cat medial gastrocnemius motoneuron pool,” J. Neurophysiol., vol. 65, pp. 952-967, 1991. [27] A. Fuglevand, D. Winder, and A. Patla, “Models of recruitment and rate coding organization in motor-unit pools,” J. Neurophysiol., vol. 70, pp. 2470-2488, 1993. [28] F.E. Zajac, “Muscle and tendon: Properties, models, scaling, and application to biomechanics and motor control,” CRC Crit. Rev. Biomed. Eng., vol. 17, pp. 359-411, 1989. [29] J. He, W.R. Norling, and Y. Wang, “A dynamic neuromuscular model for describing the pendulum test of spasticity,” IEEE Trans. Biomed. Eng., vol. 44, no. 3, pp. 175-184, 1997. [30] S. Spector et al., “Muscle architecture and force-velocity characteristics of cat soleus and medial gastrocnemius: Implications for motor control,” J. Neurophysiol., vol. 44, pp. 951-60, 1980. [31] J. Stephens, R. Reinking, and D. Stuart, “The motor units of cat medial gastrocnemius: Electrical and mechanical properties as a function of muscle length,” J. Morph., vol. 146, pp. 495-512, 1975. [32] R. Kirsch, D. Boskov, and W. Rymer, “Muscle stiffness during transient and continuous movements of cat muscle: Perturbation characteristics and physiological relevance,” IEEE Trans. Biomed. Eng., vol. 41, pp. 758-770, 1994. [33] N. Rosenthal et al., “Frequency analysis of stretch reflex and its main subsystems in triceps surae muscles of the cat,” J. Neurophysiol., vol. 33, pp. 713-49, 1970. [34] U. Windhorst, “Activation of Renshaw cells,” Prog. Neurobiol., vol. 35, pp. 135-179, 1990. [35] A. Prochazka and M. Gorassini, “Ensemble firing of muscle spindle afferents recorded during normal locomotion in cats,” J. Physiol., vol. 507, pp. 293-304, 1998. [36] M. Binder and C. Heckman, “The physiological control of motoneuron activity,” in Handbook of Physiology, Exercise: Regulation and Integration of Multiple Systems, L. Rowell and J. Shepherd, Eds. New York: Oxford Univ. Press, 1996, pp. 3-53. [37] Z. Hasan, “A model of spindle afferent response to muscle stretch,” J. Neurophysiol., vol. 49, pp. 989-1006, 1983. [38] J.C. Houk and W. Simon, “Responses of Golgi tendon organs to forces applied to muscle tendon,” J. Neurophysiol., vol. 30, pp. 1466-1481, 1967. [39] C.C. Hunt and R.S. Wilkinson, “An analysis of receptor potential and tension of isolated cat muscle spindles in response to sinusoidal stretch,” J. Physiol., vol. 302, pp. 241-282, 1980. [40] E. Jankowska and S. Edgley, “Interactions between pathways controlling posture and gait at the level of spinal interneurones in the cat,” Prog. Brain Res., vol. 97, pp. 161-171, 1993. [41] R. Fitzpatrick et al., “Postural proprioceptive reflexes in standing human subjects: Bandwidth of response and transmission characteristics,” J. Physiol. Lond., vol. 458, pp. 69-83, 1992. [42] C.X. Tian and J. He. “Posture stability: Control strategies and their boundaries,” in Proc. IEEE Conf. Decision and Control, San Diego, CA, 1997. [43] M. Raibert, “Dynamically Stable Legged Robots,” MIT, Cambridge, MA, Tech. Rep., 1987.

Jiping He was born in Shanghai, China. He received the B.S. degree in control engineering from Huazhong University of

August 2001

Science and Technology, Wuhan, China, in 1982 and the M.S. and Ph.D. degrees in electrical engineering from the University of Maryland, College Park, in 1984 and 1988, respectively. He was a postdoctoral fellow at MIT, Cambridge, MA. In 1990, he was with Thomas Jefferson University, Philadelphia, PA, as a Research Assistant Professor. He was a visiting scientist at Princeton University, NJ, during 1991 and 1992. He has been an Associate Professor of Bioengineering at Arizona State University, Tempe, AZ, since 1994 and the Director of the IGERT program since 2000. He is a Senior Member of the IEEE. Mitchell Gil Maltenfort received his B.S. in biomedical engineering at Tulane University in 1984 and his M.S. and Ph.D. in the same field at Northwestern University in 1991 and 1995. Following Northwestern, he joined Tom Hamm’s lab at Barrow Neurological Institute in Phoenix, AZ. Since 1999, he has also been working at the National Institute of Neurological Disorders and Stroke in another modeling study, to identify possible unique roles of beta innervation of spindles in contrast with Type II afferent feedback on spindles and alpha-gamma co-activation. Qingjun Wang is a Ph.D. candidate in bioengineering at Arizona State University. He is a Research Assistant working on the restoration of gait in chronic spinal cord injury patients by epidural spinal cord stimulation. He received the M.S. in electrical engineering from Arizona State University and the B.S. in mechanical engineering from TsingHua University (China). He received the Third-Class Award for Progress of Chinese National Science and Technology in 1994 and the IOC Scholarship from the Olympic Scientific Congress in Dallas, TX, in 1996. His professional interests include research and development of biomedical instrumentation, real-time signal processing, and human rehabilitation. Mr. Wang is a Member of IEEE Thomas M. Hamm received a B.S. in physics and mathematics from the University of Memphis and a Ph.D. in physiology from the University of Tennessee Center for Health Sciences. He subsequently received postdoctoral training in the neurobiology of segmental and spinal systems at the University of Arizona. After postdoctoral training, he took a position in the Division of Neurobiology at the Barrow Neurological Institute (BNI) in Phoenix, where he established a research program in spinal motor systems. He is currently an Associate Staff Scientist in Neurobiology at the BNI and holds joint and adjunct appointments in physiology at the University of Arizona and in bioengineering at Arizona State University. He is a member of the Society for Neuroscience and the American Physiological Society.

IEEE Control Systems Magazine

69