Real-Time Collision-aware Musculoskeletal Model for Virtual ... - CNRS

Mar 8, 2014 - As a target shape for each map, two dimensional outlines of ... provided an added value of accuracy for estimating moment arms of muscle ... Biomechanical models include data that are relevant to the physical aspects of the.
14MB taille 5 téléchargements 274 vues
Utrecht University

Master Thesis

Real-Time Collision-aware Musculoskeletal Model for Virtual Human Animation

Author:

Supervisor:

´ Francis P. Lacle

Dr. Nicolas G. Pronost

ICA-3503534

A thesis submitted in fulfilment of the requirements for the degree of Master of Science in the Virtual Human Technology Lab Department of Information and Computing Sciences

March 2014

Declaration of Authorship ´, declare that this thesis titled, ’Real-Time Collision-aware MuscuI, Francis P. Lacle loskeletal Model for Virtual Human Animation’ and the work presented in it are my own. I confirm that:



This work was done wholly or mainly while in candidature for a master’s degree at this University.



Where any part of this thesis has previously been submitted for a degree or any other qualification at this University or any other institution, this has been clearly stated.



Where I have consulted the published work of others, this is always clearly attributed.



Where I have quoted from the work of others, the source is always given. With the exception of such quotations, this thesis is entirely my own work.



I have acknowledged all main sources of help.



Where the thesis is based on work done by myself jointly with others, I have made clear exactly what was done by others and what I have contributed myself.

Signed:

Date:

i

“As computers have become more powerful, computer graphics have advanced to the point where it’s possible to create photo-realistic images. The bottleneck wasn’t, ’How do we make pixels prettier?’ It was, ’How do we engage with them more?’”

Jefferson Han

UTRECHT UNIVERSITY

Abstract B`etawetenschappen Department of Information and Computing Sciences Master of Science Real-Time Collision-aware Musculoskeletal Model for Virtual Human Animation ´ by Francis P. Lacle

Keywords: biomechanics, computational geometry, computer animation, computer graphics, the Ultimate Human Model. This MSc project focused on combining a biomechanical musculoskeletal model with a high poly musculotendon model for the lower human body. A low-poly cylinder-based musculotendon unit is constructed that segments muscle and tendon into separate compartments. The shape of high poly meshes is approximated through ray-triangle intersection tests. Point samples are then used to move action lines of the biomechanical model within the high poly model. Once inside, ray-triangle intersection tests are carried out again to get a more accurate approximation of the surface of the high poly model. The method was developed in order to be invariant to spatial and polygonal configurations. Results with 48 musculotendons for the lower body show a drop of ≈ 84% with respect to the amount of vertices when compared to the high poly model. The project also considered real-time as a criterion and introduced a scalar for the geometrical model allowing each cylinder to be scaled in both longitudinal and latitudinal directions. The amount of information could therefore be increased or decreased to retain real-time performance on faster or slower computer hardware. An experiment with 10 muscles resulted in a runtime of 164Hz on average having a total of 25 constraints. Finally, a custom collision constraint between musculotendons was introduced, coupled with an area preservation technique used during deformation, which takes advantage of the cylindrical construction of the real-time model.

Acknowledgements First I would like to express my gratitude towards my supervisor. He provided me with all the necessary resources and support that was always accompanied by great enthusiasm. His remarks and suggestions helped me to focus in the right direction throughout the course of this MSc project. Second, I would like to thank all my family and friends. My family for constant support, and friends for giving me the space and bearing the incremental social distance while I was working on this thesis. A special thank you goes to Robin, Bernd, and Michal for proof reading this thesis, which was a great help in pointing out some obvious grammatical and notational errors. I am also grateful to live in the 21st century and in a developed country where one can shower with warm water, which for me served as an idea-incubator for most of the breakthroughs implemented within this MSc project. The most important gratitude goes to Nora, whom I love very much. Being someone quite stubborn, she showed me how to change mindsets and disregard previous erroneous ideas, and also how to strike the right chord between perfection and efficiency against the always persistent arrow of time. Finally, I would like to address our little one (I’m writing this while you are making funny noises in the background.) Henry, regardless of which path you choose in life, remember that you only need two things, a thirst for knowledge and perseverance.

iv

Contents Declaration of Authorship

i

Abstract

iii

Acknowledgements

iv

Contents

v

List of Figures

viii

List of Tables

ix

List of Algorithms

ix

Abbreviations

x

Symbols

xi

1 Introduction 1.1 Motivations and Contributions . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Chapters overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

1 1 3

2 Related Work 2.1 Musculoskeletal Models in Computer Graphics . . . . . . . . . . . . . . . 2.2 Musculotendon Models for Volumetric Deformation . . . . . . . . . . . . .

4 4 5

3 The 3.1 3.2 3.3

3.4

Musculoskeletal Model Anatomical Accuracy and Real-Time Performance The Biomechanical Musculoskeletal Model . . . . . The Polygonal Skeleton Meshes . . . . . . . . . . . 3.3.1 Attaching the Skeleton Meshes . . . . . . . 3.3.2 OpenSim Bodies in Cartesian Coordinates . 3.3.3 Semi-Automatic Skeleton Mesh Alignment . A Simple Geometric Musculotendon Model . . . . 3.4.1 Adding Action Lines and Via Points . . . . 3.4.2 Adding Volume with Cross Sections . . . . v

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

7 7 7 9 10 11 13 15 16 19

Table of Contents

3.5

3.6

3.7

vi

3.4.3 Including Tendons into the Model . . . . . . . . . . . . . . Geometrical Enhancement with the UHM . . . . . . . . . . . . . . 3.5.1 Preparing the UHM Meshes . . . . . . . . . . . . . . . . . . 3.5.2 The UHM Mesh Shape Analysis . . . . . . . . . . . . . . . 3.5.3 A Technique for Musculotendon Enhancement . . . . . . . 3.5.3.1 Repositioning C’s on the Surface of U . . . . . . . 3.5.3.2 Repositioning Elements of A within the Volume of Match Detection . . . . . . . . . . . . . . . . . . . . Permuting Matches . . . . . . . . . . . . . . . . . . . Determining an Interior ”Inner” Point . . . . . . . . 3.5.4 Iterating the Enhancement Process . . . . . . . . . . . . . . Adapting the LOD of the Musculotendon Model . . . . . . . . . . 3.6.1 Longitudinal and Latitudinal Scaling . . . . . . . . . . . . . 3.6.2 Hexagons as a Lower Bound for Ck . . . . . . . . . . . . . . Drawing the Cylinder-based Musculotendon Unit . . . . . . . . . . 3.7.1 Triangle Fans and Triangle Strips . . . . . . . . . . . . . . . 3.7.2 Applying Textures . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . U . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . .

20 23 24 26 28 29 30 30 32 33 34 35 35 36 37 38 38

4 Real-Time Collision Detection and Response 4.1 Custom Strategy for Discrete Collision Detection . . . . . 4.1.1 Collision Constraints for Muscle and Skeleton . . . 4.1.2 Discrete Collision Detection with Ray Casting . . 4.2 Collision Response . . . . . . . . . . . . . . . . . . . . . . 4.2.1 Retaining Cross Sectional Areas through Inflation

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

. . . . .

40 41 41 42 44 44

5 Results and Discussion 5.1 The UHM Enhancement . . . . . . . . . . . . . . . 5.1.1 Non-linear Longitudinal Scaling . . . . . . . 5.1.2 Approximating the UHM at Different Scales 5.1.3 Match Detection . . . . . . . . . . . . . . . 5.2 Collision Detection and Response . . . . . . . . . . 5.2.1 Radii Inflation . . . . . . . . . . . . . . . . 5.3 Real-Time Performance . . . . . . . . . . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

47 47 48 48 51 52 53 53

. . . . . . . . .

56 57 57 58 58 59 59 59 60 60

. . . . . . .

. . . . . . .

. . . . . . .

. . . . . . .

6 Conclusion and Future Work 6.1 Improvements and Future Considerations . . . . . . . . . . . . . . 6.1.1 Biomechanical Accuracy . . . . . . . . . . . . . . . . . . . . 6.1.2 Extending the Musculotendon Model . . . . . . . . . . . . . 6.1.2.1 Linked Transformation Correction for A . . . . . . 6.1.2.2 Multiple Action Lines for Wide Attachments . . . 6.1.2.3 Distinct Heads . . . . . . . . . . . . . . . . . . . . 6.1.2.4 A Geometric Representation for Ligaments . . . . 6.1.3 Complete Collision Detection and Response . . . . . . . . . 6.1.4 The Musculotendon Model for FEM and Inverse Dynamics Bibliography

. . . . . . . . .

. . . . . . . . .

. . . . . . . . .

62

Table of Contents

vii

A List of Included Musculotendons (OpenSim)

1

B Ralph’s Bone Hierarchy (RAGE)

2

C Upper and Lower Body Skeleton Hierarchy (OpenSim)

3

D List of Included UHM Meshes

4

E Edge List Sort Algorithm

6

F Area and Centroid Calculation

8

List of Figures 1.1

Schematic plot between real-time performance and computational accuracy

1

3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11 3.12 3.13 3.14 3.15 3.16 3.17 3.18 3.19 3.20 3.21 3.22 3.23 3.24 3.25

Screenshots showing three muscle paths rendered in OpenSim and RAGE Different coordinate systems used in OpenSim and OGRE/RAGE . . . . Kinematic relationship between OpenSim bodies in generalised coordinates Rotation of OpenSim bodies along a curved path using spline functions . Visible problems after attaching OpenSim skeleton meshes . . . . . . . . . Difference between automatic and semi-automatic skeleton alignment . . . Walk-cycle animation history plot showing attached skeleton . . . . . . . Screenshots showing one action line with and without via points . . . . . Three figures showing different up-vectors for cross sections . . . . . . . . Schematic representation of a musculotendon actuator . . . . . . . . . . . Schematic representation of the musculotendon model . . . . . . . . . . . Comparison between closed and open UHM mesh . . . . . . . . . . . . . . Self-intersection issues with the UHM . . . . . . . . . . . . . . . . . . . . Dividing the UHM meshes for multiple action lines for one musculotendon. Two results from the analysis of the UHM meshes . . . . . . . . . . . . . General idea for enhancing the musculotendon model . . . . . . . . . . . . Schematic figures showing the first preference for P ’s . . . . . . . . . . . . Schematic figure showing the second preference for pi ’s . . . . . . . . . . . Schematic figure showing a key example of the enhancement process . . . Schematic figure showing same set E1 for two different outcomes . . . . . An example illustrating two types of inner point determination. . . . . . . Illustration of polygons inscribed in closed conic sections . . . . . . . . . . Hierarchy of drawing steps and illustration of internal coordinate system . Illustration of cylinder buildup using triangle fans and strips. . . . . . . . Screenshots showing different textured tendons and muscles . . . . . . . .

8 9 11 12 13 15 15 18 19 20 23 24 25 25 27 28 29 30 31 32 34 37 38 39 39

4.1 4.2

Illustration of collision constraints between two muscles Mx and My . . . 42 Direction of cast rays confined by transversal constraints . . . . . . . . . . 43

5.1 5.2 5.3 5.4 5.5 5.6

Pre and post UHM enhancement of the lower body musculoskeletal model Some examples for inner point determination . . . . . . . . . . . . . . . . Misplaced action line due to partial convergence . . . . . . . . . . . . . . UHM approximation of the Tibialis Anterior in eight different scales of Ck Three results without and with constraints . . . . . . . . . . . . . . . . . Displacement for two consecutive cki ’s shown as black line segments . . . .

6.1 6.2

Linked transformation correction for a muscle path . . . . . . . . . . . . . 58 Schematic figures showing how volume of segments can be computed. . . . 60 viii

47 49 49 50 54 55

List of Tables 3.1 3.2 3.3 3.4

Biomechanical parameters included in this project . . . . . . . . . . . . . 9 Summarised description of muscle path constraints in OpenSim . . . . . . 17 One result for the scaled tendon slack length of the Tibialis Anterior . . . 22 Including the sign of algebraic radii allows Ek to be permuted accordingly 33

5.1 5.2 5.3

Vertices of 48 musculotendons in 12 different scales compared to the UHM 48 Amount of ak outside an UHM mesh for LSS = 1 . . . . . . . . . . . . . . 51 Amount of ak outside an UHM mesh for LSS = 2 . . . . . . . . . . . . . . 51

List of Algorithms

1

Permutation σ given in pseudo-code that returns EK containing a sequence of matches winding through from one end till the other end of U Ck . . . . . 33

2

The order of steps executed during the UHM enhancement process. . . . . 35

ix

Abbreviations CoM

Centre of Mass

CSS

Cross Sectional Scalar

DOF

Degree Of Freedom

FE

Finite Element

FEM

Finite Element Method

FFD

Free Form Deformation

FPS

Frames Per Second

FVM

Finite Volume Method

GPU

Graphics Processing Unit

HACD

Hierarchical Approximate Convex Decomposition

IAR

Instantaneous Axis of Rotation

LOD

Level Of Detail

LSS

Longitudinal Segment Scalar

OGRE

Open-source 3d GRaphics Engine

RAGE

Real-time Animation Game Engine

UID

Unique IDentifier

UHM

Ultimate Human Model

XML

eXtensible Markup Language

x

Symbols A

vector space defining action-line of a musculotendon

ak = (x, y, z)T in R3

BR

OpenSim body in relative coordinates

(x, y, z)T coordinates in R3

BW

OpenSim body in world coordinates

(x, y, z)T coordinates in R3

Ck

vector space defining each cross section of ak

cki = (x, y, z)T in R3

Dk

vector space defining a deformed cross section of ak

dki = (x, y, z)T in R3

M

main vector space defining a musculotendon

mg = (x, y, z)T in R3

O

transformed coordinates from OpenSim into RAGE

vertex positions in R3

U

high poly mesh from the UHM

vertex positions in R3

rik

radius that determines location of cki

R

V

vector space defining via points of a musculotendon

equal to a particular ak

α

pennation angle

rad



function for triangle area with given parameters

R

κ

transversal line segment used for collision constraint

two tagged cki ’s

µk

displacement of ak due to collision

(x, y, z)T coordinates in R3

φ

alternate interior angle determined by κ

rad

ρ

penetration depth determined by κ

R

σ

permutation on Ek

indices of matches ek ∈ N

ω

free area for deforming Ck

R

Note: The above lists some of the more important symbols present within this thesis. In addition, arrows for denoting vectors are omitted in order to improve readability.

xi

To my pioneering father. May he rest in peace.

xii

Chapter 1

Introduction 1.1

Motivations and Contributions

In computer science there exists the underlying difficulty of balancing real-time performance versus computational accuracy, as illustrated in Figure 1.1. This tug of war is also prevalent in the fields relevant to this MSc project, namely computer graphics and

Comput at i onalAc c ur ac y

animation, where visualisation and naturalness of character motion plays a large role.

Fu t u r e Un c e r t a i n t y Qu a d r a n t

Re a l Ti meFr a me r a t e Ge o me t r i cAc c u r a c y / Nu me r i cPr e c i s i o n

c u r r e n t c o mp r o mi s e Re a l Ti mePe r f o r ma n c e

Figure 1.1: Schematic plot of real-time performance of commodity hardware versus computational accuracy. As hardware improves, the middle way (illustrated as the yellow dot) will shift to an undetermined point in the upper right quadrant with ever increasing accuracy. Therefore, this intersection point is dependent on the current state of computational accuracy and graphical performance.

For this MSc project, this meant creating a middle way between having an accurate anatomical musculoskeletal model suitable for FE simulations and real-time performance 1

Chapter 1. Introduction

2

of the animation/simulation. Because the compromise will shift in the future depending on the hardware being used, it is useful to create a musculotendon model that is scalable with respect to accuracy. This scaling for real-time applications was incorporated into the project, next to a Hill-type biomechanical model that brings with it certain benefits. For instance, muscle paths are usually prearranged in models and thus prevents the tedious work of manually attaching each muscle with guidance of an anatomist. It also allows the validation of approaches that use individual as well as groups of muscles for static-optimisations on top of kinematic-based motions. An even more exciting possibility is using models to help solve dynamic-optimisation problems and achieve realistic physics-driven motion. Another advantage for students and researchers new to the field is that these rigorously tested biomechanical models can be freely obtained through open source simulators such as OpenSim (The National Center for Simulation in Rehabilitation Research, 2010), speeding up collaboration and progress. As Millard et al. (2013) described, biomechanical models usually come with a myriad of data such as coordinates for attachment points, maximum isometric forces, optimal fibre lengths, pennation angles, and so forth. Current biomechanical models, although accurate and fast for real-time applications (Millard et al., 2013), are also heavily simplified and usually represent musculotendons as polylines without any polygonal geometry. This MSc project is meant to bridge that gap by allowing high poly meshes to be approximated with embedded biomechanical polylines. The use of an underlying biomechanical model removes the need to manually align anatomical musculotendons, while the approximation of high poly models remove the need for mesh preparation, as approximated versions remain simple with segmented area and volume for muscle and tendon compartments. Thus making them ready to be used in real-time applications where volumetrically discretised musculotendons are required. The original aim of this MSc project was to investigate the possibility of simulating musculoskeletal injuries in real-time using accurate numerical techniques such as FEM(Lacl´e, 2013). However, as the project progressed it became clear that there are other problems obscured by the real-time simulation of injuries: 1. The inclusion of anatomical polygonal models for higher geometric accuracy. 2. The inclusion of biomechanical models for biomechanical accuracy. 3. Implementing a collision-free system coinciding with real-time performance. Applications in real-time computer animation that would use biomechanical musculoskeletal models, such as musculoskeletal injury simulation, orthopaedic training, and

Chapter 1. Introduction

3

validation of biomechanical effects for animated characters are thus directly dependent on the solution to these problems. Hence, this project focused mainly on addressing these three problems. The main contribution of the MSc project and this thesis are a novel way to approximate high poly anatomical meshes of musculotendons in a way that can be discretised into tetrahedrons while retaining a low amount of vertices with as much features of its high poly counterpart as possible. Making it possibly useful for real-time FE simulations. Moreover, a collision detection and response strategy was designed for geometric musculotendon models in real-time with an accurate area and consequently volume preservation technique. Additionally a strict consideration for sustainable real-time deformation was given, using a cylinder-based musculotendon construction that is similar to the early work of Wilhelms and Van Gelder (1997), but created using triangle fans and strips for an optimal GPU footprint. In addition, for finer accuracy on more capable hardware, an adaptive LOD scheme is developed. What is not included within the scope of this MSc project are a physics-based representation and a FE solver. Although a physics-based representation was developed in Bullet Physics (Coumans, 2013) with a spring-mass scaffold controlled by a couple of kinematic constraints. However, as discussed in Chapter 4, this approach turned out be unsuitable for musculoskeletal modelling and was left out of scope. Nonetheless, the model presented here was constructed in a way that makes it portable for real-time physics-engines.

1.2

Chapters overview

The remainder of this thesis is structured as follows. Chapter 2 briefly discusses previous musculoskeletal models used in the field of computer animation and computer graphics, followed by a brief review of volumetric models developed for FE simulations. Chapters 3 and 4 will thoroughly discuss the musculoskeletal model and the collision strategy that were developed for this project. The elaborate chapters are also meant for readers who are considered as newcomers to this particular area within the field of computer animation and computer graphics. Chapter 5 discusses the results, and finally Chapter 6 ends with concluding remarks with hints at future applications and improvements for the combined musculotendon model.

Chapter 2

Related Work 2.1

Musculoskeletal Models in Computer Graphics

Polygonal musculotendon models for computer graphics and animation are visualised and controlled using different approaches to augment character movement and simulate musculoskeletal deformation (Lee et al., 2010; Ramos and Larboulette, 2009; Aubel and Thalmann, 2001a,b; Albrecht et al., 2003; Mohr and Gleicher, 2003; Lee and Ashraf, 2007; Sueda et al., 2008). One of the first approaches to incorporate muscles into this domain was Chadwick et al. (1989) that used an FFD lattice to kinematically deform skin with underlying muscle. Later, Thalmann et al. (1996) used meta-balls to define the underlying musculature. At specific intermediate distances, along each bone segment they create orthogonal cross sections to define the shape of the combined meta-balls. This is similar to the musculotendon construction discussed in Chapter 3, except here cross sections are formed along musculotendon action lines obtained from a biomechanical model. Thalmann et al. (1996) also used ray-casting to extract contours in order to form the skin of the character, which is the basic principle used in this MSc project to approximate features of high poly musculotendon meshes. A year later, Scheepers et al. (1997) published another geometric-based approach that included a procedurally created skeleton and elliptical muscles with added volume preservation. In that same year, Wilhelms and Van Gelder (1997) publishes a different geometric approach that uses discretised cylinders for musculotendons attached unto skeleton meshes. It was one of the first works to use the concept of a pivot point, which is equivalent to the contemporary labelled via point. Their work was also the primary inspiration for the cylinder-based musculotendon model presented here, as it brought with it some practical real-time benefits such as radii-based deformation, as discussed in Chapter 4.

4

Chapter 2. Related Work

2.2

5

Musculotendon Models for Volumetric Deformation

Early work of Chen and Zeltzer (1992) showed that FEM could be combined with a classical biomechanical model for calculating muscle forces, which paved the way for a new generation of FEM and FVM-based approaches such as Zhu et al. (1998); Hirota et al. (2001); Lemos et al. (2001); Teran et al. (2003, 2005); Lee et al. (2009). Regarding FEM mesh generation, Jacobson et al. (2013) introduced in 2013 a novel method that is slightly similar to the winding-principle used in the match detection technique discussed in subsection 3.5.3.2 for the creation of volumetric meshes. While their method outputs FEM-ready meshes that are watertight, they still contain at least three times more vertices than the original input mesh, and thus not suitable for real-time applications. Berranen et al. (2012) used both FEM meshes and a modified Hill model for realtime deformation analysis, where contractile muscle forces operate between adjacent nodes of the FEM mesh. This is one of the few recent work found that combined a Hill-type biomechanical model with a volumetric FEM mesh with promising real-time performance, albeit the result included just a single musculotendon. Around half a decade earlier, Lee et al. (2009) published an elaborate upper body model, using the same dataset as this MSc project, namely the UHM (Snoswell, 2003). The UHM geometry was processed, as is the case here (refer to subsection 3.5.1), and volumetrically discretised for FEM. They also incorporated a Hill-type biomechanical model on the action line for the computation of muscle forces for forward dynamics. In general, FEM is still considered to be too computationally expensive for real-time simulations. Even more so when not one but several musculotendon units that are attached to the skeleton have to be simulated in concert, with active collision detection and response strategies next to the FE solver. To start looking at these challenges, it is handy to set a lower bound for the least amount of vertices that a musculotendon model should contain. Such a lower bound is discussed in subsection 3.6.2 having the goal of allowing a very low amount of vertices to be considered but still with an acceptable limit of accuracy for real-time applications. Work from Blemker and Delp (2005) also created volumetric meshes by mapping a template hexahedral mesh to another target hexahedral mesh using projections and Delauney triangulation. As a target shape for each map, two dimensional outlines of cross sections, sampled with MR imaging, were used and projected in three dimensions, which is similar to the UHM enhancement technique discussed in section 3.5. Fibre geometries were also created separately as rational Bezier spline curves with template schemes for parallel, pennate, and fan based muscle fibre architectures. Blemker and Delp (2005) concluded that this three dimensional approach to musculotendons such as the Gluteus group, Psoas, and Iliacus exhibited enough diverse behaviours that it

Chapter 2. Related Work

6

provided an added value of accuracy for estimating moment arms of muscle fibres; at least for this tested group of musculotendons. While this approach could ultimately render the use of classical Hill-type models, which are polyline-based with via points obsolete, it is still considered early work particularly when real-time usage of classical models are increasing in accuracy as the latest benchmark result of Millard et al. (2013) shows. Similarly, work of Kohout et al. (2012) took a combined approach labelled as surfacefirst and fibres-first. With surface-first, volume preservation is solved followed by decomposing the muscle compartment into fibres, where they are arranged using the template-based approach from Blemker and Delp (2005). Fibres-first starts from muscledecomposition and represents fibres as a spring-mass system. They claim to solve collisions for both approaches, but unfortunately not much detail is given on how this is done. Furthermore, their result is not fast enough for real-time applications requiring above 60Hz performance on commodity hardware. To improve realism, recent work of S´ anchez et al. (2014) stepped entirely away from template-based approaches and proposed a new workflow for embedding subject-specific fibre fields in FE models of musculotendons. They show that incorporating this information into their models led to a 10%-20% difference in predicting net muscle forces of specific patients. As these last references and others such as Tan et al. (2012) imply, the next frontier is indeed modelling on the scale of fibres, however it could still take some years before this can be considered a possibility in the real-time domain.

Chapter 3

The Musculoskeletal Model 3.1

Anatomical Accuracy and Real-Time Performance

This project aims to develop a model that approximates anatomical accuracy and is capable of real-time or interactive performance. To help reach this objective, the following two criteria are considered: • The model should include at least one anatomical model and one biomechanical model to help approximate anatomical and physical correctness. • The geometrical model has to be flexible i.e. scalable with respect to its level of detail (LOD). On faster computer hardware this allows more geometrical information to be processed in real-time. The process that culminated in the end result of each of the above mentioned criteria is described in the following sections.

3.2

The Biomechanical Musculoskeletal Model

Biomechanical models include data that are relevant to the physical aspects of the musculoskeletal model such as positions of attachment points of muscles relative to their attached skeletal bone, length and arrangement of muscle fibres, isometric forces, etc. For this project it was decided to use the combined model from Menegolo (2011) that was developed for the open source software known as OpenSim (The National Center for Simulation in Rehabilitation Research, 2010; Delp et al., 2007; Seth et al., 2011)1 . 1 OpenSim allows researchers, students, and health professionals to develop and test biomechanical models for dynamic simulations of movement.

7

Chapter 3. The Musculoskeletal Model

8

It is based on research that was carried out for both the upper and lower halves of the human body, which became the main reason to include it in the project. The model of Menegolo (2011) is backed up by the following associated publications: 1. An interactive graphics-based model of the lower extremity to study orthopaedic surgical procedures (Delp et al., 1990). 2. A planar model of the knee joint to characterize the knee extensor mechanism (Yamaguchi and Zajac, 1989). 3. A dynamic optimization solution for vertical jumping in three dimensions (Anderson and Pandy, 1999). 4. Dynamic optimization of human walking (Anderson et al., 2001). 5. A model of the upper extremity for simulating musculoskeletal surgery and analyzing neuromuscular control (Holzbaur et al., 2005). The model of Menegolo (2011) also includes polygonal meshes for the bones of the human skeleton. Using the same meshes for the skeleton came in handy while testing and verifying the attachment of muscles to make sure they are correct, especially when the skeleton meshes were scaled and oriented semi-automatically to fit the current animated character. Placement of points relative to transformed skeleton meshes could thus be visually compared with the result in OpenSim. Figure 3.1 shows one example where this is the case.

(a)

Musculoskeletal Model in OpenSim.

(b)

Musculoskeletal Model in RAGE.

Figure 3.1: Screenshots from OpenSim (a) and custom implementation of a Musculoskeletal Model (b) within the Real-time Animation Game Engine (RAGE) from Utrecht University. Visible are three muscles from Menegolo (2011) rendered as action lines. Green cubes represent attachment points for each muscle’s origin, while yellow cubes represent attachment points of each muscle’s insertion. All three muscles collectively represent the Gluteus Medius muscle on the right side of the sagittal plane.

One drawback of Menegolo (2011) that became apparent was that not all parameters were included for the upper body, notably the parameters mass and centre of mass were missing from all skeleton meshes pertaining to the upper body. These would have been beneficial for use in a physics-based simulator. The model does include a complete set

Chapter 3. The Musculoskeletal Model

9

of upper and lower body skeleton meshes that were used within this project, however solely the biomechanical muscle data of the lower body was included as first tryout. This is because the lower body, which includes the pelvis, is the predominant actuator of active and passive forces during locomotion of the bipedal human body, and this ability to move from one place to another plays an important part in the synthesis of human movement. Moreover, data from upper body biomechanical models can always be included at a later stage. Appendix A gives a complete list of all the muscles used in this project and Table 3.1 lists all six biomechanical parameters that were used for the lower body. Parameter name

(Muscle) Dataset

Object Type

maximum isometric force

Thelen 2003 & Schutte 1993

musculotendon

optimal fiber length

Thelen 2003 & Schutte 1993

musculotendon

tendon slack length

Thelen 2003 & Schutte 1993

musculotendon

pennation angle

Thelen 2003 & Schutte 1993

musculotendon

centre of mass

Menegolo (2011) (lower body)

OpenSim body

mass

Menegolo (2011) (lower body)

OpenSim body

Table 3.1: Biomechanical parameters from Menegolo (2011) included in this project.

3.3

The Polygonal Skeleton Meshes

The polygonal meshes from Menegolo (2011) representing the human skeleton had to be prepared and converted prior being able to use them in OGRE/RAGE. The conversion process started with transforming the skeleton meshes from the Cartesian coordinate system in OpenSim to the Cartesian coordinate system used in OGRE/RAGE. OpenSim utilises the laboratory coordinate system (Lund and Hicks, 2012) that is different than what is used in motion capture systems or popular 3D authoring software. Figure 3.2 illustrates orthonormal basis vectors of OpenSim versus ones used in OGRE/RAGE. Op e n Si m

OGRE/ RAGE

y

y

z

x

x

(a)

Orthogonal basis vectors in OpenSim.

z

(b)

Orthogonal basis vectors in OGRE/RAGE.

Figure 3.2: Different orthogonal bases in OpenSim compared to OGRE/RAGE. OpenSim uses the laboratory coordinate system where the z axis is switched with the x axis.

Chapter 3. The Musculoskeletal Model

10

The orthonormal transformation O equals the multiplication of coordinates of every vertex of every skeleton mesh in OpenSim and is given by the 3x3 rotation matrix:      0 0 1 xo xr          O=  yr  = 0 1 0  yo  zo zr 1 0 0

(3.1)

where (xo , yo , zo )T are the coordinates of vertices of the skeleton meshes in OpenSim and (xr , yr , zr )T are the new transformed coordinates of these vertices, ready to be used.

3.3.1

Attaching the Skeleton Meshes

Once the coordinates were transformed and exported came the task of attaching the polygonal meshes unto an animated character. Throughout this project, one of the characters of the Virtual Human Technology Lab, nicknamed Ralph, was used as the character that incorporated the musculoskeletal model. Ralph comes with its own joint (or bone) hierarchy for animation2 . As reference, Ralph’s complete bone hierarchy is given in Appendix B. In order to attach each skeleton geometry on a particular joint or bone of Ralph, the two hierarchies first had to be mapped. This mapping between the two separate hierarchies occurred offline as both OpenSim and OGRE have different ways of representing each. The OpenSim framework stores attached skeletal geometry in so-called bodies provided by the Simbody toolkit. Bodies are linked through OpenSim’s joint system. Each can have multiple geometries attached forming a hierarchical tree where each linked body represents one node and each node can consist of zero or more leafs that represent skeleton meshes3 . All the relationships between parents and children were mapped using the Extensible Markup Language (XML) and added via OGRE using Ogre::TagPoint objects. The following snippet gives a snapshot of the end result: 22 23 24 25 26 27 2

To prevent confusion, the term bone hierarchy will be used throughout this thesis to represent the matrix palette skinning technique typically used for skinned animations in OGRE and RAGE. 3 The OpenSim hierarchy of Menegolo (2011) is visualised in Appendix C for a complete overview

Chapter 3. The Musculoskeletal Model

11

28 29 30 31 32

Due to the difference in hierarchies, an XML schema proved to be versatile in the creation of a quick mapping scheme. Unique identifiers (UID’s) of skeleton meshes could be added to specific bones satisfying the 1 : N cardinality requirement, where e.g. both tibia and fibula would fall under the right knee bone of Ralph. In addition, relationships between parent and child could be mapped by adding other UID’s as children. −1’s were added for user friendliness, indicating unused values.

3.3.2

OpenSim Bodies in Cartesian Coordinates

As mentioned in the previous subsection, OpenSim encapsulates skeleton meshes within body objects for their physics-based simulator. Each body is defined in internal coordinates that are generalised in order to help solve multi-body dynamics (Sherman and Eastman, 2012), shown in Figure 3.3. As each coordinate of a body is linked and parameterised to another body in a relative frame of reference (Hicks, 2012b), it was useful to know where OpenSim bodies were positioned relative to OpenSim’s world origin. This unlinking of bodies came in handy during the skeleton alignment algorithm described in subsection 3.3.3 and also when verifying the transformation of custom OpenSim joints. c hi l dbodyB ( i )

o i 1

B( i 1 )parentbody o

2 i 1

c ont ai ns s k e l e t on me s he s

3 oi 1

I AR

di

di-1

3 oi

2 o i

c ont ai ns s k e l e t on me s he s

o i -

1 1

y T

Gr o u n d( 0 , 0 , 0 ) z x

Figure 3.3: Kinematic relationship between two bodies in OpenSim defined in a relative frame of reference and connected by a joint, the instantaneous axis of rotation (IAR). Adapted from Christophy (2010); Hicks (2012b).

Chapter 3. The Musculoskeletal Model

12

The Cartesian coordinates of each OpenSim body are given by:  BR (0) BW (i) = BR (i − 1) + BW (i − 1) − BR (i)

if i = 0

(3.2)

if i > 0

where BW gives world coordinates for position and rotation of a bodies, i ∈ N is an index in the body’s hierarchy, and BR represents the relative coordinates with respect to the IAR. Which define the displacement vectors d and co-rotational bases {o1 , o2 , o3 } that both displace and rotate each body about the connecting point of the joint (the IAR) in Figure 3.3. OpenSim also includes complex biomechanical joints next to the traditional variety of mechanical joints (such as welded, pin, slider, etc.) to model joints more accurately (Seth et al., 2011). One type of complex joint uses natural cubic splines to let the IAR interpolate along curved paths. One example for rotation of the knee along xy-plane with a fitted natural cubic spline is shown in Figure 3.4.

Figure 3.4: xy planar displacement of the fibula and tibia interpolated along a spline curve with respect to the knee joint angle θ. The position of the knee is illustrated at three flexion (negative) angles, namely 0◦ , 60◦ , and 120◦ . From Seth et al. (2011).

The natural cubic spline function used in OpenSim had to be reimplemented in this project in order to complete the unlinking of bodies to acquire the world positions and orientations of all subsequent children bodies. With all bodies unlinked it is possible to compute the length of each skeleton mesh for the semi-automatic skeleton alignment described in the next subsection.

Chapter 3. The Musculoskeletal Model

3.3.3

13

Semi-Automatic Skeleton Mesh Alignment

The next step, once the skeleton was attached, was to align each mesh along Ralph’s bone hierarchy. A couple of problems that needed to be addressed, shown in Figure 3.5, became apparent after the attachment routine. The first problem was dealing with different starting poses. The starting pose used in Menegolo (2011) is the relaxed-pose with both arms hanging loose, while in Ralph the starting pose is the more common t-pose. Next, scaling issues were also noticeable due to different anthropometric proportions in the hierarchical body compared to Menegolo (2011). Each skeleton mesh is also relatively positioned from its own world origin, which caused differences at certain areas.

(a)

Front-view with complete skeleton attached.

(b)

Close-up of Ralph’s head with floating skull.

Figure 3.5: The bone hierarchy is visualised as black line segments connected by orange cubes. (a) shows the difference in initial poses between OpenSim and Ralph, scaling issues such as gaps at the knee joint, and a more obvious problem highlighted in (b) with the skull base meshes due to relative transformations present in mesh files.

In addition, certain bones were not correctly placed anatomically. For instance, the skullbase bone was artistically placed at the top of the head to include all vertices during the skinning process, however this placement is anatomically incorrect because it lies too much on the top end of the skull. A more representative location would be in the centre of the skull, placing the skullbase bone equally distanced between the anatomical occipital and frontal bones. One solution for the skull base meshes is to simply reattach them on another bone, such as the last neck bone vc2. The hierarchy map written in XML is flexible enough to allow this, but unfortunately preliminary tests showed that the skull was still not entirely at the centre of Ralph’s head. These unresolved complications resulted in the development of a semi-automatic approach where certain transformations would be initially carried out online and supplemented with manual adjustments.

Chapter 3. The Musculoskeletal Model

14

The automatic alignment of skeleton meshes depended on the two transformations rotation and scale as these required less granular adjustments. To address scale, the transformation from internal coordinates to Cartesian coordinates for OpenSim bodies opened up the possibility to calculate anthropometric distances in Menegolo (2011) and compare these with distances from the bones of Ralph. A scaling ratio was then calculated and applied in the transformation of each mesh. To address rotation, the shortest rotational arc between bones and bodies from the separate hierarchies were calculated using quaternion functions from OGRE4 . The final result can be seen in Figure 3.6a. Scaling has improved for some areas such as the gap at the knee joints, but there are still visible problems with rotation (e.g. with the arm attachments) due to relative transformations present in the skeleton mesh files. Figure 3.6a makes clear that although the automatic scaling and rotation techniques improves initial errors, the following pair of problems still remains to be solved: • Relative displacements present in each skeleton mesh file. • Artistic differences in the bone hierarchy compared to the OpenSim hierarchy. Furthermore, one requirement with automatic alignment is that each transformation requires a link with a child in both hierarchies in order to compare distances. Such cases are not always present, such as skeleton meshes of the forefoot that are attached to leaf nodes in the bone hierarchy of Ralph. Hence, to resolve the above mentioned issues, it was decided to add an extra pre-processing step that would adjust all three transformations, namely translation, rotation, and scale with hard coded values. For translations, manual adjustments were already planned as automating translations requires techniques to automatically determine boundaries of each skeleton mesh to determine how far away a mesh should reside from a bone. In addition, translations require fine control (on each separate axis) to achieve a good result. Manual adjustments also created the possibility to interactively alter each mesh, which speeded up the manual alignment process. The outcome of the semi-automatic alignment process is visible in Figures 3.6b and 3.7. 4 As a reference, the function that computes the rotational difference in quaternions can be studied in Torus Knot Software Ltd (2013).

Chapter 3. The Musculoskeletal Model

(a)

Front-view showing automatic alignment.

15

(b)

Front-view showing semi-automatic alignment.

Figure 3.6: Automatic skeleton alignment with respect to scale and rotational transformation is not enough as shown in (a). The final result is shown in (b). Adjustments can be made for each of the three transformations with three DOF totalling nine DOF.

Ralph’s skin-transparency was also set to be interactively adjusted, which demonstrated to be an effective way to visualise penetrations of the inner skeleton and later on the musculoskeleton.

Figure 3.7: Once the semi-automatic skeleton alignment was completed on the initial pose, an animation for Ralph was loaded to further look for penetrations with the mesh of Ralph, i.e. if the skeleton was fully contained. As a visual aid, an extra widget was added to RAGE’s GUI to interactively change the transparency of Ralph’s mesh in OGRE. The above figure shows an animation history plot of Ralph loaded with a walk-cycle animation and Ralph’s mesh transparency set to 50%.

3.4

A Simple Geometric Musculotendon Model

The geometrical musculotendon model combines muscle and tendons into a single geometrical object. Each musculotendon can be defined by the following vector space M

Chapter 3. The Musculoskeletal Model

16

with dimension n as the index counter is positive and zero-based:  M = m0 , m1 , ..., mn−1 | mg ∈ R3

(3.3)

where each element mg is a position vector in R3 Cartesian coordinates. The initial shape of the musculotendon can be thought of as a closed cylinder with the starting point, the origin point, lying at the centre of the top cap. This point is attached relative to a specific bone in the bone hierarchy, which can also contain other siblings representing other musculotendons or other skeleton meshes. The end point, or insertion point, at the centre of the bottom cap is attached relative to either the same or another bone in the hierarchy. OpenSim refers to these two attachments as fixed points in their framework (Hicks, 2012b)5 . All attachments are children of a bone where the attached skeleton meshes are part of the same OpenSim body as the musculotendons in the data of Menegolo (2011)6 .

3.4.1

Adding Action Lines and Via Points

Biomechanical models, such as those used in OpenSim (The National Center for Simulation in Rehabilitation Research, 2010), usually simplify a musculotendon as a line or a set of connected lines, referred as the action line between the origin and insertion coordinates (Delp et al., 1990) that defines each muscle’s path. The action line, shown in Figure 3.1a, lies at the core of the models in Menegolo (2011), which for some parts was designed to match moment arms that where measured experimentally (Holzbaur et al., 2005). An action line is defined as a vector space of subset A in this thesis and is always contained within the boundary of M , hence A ( M , and given as:  A = a0 , a1 , ..., an−1 | ak ∈ R3 , |A| ≥ 2 and

(3.4)

where each ak is a position vector on the action line representing coordinates in R3 . The first and last element of A represent the attachment coordinates linking musculotendons to their respective meshes of the skeleton. During animation, the origin and insertion coordinates are updated kinematically as each attached skeleton mesh also moves with the character’s animated bones. Besides origin and insertion attachments, the biomechanical model of Menegolo (2011) includes an additional set of constraints that are supported by The National Center for Simulation in Rehabilitation Research (2010), such as wrapping surfaces, via points, and 5

Points are sometimes used as simplified constructs in real-time simulators. In actual biological systems musculotendons are usually attached at sites, known as entheses, and not at a single point (McGonagle and Benjamin, 2013). 6 Refer to section 3.3 for more information on the bone hierarchy used for this project.

Chapter 3. The Musculoskeletal Model

17

moving muscle points that allow the model to approximate empirical data more closely. A brief description of each is given in Table 3.2. Constraint Type

Description

Wrapping Surface

Simple primitives (spheres, ellipsoids, cylinders, and torii) that allow the action line to wrap around the primitive’s surface. Uses two extra points placed on tangents of the primitive that define the point of contact initiation and of contact breaking.

Via Point

Extra attachment points fixed to an OpenSim body and activated when a specific parameter (such as a joint’s knee angle) lies in a given range.

Moving Muscle Point

The coordinates of this constraint are passed as functions such as natural cubic splines instead of constants. Useful for moving the action line when an articulate body is being flexed or extended and a wrapping surface is not sufficient to recreate the proper motion.

Table 3.2: Summarised descriptions of constraints within OpenSim (Hicks, 2012a).

For this project it was decided to include solely the via point fixed constraint due to the following reasons. An action line, as used in OpenSim is not enough to create a polygonal representation of a musculotendon. To approximate a more accurate shape of a musculotendon, an enhancement process was also integrated into the musculotendon model7 . The exclusion of other constraints, such as moving muscle points, gave the geometrical model more freedom to realign each action line A within a high poly mesh. Furthermore, no physics-driven biomechanical simulation was required for this project and therefore no accurate force-length and force-velocity relationships were needed, i.e. the derivative of an action line’s length was not dependent on neither forces driving each musculotendon unit nor static optimisations. For that reason, the inclusion of fixed via points as a way to approximate the curve of A before executing the enhancement routine was sufficient. Finally, Rankin and Neptune (2012) states that the algorithms used for wrapping surfaces are computationally expensive and thus computational overhead was also reduced by excluding this type of constraint. As briefly described in Table 3.2, via points also exhibit a limited activation range. As a result, via points are not present all the time on A. Instead of switching via points on and off during gait, it was opted to discard the activation range and activate all via 7

Refer to section 3.5 for a thorough overview of enhancement with the UHM.

Chapter 3. The Musculoskeletal Model

18

points by default. This would not only simplify the simulation’s update loop but more importantly set the amount of points along the action line as a constant finite set. This brings two benefits, namely: 1. LOD techniques for the action line can be introduced now that the model uses a fixed amount of via points. 2. Musculotendons defined as a system of spring-masses have a fixed amount of kinematic nodes from the get-go, which is beneficial for real-time physics engines. The drawback is that for some intervals, there is the risk that A would not wrap around a skeleton mesh and would instead cause penetration. However, seeing that the geometrical model is represented as a simple polygonal mesh, a collision management system would have to prevent penetration for all vertices and this would automatically solve all wrapping issues. In addition, the ability to include LOD techniques on a constant amount of points to adapt the surface and volume of the musculotendon brings more accuracy to the geometrical model than not including LOD but dynamically adding and removing points on and off of A. With the addition of via point constraints, the model now provides finer kinematic control over the direction of each muscle along the skeleton. Via points also approximate the continuity of a muscle more closely than a mere line segment effectively turning each muscle’s action line into a polyline as shown in Figure 3.8.

(a)

Action line without via points.

(b)

Action line with via points.

Figure 3.8: Screenshots from RAGE showing the action line of one section of the Gluteus Maximus muscle from Menegolo (2011) without via points (a) and with via points (b). The via points, visualised as pink cubes, add extra kinematic control to the geometrical model and better approximation to the curve of the action line.

Not all musculotendons present in Menegolo (2011) contain via points, the Gluteus Medius, Rectus Femoris, and the Pectineus are some examples. Therefore, the least amount of elements that can be contained in A is two for the origin and insertion points. When via points are present in a muscle, each element v from the set V lies always at another point on the action line in the geometrical model presented in this thesis meaning V ⊆ A.

Chapter 3. The Musculoskeletal Model

3.4.2

19

Adding Volume with Cross Sections

The inclusion of via point constraints in the model is not enough to give each muscle volume. At this moment, M is still a polyline defined by A. To actually add volume, extra vertices are needed for its surface. The initial geometric state of the model, a polygonal mesh constructed as a closed cylinder, would already include two cross sections for the top and bottom caps. To also account for muscles that contain via points, each closed cylinder can be split into smaller segments by introducing bisecting cross sections at each element of A. The subset Ck containing each element of an arbitrary cross section that is responsible for volume, of which none are on the action line, is defined as: n o Ck = ck0 , ck1 , ..., ckn−1 | cki ∈ R3 , Ck ( M with

(3.5)

Ck ∩ A = ∅.

(3.6)

With Ck defined as being another proper subset of M , the relationship A ( M is now also conformed. The cardinality of Ck is also proposed as |Ck | ≥ 6 ∧ |Ck | = 2ℵ for reasons discussed in subsection 3.5.3.2. Another area of Ck that is also worth defining is the new direction of the up-vector for ak . This would allow the segmented cylinder to ”follow” the curve of A more closely, as shown in Figure 3.9b and 3.9c.

(a)

Parallel to y (default).

(b)

 Parallel to ak+1 − ai T .

(c)

 Parallel to ak+1 − ak−1 T .

Figure 3.9: Three possibilities for choosing an up-vector for bisecting cross sections. The chosen computation is (c) which takes the up-vector (ak+1 −ak−1 )T as this approximation includes information from the point before and the point after the current point. Bisecting cross sections are defined such that a ∈ A and k ∈ N with 0 < k < (|A| − 1).

The default up-vector is shown in Figure 3.9a, which in this case coincides with the −y axis for this particular muscle, the Gluteus Maximus 3. This is because each muscle is built starting at the origin and ending at the insertion point, and for this particular musculotendon from Menegolo (2011), the origin point lies above the insertion point in the y direction. Hence the direction is downwards in the −y direction given as (0, −1, 0)T . Next is Figure 3.9b showing up-vectors pointing in the forward direction parallel to a segment of the action line between ak+1 and ak . The last tried approach, Figure 3.9c,

Chapter 3. The Musculoskeletal Model

20

includes information from both sides of ak with the vector (ak+1 − ak−1 )T resulting in an approximation that is centralised and therefore more robust when dealing with acute angles between two other cross sections.

3.4.3

Including Tendons into the Model

Before the model can be recognised as a musculotendon model, a geometrical representation of the tendon must be included. Unlike graphical representations of musculotendons such as Sueda et al. (2008); Geijtenbeek et al. (2013); Teran et al. (2005); Scheepers et al. (1997), tendons are considered as a single piece lying on one side of the musculotendon unit in classical models such as Hill’s and Zajac’s (Zajac, 1988). However, like their graphical counterparts most real tendons are located on each side of a muscle (Encyclopedia Britannica, 2014b). The biomechanical models in Menegolo (2011) include parameters that would satisfy the equations in classical models, yet are not sufficient to provide an accurate geometric representation for each tendon. For instance, the tendon on the side of the origin might take 80% of the total tendon length with the remaining 20% by the tendon on the insertion side, or vice versa. Therefore, the length distribution of each individual tendon remains unknown. In addition, the parameter tendon slack length not only represents the total length of both tendons but also combines the length of free and aponeurotic tendons (Delp et al., 1990), and there is no distinction as to where the free tendon stops and the aponeurotic tendon begins. As a result, no dataset was found for this project that included lengths for each separate tendon and for each component of the tendon, dividing each slack length parameter into the two separate lengths. Figure 3.10 illustrates a schematic representation of the problem. u n k n o wnmm Apone ur ot i cTe ndon

or i gi n

Mus c l e

Fr e eTe ndon

Apone ur ot i cTe ndon

u n k n o wnmm

unknownmm

u n k n o wnmm Fr e eTe ndon

i ns e r t i on

Figure 3.10: Classic musculotendon models such as Zajac’s (Zajac, 1988) combine all four unknown lengths into a single term for the tendon. While accurate in biomechanics, these unknowns make it challenging to define an anatomical three dimensional model of a musculotendon with two individual tendons. Adapted from Hoy et al. (1990).

Without the necessary biomechanical data to calculate the correct anatomical length of each individual tendon, it became a challenge to define a way of incorporating tendons into the model. Although anatomically inaccurate, it was decided to implement just one

Chapter 3. The Musculoskeletal Model

21

tendon under the assumption that, at least for the lower body, most free tendon lengths seemed to be distributed on the insertion side in anatomical illustrations such as Taylor (2013). Hence, within this model, the length of the tendon is computed backwards, starting at the insertion point and traversing along the action line in the direction of the origin point for all musculotendons. To compute the tendon length, two calculations must be preceded beforehand, which are: 1. Calculate the rest length of the musculotendon. 2. Calculate the scaled rest length of the musculotendon. This generates a total of four separate variables for lengths, namely the length of the action line and the length of the scaled action line, denoted by lA and lAS respectively, the rest length denoted by lR , and finally the scaled rest length denoted by lRS . Scaled versions of lA and lR are used in this model because the skeleton was transformed during the semi-automatic alignment process and so were all the attachment points in order to fit Ralph. It is also worth noting that due to the use of a simple kinematic system for animation, the model does not include any static nor dynamic optimisation pipelines. This fact combined with knowledge from Nordin and Frankel (2012); Benjamin et al. (2006) that the elastin proportion is approximately 2% and maximum strain around 6% led to the decision that tendons are assumed to be infinitely stiff in this model. The assumption that the tendon is rigid would allow its length to be set equal to its slack length, denoted by lsT . This opened up the possibility to adapt the implemented version of the classical Zajac model given in Delp et al. (1990) to derive the rest length lR as follows: lR = lsT + loF cos α



(3.7)

where loF represents the optimal fiber length and α the pennation angle present in pennate musculotendons. All three parameters including the lsT were used from Menegolo (2011). To derive the scaled rest length lRS , the ratio between the |A| in Menegolo (2011) and the scaled |A| in OGRE must be calculated first. The generic calculation for the |A| is given by the sum of the Euclidian length between two ak elements: n−2 X

T (ak+1 − ak )

(3.8)

k=0

where the outcome can be assigned to lA or lAS depending on which state of A the calculation was executed. Next, lRS is given by: lRS =

lR lAS . lA

(3.9)

Chapter 3. The Musculoskeletal Model

22

With lRS assigned, the scaled tendon slack length lsT S can also be obtained with: lsT S =

lRS lsT . lR

(3.10)

After the UHM enhancement, which will be discussed in section 3.5, it was found that for most musculotendons the tendon length is longer than a noticeable free tendon length in the high poly meshes. For the purpose of representation, it was then decided to use 1 T 2 ls

which gave visually a more correct result but still remained anatomically inaccurate.

Table 3.3 demonstrates a result with the Tibialis Anterior of the right leg. loF

α

lsT

lA

lAS

lR

lRS

lsT S

1 TS 2 ls

0.098

0.087

0.223

0.303

0.292

0.320

0.309

0.215

0.107

Table 3.3: For the Tibialis Anterior, its scaled tendon slack length lsT S was found to be ≈ 0.215 with half of that length being ≈ 0.107 using Equations 3.7 till 3.10. These were computed after the skeleton was attached and aligned to fit Ralph. All values are given in metres and rounded up to millimetres for this thesis.

Equation 3.10 is then used to determine which segment of A contains the tendon point. Let first the function d (k) equal a length of a negative vector between two points on A: d (k) = (ak − ak−1 )T

(3.11)

where k is the index iterating over A in N. Starting from the insertion point n − 1, the travelled distance dtg along A is then computed with:

dtg =

1 X k=n−1

  1 TS d (k) d (k) ≤ ls 2

adding only the lengths where the condition d (k) ≤

1 TS 2 ls ,

(3.12)

contained within Iverson

brackets ‘[]’, is satisfied. To return the last index j that represents the last point before the tendon point, a final condition is required on j:   1 TS j = k for d (k) ≤ ls . 2

(3.13)

To give an example, let the number of points n for a particular musculotendon be equal to 5. During a particular computation, the condition in Equation 3.12 returned a 1 up until k = 3, consequently leaving j = 3 as well. Because of the ≤ sign present in the condition, the tendon point in A would lie somewhere between a2 and a3 . This example is further illustrated in Figure 3.11. This figure also illustrates schematically what was discussed previously with lack of biomechanical data prompting to implement

Chapter 3. The Musculoskeletal Model

23

a single tendon using 21 lsT S as a way to visually represent the free tendon attached on the insertion side, but this remains anatomically inaccurate.

0 o r i g i n p o i n t

1

v i ap o i n t s

2

dtg

3

4

t e n d o np o i n t

i n s e r t i o n p o i n t _ 1 TS 2s

l

Figure 3.11: A schematic representation of the musculotendon model including a single tendon point lying somewhere between a2 and a3 .

The actual position of the tendon point tg based on 21 lsT S can now be computed using linear interpolation between two points on A: tg = aj−1 + s (j) with

(3.14)

 d (j) + dtg − 21 lsT S (aj − aj−1 )T . s (j) = d (j)

(3.15)

s (j) computes the ratio between scaled tendon slack length and the distance until the point aj−1 and then scales the magnitude of the vector between aj and aj−1 . The position of aj−1 plus this outcome is used to calculate tg . At tg , a cross section Ct is added to divide tendon and muscle. Each element of Ct is also linearly interpolated between the previous and next cross sections as each cross section can have different shapes. To retain the assumption that the tendon is infinitely stiff in this model, 21 lsT S is computed only once to fix the length of the free tendon. This is done after the skeleton is aligned and the UHM enhancement process are completed, but prior to loading any animations, i.e. when Ralph is still in his t-pose configuration.

3.5

Geometrical Enhancement with the UHM

The Ultimate Human Model data set (UHM) is according to Snoswell (2003), ”the most complete and accurate set of 3D models of the human musculoskeletal system that has ever been created”. However, it is also possible that the data set is artistically tweaked as no reference to any work is given at www.cgcharacter.com (Snoswell, 2003). Nonetheless, it does state that its accuracy comes from anatomical texts, papers, and custom MRI scans that went into its more than 12 man years of creation.

Chapter 3. The Musculoskeletal Model

24

The goal of using the UHM in this project is to enhance the simple cylindrical shape of each musculotendon with detailed geometries, thereby leveraging the accuracy of the real-time musculotendon model. The enhancement starts by preparing each mesh for import into RAGE, where they will be attached to their equivalent musculotendon of OpenSim8 . Once the UHM meshes are all attached and oriented in the right configuration, the enhancement process discussed in subsection 3.5.3, can commence. The first step, mesh preparation, is discussed in the following subsection.

3.5.1

Preparing the UHM Meshes

Because the musculotendon model should stay simple and volumetric for future volume discretisation techniques, it was required to adapt certain meshes of the UHM. Lee et al. (2009) confirms this adaptation by stating that the UHM meshes are not well conditioned for a physics-based simulator and report aspect ratios for elements as high as 60:1 and edge length ratios between 1000:1. Certain meshes also exhibit open areas, which could pose problems for volume discretisation techniques as no closed surface can be defined. Thus, similarly to Lee et al. (2009), holes present in meshes representing musculotendons needed to be closed. For this project, this was accomplished using the 3D authoring software Blender Foundation (2006). One example is shown in Figure 3.12.

(a)

Right Soleus represented as an open mesh.

(b)

Right Soleus represented as a closed mesh.

Figure 3.12: Comparison between an open and closed UHM mesh of the Soleus. The large open area on the front is due to Gastrocnemius and Calcaneal that were occupying that space. The final result, shown in (b), was manually processed using Blender (Blender Foundation, 2006). 8

Appendix D shows the final result of the mapping process.

Chapter 3. The Musculoskeletal Model

25

Furthermore, some meshes would exhibit self-intersections and these would give incorrect results after the enhancement process was completed for reasons discussed in subsection 3.5.3. To ameliorate potential risk areas, vertices that were positioned on opposite sides were repositioned so as to keep the polygonal UHM mesh penetration free. An example is given in Figure 3.13.

(a)

Self-intersection of one vertex (top).

(b)

Corrected mesh without self-intersections.

Figure 3.13: Self-intersection issues with the UHM meshes such as the Iliotibial Band. The corrected result, shown in (b) was manually processed using Blender.

Finally, some UHM meshes were combined to fit a single musculotendon unit in OpenSim, while others where divided for cases where OpenSim represents single musculotendons with multiple action lines, such as the Gluteus muscles that are characterised with broad attachment sites (Blemker and Delp, 2005). Figure 3.14 shows one example for the divided Gluteus Maximus processed in Blender. A complete list, overviewing both divided and combined the UHM meshes, is given in Appendix D.

Figure 3.14: An example of the Gluteus Maximus divided (or cut) in Blender for cases where OpenSim uses multiple action lines due to large attachment sites.

Chapter 3. The Musculoskeletal Model

3.5.2

26

The UHM Mesh Shape Analysis

With the inclusion of the UHM meshes for the lower body, a more accurate representation could now be achieved for each musculotendons’ custom shape. As musculotendons have different architectures, and each musculotendon has its own unique anatomical shape, a dilemma arose as to what modelled shape would cover the most space contained within each UHM mesh. To address the above dilemma, it was decided to make a small analysis of each high poly mesh borrowed from the UHM. As an input source for the analysis, three slices from each processed mesh were sampled at approximately first quarter, half way, and third quarter lengths, each orthogonal to the longitudinal axis of each musculotendon. To get an idea for what kind of shapes the high poly meshes would consists of, its isoperimetric quotient is calculated and convex-concave tests are performed on each slice. The isoperimetric quotient, denoted by q, is used to calculate the ratio between the area a of an arbitrary slice U Sk and the area of a circle with the same perimeter p as U Sk . It is calculated as: q=

4πa U Sk p (U Sk )2

 .

(3.16)

Convex-concave tests were then calculated by first sorting the edge list of U Sk using a method described in Appendix E. After the edges were sorted in a cyclic manner, the z component of the cross product was used to determine whether the simple polygons represented by U Sk were convex or concave. The computation starts first by aligning all slices in the same XY plane. For each vertex ui of U Sk with coordinates uxi , uyi , and uzi = 0, the following calculation is performed:       uxi+1 − uxi uyi+2 − uyi+1 − uyi+1 − uyi uxi+2 − uxi+1        Zi = uxi+1 − uxi uy0 − uyi+1 − uyi+1 − uyi ux0 − uxi+1     x (u0 − uxi ) (uy1 − uy0 ) − (uy0 − uyi ) (ux1 − ux0 )

if 0 ≤ i < (n − 2) if i = (n − 2) if i = (n − 1) (3.17)

with i denoting a zero-based index in N,n equals the number of vertices in

U Sk ,

and

Z representing the z component of the cross product. If all Zi ’s have the same sign, i.e. they are all either positive or negative, then no angle has surpassed the [0◦ , 180◦ ] or [−180◦ , 0◦ ] thresholds. In this case the simple polygon is considered convex, otherwise it is considered concave. Figures 3.15a and 3.15b shows the results of the analysis9 . 9

It is worth noting that the area of each slice was normalised to 1, making it possible to overlay each slice on top of each other to get the composited image render shown in Figure 3.15b.

Chapter 3. The Musculoskeletal Model

27

Perimeters with normalised area (1m)

12 11

Concave

10

Convex

9 8 7 6 5 4 3 0

0.2

0.4

0.6

0.8

1.0

Isoperimetric Quotient (q)

(a)

Plot of perimeters versus isoperimetric quotients.

(b)

Top-view of 72 slices from 24 UHM meshes overlaid.

Figure 3.15: Two results from the analysis of 72 slices taken from 24 UHM meshes for the lower body. (a) shows a plot where for each slice its isoperimetric quotient was calculated from its perimeter with a normalised area. (b) shows overlaid renders, also from 72 slices with normalised areas. Each slice was centred around their centroid and positioned on the same plane. The composite image was blended with the difference mode in inverted RGB colours in order to highlight intersections between slices. All UHM meshes that were used in this analysis are given in Appendix D.

Apparent from Figure 3.15a is that there are more concave than convex slices, but as the normalised perimeter gets smaller, there appears to be more convex slices, with convexity increasing for q > 0.6. Furthermore, 75% (or 54) of the 72 slices are more inclined towards circular than elongated shapes, i.e. when q > 0.5. Finally, the result also indicates that there is a concentration of shapes with an isoperimetric quotient around 60%. This can be inferred from the difference of 1% between the median and arithmetic mean of q (Figure 3.15b) that presumes a symmetrical distribution for the 72 slices that were sampled from 24 UHM meshes for the lower body. Although the results of the analysis show circular tendencies, there are also highly concave slices, as shown in Figure 3.15b (bottom left). This poses a problem for realtime musculotendon models, as more vertices are required to approximate concave than convex shapes. Nonetheless, by taking the circular tendencies and real-time performance into consideration, it was decided to approximate the high poly geometry by trying to reposition each element of Ck at the surface of its corresponding UHM mesh. This is accomplished with ray-triangle intersection tests described in the next subsection with the goal of having Ck remaining as a simple polygonal representation of its high poly counterpart. In addition, if the real-time model requires better approximation, it would be possible to increase the amount of elements in Ck before casting rays for the intersection tests with the UHM.

Chapter 3. The Musculoskeletal Model

3.5.3

28

A Technique for Musculotendon Enhancement

The general idea for enhancing the musculotendon model is illustrated in Figure 3.16. It starts by scaling the UHM mesh so that the attachment points of the cylinder-based musculotendon are corresponding to similar areas within the high poly mesh. Each mesh is then oriented to face its right direction during t-pose. Next, for each surface vertex, or each element of an arbitrary cross section Ck , rays are shot in the normal direction to search for intersections with the high poly mesh. Once intersection points are found, each element of Ck is translated towards each intersection point along the direction of the ray. The intersection tests can be repeated a couple of times in order to determine whether elements of A should also be repositioned at a new inner point of a cross section (or slice) of an UHM mesh. This is further discussed in subsection 3.5.3.2.

(a)

(c)

Cylinder-based musculotendon with via points.

Rays in normal direction intersecting high poly mesh.

(b)

High poly mesh aligned between origin and insertion.

(d)

Final enhanced musculotendon.

Figure 3.16: General idea for enhancing the musculotendon model with a high poly reference mesh with the help of ray-triangle intersection tests, in the order of (a) to (d).

Chapter 3. The Musculoskeletal Model

29

It was chosen to use the fast ray-triangle intersection algorithm of M¨oller-Trumbore (M¨oller and Trumbore, 1997), instead of contour hull algorithms such as Gurram et al. (2007) and Duckham et al. (2008) as both the UHM meshes and cylindrical musculotendons are configured differently with respect to the amount of vertices and with respect to the three-dimensional position and orientation of their vertices.

3.5.3.1

Repositioning C’s on the Surface of U

Generally speaking, an element cki for an arbitrary Ck is repositioned when an intersection point pi is found between a surface triangle of an UHM mesh U and a ray that has been cast from cki . For each ck , rays are cast in two normal directions on the plane of every Ck , namely positive and negative. With the positive direction being the normalised T T k−a vector c\ and the negative direction being the normalised vector a\ − ck . In k

i

k

i

addition, each surface triangle of U that intersects this plane creates a Jordan curve, denoted as U Ck , in the same plane as Ck . The ray-triangle intersection tests are used to find three possible outcomes, which is when: 1. U Ck lies completely outside of Ck in the plane containing Ck . 2. U Ck lies completely inside of Ck in the plane containing Ck . 3. U Ck lies partly outside of Ck in the plane containing Ck . The technique incorporated two additional preferences to cope with the two sides of each triangle’s face, i.e. when an intersected triangle was either back or front-faced. The first preference is for choosing intersections found through rays being cast in the positive direction P above the negative direction N , shown schematically in Figure 3.17.

N

p5

P

N

p0 p4

N

N

N

p1 ak

N

P

p2

N

p5

p3 N

p0 p4

N

N

p1 ak p2 p3

N

N

Figure 3.17: Showing ray-triangle intersection outcome number three (left) and outcome number two (right). U Ck is represented by the green shapes and pi ’s denote intersection points. pi ’s can be found with a ray cast in the positive direction P or in the negative direction N . The left figure shows that here N of p5 equals p2 and N of p2 equals p5 , illustrated with the light-red line segment. The figure on the right illustrates a case where no P -points were found, and instead resorted to finding N -points.

Chapter 3. The Musculoskeletal Model

30

The second preference is that for P , the furthest pi of each back-face triangle is returned, while for N the closest pi of each front-face triangle is returned, shown in Figure 3.18. The combination of these two preferences allows each cki to find its corresponding pi .

pi P k c i

pi a k

N

a k

k c i

Figure 3.18: Schematic figures showing outcome one and highlighting the second preference: If more than one pi is found for P , choose the furthest back-face triangle. Conversely, if more than one pi is found for N , choose the closest front-face triangle.

3.5.3.2

Repositioning Elements of A within the Volume of U

Ck ’s are constructed with algebraic radii, denoted by the set Rk , between elements of T 10 k Ck and corresponding elements ak of A . Each radius ri is obtained with cki − ak T or ak − cki . Using radii proved to be versatile as it allowed Ck to have an initial shape that was useful for the intersection tests and allowed each deformation to be kept in memory by storing distances as radii. This brought the benefit that a point within U Ck could be determined by checking which radii have been updated and consequently iterate the process discussed previously to achieve a better approximation for each U Ck . To explain the approximation process visually, a key example is introduced that will be used throughout the remaining subsections. The example uses the index k = 1 in the longitudinal direction where U C1 turns out to be a closed concave shape positioned outside of C1 and is schematically illustrated in Figure 3.19. The illustration shows the end result indicating why locating a point within U Ck (an inner point) is necessary as the result equals a false representation with the deformed hexagon being complex instead of simple.

Match Detection

First comes the task of detecting a match between two algebraic

radii that have been updated. A match is denoted by ek and is defined as an ordered pair of indices (i+ , i− ) that correspond to an index i of p where its distance is established with either a positive or negative algebraic radius for two opposing elements of Ck . In this respect, the or-relationship is exclusive and imposes the following condition: One of the two opposing algebraic radii has a negative sign. 10

For a more detailed description of this process, refer to section 3.7.

Chapter 3. The Musculoskeletal Model

31 1 d 0

p 0

1 c 2o

c

1 5

p 3 1 c 0

1 r 1 3r 0 1 r 1 4 1 r 1 1 1 r 15 r 2

1 c 4 a 1 o c 1 C 1 c 3 1 o c 0

c

1 1

1 c 2

0 1 d d 3

1 o

UC1 p 4

p 1

p 5 p 2

D1 1 o 1 d 1 d 4

1 d 1

1 d 5 1 d 2o 1 d 2

Figure 3.19: Showing a key example with a concave slice of an UHM mesh, denoted as U C1 . Applying the preferences and conditions discussed previously results in the complex polygon D1 , which is a false representation due to a1 being positioned outside of the targeted cross section U C1 .

In order for a pair of cast rays to be considered equal, for instance c11 and c14 in Figure 3.19, it is important that both c1 ’s are symmetrical counterparts with respect to a1 , i.e. opposites. This is ensured by requiring that the cardinality for every Ck is always even and never odd, which is the case here for the current example that uses six vertices, i.e. a hexagon. From this example it is also visible that for every Ck with |Ck | ∈ 2ℵ and |Ck | = 6, the maximum amount of matches would be always equal to 12 |Ck | thereby reducing the amount of match-lookups to half. Hence, each opposing element of cki can be deduced by taking the opposing index io with:   1 1 i = i + |Ck | where i ∈ N i < |Ck | . 2 2 o

(3.18)

Each detected match helps to determine when a ray passed a centre point ak as this suggests the possibility that U Ck lies somewhere (at least partly) outside of Ck . Because each surface vertex or cki is drawn by computing its radius, a ray that surpassed ak in the negative direction N needs a negative algebraic radius to correctly represent the intersection point’s mirrored position. This is accomplished with the following condition:  T pi − cki > rik . In this case, a negative sign is added to update the radius with the following new result: − (pi − ak )T . In all other cases, the radius remains positive. In the current example, radii r31 , r41 , and r51 become negative, depicted with orange line segments in Figure 3.19 and overlapping the positive radii r01 , r11 , and r21 lying in the same direction.

Chapter 3. The Musculoskeletal Model Permuting Matches

32

Once all matches are detected, they are stored in a single

container, denoted by the set Ek with |Ek | ≤ 12 |Dk |, and where Dk represents a deformed cross section at index k. Permuting the elements of Ek is necessary for the solution to locate an inner point that is discussed in the next paragraph. Visually, the result of the  permutation on Ek is a sequence of matches so that the first term 0, ek0 represents a match at one end of U Ck , winding through the continuity of the shape, and reaching the  final term n − 1, ekn − 1 at the other end of U Ck ; given that the amount of elements is greater than one. Because relative spatial configurations of U Ck with respect to Ck have many possibilities, the intersections tests will not necessarily return a sequence of matches in an ordered fashion when the tests always start at ck0 . One example is when U Ck is intersected by rays cast from elements ck5 and ck0 . The example given in Figure 3.20 illustrates that when starting the search at c10 , the elements of E1 would contain the same set of matches. Thus without taking the sign of each algebraic radius into consideration, there is no easy way of determining whether the first match starts with the rays cast initially through c10 or through c15 . 1 e 0 1 c 5 1 c 4

a1 C1 1 c 3

1 c 1 1 c 2

UC1

UC1

1 c 0

1 e 2

1 e 1

1 e 2

1 e 0

1 c 5

1e 1e 1 E1={ e 0, 1, 2}

1 c 4

1 c 0

a1 C1 1 c 3

1 c 1

1 e 1

1e 1e 1 E1={ e 0, 1, 2}

1 c 2

Figure 3.20: Schematic figure showing the same set E1 with matches detected in the same order but for two different shape configurations, prompting the need to track if a c1i was repositioned to d1i with a positive or negative algebraic radius.

By filtering the type of intersection that produced either a positive or negative algebraic radius, the elements of Ek can be rearranged so that the search winds correctly, similar to the concept of chain codes. Searching with index i = 0 up until i = 12 |Dk | − 1, a sign of the computed radius for all dki is chosen. If the new position dki was established with a negative algebraic radius, then its opposing element dkio is chosen such that this would contain the correct point, i.e. i+ = io . This example is further elaborated in Table 3.4 that shows all possible permutations with i+ using the same configuration as previous examples with U Ck lying outside and intersected by three pairs of rays cast from Ck with |Ck | = 6. Note that only half of the points of Dk (or 12 |Dk |) have to be searched to get all six permutations.

Chapter 3. The Musculoskeletal Model Sign of r for dk0 , dk1 , dk2 f : (+ + +) → f : (+ + −) → f : (+ − −) → f : (− − −) → f : (− − +) → f : (− + +) →



 dk0 , dk1 , dk2  dk0 , dk1 , dk2o  dk0 , dk1o , dk2o  dk0o , dk1o , dk2o  dk0o , dk1o , dk2  dk0o , dk1 , dk2

33

Permutation with i+ of dki

Rearranged matches

σ (0, 1, 2) = (0, 1, 2)

hek0 , ek1 , ek2 i

σ (0, 1, 5) = (5, 0, 1)

hek2 , ek0 , ek1 i

σ (0, 4, 5) = (4, 5, 0)

hek1 , ek2 , ek0 i

σ (3, 4, 5) = (3, 4, 5)

hek0 , ek1 , ek2 i

σ (3, 4, 2) = (2, 3, 4)

hek2 , ek0 , ek1 i

σ (3, 1, 2) = (1, 2, 3)

hek1 , ek2 , ek0 i

Table 3.4: Taking the sign of r into consideration allows Ek to be rearranged.

With the example given in Table 3.4, the actual permutation σ computed on indices of Ek (visible in the middle column) is given in pseudo-code with Algorithm 1. − ). input : A set Ek containing ordered pairs of matches (i+ , i k k output: A sequence hen i with a range of en : 0 ≤ n < |Ek | . 1 2 3 4 5 6

if |Ek | > 1 then Ek ← SortAscending(Ek ); // sort numerically from low-high hetemp i ← Ek ; n  temp match ← ek0 ;  mindex ← ek01 ; // note that ekn1 represents i+ for every ekn nshif ts ← 0; // index where to insert new term in hetemp i n

for m ← 1 to |Ek | − 1 do if ekm1 − mindex  6= 1 then 9 temp match ← ekm ; 10 hetemp i ← hetemp i − (m, etemp n n m ); temp i + (nshif ts, temp match); i ← he 11 hetemp n n 12 nshif ts ← nshif ts + 1; 13 end 14 mindex ← temp match1 ; 15 end 16 hekn i ← hetemp i; n 17 end Algorithm 1: Permutation σ given in pseudo-code that returns EK containing a sequence of matches winding through from one end till the other end of U Ck . 7

8

σ can also be used to calculate the area and consequently the centroid (or centre of gravity) with equations given in Appendix F. It is worth pointing out however that the centroid is not suitable for this application, as not always its derivation will result in a point located within U Ck , e.g. in boomerang like shapes.

Determining an Interior ”Inner” Point inside of

U Ck ,

As discussed previously, when ak lies

the amount of false positives is presumed to be less thereby reducing

Chapter 3. The Musculoskeletal Model

34

the initial problem illustrated in Figure 3.19. Each detected match therefore suggests the possibility of Dk resulting into a complex shape, which should not be the case as polygonal meshes suited for deformation should remain simple. To determine an inner point, a heuristic is used on the number of matches, dividing them into even and odd after being permuted by σ. When odd, the mid point of the middle match is taken as the inner point where ak should be repositioned. When even, a single match cannot be determined, thus a more complicated approach is used. Here, the two middle matches are taken that collectively form a quadrilateral. Each vertex of this quadrilateral is then projected into two dimensional space, where its centroid can be calculated using the method of Patterson (2003). The method of Patterson (2003) requires an intersection test for two line segments between pairs of centroids of each of the four triangles present in every quadrilateral. This intersection point is computed using the method of Goldman (1990) and represents the centroid of the quadrilateral. Thus, this point is the inner point where ak has to be translated to. Figure 3.21 shows three examples for different dimensions (or cardinality) of Ck with Figure 3.21c illustrating an example for the centroid approximation of a middle quadrilateral.11 .

UC1

1 c 5 1 c 4

1 c 0

a1 C1 1 c 3

1 c 1

1 c 2

i n n e r a 1

1c 1 1 c 1 c 1 0 0 1

1 c 9 1 c 8

a1

1 c 1 1 c 2

i n n e r a 1

1 C1 c 3 c 1 c 1 4 c 1 c 6 5 1 7

b )Sa mea s( a )f o raDo d e c a g o n ( a )Mi d p o i n to fme d i a lma t c h( u n e v e n ) (

a1 C1

i n n e r a 1

( c )Ev e nr e s u l t st a k et h ec e n t r o i do faq u a d

Figure 3.21: An example illustrating two types of inner point determination. (a) shows the inner point determined by the midpoint of the middle match for an uneven number of matches. (b) shows the same situation for a Dodecagon. (c) shows the resulting centroid determined with an even number of matches for a Tetradecagon.

3.5.4

Iterating the Enhancement Process

With ak now repositioned to aCentroid , the enhancement process can be repeated in order k to get a simple and more accurate polygonal representation of U Ck , given in Algorithm 2. It is also worth mentioning that Line 14 and Line 15 were included to give preference to the attachment positions of OpenSim instead of the UHM. So that even after the origin and insertion points have been repositioned, they will still go back to their initial OpenSim locations. 11

Refer to section 3.6 for more information on scaling the dimension of Ck .

Chapter 3. The Musculoskeletal Model

1 2 3

35

foreach M do k ← 0; for k ← 0 to k = |A| − 1 do

Ck ←SetRadii(Ck ); // initial configuration ComputeRayTriangleTests(Ck ); 6 Dk ←UpdateRadii(Ck ); 7 Ek ← DetectMatches(Dk ); 8 hekn i ← σ; 9 ak ← ComputeInnerPoint(hekn i); 10 Ck ←ResetRadii(Dk ); 11 ComputeRayTriangleTests(Ck ); 12 Dk ←UpdateRadii(Ck ); 13 end 14 a0 ←ResetActionPoint(a0 ); // origin position 15 a|A|−1 ←ResetActionPoint(a|A|−1 ); // insertion position 16 end Algorithm 2: The order of steps executed during the UHM enhancement process. 4

5

3.6

Adapting the LOD of the Musculotendon Model

Most musculotendons in Menegolo (2011) contain zero via points with the Sartorius having a maximum of three via points. This creates a problem for leveraging the accuracy during the enhancement process discussed in subsection 3.5.3 as not much information is present in the latitudinal and longitudinal dimension, i.e. the amount of detail for each Ck ’s at every ak of A. To solve this problem, it was decided to adapt the musculotendon model to be scalable in both longitudinal and latitudinal dimensions, thereby introduction a LOD technique. Allowing the LOD to be adapted in both directions also satisfied the requirement to simulate a more accurate model in real-time on faster computer hardware.

3.6.1

Longitudinal and Latitudinal Scaling

As discussed in subsection 3.4.2, musculotendons where via points are present can be split into smaller segments by introducing extra bisecting cross sections that lie orthogonal to the vector (ak+1 − ak−1 )T for a ∈ A ∧ 0 < k < (|A| − 1). For the longitudinal segment scalar (LSS) it was decided to look at the amount of extra segments instead of cross sections. A segment is defined as the volume between two cross sections, when LSS = 0 this would equal one segment, when LSS = 1 this would equal two segments, LSS = 2 equals four segments, and so forth. The reasoning is that when a cross section is added it actually splits an existing segment into two, hence when this splitting pattern is repeated the resulting expression would be in the form of 2n . Furthermore, each newly created

Chapter 3. The Musculoskeletal Model

36

ak is added at exactly the mid point between the two opposing cross sections on each side. This allows the dimension of A to scale in a uniform fashion. The same repeating pattern is applied as well to via points, as each via point also splits an existing segment into two. Adding everything together results in the following equation to calculate the target cardinality in the longitudinal dimension, given as: tLON = 2LSS (|V | + 1) + 1 with LSS ∈ N

(3.19)

The musculotendon model can also scale in the latitudinal dimension, denoted by tLAT , with the amount of vertices at each cross section falling within the range of |Ck | ≥ 6 ∧ |Ck | = 2ℵ. This is used to locate inner points as discussed previously. A copy of M , denoted as Mcopy , is taken where the latitudinal dimension is scaled to a high enough value so that every ak of Mcopy gets repositioned inside the volume of an UHM mesh U . Once the enhanced process is finished, the repositioned ak are copied back into the original container where ray-triangle intersections tests are executed again to get the final approximated shapes. This is done by introducing a scalar, the cross sectional scalar, denoted as CSS to differentiate between latitudinal scales and is used as: tLAT = CSS |Ck | with CSS ∈ N.

(3.20)

There is also a lower bound on Ck ’s cardinality for reasons discussed in the next subsection.

3.6.2

Hexagons as a Lower Bound for Ck

With the ability to scale the amount of elements in Ck , as explained in subsection 3.6.1, it would be also beneficial to establish a lower bound for the cardinality of Ck . This limit would define the lowest amount of geometrical information that is reasonably achievable for the real-time simulation. The lower bound was chosen to be six for every Ck instead of the remaining three polygons with lesser sides, illustrated in Figure 3.22. Using hexagons as the lower bound for |Ck | brings forth two advantages: 1. As with all polygons with even Schl˚ afli numbers, hexagons have the property of being reflectively symmetric. This comes in handy for the pairwise ray-triangle intersection tests when determining whether a hexagonal-plane needs to be translated into the volume of another target musculotendon geometry with higher resolution12 . Thereby excluding pentagons and triangles as possible candidates for a lower bound. 12

An in-depth discussion on the pairwise ray-triangle intersection tests is given in subsection 3.5.3

Chapter 3. The Musculoskeletal Model

≈2. 17

(a)

inscribed hexagon

≈2. 66

(b)

≈2. 35

(e)

inscribed hexagon

37

inscribed pentagon

≈4. 73

(c)

inscribed quadrilateral

≈3. 06

(f )

inscribed pentagon

≈7. 32

(d)

inscribed triangle

≈4. 60

(g)

inscribed quadrilateral

≈7. 43

(h)

inscribed triangle

Figure 3.22: Illustration of areas covered by polygons with Schl˚ afli numbers between {6} and {3} inscribed in conic sections. Both the circle and ellipse have the same area. Clearly, both regular and irregular hexagons have a better perimeter to area ratio. The calculation of each covered area is the subtraction of standard area equations for these polygons from the area of the circle or ellipse. The approximated solution to each problem is given in the lower right corner where a lower value indicates better coverage.

2. Second, even though musculotendons consist of different architectures, the analysis in subsection 3.5.2 indicates that anatomical shapes of musculotendons have circular tendencies13 . Thus, a cross section of a musculotendon unit could be considered as a closed curve that deviates from an idealised conic section14 . Because hexagons have a higher perimeter to area ratio than quadrilaterals, hexagons are thus more fitting to a model with circular tendencies compared to quadrilaterals. 3. Hexagons have a lower amount of vertices than octagons, and because the three polygons with lesser sides are already excluded, the remaining option is the hexagon.

3.7

Drawing the Cylinder-based Musculotendon Unit

Drawing M in each time step comprises of a couple of steps outlined in Figure 3.23a. Each M is defined in its own internal coordinate system based on the world position of the bone parented to the origin point. The first routine computes local positions of A as illustrated in Figure 3.23b. Afterwards, Ck is computed with all radii from the set Rk . Similar to the internal coordinate system used for the relative positions of ak , the directions of rik are also computed in intervals relative to angle θ, given as

1 ◦ |Ck | 360 .

Next, one tendon point is interpolated and stored separately as at and Ct . Finally, the muscle is drawn, shaded, and textured using OGRE. 13

A circle is also one of the most common shapes found in nature (Whitaker, 2006). Except for aponeuroses, which are flat and broad tendons. These are not included in biomechanical models as discussed in subsection 3.4.3 and thus were considered beyond scope. 14

Chapter 3. The Musculoskeletal Model

38 W b 6 1

q p+q

(a)

Steps involved prior the drawing routine.

(b)

W v p 0W b 6 2 W v 1

Example showing the internal coordinate system.

Figure 3.23: (a) shows the order of steps before drawing each musculotendon in OGRE. In (b), an example is given with two via points where the second via point v1R (in internal coordinates) is derived with the help of p. For each constraint, a vector is taken between the world position of an attachment point and the parented bone of T T the origin point. In this example, q = v0W − bW and p + q = v1W − bW therefore 61 61 p = p + q − q and so the relative position v1R can be derived with v0R + p.

For the actual drawing routine, it was opted to use the Ogre::ManualObject class as it provides wrapper functions for the creation of custom objects. To prevent performance hits, each M is created once with the inclusion of extra vertices for the tendon point and is afterwards updated with new positions in subsequent time intervals, ensuring that GPU memory is allocated once, with pre-calculated vertex and index buffers.

3.7.1

Triangle Fans and Triangle Strips

Instead of using triangle lists, it was opted for a combination of fans and strips, illustrated in Figure 3.24. These render types give less vertex redundancy with each needing just n+2 vertices to draw n triangles. Coupling fans and strips with vertex and index buffers results in an efficient way to draw the graphical musculotendon in real-time.

3.7.2

Applying Textures

The final addition to the musculotendon model was a visual aid to inspect the length of tendons derived with the linear interpolation function discussed in subsection 3.4.3. Adding textures proved to be a practical way to solve this issue and moreover it increased the visual realism of the model. Some examples are given in Figure 3.25.

Chapter 3. The Musculoskeletal Model

39

CT

C3

C2

C4 Tr i angl e Fan

C1

Bo t t o mCa p

C0 TopCap

0

a0

Tr i angl eSt r i ps

Tr i angl e Fan

Figure 3.24: Showing the cylinder-based musculotendon with triangle fans and strips.

Figure 3.25: The top rows shows interpolated tendons up close, while the bottom row shows a back and front view of the lower body model that is embedded within Ralph.

Chapter 4

Real-Time Collision Detection and Response The spatial arrangement of musculotendons is considered problematic as they are grouped and intertwined with the skeleton, with themselves, and in tight spaces. A recent survey from Lee et al. (2010) mentions that solving collisions would provide a significant advancement in the area of musculoskeletal simulation. Indeed, a collision-free system would for example allow volumetric models to be used with the finite element method (FEM). This project tried to look for a solution using different techniques, but in the end resorted to a custom strategy that tries to isolate the problem of collision detection and response. To solve collisions, the relatively fast GJK-EPA algorithm (Cameron, 1997) that is taken for granted in most real-time contact problems could not directly be used in this case due to the concavity inherit within the musculoskeletal anatomy. An attempt was therefore made with Bullet Physics (Coumans, 2013) by representing each musculotendon as a soft body comprising of spring-masses. Unfortunately, this turned out to be impossible to model muscle behaviour. The addition of extra links to reduce bending and preserve volume did not solve the issue either and hinted at possibly reaching a limit for the stiff equations that are used by Bullet’s integrator, or lack of knowledge and experience from the author. The second issue was a exceedingly low amount of frames per second (FPS) even when using collision groups and masks for all bodies, and hierarchical approximate convex decomposition (HACD) for the rigid skeleton bodies (Mamou, 2013). One other option was to represent each musculotendon as a chain of connected rigid cross sections. However, this would create other issues such as not being able to deform cross sections and missing collisions within the inner segments, due to the thinness of the plate like cross sections. 40

Chapter 4. Collision Detection and Response

4.1

41

Custom Strategy for Discrete Collision Detection

The above mentioned problems hint at the complex arrangements within components of the musculoskeletal system that are not suitable for the standard methods and techniques within the field of real-time collision detection, such as sweep and prune and spatial partitioning. Besides the presumption that most anatomical objects are concave, there are interdependencies for contacts between skeletal muscle, skeletal bone, and between fat and adipose tissues. Even in a simplified model that uses just skeletal muscle and bone, collisions would occur between the muscles and the skeleton, within the same muscle, between different muscles, and also between different bones of the skeleton, all influenced by and depending on a certain pose of a particular animated character. With such a high degree of outcomes for colliding objects, it made sense to isolate the problem of collision detection for this specific problem-type and introduce a new pairwise collision constraint suitable for both low and high poly musculotendon models.

4.1.1

Collision Constraints for Muscle and Skeleton

Besides having a penetration free state, muscles should follow a path from origin to insertion, passing around other muscles and other skeletal bones. The musculotendon model discussed in this thesis uses via-points from OpenSim and approximated inner points of the Ultimate Human Model (UHM) to create a more accurate path1 . However, these additions alone do not guarantee a convergence to a penetration free state, especially when coupled with an animation sequence that wraps, bends, twists, and stretches musculotendons. To try to solve this problem, a new constraint denoted by κ is introduced that forms a transversal line segment between two cross sections of two arbitrary musculotendons. Each constraint would connect a vertex of a tagged cross sectional plane Ck ∈ Mx with another vertex of another tagged cross sectional plane Ck ∈ My . The aim is to try to solve two problems in real-time collision detection within this domain, namely: 1. Finding a correct location of Mx with respect to My . 2. Reducing the amount of discrete collision checks. A constraint κ consists of two tagged cki ’s, denoted as κx and κy , from two separate musculotendon meshes. Geometrically, each κ represents a transversal with two alternate interior angles φx and φy , shown in Figure 4.1 that vary with the orientation of each cross 1 OpenSim uses similar constraints such as wrapping surfaces, discussed in 3.4.1, however these are aimed at approximating empirical data more closely for a polyline representation of the action line.

Chapter 4. Collision Detection and Response

42

section. When either of the two angles are acute, the solver uses the scalar projection of T T T T κyk − cki unto ak − cki as the penetration depth ρx , or κxk − cki unto ak − cki as the penetration depth ρy . If both angles are acute, the sum of both depths is taken.

k c 4

Mx

(a)

Mx

ak oy

ak

ox

My

oy

ox

ak

k c 4

k c 1

A potential collision is detected.

k c 1

(b)

ak

My

No collision is detected.

Figure 4.1: A transversal constraint κ consisting of two tags, ck1 of Mx and ck4 of My . (a) illustrates one acute angle φy with the penetration depth ρy (marked in blue). The penetration depth is used to determine whether ak of My should move in the opposite T direction ckio − ak by distance ρy . Due to the symmetry of even dimensions for |Ck |, the opposite index denoted by io can be found. (b) illustrates a case where no acute angles are present on either side of κ, thus indicating a possible collision-free state.

ρ is then used as the distance to determine whether either ak should be moved in T the opposite direction ckio − ak , depending on which reference frame the solver is computing from2 . Using the example of Figure 4.1a, if a user-defined tag places My as its frame of reference, then for this particular κy the solver is instructed to check if the constrained cross section centred at ak should move in the opposite direction given T by ck1 − ak with a tag being placed on ck4 , of My . In addition, tagged cki ’s are used to instruct the solver to let all elements of Ck be checked for collisions against specific skeleton meshes, and not just other muscles.

4.1.2

Discrete Collision Detection with Ray Casting

A transversal constraint alone does not guarantee a penetration free state as it depends too much on where manual tags are placed, and when leading with low scale geometries this does not provide sufficient robustness. Hereto, they are only used to check if a cross section should be moved or not. To increase the accuracy of the discrete collision solver, ray-triangle intersection tests are carried out with the same M¨oller-Trumbore algorithm 2

Opposing indices denoted by io are found with Equation 3.18.

Chapter 4. Collision Detection and Response

43

(M¨oller and Trumbore, 1997) that was used for the UHM enhancement process3 . Although all triangles of a tested object could be taken into account during narrow phase detection, the amount of rays being cast was limited in this implementation to the amount of constraints placed on the object. This results in less computations, except when checking for collisions between muscle and skeletal bones as each face of a skeleton mesh has to be tested. Each intersection test starts at a Ck of a tagged cki where rays are cast from each cki in one sweep consisting of two different directions, illustrated in Figure 4.2.

k c 0

c

k 5

ak

k c 4

k c 3

k c 1

k 1

a k 1

Ck-1

k

ak

Ck

a k +1

Ck+1

k c 2 k +1

(a)

(b)

Latitudinal Sweep.

Diagonal Sweep.

Figure 4.2: Direction of cast rays confined by transversal constraints.

At each cki rays are cast in the latitudinal and diagonal normal directions with:    ck − ck if i < |Ck | − 1 i+1 i Dlat (i, k) =  k k  c0 − ci if i = |Ck | − 1

(4.1)

DdiagP

   ck+1 − ck i i+1 (i, k) =  k+1 k  c0 − ci

if i < |Ck | − 1 ∧ k < |A| − 1 if i = |Ck | − 1 ∧ k < |A| − 1

(4.2)

DdiagN

   ck−1 − ck i i+1 (i, k) =   ck−1 − cki 0



(4.3)

if i < |Ck | − 1 ∧ k = |A| − 1 if i = |Ck | − 1 ∧ k = |A| − 1

where Dlat is the normal direction of the cast ray around Ck in the latitudinal direction, and DdiagP and DdiagN represent the positive diagonal and negative diagonal directions respectively4 . The presumption here is that extra rays tests would lower the chances 3 4

Refer to subsection 3.5.3 for how these tests were used during the UHM enhancement process. Note that Equation 4.3 is only used for cases when k = |A| − 1.

Chapter 4. Collision Detection and Response

44

of false negatives during narrow phase detection. The following section describes the discrete iterative resolution, i.e. converging to a penetration-free state after penetrations were already detected.

4.2

Collision Response

In this implementation, it was determined that it would be best to move ak of Mx on three occasions, namely: T 1. when ρx + ρy > ak − cki

,

2. when radii are adjusted, 3. when penetration is detected with rays cast in a diagonal direction. The first occasion will be described here while the second and third occasions will be described in the next subsection. As mentioned previously, κ’s main purpose is to find a correct location of Mx with respect to My , and is particularly useful for cases when more than half of the volume of Mx lies in My or when Mx lies on the opposite side of My of its supposed location. In these cases, when penetration depths are above a certain criteria, ak ∈ A are displaced. The actual displacement of ak uses translation vectors that were purposed for the UHM enhancement, but because at this stage the enhancement process is already complete, their values can be adjusted for a new purpose. Each new position is calculated by first subtracting the applied translation from the UHM. Let µk denote T  denotes the original OpenSim the translation vector given by ak − aorig where aorig k k position before the UHM. Subtracting ak from µk gives then aorig k . With the obtained aorig k , the new µk is then calculated as: 

ak − µk = cki + ρx + ρy ak −

T cki T cki

  − aorig k

(4.4)

There are also cases where ak does not have to be translated, here radii of Rk are adjusted in order to preserve the same initial area prior deformation.

4.2.1

Retaining Cross Sectional Areas through Inflation

To keep the same cross sectional area after being deformed by the UHM enhancement process, radii rik that would deform again due to detected collisions, are adjusted. Here again the usefulness of using a cylindrical-based musculotendon construction comes forward as it prevents Euclidian lengths of vectors to be calculated. Since in order to

Chapter 4. Collision Detection and Response

45

deform the musculotendon all that is needed is the adjustment of a radius. To retain cross sectional area, an iterative approach is taken that incrementally expands rik a tiny bit in each time step resulting in an inflation effect for the free area of Ck . Before inflation, the deformed area after the UHM enhancement is stored and used as a target area τ . In order to know which rik can be adjusted and which are currently being deformed by another object, a distinction is made between radii, segmenting them into fixed and free. A radius is flagged as fixed when it has to expand inwards due to its surrounding surface edges detecting penetration of another object. Remaining radii are considered free and are thus able to expand outwards. Both fixed and free indices i of each rik are stored in the set Rfk ixed and Rfk ree respectively. As a rule of thumb, when confronted with a penetration, fixed radii will decrease in distances of 10% until a Ck with κ reaches a penetration free state5 . When rik approaches zero, given as numerical limit , its cenT 1 ckio − ak , which is also 10% of the radius in troid ak is translated in direction of 10 the opposite direction obtained with Equation 3.18. This allows Ck to translate slowly out of the maximum penetrated area and not create a big enough gap between the two collided objects at constrained cross sections. This same technique for displacement is also applied on the third occasion when rays are cast in the diagonal in order to push ak a bit outwards in each time step. An important aspect of radii adjustment is to not lose the initial shape of τ , which is achieved by calculating a factor for all other free radii with respect to a particular radius that is fixed, and thus represents a radius that has finished deforming or is still undergoing deformation. A radius factor fik can be calculated for each rik as the function: fik

(j) =

rjk rik

, ∀j : 0 ≤ j < |Ck | ∧ i 6= j

(4.5)

To reach convergence, areas comprising of fixed and free radii, denoted as τf ixed and τf ree are also taken into consideration. Due to the cylindrical construction, each Ck can be split into its constituent triangles, allowing areas τfkixed and τfkree to be calculated separately using scalar projections6 . τfkixed and τfkree are calculated as: i 35, for both tested scales of LSS = 1 and LSS = 2. This can be for instance when |Ck | = 6 and CSS = 6 (as shown

Chapter 5. Results and Discussion

51

in Table 5.2) or for instance when |Ck | = 36 and CSS = 12 . The latter, where |Ck | = 36 and CSS = 1, is considered the optimal choice of parameters as it contains the most information. In this configuration, the amount of vertices for all 48 musculotendons totals 8736, which is still ≈ 16% of the UHM. CSS / |Ck |

Hexagon{6}

Octagon{8}

Decagon{10}

Dodecagon{12}

1

23

19

7

8

2

8

7

7

3

3

7

3

1

0

4

3

2

0

0

5

1

0

0

0

6

0

0

0

0

Table 5.2: Amount of ak outside an UHM mesh for LSS = 1.

CSS / |Ck |

Hexagon{6}

Octagon{8}

Decagon{10}

Dodecagon{12}

1

42

24

12

17

2

17

12

10

5

3

10

5

2

0

4

5

3

0

0

5

2

0

0

0

6

0

0

0

0

Table 5.3: Amount of ak outside an UHM mesh for LSS = 2.

5.1.3

Match Detection

As discussed in Chapter 3, the classification of each match uses information from normals of each triangle’s face. A previous version disregarded this information entirely due to the fact that the normals of the UHM meshes were not correctly set. This previous version took the furthest intersection point when casting rays in the positive direction, and the closest intersection point when casting rays in the negative direction, regardless of the normals. In the end, this resulted in slightly more errors when wrong intersection points were taken in highly curved segments, as shown in Figure 5.3a. This led to the inclusion of normal information by manually checking normals of modified UHM meshes that were incorrectly set in Blender Foundation (2006) to reduce the amount of errors and adapt the conditions outlined in subsection 3.5.3.2. These conditions also impose a 2 In addition, for gathering the results, Line 14 and Line 15 of Algorithm 2 were disabled to isolate the tests from OpenSim.

Chapter 5. Results and Discussion

52

limitation when the technique encounters two separate U Ck in the same plane. In this case, depending on the spatial configuration of Ck with respect to either U Ck , a new inner point will move within either one of the two closed curves, or when no matches are detected, would result with a deformed Ck that emcompasses the empty space in between and results in an incorrect areal representation. To prevent this from happening, areas such as the head of the Medial Gastrocnemius that consists of two attachment sites had to be closed in Blender Foundation (2006)3 .

5.2

Collision Detection and Response

The strategy discussed in Chapter 4 tried to approach the problem of collision detection and response from a different angle. The main takeaways are a new type of constraint that works with alternate interior angles of the transversal for muscle reconfiguration. This type of constraint prevents expensive length calculations to be performed such as is the case with unilateral constraints or signed distance functions. Next to transversal constraints, a narrow phase detection step that uses the same fast ray-casting algorithm (M¨oller and Trumbore, 1997) is also included for both muscle versus muscle and muscle versus skeleton penetration tests. The complete strategy was tested on a small set of 10 muscles, which comprises of a total of five muscle versus muscle constraints, or transversal constraints, and 20 muscle versus skeleton constraints. Three results are given in Figure 5.5. Unfortunately, the system is not stable for the complete set of 48 musculotendons due to the exclusion of a physics engine, which forces deformations to be permanent. This creates instabilities when too many interdependencies are introduced; such as when muscle Mx affects My , and My is affected by skeleton mesh Sz that in turn affects back Mx . The instability is exaggerated with muscles in and around the pelvic area as these are heavily intersected with each other. Another drawback of the solution is that transversal constraints have to be manually placed and are scale-dependent, i.e. the same set of transversal constraints for |Ck | = 6 would not work for |Ck | = 8. The placement of each transversal constraint is also not straightforward. Care has to be taken on which cki ’s of two to be constrained M ’s have to be tagged, as wrong tags can also cause more displacement than necessary. Although this can be remedied with physics-based techniques such as springs where equilibrial displacement positions are converged to. 3

This is further discussed in subsection 6.1.2.2.

Chapter 5. Results and Discussion

53

Finally, it is worth mentioning that during the implementation of ak displacements, Catmull-Rom splines were added, where each constrained ak was used as a control point for the spline, while the remaining ak ’s are interpolated. The assumption was that the curvedness of a spline would help pull the cross sections without a constraint a bit further out thereby needing less constraints to reach a penetration free state. However, during the implementation this was found to be untrue, and so the added spline computations were removed.

5.2.1

Radii Inflation

For the iterative inflation of radii, this turned out to be a good approach in preserving the same cross sectional area prior deformation in real-time, and provides a simple implementation that takes advantage of the model’s cylindrical construction. Figure 5.6 shows one result for the three top cross sections of the Vastus Intermedius.

5.3

Real-Time Performance

Without the collision solver, the complete model with 48 loaded musculotendons having |Ck | = 8 and LSS = 1, and upper and lower body skeleton meshes, ran 181Hz on average with a loaded walk-cycle animation on the character Ralph. This was tested on an Intel i5-3210M x64 CPU running at 2.50GHz with a NVIDIA GeForce GT 645M mobile graphics card, and having OGRE compiled in release mode. With an active collision solver, having a total of 25 collision constraints and 10 muscles due to stability limitations (≈

1 5

of the complete model set), the runtime was 164Hz

on average using the same system configuration and walk-cycle animation that was mentioned previously. For this current solution, a fixed interval of one for every three frames was used for the collision solver to aid during debugging, thereby slowing down the rate of convergence to one iteration for every three cycles. However, this can be decreased to one to provide a maximum and faster resolution of one iteration for each cycle. It is worth noting that both results are without any GPU or software-based parallelisation techniques. As reported in subsection 5.1.2, the optimal choice of parameters is with |Ck | = 36 and CSS = 1. Here the performance was 135Hz on average with the same system configuration as mentioned previously. Thereby satisfying the > 60Hz requirement of real-time engines. Finally, it is also worth noting that the UHM enhancement process itself takes around 10 seconds for 48 musculotendons with LSS = 1 and |Ck | = 36, and is executed once at the beginning of each run.

Chapter 5. Results and Discussion

Figure 5.5: Three examples without and with constraints. (a) and (b) show how transversal constraint could help in reconfiguring a correct path between two muscles. (c) and (d) show how small ak displacements push the thick Gluteus Maximus outward. (b) and (f) show the transversal constraint (in yellow) with (f) also showing the line segment between ak and cki (in green), and the displacement of the right muscle from the femur bone due to inflation. The example of (e) and (f) is also given in Figure 5.6.

54

Chapter 5. Results and Discussion

(a)

Before inflation.

55

(b)

After inflation.

(c)

Displaced area shown in black.

Figure 5.6: Each displacement of the area of a triangle on Ck for two consecutive cki ’s is shown above as a black line segment of one pixel thick, offset from the base of every triangle. The thickness of culminated lines therefore gives an indication on the total displaced area after one or more radii have been pushed out of the femur bone. For this particular example, the three areas of the first three cross sections with values: 9.486 × 10−6 , 1.916 × 10−4 , and 2.587 × 10−4 where each area converged in parallel with 8, 4, and 6 iterations using Equation 4.10. Rest areas had a numerical threshold set by  = 1 × 10−6 . Areas were calculated using Equation 4.8.

Chapter 6

Conclusion and Future Work This MSc project explored the creation of a collision-free and simple musculoskeletal model that could be used in the future for applications such as musculoskeletal injury simulation. Due to computational limits imposed by a real-time criterion, models usually have to deal with the tug of war between accuracy and real-time performance. This project tried to keep the real-time performance in check by applying a lower bound on the cardinality of Ck , and using triangle fans and strips for the cylinder-based musculotendon unit. To leverage accuracy, low poly musculotendon models were systematically enhanced by adding more detail from high poly models. This process, labelled the UHM enhancement, introduced a permutation algorithm on pairs of intersection points. The permutation allows the possibility to locate a point inside a cross section of a high poly mesh by another separate cross section of a low poly mesh in a way that is invariant to the spatial and polygonal configuration between the two meshes. Thus providing a different way to represent high poly meshes from other data sets while keeping a simple low poly representation for future considerations discussed in this section. Collision management for soft bio-based materials still remains a challenge in computerbased physics simulations, especially when real-time requirements are added on top. After an unsuccessful attempt with Bullet Physics, the mental effort shifted back to basics, classifying the type of information that is available and determining how the simple musculotendon model can be exploited to detect and respond to collisions. A transversal constraint was introduced with the purpose of correctly arranging one musculotendon with respect to another musculotendon spatially. This compensated unintended side effects attained from combining a polyline representation with action lines from Menegolo (2011) and polygonal meshes from the UHM (Snoswell, 2003)1 . In addition, an iterative 1 One example is given in Figure 5.5 where action lines of OpenSim, when fitted inside geometrical models, resulted in musculotendons intersecting each other.

56

Chapter 6. Conclusion and Future Work

57

method that inflates triangulated areas of each cross section was devised to preserve cross sectional area in real time.

6.1 6.1.1

Improvements and Future Considerations Biomechanical Accuracy

This project tackled the problem of combining an action line based biomechanical model with a high poly model. As was previously discussed in subsection 3.4.3, the tendon slack length currently includes both the length of the free tendon and the length of the aponeurotic tendon (Delp et al., 1990). The latter represents the part of the tendon that is internal to a muscle’s belly, i.e. it also extends through and within the muscle fibres and does not end at the point that represents the starting point of the physical muscle. In addition, classical Hill-type models such as Zajac (1988) combine the length of each separate free tendon into a single variable, termed as ”in-series” in biomechanics, while in reality most musculotendon units consists of at least one tendon on each side. Unfortunately, during the course of this MSc project, no representation has been found that divides the tendon length into separate lengths for the origin and insertion, and also into its two constituents, free and aponeurotic. This is probably due to the complexity inherit in anatomical and biomechanical measurements and difficulties in generalising these measurements to be subject independent (Benjamin et al., 2006; Gerus et al., 2012). While simplifications such as these work well for state-of-the-art polyline-based biomechanical simulators (Millard et al., 2013), a combined tendon representation is not correct for applications such as musculoskeletal injuries as geometrical or polygonal based representations of musculotendons are indispensable2 . It is here where a crossroad between the two worlds meet as a three dimensional representation would allow volume to be segmented with differing material properties alike their real anatomical counterparts. Thus, until such progress is made, as argued by Epstein et al. (2006); Siebert et al. (2008), assumptions have to be taken such as on the length of free and aponeurotic tendons for musculoskeletal-dependent applications where musculotendon geometry plays an active role. Future endeavours in computer animation that would need to combine musculoskeletal geometry with biomechanical models should therefore investigate other ways to combine these two separate worlds. 2

For example, injuries on bi-articular muscles such as hamstrings where musculotendon geometry plays an even larger role (Lenhart and Thelen, 2012).

Chapter 6. Conclusion and Future Work

6.1.2

58

Extending the Musculotendon Model

For this project, the lower body was used as a test case for reasons mentioned in section 3.2. The next logical step would be to include both lower and upper biomechanical models for the human body. However, to accomplish this, improvements suggested to the following class of similar problems should be considered first.

6.1.2.1

Linked Transformation Correction for A

Section 3.3 in Chapter 3 covered the great deal of work that went into the successful attachment of the skeleton onto the animated character nicknamed Ralph. There is however still one remaining improvement in the area of anthropometry that is described as follows. Each musculotendon unit is attached relative to the origin of an OpenSim body object. After applying the transformations required for the semi-automatic skeleton mesh alignment technique, some attachment points of muscles, which also inherited the same transformations, were slightly off. This has to do with the skeleton hierarchy affecting relative distances after being attached to the bone hierarchy of Ralph. The illustration in Figure 6.1 gives an example. Perhaps a future experimental project could establish which approach, such as comparing marker placement of motion-capture data, is able to best solve this anthropometric scaling problem.

or i gi na l

s c a l e d f e e t

de s i r e d r e s ul t

Figure 6.1: Linked transformation correction for a muscle path. The via point is attached to the parent of the foot and not influenced by the scaling of linked children. This results in a curve deviating from the original path. In this example, the correction would entail transforming the via points such that all internal angles are respected.

Chapter 6. Conclusion and Future Work 6.1.2.2

59

Multiple Action Lines for Wide Attachments

Within the human body there are some musculotendons that consist of more than one tendon at each side of attachment, such as the Extensor Digitorum Communis and the Abductor Pollicis Longus found on the forearm and palmar regions (Holzbaur et al., 2005). There are also broad and flat tendons known as aponeuroses that join muscle with muscle or muscle with skeleton along a broader area (Encyclopedia Britannica, 2014a). The representation of aponeurotic tendons is especially important if an upper body model has to include the ventral abdominal region or the dorsal lumbar region. Finally there is the problem area for the Gluteus group of musculotendons for the lower body, where some biomechanical models, like the one used for this project, divides each musculotendon into three separate units for their action line based representation. This resulted in high quality meshes being manually, and therefore inaccurately segmented as shown in Figure 3.14. To conclude, the musculotendon model presented here should be extended to include a representation with multiple action lines and multiple attachment points to account for all types of musculotendons, but nevertheless be part of (or enclosed by) a single simple geometrical shape.

6.1.2.3

Distinct Heads

The Medial Gastrocnemius from the UHM had to be adapted in Blender (Blender Foundation, 2006) in order to prevent the match detection process from converging solely at either one of its two heads. Similar to the problems discussed in the previous subsection, the musculotendon model should be able to have a ”splitting” option to account for multiple heads, as musculotendons in the upper body (e.g. the Biceps Brachii) also exhibit this feature. All in all, these improvements would make the geometric musculotendon model more robust as it does not have to depend on the limitations imposed by action line based biomechanical models.

6.1.2.4

A Geometric Representation for Ligaments

Ligaments connect skeletal bones with each other and can therefore be considered the real physical joints within the musculoskeletal system. They extend mechanical support to joints, improving its stability, help guide joint motion, and prevent excessive motion. The inclusion of ligaments would open up new applications for musculotendon modelling in computer animation. For instance, simulating musculoskeletal injuries that include damaged ligaments (e.g. spinal injuries).

Chapter 6. Conclusion and Future Work

6.1.3

60

Complete Collision Detection and Response

The addition of transversal constraints allow musculotendons to be configured spatially with awareness of each other, however each constraint is currently being placed manually. One improvement would be to develop a technique that allows automatic placements of constraints. Furthermore, the complete collision strategy is currently stable for just 10 musculotendons with 25 constraints. This instability can certainly be improved by including the system within a real-time physics engine as section 5.3 has demonstrated that the real-time performance is not hindered by the collision manager. In this scenario, spring-mass systems with added dampers could be used to reach optimal equilibrium.

6.1.4

The Musculotendon Model for FEM and Inverse Dynamics

Future applications for this musculotendon model, such as realistic musculoskeletal injuries, require that the model should be able to deform in a physically accurate way and thus physical properties such as the volume of each compartment of the model should be accounted for. Using the current construction, this can be achieved with the tetrahedralisation approach shown in Figure 6.2. The cylinder-based geometrical model presented in this thesis is therefore suitable for deformation techniques using FEM. This approach works as follows. Each segment of the geometrical musculotendon can be subdivided into a set of irregular triangular frustums, shown in Figure 6.2a and 6.2b3 . In turn, each frustum can be further subdivided into three separate tetrahedrons, shown in Figure 6.2c. Using for instance a latitudinal dimension of six and for a particular musculotendon, the muscle compartment would encompass four segments, then the amount of tetrahedrons would be 3 × 6 × 4 = 72, which is around 1% of the total amount of tetrahedrons used in the demonstration of Berranen et al. (2012) that had a performance result of 45Hz.

( a )

k c 5

k c 4 k c 3

ak

k c 1 k c 2

k +1 c 4 k +1 c 3

( b)

k c 0

k +1 c 2

k +1 c 1

( c ) 2

3

1

Figure 6.2: Schematic figures showing how volume of segments can be computed.

With the componential representation of the total volume for every segment that represents muscle, and because all of the vertices from the musculotendon model are known, 3 Note that the graphical object does not actually render faces between segments as only the surface representation is sent to OGRE and drawn with triangle fans and strips, as described in subsection 3.7.

Chapter 6. Conclusion and Future Work

61

each tetrahedron’s volume can be calculated geometrically as: ν=

| (w − z) · ((x − z) × (y − z)) | 6

(6.1)

with w, x, y, z representing four known vertices from a musculotendon model M . Summing up the volume for every tetrahedron that occupies the segments that represents the muscle compartment (up until j from Equation 3.13) gives thus the overall volume for just the muscle part, denoted by ν M . Using an estimated muscle tissue density of e.g. 1059.7kg/m3 for mammalian muscle (Dorn, 2014), would allow the mass of each tetrahedron to be estimated as well. In addition, different material properties for every tetrahedron can be separately assigned to the muscle and tendon compartments. Because the current musculotendon model includes an embedded biomechanical model, it also has the necessary information to solve inverse dynamics. However, given the inclusion of geometric information, this additionally allows the model to estimate individual muscle forces in a geometrical way, similar to Lee et al. (2009). This works as follows. Using the optimal fibre length loF from Menegolo (2011), the alternative and faster computation of the physiological cross-sectional area, denoted as PCSA2 can be calculated for pennate muscles, given as: PCSA2 =

ν M cos α . loF

(6.2)

This in turn allows an individual muscle force FM to be estimated geometrically as: FM = PCSA2 SM

(6.3)

where FM represents the estimated muscle force and SM denotes the specific tension metric (which is a constant) for a particular muscle4 . Tension information, e.g. within a range of 32 − 61Ncm−2 , can be obtained from experimental measurements as well as biomechanical models in literature such as Dorn (2014); O’Brien et al. (2010); Maganaris et al. (2001); Arnold et al. (2010); Hansen et al. (2006); Vasavada et al. (1998); Lieber et al. (1992); An et al. (1981); Langenderfer et al. (2004); Jacobson et al. (1992).

4 Note however that the estimated force can vary widely as it depends on the accuracy of musculotendon model and estimated measurements of specific tension (Fukunaga et al., 1996; Buchanan, 1995).

Bibliography Albrecht, I., Haber, J., and Seidel, H. (2003). Construction and animation of anatomically based human hand models. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation, pages 98–109. Eurographics Association. An, K., Hui, F., Morrey, B., Linscheid, R., and Chao, E. (1981). Muscles across the elbow joint: a biomechanical analysis. Journal of biomechanics, 14(10):659–669. Anderson, F. and Pandy, M. (1999). A dynamic optimization solution for vertical jumping in three dimensions. Computer Methods in Biomechanics and Biomedical Engineering, 2(3):201–231. Anderson, F., Pandy, M. G., et al. (2001). Dynamic optimization of human walking. Transactions-American Society of Mechanical Engineers Journal of Biomechanical Engineering, 123(5):381–390. Arnold, E., Ward, S., Lieber, R., and Delp, S. (2010). A model of the lower limb for analysis of human movement. Annals of biomedical engineering, 38(2):269–279. Aubel, A. and Thalmann, D. (2001a). Efficient muscle shape deformation. In Deformable avatars, pages 132–142. Springer. Aubel, A. and Thalmann, D. (2001b). Interactive modeling of the human musculature. In Computer Animation, 2001. The Fourteenth Conference on Computer Animation. Proceedings, pages 167–255. IEEE. Benjamin, M., Toumi, H., Ralphs, J., Bydder, G., Best, T., and Milz, S. (2006). Where tendons and ligaments meet bone: attachment sites (‘entheses’) in relation to exercise and/or mechanical load. Journal of Anatomy, 208(4):471–490. Berranen, Y., Hayashibe, M., Gilles, B., and Guiraud, D. (2012). 3d volumetric muscle modeling for real-time deformation analysis with fem. In Engineering in Medicine and Biology Society (EMBC), 2012 Annual International Conference of the IEEE, pages 4863–4866. IEEE. 62

Bibliography

63

Blemker, S. and Delp, S. (2005). Three-dimensional representation of complex muscle architectures and geometries. Annals of biomedical engineering, 33(5):661–673. Blender Foundation (2006). Blender.org - home of the blender project - free and open 3d creation software. Available from: http://www.blender.org/ [cited 14th of January 2014]. Bourke, P. (1988). Calculating the area and centroid of a polygon. Available from: http://paulbourke.net/geometry/polygonmesh/centroid.pdf [cited 28th of

January 2014]. Buchanan, T. (1995). Evidence that maximum muscle stress is not a constant: differences in specific tension in elbow flexors and extensors. Medical engineering & physics, 17(7):529–536. Cameron, S. (1997). Enhancing gjk: Computing minimum and penetration distances between convex polyhedra. In ICRA, volume 97, pages 20–25. Citeseer. Chadwick, J., Haumann, D., and Parent, R. (1989). Layered construction for deformable animated characters. In ACM Siggraph Computer Graphics, volume 23, pages 243–252. ACM. Chen, D. and Zeltzer, D. (1992). Pump it up: Computer animation of a biomechanically based model of muscle using the finite element method, volume 26. ACM. Christophy, M. (2010). A detailed open-source musculoskeletal model of the human lumbar spine. PhD thesis, University of California. Coumans, E. (2013). Real-time physics simulation: Home of the open source bullet physics library and physics discussion forums. Available from: http://bulletphysics.org/ [cited 9th of February 2014].

Delp, S., Anderson, F., Arnold, A., Loan, P., Habib, A., John, C., Guendelman, E., and Thelen, D. (2007). Opensim: open-source software to create and analyze dynamic simulations of movement. Biomedical Engineering, IEEE Transactions on, 54(11):1940–1950. Delp, S., Loan, J., Hoy, M., Zajac, F., Topp, E., and Rosen, J. (1990). An interactive graphics-based model of the lower extremity to study orthopaedic surgical procedures. Biomedical Engineering, IEEE Transactions on, 37(8):757–767. Dorn, T. (2014). Opensim::musclemetabolicpowerprobeumberger2010 metabolicmuscleparameter.

Bibliography

64

Available from: https://simtk.org/api_docs/opensim/api_docs31/ classOpenSim_1_1MuscleMetabolicPowerProbeUmberger2010_ _MetabolicMuscleParameter.html [cited 20th of February 2014].

Duckham, M., Kulik, L., Worboys, M., and Galton, A. (2008). Efficient generation of simple polygons for characterizing the shape of a set of points in the plane. Pattern Recognition, 41(10):3224–3236. Encyclopedia Britannica (2014a). Aponeurosis. Available from: http://www.britannica.com/EBchecked/topic/30151/aponeurosis [cited

19th of February 2014]. Encyclopedia Britannica (2014b). Tendon (anatomy) – encyclopedia britannica. Available from: http://www.britannica.com/EBchecked/topic/587171/tendon [cited 6th of

January 2014]. Epstein, M., Wong, M., and Herzog, W. (2006). Should tendon and aponeurosis be considered in series? Journal of biomechanics, 39(11):2020–2025. Fukunaga, T., Roy, R., Shellock, F., Hodgson, J., and Edgerton, V. (1996). Specific tension of human plantar flexors and dorsiflexors. Journal of Applied Physiology, 80(1):158–165. Geijtenbeek, T., van de Panne, M., and van der Stappen, A. (2013). Flexible muscle-based locomotion for bipedal creatures. ACM Transactions on Graphics (TOG), 32(6):206. Gerus, P., Rao, G., and Berton, E. (2012). Subject-specific tendon-aponeurosis definition in hill-type model predicts higher muscle forces in dynamic tasks. PloS one, 7(8):e44406. Goldman, R. (1990). Intersection of two lines in three-space. In Graphics Gems, page 304. Academic Press Professional, Inc. Gurram, P., Rhody, H., Kerekes, J., Lach, S., and Saber, E. (2007). 3d scene reconstruction through a fusion of passive video and lidar imagery. In Applied Imagery Pattern Recognition Workshop, 2007. AIPR 2007. 36th IEEE, pages 133–138. IEEE. Hansen, L., De Zee, M., Rasmussen, J., Andersen, T., Wong, C., and Simonsen, E. (2006). Anatomy and biomechanics of the back muscles in the lumbar spine with reference to biomechanical modeling. Spine, 31(17):1888–1899.

Bibliography

65

Hicks, J. (2012a). Muscle editor - opensim documentation. Available from: http: //simtk-confluence.stanford.edu:8080/display/OpenSim/Muscle+Editor

[cited 23th of December 2013]. Hicks, J. (2012b). Opensim models: Documentation. Available from: http://simtkconfluence.stanford.edu:8080/display/OpenSim/OpenSim+Models [cited

29th of December 2013]. Hirota, G., Fisher, S., State, A., Lee, C., and Fuchs, H. (2001). An implicit finite element method for elastic solids in contact. In Computer Animation, 2001. The Fourteenth Conference on Computer Animation. Proceedings, pages 136–254. IEEE. Holzbaur, K., Murray, W., and Delp, S. (2005). A model of the upper extremity for simulating musculoskeletal surgery and analyzing neuromuscular control. Annals of biomedical engineering, 33(6):829–840. Hoy, M., Zajac, F., and Gordon, M. (1990). A musculoskeletal model of the human lower extremity: the effect of muscle, tendon, and moment arm on the moment-angle relationship of musculotendon actuators at the hip, knee, and ankle. Journal of Biomechanics, 23(2):157–169. Jacobson, A., Kavan, L., and Sorkine-Hornung, O. (2013). Robust inside-outside segmentation using generalized winding numbers. ACM Trans. Graph, 32:4. Jacobson, M. D., Raab, R., Fazeli, B., Abrams, R., Botte, M., and Lieber, R. (1992). Architectural design of the human intrinsic hand muscles. The Journal of hand surgery, 17(5):804–809. Kohout, J., Clapworthy, G., Tao, Y., Dong, F., Kelnhoffer, P., Cholt, D., and Zhao, Y. (2012). Technologies for modelling fibrous muscle in motion. In VPH 2012 London. Virtual Physiological Human. Lacl´e, F. P. (2013). A literature review for the master project: Real-time musculoskeletal model for injury simulation on 3d human characters. Available from: http://vhtlab.nl/sites/default/files/A%20Literature%20Review%20on% 20Real-Time%20Musculoskeletal%20Model%20for%20Injury%20Simulation% 20on%203D%20Human%20Characters.pdf [cited 21th of February 2014].

Langenderfer, J., Jerabek, S., Thangamani, V., Kuhn, J., and Hughes, R. (2004). Musculoskeletal parameters of muscles crossing the shoulder and elbow and the effect of sarcomere length sample size on estimation of optimal muscle length. Clinical Biomechanics, 19(7):664–670.

Bibliography

66

Lee, D., Glueck, M., Khan, A., Fiume, E., and Jackson, K. (2010). A survey of modeling and simulation of skeletal muscle. ACM Transactions on Graphics, 28(4). Lee, K. and Ashraf, G. (2007). Simplified muscle dynamics for appealing real-time skin deformation. In CGVR, page 160. Citeseer. Lee, S., Sifakis, E., and Terzopoulos, D. (2009). Comprehensive biomechanical modeling and simulation of the upper body. ACM Transactions on Graphics (TOG), 28(4):99. Lemos, R., Epstein, M., Herzog, W., and Wyvill, B. (2001). Realistic skeletal muscle deformation using finite element analysis. In Computer Graphics and Image Processing, 2001 Proceedings of XIV Brazilian Symposium on, pages 192–199. IEEE. Lenhart, R. and Thelen, D. (2012). Influence of musculoskeletal geometry on model-based predictions of plantarflexor function during gait. In GCMAS Annual Conference. Lieber, R., Jacobson, M., Fazeli, B., Abrams, R., and Botte, M. (1992). Architecture of selected muscles of the arm and forearm: anatomy and implications for tendon transfer. The Journal of hand surgery, 17(5):787–798. Lund, K. and Hicks, J. (2012). Coordinates and utilities - opensim documentation. Available from: http://simtk-confluence.stanford.edu: 8080/display/OpenSim/Coordinates+and+Utilities [cited 27th of December

2013]. Maganaris, C., Baltzopoulos, V., Ball, D., and Sargeant, A. (2001). In vivo specific tension of human skeletal muscle. Journal of applied physiology, 90(3):865–872. Mamou, K. (2013). Hierarchical approximate convex decomposition of 3d meshes. Available from: http://sourceforge.net/projects/hacd/ [cited 9th of February 2014]. McGonagle, D. and Benjamin, M. (2013). Muscle attachments to bone as an unrecognised cause of pain. Available from: http://www.enthesis.info/anatomy/muscle_and_enthesis.html [cited 4nd of

January 2014]. Menegolo, A. (2011). Simtk.org: Upper and lower body model: Overview. Available from: https://simtk.org/home/ulb_project [cited 26th of August 2013]. Millard, M., Uchida, T., Seth, A., and Delp, S. (2013). Flexing computational muscle: Modeling and simulation of musculotendon dynamics. Journal of Biomechanical

Bibliography

67

Engineering, 135(2):021005–021005. Available from: http://dx.doi.org/10.1115/1.4023390.

Mohr, A. and Gleicher, M. (2003). Building efficient, accurate character skins from examples. In ACM Transactions on Graphics (TOG), volume 22, pages 562–568. ACM. M¨oller, T. and Trumbore, B. (1997). Fast, minimum storage ray-triangle intersection. Journal of graphics tools, 2(1):21–28. Nordin, M. and Frankel, V. (2012). Basic Biomechanics of the Musculoskeletal System. Wolters Kluwer Health, 4th edition. O’Brien, T., Reeves, N., Baltzopoulos, V., Jones, D., and Maganaris, C. (2010). In vivo measurements of muscle specific tension in adults and children. Experimental physiology, 95(1):202–210. Patterson, M. (2003). The centroid of the quadrilateral. Available from: http://jwilson.coe.uga.edu/emt668/EMT668.Folders.F97/Patterson/EMT% 20669/centroid%20of%20quad/Centroid.html [cited 8th of March 2014].

Ramos, J. and Larboulette, C. (2009). Real-time anatomically based character animation. In Proc. of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games, Posters and Demos. Rankin, J. and Neptune, R. (2012). Musculotendon lengths and moment arms for a three-dimensional upper-extremity model. Journal of biomechanics, 45(9):1739–1744. S´anchez, C., Lloyd, J., Fels, S., and Abolmaesumi, P. (2014). Embedding digitized fibre fields in finite element models of muscles. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization, pages 1–14. Scheepers, F., Parent, R., Carlson, W., and May, S. (1997). Anatomy-based modeling of the human musculature. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques, pages 163–172. ACM Press/Addison-Wesley Publishing Co. Seth, A., Sherman, M., Reinbolt, J., and Delp, S. (2011). Opensim: a musculoskeletal modeling and simulation framework for in silico investigations and exchange. Procedia IUTAM, 2:212–232. Sherman, M. and Eastman, P. (2012). Simbody: Multibody physics api: Overview. Available from: https://simtk.org/home/simbody [cited 27th of December 2013].

Bibliography

68

Siebert, T., Rode, C., Herzog, W., Till, O., and Blickhan, R. (2008). Nonlinearities make a difference: comparison of two common hill-type models with real muscle. Biological cybernetics, 98(2):133–143. Snoswell, M. (2003). Cg character - the ultimate human. Available from: http://www.cgcharacter.com/ultimatehuman.html [cited 12th of January

2014]. Sueda, S., Kaufman, A., and Pai, D. (2008). Musculotendon simulation for hand animation. In ACM Transactions on Graphics (TOG), volume 27, page 83. ACM. Tan, J., Turk, G., and Liu, C. (2012). Soft body locomotion. ACM Transactions on Graphics (TOG), 31(4):26. Taylor, T. (2013). Muscles of the leg and foot – male view. Available from: http://www.innerbody.com/anatomy/muscular/leg-foot-male [cited 6th of

January 2014]. Teran, J., Blemker, S., Hing, V., and Fedkiw, R. (2003). Finite volume methods for the simulation of skeletal muscle. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation, pages 68–74. Eurographics Association. Teran, J., Sifakis, E., Blemker, S., Ng-Thow-Hing, V., Lau, C., and Fedkiw, R. (2005). Creating and simulating skeletal muscle from the visible human data set. Visualization and Computer Graphics, IEEE Transactions on, 11(3):317–328. Thalmann, D., Shen, J., and Chauvineau, E. (1996). Fast realistic human body deformations for animation and vr applications. In Computer Graphics International, 1996. Proceedings, pages 166–174. IEEE. The National Center for Simulation in Rehabilitation Research (2010). Official opensim website. Available from: http://opensim.stanford.edu/ [cited 24th of August 2013]. Torus Knot Software Ltd (2013). Ogre source: Ogrequaternion.cpp — bitbucket. Available from: http: //www.ogre3d.org/docs/api/1.9/OgreVector3_8h_source.html#l00655

[cited 2nd of January 2014]. Vasavada, A., Li, S., and Delp, S. (1998). Influence of muscle morphometry and moment arms on the moment-generating capacity of human neck muscles. Spine, 23(4):412–422.

Bibliography

69

Whitaker, A. (2006). Sacred-geometry: The first step. Available from: http://www.ancient-wisdom.co.uk/sacredgeometry.htm [cited 13th of

January 2014]. Wilhelms, J. and Van Gelder, A. (1997). Anatomically based modeling. In Proceedings of the 24th annual conference on Computer graphics and interactive techniques, pages 173–180. ACM Press/Addison-Wesley Publishing Co. Yamaguchi, G. and Zajac, F. (1989). A planar model of the knee joint to characterize the knee extensor mechanism. Journal of Biomechanics, 22(1):1 – 10. Available from: http://www.sciencedirect.com/science/article/pii/0021929089901796.

Zajac, F. (1988). Muscle and tendon: properties, models, scaling, and application to biomechanics and motor control. Critical reviews in biomedical engineering, 17(4):359–411. Zhu, Q., Chen, Y., and Kaufman, A. (1998). Real-time biomechanically-based muscle volume deformation using fem. In Computer Graphics Forum, volume 17, pages 275–284. Wiley Online Library.

Appendix A

List of Included Musculotendons (OpenSim) Musculotendons from Menegolo (2011) for the lower body, sorted alphabetically. Each musculotendon is included twice in the project to account for both sides (left and right) of the human body reaching a combined total of 48 biomechanical musculotendons during the simulation’s runtime. ID

Anatomical Label

ID

Anatomical Label

6

Adductor Magnus 2

20

Medial Gastrocnemius

3

Biceps Femoris-Long Head

8

Pectineus

4

Biceps Femoris-Short Head

17

Piriformis

10

Gluteus Maximus 1

14

Psoas Major

11

Gluteus Maximus 2

15

Quadratus Femoris

12

Gluteus Maximus 3

18

Rectus Femoris

0

Gluteus Medius 1

5

Sartorius

1

Gluteus Medius 2

21

Soleus

2

Gluteus Medius 3

7

Tensor Fasciae Latae

9

Gracilis

23

Tibialis Anterior

13

Iliacus

22

Tibialis Posterior

16

Inferior Gemellus

19

Vastus Intermedius

1

Appendix B

Ralph’s Bone Hierarchy (RAGE)

root

vl5

vc4

vl3

r_knee

l_knee

vl1

r_ankle

vt10

r_subtalar

l_subtalar

vt1

r_foot

l_foot

l_eyeball_joint

l_eyeball

l_ankle

l_sternoclavicular

r_shoulder

r_eyeball_joint

r_eyeball

l_hip

r_sternoclavicular

vc2

skullbase

r_hip

l_shoulder

r_elbow

l_elbow

r_wrist

l_wrist

r_thumb0

r_index0

r_middle0

r_ring0

r_pinky0

l_thumb0

l_index0

l_middle0

l_ring0

l_pinky0

r_thumb1

r_index1

r_middle1

r_ring1

r_pinky1

l_thumb1

l_index1

l_middle1

l_ring1

l_pinky1

r_thumb2

r_index2

r_middle2

r_ring2

r_pinky2

l_thumb2

l_index2

l_middle2

l_ring2

l_pinky2

r_thumb3

r_index3

r_middle3

r_ring3

r_pinky3

l_thumb3

l_index3

l_middle3

l_ring3

l_pinky3

2

Appendix C

Upper and Lower Body Skeleton Hierarchy (OpenSim) A digit prefixing each node’s label indicates the amount of OGRE meshes attached to each OpenSim body.

1:ground

3:pelvis

4:thorax

1:femur_l

2:tibia_r

1:clavicle_l

2:tibia_l

1:talus_r

1:scapula_l

1:talus_l

1:calcn_r

1:humerus_l

1:calcn_l

1:toes_r

1:ulna_l

1:clavicle

1:toes_l

1:radius_l

1:scapula

1:humerus

1:ulna

1:lunate_l

1:scaphoid_l

1:trapezium_l

1:radius

1:pisiform_l

1:triquetrum_l

1:capitate_l

1:trapezoid_l

1:hamate_l

1:secondmc_l

1:thirdmc_l

1:fourthmc_l

1:fifthmc_l

1:2proxph_l

1:3proxph_l

1:4proxph_l

1:5proxph_l

1:2midph_l

1:3midph_l

1:4midph_l

1:5midph_l

1:firstmc_l

1:lunate

1:scaphoid

1:trapezium

1:femur_r

1:pisiform

1:proximal_thumb_l

1:triquetrum

1:trapezoid

1:distal_thumb_l

1:2distph_l

1:3distph_l

1:secondmc

1:thirdmc

1:fourthmc

1:fifthmc

1:2proxph

1:3proxph

1:4proxph

1:5proxph

1:proximal_thumb

1:2midph

1:3midph

1:4midph

1:5midph

1:distal_thumb

1:2distph

1:3distph

1:4distph

1:5distph

1:firstmc

1:hamate

1:capitate

3

1:4distph_l

1:5distph_l

Appendix D

List of Included UHM Meshes Musculotendon mesh files from the Ultimate Human Model data set Snoswell (2003) for the lower body sorted alphabetically. Note that some mesh files were combined to fit a single musculotendon unit in OpenSim, while others where divided for cases where OpenSim represents a single muscle with multiple action-lines, such as the Gluteus Maximus Blemker and Delp (2005). The ID field in the table below refers to mapped ID’s of the OpenSim musculotendon where a particular UHM mesh was attached to, given in Appendix A. In addition, each processed mesh is included twice in the project to account for both sides (left and right) of the human body. ID

UHM Mesh

0

r mbu002

Combined with

Divided into

Anatomical Label

r mbu002.1

Gluteus Medius

1

r mbu002.2

2

r mbu002.3

3

r mul004 b

Biceps Femoris (b)

4

r mul004 a

Biceps Femoris (a)

5

r mul007

Sartorius

6

r mul003 b

Adductor Magnus (b) r mul003 c

7

Adductor Magnus (c)

r mul010

Tensor Fasciae Latae r lig027

8

Iliotibial Band

r mhi006

Pectineus 4

Appendix D. List of Included UHM Meshes ID

UHM Mesh

9

r mul005

10

r mbu001 c

Combined with

5 Divided into

Anatomical Label Gracilis

r mbu001 c.1

11

r mbu001 c.2

12

r mbu001 c.3

Gluteus Maximus (c)

13

r mhi003 f

Iliospoas (f)

14

r mhi003 d

Iliospoas (d)

15

r mhi009

Quadratus Femoris

16

r mhi001

Inferior Gemellus

17

r mhi007

Piriformis

18

r mul006 b

Rectus Femoris (b)

19

r mul011 b

Vastus Intermedius (b) r mul011 a

20

Vastus Intermedius (a)

r lig044

Calcaneal Tendon r mll005 a

Gastrocnemius (a) r mll005 b

21

r mll010 a

Soleus (a) r mll010 b

22

23

Gastrocnemius (b)

r mll012 g

Soleus (b) Tibialis Posterior (g)

r mll012 a

Tibialis Posterior (a)

r mll012 b

Tibialis Posterior (b)

r mll012 c

Tibialis Posterior (c)

r mll012 d

Tibialis Posterior (d)

r mll012 e

Tibialis Posterior (e)

r mll012 f

Tibialis Posterior (f)

r mll011 c

Tibialis Anterior (c) r mll011 a

Tibialis Anterior (a)

r mll011 b

Tibialis Anterior (b)

Appendix E

Edge List Sort Algorithm 1

# Script to sort edge list

2

# of a simple polygon with

3

# 1 face.

4

# Something like this:

5

#

6

#

7

#

4/

8

#

\

9

#

5\_________/1

0_________3 /

\ \2 /

10

# becomes sorted as {[4,5],[5,1],...,[0,4]}.

11

# Tested with Blender v2.67.

12

# ##! Make sure to be in Edit Mode !##

13 14

import bpy

15

import os

16 17

#clear screen (command for windows)

18

os.system(’cls’)

19 20

scn = bpy.context.scene

21

obj = bpy.context.object

22 23

#get mesh data from object

24

mesh = obj.data

25 26

edge_count = mesh.total_edge_sel

27

print("number of elements in m:", edge_count)

28 29

if(edge_count > 0):

30 31

e = []

32

e.append(mesh.edges[0])

33

flip = []

34 35

#main loop for edge e

36

for e_iter in mesh.edges:

37

eb = e[-1].vertices[1] #end vertex of last edge of e

38

found = False

6

Appendix E. Edge List Sort Algorithm (for Blender 3D Authoring Software) 39 40

#first search

41

for m in mesh.edges:

42

ma = m.vertices[0] #begin vertex of edge m

43

mb = m.vertices[1] #end vertex of edge m

44

if(eb == ma):

45

flipFound = False

46

for n in e: if(n.vertices[0] == mb and n.vertices[1] == ma):

47

flipFound = True

48

break

49

if(flipFound == True):

50

continue

51

else:

52 53

e.append(m)

54

found = True

55

break

56 57

#check for reverse direction in case first search failed

58

if(found == False): for m in mesh.edges:

59 60

ma = m.vertices[0] #begin vertex of edge e

61

mb = m.vertices[1] #end vertex of edge e

62 63

#...also exclude existing m’s in e

64

if(mb == eb and m not in e):

65

#create duplicate to reverse vertex indices

66

m_dup = mesh.copy()

67

f = m_dup.edges[m.index]

68

f.vertices[0] = mb

69

f.vertices[1] = ma e.append(f)

70

else:

71

continue

72 73 74

#remove last element (was added twice)

75

del e[-1]

76 77

print("number of elements in e:",len(e))

78

print()

79

print("--- sorted edge list ---")

80

for idx, val in enumerate(e):

81

print("[",val.vertices[0],",",val.vertices[1],"]")

82

print()

83

print("--- original edge list ---")

84

for i in mesh.edges:

85

print("[",i.vertices[0],",",i.vertices[1],"]")

7

Appendix F

Area and Centroid Calculation For cases where the centroid (also known as the centre of gravity) is needed, one can use the equations from Bourke (1988) and the permutation σ in Algorithm 1. This goes as follows. The projected two dimensional area of Ck is first calculated with: n−1

area =

1X xi yi+1 − xi+1 yi 2

(F.1)

i=0

where x and y represent the two coordinates lying in the two dimensional coordinate system1 . Then, the centroid of each U Ck can now be calculated as well with: n−1

centroidx =

X 1 (xi + xi+1 ) (xi yi+1 − xi+1 yi ) and 6(area)

(F.2a)

i=0

n−1

X 1 centroidy = (yi + yi+1 ) (xi yi+1 − xi+1 yi ) 6(area)

(F.2b)

i=0

However, as discussed in Chapter 3, the centroid does not always lie within a slice taken from a UHM mesh and therefore is not suitable for this purpose.

1 Note that the centroid is a different calculation than the standard arithmetic mean seen in other contour extraction approaches and represents the centre of gravity of the Ck .

8