A Case Study: Examination of RAM/CAM Application for Evaluation of a Coupled Musculoskeletal-FEA Model

Anthony Petrella, PhDColorado School of Mines, Golden, CO

AnyBody Technology Inc, Cambridge, MA

Ruxi Marinescu, PhD

Brian McKinnonSmith & Nephew, Inc, Memphis, TN

Jeff Bischoff, PhDZimmer, Inc, Warsaw, IN

SwRI, San Antonio TX

January 22, 2014

VV40: Committee on Verification & Validation for Modeling & Simulation of Medical Devices

Technical Symposium, Subgroup: Orthopaedics

Aims of this Case Study

Explore application of RAM/CAM to realistic modeling scenario used in orthopaedic implant development

Attempt to consider a more complex modeling workflow comprised of multiple scales and simulation methods

Consider a common modeling context in device new product development – comparison to predicate device reference

Specifically, we sought to examine the question...

How well do the RAM/CAM evaluation criteria work in their current form for a coupled musculoskeletal-FE analysis for purposes of device evaluation?

Literature Used

Kim et al., “Evaluation of Predicted Knee-Joint Muscle Forces during Gait Using an Instrumented Knee Implant,” JOR, pp.1326-1331, Oct 2009.

Lin et al., “Simultaneous prediction of muscle and contact forces in the knee during gait,” J Biomech, 43, pp.945-952, 2010.

Pegg et al., “Evaluation of Factors Affecting Tibial Bone Strain afterUnicompartmental Knee Replacement,” JOR, pp.821-828, May 2013.

Disclaimers

The authors did not organize the details of their articles with the intention of being “scrutinized” in the context of the CAM

Evaluators have limited experience with RAM/CAM

Summary of Modeling Workflow

(Pegg et al., 2013)

(Kim et al., 2009)

(Lin et al., 2013)

COU

Decision/Question… Have we satisfied design verification requirements for this UKR design?

Some patient needs:a) Avoid subsidence of device

b) Avoid chronic pain

Related design inputs:a) Good coverage, modest resection, no excessive rise in

periprosthetic bone strain relative to successful predicate(s)

b) Modest resection, no excessive rise in periprosthetic bone strain relative to successful predicate(s)

Model serves as sole source of info to test whether design inputs RE: bone strain have been met

V&V Workflow

COU not explicitly defined, Draft v1.1

One definition of COU is:1. Decision/question to be addressed

2. Influence of model on decision

3. Risk to patient

Any of the three elements can change independently and affect COU

How can risk be separate from COU?

RAM

RAM – VV40 Guide Draft v1.1

Moderate: M&S is considered to address only a part of the decision... There are ample data from similar sources...

Major: M&S is not the sole source of information... Data are available from similar sources to support the decision but no data are available from the actual environment...

Controlling: no data are available from other sources for essential aspects of the system and the M&S plays a key role in the decision.

A. No adverse health consequences

B. Limited (transient, minor impairment or complaints)

C. Temporary or reversible (without medical intervention)

D. Necessitates medical or surgical intervention

E. Results in permanent impairment of body function or permanent damage to a body structure

F. Life-threatening (death could occur)

G. Hazard cannot be assessed

COU Summary

Decision/Question… Have we satisfied design verification requirements (inputs a,b) for this UKR design?

Related design inputs:a) Good coverage, modest resection, no excessive rise in

periprosthetic bone strain relative to successful predicate(s)

b) Modest resection, no excessive rise in periprosthetic bone strain relative to successful predicate(s)

Summary of Modeling Workflow

(Pegg et al., 2013)

(Kim et al., 2009)

(Lin et al., 2013)

1

2

CAMA. Software VerificationVerification of the Software Code & Solution:

0. Insufficient1. Minimal2. Some testing conducted3. Some peer review conducted4. All algorithms tested, independent peer

review conducted

Matlab model, geometry registration Geomagics. Based on the information provided we don’t know if the author reviewed the verification activities to determine if those were relevant to this

application. Based on prior analysis.

CAM A = 1

Note: element types, adequate mesh size, not applicable.

CAMB. Validation computational modelB1. System configuration:0. Insufficient1. Minimal – abstraction of geometry2. Simplified – single patient-specific case, captures major features3. Minor/major features captured, ranges of possible geometry, multiple

cases4. All features captured, multiple cases, statistically relevant

Skeletal model – o Implant geometric model – patient-specific post-surgery CT data, CAD

models of the patient’s implant components (Patient 1)o Bone geometric model – MRI-derived bone models from another

patient (Patient 2) Muscle model – 11 muscles (values from literature), MRI-derived

models from Patient 2 with muscle and patellar ligament origin/insertion locations

Articular contact model – TF and PF, elastic model

CAM B1 = 3?

Patient specific inverse dynamic model. The equations of motion were derived using Autolev symbolic manipulation software. The complete

knee model was implemented in Matlab (OpenSIM for Pegg et al. 2012)

CAM B2 = 3?

B2. Governing equations:0. Insufficient1. Substantially simplified2. Model forms are based and tuned on data from

related systems3. Representation of all important processes, tuning

needed4. Key physics captured, minimal need for tuning

CAMB. Validation computational model

Skeletal model – o Implant geometric model – linear elastic isotropic

materials (Pegg et al. 2012)o Bone geometric model – Calculated from HU in CT scans?

Pegg et al. 2012) Muscle model – muscles (strength values from

literature), patellar ligament (data from literature); mapping of the muscle attachment sites (Matlab, Pegg et al. 2012)

Articular contact model – TF and PF, elastic model

CAM B3 = 3?

B3. System properties (biological, physical properties)0. Insufficient1. Simplified properties, sensitivities not

addressed2. Nominal properties, uncertainties3. Distribution of properties, uncertainties

identified4. Key properties captured, sensitivity analysis

CAMB. Validation computational model

The femur was fixed to ground, where the tibia and patella were allowed to move

relative to it; Two-level optimization approach (Matlab – min of the sum of 3

compressive contact forces).

CAM B4 = 3?

B4. Boundary conditions (e.g., applied loading)0. Insufficient1. Significant simplification2. Some simplification of BCs3. Representative BCs, uncertainties identified4. No simplifications, appropriate distribution

of variation, comprehensive sensitivity analysis

CAMB. Validation computational model

CAMC. Validation: Evidence-based comparator

Pegg et al. 2012Force-measuring tibial prosthesisOver ground walking trials (normal and medial-lateral trunk sway)Experimentally measured contact forces (medial and lateral sides of the tibial tray – loads magnitude, direction, position and contact area recorded; custom python script)

CAM C1 = 3

C1. System configuration:0. Insufficient1. Locations for data collection are roughly measured; geometry of parts

is assumed2. Locations for data collection are prescribed and measured; geometry of

parts is coarsely measured (?); calibrated system or signal/noise ratio>1

3. Locations for data collection are prescribed and error collected; geometry of parts is measured to machine tolerance; signal/noise ratio is high

4. All dimensions known to greater than machine precision; high precision

CAMC. Validation: Evidence-based comparator

Pegg et al. 2012Adult male subjectForce-measuring tibial prosthesisGait analysisExperimentally measured contact forces

CAM C2 = 4

C2. System Properties:0. Insufficient1. Material properties are average, homogeneous non-specific;

environment conditions unknown2. Material properties are average, homogeneous specific to the

system; environment conditions known3. Key material properties are measured and heterogeneity

captured4. All material properties are measured, environmental effects

accounted for.

CAMC. Validation: Evidence-based comparator

Pegg et al. 2012Adult male subjectForce-measuring tibial prosthesisGait analysis, over ground walking trials (normal and medial-lateral trunk sway)Experimentally measured contact forces

CAM C3 = 4?

C3. Boundary Conditions:0. Insufficient1. System states are not specifically measured2. System states are specifically measured or perturbations are

measured3. System states are specifically measured, affected degrees of

freedom are known and perturbations are measured4. System states are specifically measured and degrees of

freedom are known and perturbations are measured; variability known

CAMC. Validation: Evidence-based comparator

Pegg et al. 2012Single adult male subject

CAM C4 = 1

C4. Sample Size:0. Insufficient1. Single case or few cases2. Several cases or statistically relevant sample size3. Several cases and statistically relevant sample size4. Comprehensive parameter variability and statistically

relevant sample size for all parameters

CAMD. Determine model credibility

Discrepancy between the model and the comparator “Model was implemented with the equivalent comparator conditions”

(3 or 4? Variability?)

Comparison: qualitative or quantitative “Quantitative comparison of single achievable case” (2)

Applicability of V&V activities to Context of Use “Embodies key CoU features and captures key system properties” (3)

Single adult male subject (Patient 1) Musculoskeletal model Skeletal model –

o Implant model – patient-specific post-surgery CT (Patient 1)

o Bone geometric model – MRI-derived bone models (Patient 2)

Muscle model – 11 muscles, MRI-derived models (Patient 2) Articular contact model – TF and PF, elastic model

CAM D = 2.666666667 or 66.67% (8/12)

Credibility assessment matrix: MS Model

3

2

4

1

2

3 3

4

1

CAM. Verification

A. Code Mimics, CT segmentation MATLAB, ICP & muscle force sites SolidWorks, bone resection Mimics + custom, material calcs & mapping Abaqus, FE solution Python scripting for Abaqus

Application region for native contact load Analytical field for implant load to bone von Mises strain values Probabilistic variation of loading

PASW Statistics, statistical analysis

Score = 1

CAM. Verification

B. Solution FE mesh convergence study performed FE simplifications, no sig affect on results

Direct load vs. using actual implant Implant interface, tie vs. rough/friction Full length tibia vs. truncated

Probabilistic variation in load magnitudes based on errors reported in MS model

Material properties from CT mapping reported consistent with previous pub

Score = 2

CAM. Model Validation

A. Configuration Tibia bone geometry from single patient

extracted from CT using “previously validated method”

No information about implant model Loads applied directly to the bone surfaces Bone cuts made in accordance with surgical

technique published by implant company

Score = 2

CAM. Model Validation

B. Governing Equations Structural FE methods well established for

stress / strain calculation Physics…

Static FE simulation Bone modeled as linear elastic and

isotropic – no rate effects Non-homogeneous material property

mapping from CT Muscle and joint loading derived from

gait simulation and instrumented TKR

Score = 3

CAM. Model Validation

C. System Properties Non-homogeneous material properties

mapped from CT based on published equations

Properties not compared to human subject, but “consistent with previous” published data

No sensitivity analysis reported in relation to properties

Score = 2

CAM. Model Validation

D. Boundary Conditions Contact loading applied directly to bone surface;

compared to case with implant to confirm no significant effect on outcomes

Muscle forces from MS model applied to bone Load BC’s associated with level gait only Uncertainty in load magnitudes due to upstream

errors (MS model) incorporated using Monte Carlo simulation with (only) 40 random cases

Several custom Python scripts employed with no direct verification – creates doubt

Score = 2

CAM. Comparator Validation

Comparator A. Configuration

Comparator B. Governing Equations

Comparator C. Properties

Comparator D. Sample Size

There was NO COMPARATOR, model only used to assess relative change in outcome metric (bone strain)

Scores = 0, 0, 0, 0

CAM. Validation – Model/Comparator

A. Discrepancy

B. Comparison of Outputs

There was NO COMPARATOR, model only used to assess relative change in outcome metric (bone strain)

Scores = 0, 0

CAM. Validation – Model/Comparator

C. Applicability of V&V to COU…

Decision/Question… Have we satisfied design verification requirements (design inputs a,b) for this UKR design?a) Good coverage, modest resection, no excessive rise in periprosthetic bone

strain relative to successful predicate(s)

b) Modest resection, no excessive rise in periprosthetic bone strain relative to successful predicate(s)

Score = 4

CAM Summary

1

2 2

3

2 2

0 0 0 0 0 0

4

CAM Summary – Complete Workflow

Is the multiscale workflow acceptable?

Not obvious how to create composite score

Does precursor (MS) model even need to be evaluated, or simply captured in BC’s eval for the FE model?

Comments

RAM/CAM application for a coupled MS-FEA modeling workflow Value in looking at published models? Probably will be common in practice, and info often lacking in literature Perhaps publication standards need to evolve Library of Models, MS model repositories will help

A model is rarely based on a single piece of code How best to apply guidelines? CAM separate or combined? Is CAM even needed explicitly for both (all) models? When?

An explicit definition of COU and how to identify it will probably be needed for general users We defined COU as: Question/Decision + RAM (influence, risk) Any of three elements can independently change COU

Utility of RAM for orthopedic applications may be limited Always “Medium”? Same influence, same risk for any COU? Spine, joints, same risk for all? Will some standard simplifications evolve for specific industries?

Comments

Should CAM be a measuring tool or a checklist? “Credibility = 2.6” could be misleading Acceptance criteria? What is good enough? What does a “4” look like? Perhaps some “grandfathering” of accepted modeling paradigms will occur What impact does an incremental shift in COU have on acceptance? Removing numbers could make a subtle but positive psychological difference, not just

post-hoc scoring but planning for specific CAM levels before model development Moving “applicability” considerations to beginning of CAM may be more effective

Model Validation, BC’s Level 4 = “no simplifications” Does this make sense for a “model”, which is inherently simplified?

Comparator evaluation difficult for human subjects Especially for subject-specific modeling efforts If model compares well to single subject, does that mean it is extensible to others? Are all comparators equal? Is a weighting factor appropriate for human subjects?

Comments

510(k) pathway with comparison to predicate device extremely common in orthopaedics

Predicate device comparison (relative analysis) Strictly speaking, no direct comparator for outcome metric, but… Predicate will have controlling influence on “decision” Should predicate model be evaluated separately from primary model? Should predicate evidence be critically considered, where/how?

Incremental increase in value (CAM score) vs. increased cost to improve model is a consideration

Clinical History

Predicate Model

COU Model