Reviewing for model parameters workshop

7 February 2011Eva KaltenthalerPaul Tappenden

Suzy Paisley

DSU Technical Brief

• Practical guidance on reviewing for model parameters

• March 2011

Background

• HTA work for NICE/NETSCC

• “For all parameters (including effectiveness, valuation of HRQL and costs) a systematic consideration of possible data sources is required” (NICE, 2008)

• Perspective as a systematic reviewer

TARs

• Focused question (PICO)-clinical effectiveness review

• Decision problem-cost effectiveness model

• Additional information needed for model parameters

Points to consider

• Rapid timelines

• Transparent

• Reproducible

• Systematically done

• Attempts to reduce bias

• Not just a series of rapid reviews

Six themes from Focus Groups• Current practice: discussions from

the beginning, iterations

• Adequate information: how much is enough?

• Timing: need for information on parameters can happen at different stages

Themes (continued)

• Ideal practice: team involved in problem structuring, “tagging of references”, focussed searching

• Problem structuring: what parameters are likely to be important

• Areas for further research: focussed searches, quality and selection decisions

Conclusions

• Reviewing, searching and modelling were seen as integrated tasks

• Whole team should be involved in structuring the decision problem

• Good communication is important

• Assessments of the quality and relevance of information are important

Conclusions

• This preliminary investigation highlights numerous concerns and potential deficiencies in the process of identifying, selecting and using evidence to inform models.

Future research

• Training is needed for focussed searching, problem structuring, quality assessment and the validation of parameter estimates.

• Guidance is required to ensure that such research activity is transparent, timely and rigorous.

DSU Technical Brief

1. What methods for problem structuring can be recommended?

2. How should evidence be systematically identified to inform model parameter estimates?

3. What guidance can be provided about the methods to use for reviewing model parameter data in a systematic fashion?

4. What recommendations concerning reporting can be made?