The Very Model of a Modern Major Modeler: An Interview with Kit N. Simpson, DrPH
on her New Cost-of-Care Model for HIV Care
 

Dr Kit Simpson from the Medical University of South Carolina in Charleston has been energetically and enthusiastically working with cost models specific to HIV since the late 1980s and this year, at both the recent ICAAC and EACS conferences, she presented intriguing results from a model that she feels is superior to the current model used to predict clinical and cost outcomes.

The ADAP Fund's interest in Dr Simpson’s research is twofold. From a global perspective, any model that can more accurately predict the costs of effective treatment is critical for those governments and other funders that pay for HIV care. When such budgets are limited, looking at the most cost-efficient effective treatment regimens may allow more people to access care within budget restrictions. From a US perspective, there are some states with limited AIDS Drug Assistance Program (ADAP) budgets that are not always able to provide HIV/AIDS drugs for all who need them. These states may, at some future date, need to explore treatment protocols that are based on the most cost-efficient regimens as an option for maximizing treatment access. They may also need to evaluate drug studies to determine which drugs may be most appropriate to include on their state formularies. In the event of a possible recession, the federal and state governments may have to consider the use of treatment protocols that are based on the most cost-efficient of the effective treatment regimens available.
 

The ADAP Fund recently caught up with Dr. Simpson to discuss the results of her new model and its implications for future funding for the care of persons with HIV/AIDS.

ADAP Fund: Your modeling data that you presented at this year's ICAAC and EACS conferences seem to have important implications for more accurately predicting the cost of care for HIV-infected persons, which could affect decisions on the level of funding needed to cover this cost in the future. But before we specifically talk about your model's cost predictions, tell us about why you have been doing research in this area, the new model you have developed, and how it compares with current standard models?

Simpson: What I have been doing for the past 19 years is to try to determine an HIV-infected person’s outcome based on data from completed clinical trials. I use the results from antiretroviral clinical trials in which patient data has been collected at 24 or 48 weeks of treatment and bring this data, such as the percentage of patients with undetectable viral loads and the magnitude of increase in CD4⁺ cells, together mathematically with epidemiologic data, so that predictions can be made on the average length of survival for a patient and the average cost of care for this patient.

In the study I presented at ICAAC, two models were compared—the Markov model and the Discreet Event Simulation model—to evaluate which model was a better predictor of patient outcome at certain time points, such as at 5 and 10 years after initiating antiretroviral treatment. These models take pieces of data to build a synthetic cohort of patients so that we can predict under known assumptions what will be going on as a result of differences in clinical trials. 
 

The Markov model is the old model, where patients are put into categories, and cost and AIDS events for each patient category are predicted for 3 months. After 3 months, known proportions of patients are then assumed to progress or get better. These progression rates are based on data from large patient cohorts. So a Markov model produces a “picture” of what is happening to a group of patients every 3 months until the patients die from either AIDS or cardiovascular disease events. In the new model, the discrete event model, the model is told to construct a thousand patients with certain patient characteristics, such as the average age is 37 years old, 30% of the patients are female, 20% of the patients do not adhere very well to their drug regimen, and so on. In other words, the model generates one thousand “cyber people” based on with these characteristics and then constructs a flow chart with a series of treatment decision points that are representative of how patients get medical care. At each decision point, the set of characteristics that each “synthetic” patient has determines which pathway is chosen for that patient. Each time patients come to a decision point, which is like a fork in the road, depending on their characteristics, they may go in different directions until a specified endpoint is reached. When the model is used to evaluate two different treatment interventions, the model is run for each drug regimen using the same patient population, allowing a comparison between the two regimens. This model produces a thousand individual “patients” with lifetime data that reproduces the major risks and costs that then can be estimated for each patient. It includes such risks as progressing to AIDS, having a heart attack, or dying from “old age”.

ADAP Fund: But in the ICAAC presentation, you just compared your model to the Markov model to see how well each predicted long-term outcomes when compared to data generated from an actual long-term study of HIV-infected persons.

Simpson: That’s right. What I did was I ran both models with the same synthetic study group and then compared what each model said would happen to patients at 5 years to what actually happened to patients in a well-known clinical trial. In this case, I used the 5-year data from the Abbott 720 study, which now has data out to 7 years.

ADAP Fund: This was a validation study for your Discrete Event Simulation [DES] model, yes?

Simpson: That’s correct. In this study, we found that both models were able to closely predict the outcomes seen in an actual cohort at 5 years but that the DES model was slightly better than the current Markov model.

What is great also is that this study represents the first time I know of that a model developed specifically for HIV disease has been actually validated against real cohort data.

ADAP Fund: You also looked at 1-year data. Why did you look at 1-year data if you are interested in long-term outcomes and costs?

Simpson: One-year data from the actual study cohort was used to calibrate the models. The models were adjusted at 1 year to make sure that the variables inputted into the model make the same predictions compared to the 1-year data reported from the actual clinical trial. After this point, the models were not adjusted further.

ADAP Fund: You put the baseline CD4 count and viral load into the models. How important are these in terms of predicting outcome and what are other key characteristics that you added into your model?

Simpson:  The CD4 count and viral load surrogate markers are the main drivers to predict outcomes. The CD4 cell count clearly determines the risk of an AIDS-defining event and hospital admission. The viral load clearly determines what happens to the CD4 count, the magnitude of change and how fast it changes. My objective with modeling is to evaluate new antiretroviral treatments. The model is a piece of software, and if it works, then a regimen’s 24- or 48-week clinical trial data put into the model will predict lifetime outcomes. The whole point here is to take these two very different pieces of software and compare them to what happened in the actual cohort. We found that both models closely predicted what happened in the actual cohort over time.

The problem with the Markov model is that it is categorical and divides a study population into 12 categories, which are 12 different health states defined by a set range for CD4 cell count and viral load. So, for example, the Markov model can look at 100 patients who receive Drug A and another 100 who received Drug B, and at baseline, all of the patients are separated out into one of 12 “boxes.” They begin treatment and are told to come back in 3 months for a clinic visit. At each 3-month clinic visit, their CD4 counts and viral loads are determined, and they are reassigned back into the same category or into a different one depending which of the 12 boxes a change, if any, in their CD count and viral load places them since the last clinic visit. Although the Markov model is able to stratify the patients over a specified treatment duration, it uses 12 mean values to represent what happens to the entire patient population at each 3-month interval. However, with the DES model, you can input into the model a wide distribution of CD4 counts and HIV RNA levels so that the model can construct a thousand different patients with different CD4 counts and viral loads that change over time, which of course is much more like that of an actual study population. The DES model is much more fine-grained—like real life—than the Markov model, and the time points do not have to be every 3 months. In this comparison, I used 8 weeks as the first time period.

ADAP Fund: Why was 8 weeks an important time point?

Simpson: Because in the clinical trial that I wanted to test here, that was the time point where subjects could crossover to the other treatment if they were not responding to their originally assigned treatment. In most other cases, I would set the first analysis at 12 weeks. If you think about what happens in the first 8 to 12 weeks after starting antiretroviral treatment, there is an abrupt decrease in viral load and then a slow rise in the CD4 count. And the DES model can capture these data, while the Markov model, which just provides average values of these variables every 3 months, cannot. In other words, the DES model is much more sophisticated in that it allows one to see what is happening to every patient everyday based on the actual distributions of patients in the clinical trials

ADAP Fund: How do you determine the distribution of the patients over time in these models?

Simpson: I analyze the clinical trial data with a statistical software program that identifies the distribution of the data based on actual trial data distributions. In fact, data from several different cohorts are used to specify how the data are distributed in the DES model. Over the past 19 years, I have analyzed clinical trial data from thousands of patients. I also use published cohort data from the Medical University of South Carolina.

ADAP Fund: Okay, so what is the purpose of the Markov and DES models? Why are models developed?

Simpson: The model helps us get an understanding of what the statistical results reported for clinical trials actually mean in numbers that we care about. Most of us have no real understanding of what an 8% or 10% or 12% or 5% difference in virologic suppression at 48 weeks actually means in terms of the average length of life of the patient, the average time that they stay on their first antiretroviral therapy, and the cost of their care over the next 5 years. The model can predict the outcomes that are meaningful for planning purposes.

ADAP Fund: So the Markov and DES models are ultimately used to get predictions of how long a specific patient might stay on treatment with regard to reaching a clinical end point. The DES model seems also to be able to account for different levels of patient adherence to treatment.

Simpson: That’s correct. In terms of reaching a clinical end point, an adherence variable can be added, for example, telling the model that there is a 20% probability that subjects will not adhere resulting in viral breakthrough.

ADAP Fund: Did you include quality-of-life measures in your model?

Simpson: I do look at quality of life. As you know, quality of life is not a very good measurement because it is subjective, but there are assumptions that can be made, with some clinical pictures where we can all agree on, such as the person who has a CD4 cell count of 900 and has never had an AIDS-defining condition. This person probably has a better quality of life than one who has a CD4 cell count of 50 and has CMV disease.

ADAP Fund: How do you adjust for quality of life in your model?

Simpson: When I developed the DES model in 2006, I obtained all the clinical trial data related to quality-of-life measurements that Abbott Laboratories had on file from past clinical trials. I analyzed about 21,000 observations in order to give the model the capability to include a quality of life measurement for individual patient based on CD4 cell count and viral load, so this is a very strong data set.

ADAP Fund: One of your conclusions is that the DES model is better than the Markov model at five years.

Simpson: Yes, not only was the DES model slightly better but also the Markov model could not predict the CD4 cell changes and the DES model could. For viral load, the models are looking at viral suppression below a specific undetectable level that was clearly defined. But for CD4 changes, the Markov model is not good because it groups patients into categories that are too big. So, the Markov model is limited because it could not predict certain important variables at different time points.
 

This comparison of the Markov and the DES model presented at ICAAC is not only about the DES model being better. It is also exciting because it compares two modeling estimates to what actually happened to patients, and thus validates the model predictions. This type of validation is quite rare in modeling. I have been doing modeling since 1988, and this is the first time that I have been able to validate a model’s predictions against real long-term data. So what is really exciting is that the DES model was able to predict very accurately what happens over time in a real patient cohort

ADAP Fund: Let’s talk about how models might help policy makers make the best decisions regarding the use of limited resources to fund such government programs as ADAP. These models can be used in cost-effectiveness studies to show comparisons in the cost of different antiretroviral drug regimens over time, yes?

Simpson: Yes. The Markov model is generally used to predict total cost and outcomes and budget impact at 5 and 10 years and lifetime cost effectiveness in sort of a coarse manner. The DES model can make the same predictions, but can provide more information and appears to do so more accurately than the Markov model. For example, as I mentioned earlier, a range of expected adherence rates can be put into the DES model to see how that impacts the cost-effectiveness of treatment. If initial treatment fails in a patient because of poor adherence, subsequent regimens will be more expensive for several reasons, including the need to use more expensive regimens because of drug resistance. However, both models can provide the budget impact for a fixed cohort over 5 and 10 years and examine the cost-effectiveness. And this is very important for understanding the impact of newer drugs.

Essentially, what I do with models is to make comparisons between two drug regimens. For example, I can take any new drug, such as maraviroc, and use the model to compare it to tipranavir. If I can make sure that the results from the clinical trials are from similar study populations, then I can use the data from each trial and put that in the model with the cost of each drug to make long-term predictions.

ADAP Fund: Are you able to compare different patient subgroups?

Simpson: Yes, I can do the modeling based on data from different subgroups if the clinical trials are designed to examine data from special patient groups. In clinical trials like the maraviroc and tipranavir trials where the results were reported for different suppression levels for different subgroups, the model can predict cost and outcome differences for a particular subgroup.

ADAP Fund: Who are the decision makers that generally use these types of cost-effectiveness studies?

Simpson: There are two sets of decision makers. The studies are used in Europe, Canada, and Australia for funding decisions related to drug approval because their regulatory bodies, such as the European National Institute for Clinical Excellence and the Australian Pharmaceutical Benefits Assessment Commission, require data on a product’s cost-effectiveness. Basically, in those countries the government cannot pay for a drug without this kind of analysis performed by the manufacturer. The second set of decision makers is US payers and policy organizations, such as ADAP.

ADAP Fund: Do these international agencies use the Markov model?

Simpson: The Markov model has been used to inform reimbursement decisions by governments in 13 countries. In the United States, the HMOs and Medicaid prefer to see the 5- and 10-year budget projections.

The NICE agency has used the Markov model for approving the price of the drug at launch. It is not used for drug approval. In other words, you can buy the drug in a country, but the government will not pay for the drug if it is not cost-effective. One of the benefits of running these models is if a pharmaceutical company is charging too much for their drug, the cost-effectiveness ratio would look very high compared to the drug’s benefit. As a matter of fact, recently an approval for drug cost reimbursement was delayed in the United Kingdom because a model showed that the price of the drug was too high for the extra benefit provided compared to other drugs that available.

ADAP Fund: None of the agencies are presently using the DES model?

Simpson: Not yet. It is brand new.

ADAP Fund: Do you think there will be resistance by decision makers to adopt this new model even though it appears to be more accurate?

Simpson: No, I think they would not mind a more accurate model. The only problem is that understanding these models has not been easy for government regulators. Further, all of the models to date for evaluating antiretrovirals have been Markov models; so, the agencies are familiar with data from Markov models, and they will not be quite sure how the DES model works, and thus how accurate the model’s predictions are, until they get used to this new model.

ADAP Fund: How does the impact of the virologic and immunologic responses to initial therapy affect a drug’s cost-effectiveness?

Simpson: With first-line therapy, there are two factors. Using the intent-to treat data, the first factor is how many subjects dropout during the first 12 weeks of treatment. All the clinical trials report a huge dropout each week during this initial period, sometimes as much as 30%, and these patients are counted as failures. So no drug is going to look all that good if a whole lot of people stop using it because of side effects or because they do not want to stay in the trial for other reasons, and this early analysis is very important. After that, the next factor—and real critical determinant—is, how powerful is the drug and how sustained is its effect. These are the main factors in a cost-effectiveness model.

ADAP Fund: What about second-line therapy?

Simpson: As we know, second-line therapy is critical in terms of its ability to overcome the development of drug resistance by the virus to previous therapies. And it is easy to grasp that a drug whose efficacy is not diminished by the presence of drug-resistant viral mutants is much more cost-effective than one that cannot overcome drug resistance. The costs in this case are the increased costs related to hospitalization and the progression to salvage therapy. Once the patient needs to be treated with an increased number of drugs or with high-priced drugs like enfuvirtide at a multidrug regimen cost of $100 per day, it gets very expensive. Modeling can predict these costs over time. Again, the major variables for a cost-effectiveness model to make such predictions are the baseline CD4 count and HIV RNA level and their change over time.

ADAP Fund: Could cost-effectiveness studies of antiretroviral therapies help state ADAP administrators better meet the needs of people enrolled in state ADAP programs?

Simpson: They probably could, especially if the study’s model results are used to predict the time point at which to switch to a different drug regimen given the patient’s degree of adherence to treatment, for example. That is what really matters in the clinic. The DES model can identify the people who do not adhere to their antiretroviral regimen and are going to be burning up their drug options much quicker. And the model can show the budget impact of this because the drug costs and other costs, such as hospital admissions related to AIDS-defining illnesses, are put into the model.

ADAP Fund: So, let’s assume that a new antiretroviral has just been approved by the FDA and is very expensive. How is the decision-maker, for example, an ADAP administrator with a limited budget, going to make the most cost-effective decision on whether to include this new drug on the formulary?
 

Simpson: This is exactly what these models can show. In this case, the model very often shows an increase in costs in the first year or two but then shows cost savings with sustained treatment. The Markov model has actually shown that. And one of the papers that I have published on the Markov model has shown exactly what those cost savings could be (Simpson KN, Jones WJ, Rajagopalan R, Dietz B. Cost-effectiveness of lopinavir/ritonavir compared to atazanavir plus ritonavir in antiretroviral-experienced patients in the U.S. Clin. Drug Invest. 2007;27:807-817).

Simpson: This is where an administrator, for example, can see that he or she will be spending more during the first 18 months but after that, there will be cost savings and at the end of 5 years, it may be as much as $18,000 per patient. The models will predict this. Valid prediction of cost and outcomes is important for showing the magnitude of the benefits of ADAP funding, as well as for showing that ADAP is a cost-effective policy solution.

ADAP Fund: Is this type of modeling data now being looked by state agencies with regard to HIV?

 Simpson: I used it once while I was at the University in North Carolina at Chapel Hill when I was in the School of Public Health there. Dr. Josephine Mauskopf and I were asked to advise the State’s Medicaid Administrators on which 1 of 5 new antiretroviral agents they should put on their formulary. After putting their data into an older Markov model, we told them that they should put all 5 new antiretrovirals on their formulary, and that they should also pay for prevention of opportunistic infections. Medicaid came back and said that it could not afford to do that. We told them our modeling data showed that if they put all of their patients on regimens that included at least 1 of the 5 new antiretrovirals, the annual cost savings would be $100,000 plus or minus $17,000 in hospital admissions for AIDS-related events, even though the drug costs were higher. And that old model actually was the one that ADAP started using for predicting outcome

ADAP Fund: So, let’s say Medicaid or ADAP tells you that they do not have the budget to put all 5 new drugs on formulary. Give us some advice on which ones to use. How would you differentiate among new drugs besides the drug cost?

Simpson: Viral load suppression. This includes time to viral suppression as well as time to any subsequent virologic breakthrough. And because different drugs have different levels of power in treatment-naïve and treatment-experienced patients, we look at viral load suppression and drug cost in both populations. The DES model could estimate the expected cost and outcomes for the actual ADAP population distribution.

ADAP Fund: I assume you use the 48-week data from the drug’s registrational trials in the models?

Simpson: Yes, I do, but I also “scrounge up” all the data I can from anywhere and put this information into the model or use it for validating the model’s predictions.

ADAP Fund: With data used from different studies, how do you account for differences across the different studies? For example, differences in their study populations and study methodologies?

Simpson: I gather all the data, but only data where the populations are similar. I can’t use data from some studies. For example, some comparative trial results show that at 96 weeks, one drug performs equally as well as the other. But if 40% of the patients in one treatment group dropped out by 24 weeks, the data are not really worth anything for modeling. With the models I use, I do not have to use data beyond 24 weeks, so I can take 24-week or 48-week data in the model to predict outcomes after 5 or 10 years of treatment under a reasonable set of assumptions.

Also, if a trial’s methodology has what I call “informative dropout,” that is, if the trial was designed in a such way that patients knew what drug regimen they were receiving and could crossover to a rescue arm at a predetermined time point, such as after 24 weeks, then I do not use this data because all of the seriously ill patients have likely left the control group. The patients who are left in the control group are going to look good, but they are only the subjects who responded to the treatment. One advantage of using a model is that you can examine the differences that would be expected at, say, 96 weeks with those actually published after 96 weeks of follow up. Thus, modeling can give us a good idea of the bias from informative dropout, something you often cannot do with the standard statistical analysis of clinical trial data.

ADAP Fund: What kinds of published studies would you recommend for state ADAP directors consider?

ADAP Fund: Given that it is becoming an issue of multidrug regimen costs—not single-drug costs—how well does the DES model lend itself to not just comparing one drug to another but comparing different regimens in terms of costs and health outcomes?

Simpson: The DES model is well suited to do this, and it is specifically constructed to look at a key objective that has to do with new drugs and what kind of resistance they generate. The DES model is the type of model that can take resistance data and translate it into clinical and economic outcomes. So, the DES model can show not only the effect you get from the first drug regimen, based on resistance patterns, but it can also provide information about how much damage there will be to the success of subsequent drug regimens. The limitations for the DES model are not really related to the model; its limits are mainly due to lack of data on what actually happens to a subgroup of patients who are treated with specific sequences of drugs. None of the models are magically able to predict outcomes if no data are available to help with the prediction.

ADAP Fund: Let me understand this better. Let’s say you are evaluating a new drug, such as the CCR5 coreceptor blocker maraviroc, and it is going to be combined with a protease inhibitor plus two NRTIs. Also some patients will also get enfuvirtide and some will not. So, with this rather unique regimen for which you cannot grab a lot of clinical data, how do you use a model to make cost predictions and comparisons to more common salvage regimens?

Simpson:  The problem with models like this is that they have to be run on data; in other words, there has to be some data available somewhere. If clinicians can’t point me to data on the individual drugs, then the data requirements for the model cannot be met. However, in recent clinical trial reports, a drug’s effects are reported separately for each patient pre-treatment subgroup in a very responsible and much more informative manner than even 2 years ago. These types of reports often provide excellent data for our models.

ADAP Fund: How are they reported differently now?

Simpson: These clinical trials now routinely report the actual responses rates for subgroups of patients not only by baseline CD4 cell count but also for the magnitude of viral load, such as above and below 100,000 copies/mL at study entry and, often by resistance score based on the number of active drugs in the regimen. Once you have this information, then you can construct individual patient groups and put each individual patient subgroup’s data in the model and run the model for the combined subgroups. You can do this as long as the clinical trial results are reported for valid subgroups.

ADAP Fund: So you get 5-year and 10-year estimates of the cost and outcome of treatment and compare these predictions to the cost and outcome of a different regimen.

Simpson: Yes, there are always two comparisons in the model because you want to make a decision of which drug is the most cost-effective. Doing modeling like this is like a puzzle. Let’s say there are 20 different papers on atazanavir and 40 papers published on Kaletra. All of these reports are searched to find the most valid pieces of information to populate the model, and this is done for two drugs that have not necessarily been compared head-to-head in a clinical trial. For example, one drug might be compared to nelfinavir and the other drug might also have been compared to nelfinavir in a different study. In this case, the model would be adjusted for the response to nelfinavir. This is called an indirect comparison.

ADAP Fund: Can the model be used to compare two regimens that have a different number of drugs, such as a 3-drug regimen versus a 5-drug regimen?

Simpson: Yes. The 5-drug regimen has 5 drug costs and will have a certain percentage of patients with viral load suppression below 400 copies/mL and a certain percentage below 50. So the data for viral load below 400 and below 50 from one arm are plugged into the model along with the cost of the 5 drugs, and the corresponding data and drug costs from the 3-drug arm are also plugged in, and the model is run. But it’s not possible to then remove or add a drug and then run the model to see the effect. You could not even get this information in an actual clinical trail because you have to look at each individual patient’s resistance score and see which drugs they are sensitive to. One of the things that these models can do is actually show clinicians what would be expected to happen clinically and costwise if they were to stop treatment with an ineffective multidrug regimen that contains 5 or 6 or 7 drugs and instead use a simple regimen for 6 months (until one or more new drugs become available). In this case, such patients might be better off taking a very simple regimen for 6 months that just keeps the virus suppressed in the CNS, until a new drug becomes available. So the modeling results may help inform clinicians’ discussion of which policies to choose, although, individual treatment choices should be decided by the patient and the doctor together.

ADAP Fund: Are there any studies on medical cost savings resulting in improved outcomes that show early HIV treatment intervention is more cost-effective than later intervention?

Simpson: There is one study that was done by a group from Boston University and Harvard. Ken Freedberg and Milt Weinstein were the ones involved with that (Schackman BR, Freedberg KA, Weinstein MC. et al. Cost-effectiveness implications of the timing of antiretroviral therapy in HIV-infected adults.. Arch Intern Med. 2002;162:2478-2486). They actually ran a model similar to the models I use, looking at starting early with antiretroviral therapy versus starting late, and they found that it was quite cost-effective to start early. My DES model can do the same analysis; I just have not done it yet. 

ADAP Fund: Thank you, Dr Simpson, for your time today and for continuing this important research. We at the ADAP Fund are looking forward to your work on cost models being adopted by state ADAP administrators and possibly bringing a greater benefit to those persons with HIV/AIDS who need ADAP to obtain life-saving medication.