Dr Kit Simpson
from the Medical University of South Carolina in Charleston has been
energetically and enthusiastically working with cost models specific to HIV
since the late 1980s and this year, at both the recent ICAAC and EACS
conferences, she presented intriguing results from a model that she feels is
superior to the current model used to predict clinical and cost outcomes.
The ADAP Fund's
interest in Dr Simpson’s research is twofold.
From a global perspective, any model that can more accurately predict the
costs of effective treatment is critical for those governments and other
funders that pay for HIV care. When such budgets are limited, looking at the
most cost-efficient effective treatment regimens may allow more people to
access care within budget restrictions. From a US perspective, there are
some states with limited AIDS
Drug Assistance Program (ADAP)
budgets that are not always able to provide HIV/AIDS drugs for all who need
them. These states may, at some future date, need to explore treatment
protocols that are based on the most cost-efficient regimens as an option
for maximizing treatment access. They may also need to evaluate drug studies
to determine which drugs may be most appropriate to include on their state
formularies. In the event of a possible recession, the federal and state
governments may have to consider the
use of treatment
protocols that are based on the most cost-efficient of the effective
treatment regimens available.
The ADAP Fund
recently caught up with Dr. Simpson
to discuss the results of her new model and its implications for
future funding for the care of persons with HIV/AIDS.
Your modeling data that you presented at this year's ICAAC and EACS
conferences seem to have important implications for more accurately
predicting the cost of care for HIV-infected persons, which could affect
decisions on the level of funding needed to cover this cost in the future.
But before we specifically talk about your model's cost predictions, tell us
about why you have been doing research in this area, the new model you have
developed, and how it compares with current standard models?
Simpson: What I have been doing for the past 19 years is to
try to determine an HIV-infected person’s outcome based on data from
completed clinical trials. I use the results from antiretroviral clinical
trials in which patient data has been collected at 24 or 48 weeks of
treatment and bring this data, such as the percentage of patients with
undetectable viral loads and the magnitude of increase in CD4+
cells, together mathematically with epidemiologic data, so that predictions
can be made on the average length of survival for a patient and the average
cost of care for this patient.
In the study I
presented at ICAAC, two models were compared—the Markov model and the
Discreet Event Simulation model—to evaluate which model was a better
predictor of patient outcome at certain time points, such as at 5 and 10
years after initiating antiretroviral treatment. These models take pieces of
data to build a synthetic cohort of patients so that we can predict under
known assumptions what will be going on as a result of differences in
model is the old model, where patients are put into categories, and cost and
AIDS events for each patient category are predicted for 3 months. After 3
months, known proportions of patients are then assumed to progress or get
better. These progression rates are based on data from large patient
cohorts. So a Markov model produces
a “picture” of what is happening to a group of patients every 3 months until
the patients die from either AIDS
or cardiovascular disease
events. In the new model, the discrete event model, the model is told
to construct a thousand patients with certain patient characteristics, such
as the average age is 37 years old, 30% of the patients are female, 20% of
the patients do not adhere very well to their drug regimen, and so on.
In other words, the model generates
one thousand “cyber people” based on with these characteristics and then
constructs a flow chart with a series of treatment decision points that are
representative of how patients get medical care.
At each decision point, the set of characteristics that each “synthetic”
patient has determines which pathway is chosen
for that patient.
Each time patients come to a decision point, which is like a fork in the
road, depending on their characteristics, they may go in different
directions until a specified endpoint is reached. When the model is used to
evaluate two different treatment interventions, the model is run for each
drug regimen using the same patient population, allowing a comparison
between the two regimens. This model produces a thousand individual
“patients” with lifetime data that reproduces the major risks and costs
that then can be estimated for each
patient. It includes such
risks as progressing to AIDS,
having a heart attack,
or dying from “old age”.
But in the ICAAC presentation, you just compared your model to the Markov
model to see how well each predicted long-term outcomes when compared to
data generated from an actual long-term study of HIV-infected persons.
Simpson: That’s right. What I did was I ran both models with
the same synthetic study group and then compared what each model said would
happen to patients at 5 years to what actually happened to patients in a
well-known clinical trial. In this case, I used the 5-year data from the
Abbott 720 study, which now has data out to 7 years.
This was a validation study for your Discrete Event Simulation [DES] model,
Simpson: That’s correct. In this study, we found that both
models were able to closely predict the outcomes seen in an actual cohort at
5 years but that the DES model was slightly better than the current Markov
What is great
also is that this study represents the first time
I know of that
a model developed specifically for HIV disease has been actually
validated against real cohort data.
You also looked at 1-year data. Why did you look at 1-year data if you are
interested in long-term outcomes and costs?
Simpson: One-year data from the actual study cohort was used
to calibrate the models. The models were adjusted at 1 year to make sure
that the variables inputted into the model make the same predictions
compared to the 1-year data reported from the actual clinical trial. After
this point, the models were not adjusted further.
You put the baseline CD4 count and viral load into the models. How important
are these in terms of predicting outcome and what are other key
characteristics that you added into your model?
count and viral load surrogate markers are the main drivers to predict
outcomes. The CD4 cell count clearly determines the risk of an AIDS-defining
event and hospital admission. The viral load clearly determines what happens
to the CD4 count, the magnitude of change and how fast it changes. My
objective with modeling is to evaluate new antiretroviral treatments. The
model is a piece of software, and if it
works, then a regimen’s 24-
or 48-week clinical trial data put into the model will predict lifetime
outcomes. The whole point here is to take these two very different pieces of
software and compare them to what happened in the actual cohort. We found
that both models closely predicted what happened in the actual cohort over
The problem with the Markov model is that it is categorical and divides a
study population into 12 categories, which are 12 different health states
defined by a set range for CD4 cell count and viral load. So, for example,
the Markov model can look at 100 patients who receive Drug A and another 100
who received Drug B, and at baseline, all of the patients are separated out
into one of 12 “boxes.” They begin treatment and are told to come back in 3
months for a clinic visit. At each 3-month clinic visit, their CD4 counts
and viral loads are determined, and they are reassigned back into the same
category or into a different one depending which of the 12 boxes a change,
if any, in their CD count and viral load places them since the last clinic
visit. Although the Markov model is able to stratify the patients over a
specified treatment duration, it uses 12 mean values to represent what
happens to the entire patient population at each 3-month interval. However,
with the DES model, you can input into the model a wide distribution of CD4
counts and HIV RNA levels so that the model can construct a thousand
different patients with different CD4 counts and viral loads that change
over time, which of course is much more like that of an actual study
population. The DES model is much more fine-grained—like real life—than the
Markov model, and the time points do not have to be every 3 months. In this
comparison, I used 8 weeks as the first time period.
Why was 8 weeks an important time point?
Simpson: Because in the clinical trial that I wanted to test
here, that was the time point where subjects could crossover to the other
treatment if they were not responding to their originally assigned
treatment. In most other cases, I would set the first analysis at 12 weeks.
If you think about what happens in the first 8 to 12 weeks after starting
antiretroviral treatment, there is an abrupt decrease in viral load and then
a slow rise in the CD4 count. And the DES model can capture these data,
while the Markov model, which just provides average values of these
variables every 3 months, cannot. In other words, the DES model is much more
sophisticated in that it allows one to see what is happening to every
patient everyday based on the actual distributions of patients in the
How do you determine the distribution of the patients over time in these
I analyze the clinical trial data with a statistical software program that
identifies the distribution of the
data based on actual trial data distributions.
In fact, data from several different cohorts are used to
specify how the data are distributed in
the DES model. Over the past
19 years, I have analyzed clinical trial data from thousands of patients. I
also use published cohort data from the Medical University of South
Okay, so what is the purpose of the Markov and DES models? Why are models
Simpson: The model helps us get an understanding of what the
statistical results reported for clinical trials actually mean in numbers
that we care about. Most of us have no real understanding of what an 8% or
10% or 12% or 5% difference in virologic suppression at 48 weeks actually
means in terms of the average length of life of the patient, the average
time that they stay on their first antiretroviral therapy, and the cost of
their care over the next 5 years. The model can predict the outcomes that
are meaningful for planning purposes.
So the Markov and DES models are ultimately used to get predictions of how
long a specific patient might stay on treatment with regard to reaching a
clinical end point. The DES model seems also to be able to account for
different levels of patient adherence to treatment.
Simpson: That’s correct. In terms of reaching a clinical end
point, an adherence variable can be added, for example, telling the model
that there is a 20% probability that subjects will not adhere resulting in
Did you include quality-of-life measures in your model?
Simpson: I do look at quality of life. As you know, quality
of life is not a very good measurement because it is subjective, but there
are assumptions that can be made, with some clinical pictures where we can
all agree on, such as the person who has a CD4 cell count of 900 and has
never had an AIDS-defining condition. This person probably has a better
quality of life than one who has a CD4 cell count of 50 and has CMV disease.
How do you adjust for quality of life in your model?
Simpson: When I developed the DES model in 2006, I obtained
all the clinical trial data related to quality-of-life measurements that
Abbott Laboratories had on file from past clinical trials. I analyzed about
21,000 observations in order to give the model the capability to include a
quality of life measurement for individual patient based on CD4 cell count
and viral load, so this is a very strong data set.
One of your conclusions is that the DES model is better than the Markov
model at five years.
Simpson: Yes, not only was the DES model slightly better but
also the Markov model could not predict the CD4 cell changes and the DES
model could. For viral load, the models are looking at viral suppression
below a specific undetectable level that was clearly defined. But for CD4
changes, the Markov model is not good because it groups patients into
categories that are too big. So, the Markov model is limited because it
could not predict certain important variables at different time points.
This comparison of the Markov and
the DES model presented at ICAAC is not only about the DES model being
better. It is also exciting because it compares two modeling estimates to
what actually happened to patients, and thus validates the model
predictions. This type of validation is quite rare in modeling. I have been
doing modeling since 1988, and this is the first time that I have been able
to validate a model’s predictions
against real long-term data.
So what is really exciting is that the DES model was able to predict very
accurately what happens over time in a real patient cohort
Let’s talk about how models might help policy makers make the best decisions
regarding the use of limited resources to fund such government programs as
ADAP. These models can be used in cost-effectiveness studies to show
comparisons in the cost of different antiretroviral drug regimens over time,
Simpson: Yes. The Markov model is generally used to predict
total cost and outcomes and budget impact at 5 and 10 years and lifetime
cost effectiveness in sort of a coarse manner. The DES model can make the
same predictions, but can provide more information and appears to do so more
accurately than the Markov model. For example, as I mentioned earlier, a
range of expected adherence rates can be put into the DES model to see how
that impacts the cost-effectiveness of treatment. If initial treatment fails
in a patient because of poor adherence, subsequent regimens will be more
expensive for several reasons, including the need to use more expensive
regimens because of drug resistance. However, both models can provide the
budget impact for a fixed cohort over 5 and 10 years and examine the
cost-effectiveness. And this is very important for understanding the impact
of newer drugs.
I do with models is to make comparisons between two drug regimens. For
example, I can take any new drug, such as maraviroc, and use the model to
compare it to tipranavir. If I can make sure that the results from the
clinical trials are from similar study populations, then I can use the data
from each trial and put that in the model with the cost of each drug to make
Are you able to compare different patient subgroups?
Simpson: Yes, I can do the modeling based on data from
different subgroups if the clinical trials are designed to examine data from
special patient groups. In clinical trials like the maraviroc and tipranavir
trials where the results were reported for different suppression levels for
different subgroups, the model can predict cost and outcome differences for
a particular subgroup.
Who are the decision makers that generally use these types of
Simpson: There are two sets of decision makers. The studies
are used in Europe, Canada, and Australia for funding decisions related to
drug approval because their regulatory bodies, such as the European National
Institute for Clinical Excellence and the Australian Pharmaceutical Benefits
Assessment Commission, require data on a product’s cost-effectiveness.
Basically, in those countries the government cannot pay for a drug without
this kind of analysis performed by the manufacturer. The second set of
decision makers is US payers and policy organizations, such as ADAP.
Do these international agencies use the Markov model?
The Markov model
used to inform reimbursement decisions by governments in 13 countries. In
the United States, the HMOs and Medicaid prefer to see the 5- and 10-year
NICE agency has used
the Markov model for approving the price of the drug at launch. It is not
used for drug approval. In other words, you can
buy the drug in a country,
but the government will not pay for the drug if it is not cost-effective.
One of the benefits of running these models is if a pharmaceutical company
is charging too much for their drug, the cost-effectiveness ratio would look
very high compared to the drug’s benefit. As a matter of fact, recently an
approval for drug cost reimbursement was delayed in the United Kingdom
because a model showed that the price of the drug was
too high for the extra benefit provided
compared to other drugs that available.
None of the agencies are presently using the DES model?
Simpson: Not yet. It is brand new.
Do you think there will be resistance by decision makers to adopt this new
model even though it appears to be more accurate?
Simpson: No, I think they would not mind a more accurate
model. The only problem is that understanding these models has not been easy
for government regulators. Further, all of the models to date for evaluating
antiretrovirals have been Markov models; so, the agencies are familiar with
data from Markov models, and they will not be quite sure how the DES model
works, and thus how accurate the model’s predictions are, until they get
used to this new model.
How does the impact of the virologic and immunologic responses to
initial therapy affect a drug’s cost-effectiveness?
Simpson: With first-line therapy, there are two factors.
Using the intent-to treat data, the first factor is how many subjects
dropout during the first 12 weeks of treatment. All the clinical trials
report a huge dropout each week during this initial period, sometimes as
much as 30%, and these patients are counted as failures. So no drug is going
to look all that good if a whole lot of people stop using it because of side
effects or because they do not want to stay in the trial for other reasons,
and this early analysis is very important. After that, the next factor—and
real critical determinant—is, how powerful is the drug and how sustained is
its effect. These are the main factors in a cost-effectiveness model.
What about second-line therapy?
As we know, second-line therapy is critical in terms of its ability to
overcome the development of drug resistance by the virus to previous
therapies. And it is easy to grasp that a drug whose efficacy is not
diminished by the presence of drug-resistant viral mutants is much more
cost-effective than one that cannot overcome drug resistance. The costs in
this case are the increased costs related to hospitalization and the
progression to salvage therapy. Once the patient needs to be treated with an
increased number of drugs or with
like enfuvirtide at a multidrug
regimen cost of $100
per day, it gets very expensive. Modeling can predict these costs over time.
Again, the major variables for a cost-effectiveness model to make such
predictions are the baseline CD4 count and HIV RNA level and their change
Could cost-effectiveness studies of antiretroviral therapies help state ADAP
administrators better meet the needs of people enrolled in state ADAP
could, especially if the study’s model results are used to predict the time
point at which
to switch to a different drug regimen given the patient’s degree of
adherence to treatment, for example. That is what really matters in the
clinic. The DES model can identify the people who do not adhere to their
antiretroviral regimen and are going to be burning up their drug options
much quicker. And the model can show the budget impact of this because the
drug costs and other costs, such as hospital admissions related to
AIDS-defining illnesses, are put into the model.
So, let’s assume that a new antiretroviral has just been approved by the FDA
and is very expensive. How is the decision-maker, for example, an ADAP
administrator with a limited budget, going to make the most cost-effective
decision on whether to include this new drug on the formulary?
This is exactly what these models
can show. In this case, the model very often shows an increase in costs in
the first year or two but then shows cost savings with sustained treatment.
The Markov model has actually shown that. And one of the papers that I have
published on the Markov model has shown exactly what those cost savings
could be (Simpson KN, Jones
WJ, Rajagopalan R, Dietz B.
Cost-effectiveness of lopinavir/ritonavir compared to atazanavir plus
ritonavir in antiretroviral-experienced patients in the U.S.
Clin. Drug Invest. 2007;27:807-817).
Simpson: This is where an administrator, for example, can
see that he or she will be spending more during the first 18 months but
after that, there will be cost savings and at the end of 5 years, it may be
as much as $18,000 per patient. The models will predict this. Valid
prediction of cost and outcomes is important for showing the magnitude of
the benefits of ADAP funding, as well as for showing that ADAP is a
cost-effective policy solution.
Is this type of modeling data now being looked by state agencies with regard
I used it once while I was at the University in North Carolina at Chapel
Hill when I was in the School of Public Health there. Dr. Josephine Mauskopf
and I were asked to advise the State’s Medicaid Administrators on which 1 of
5 new antiretroviral agents they should put on their formulary. After
putting their data into an older Markov model, we told them that they should
put all 5 new antiretrovirals on their formulary, and that they should also
pay for prevention of opportunistic infections.
back and said that it could not afford to do that. We told them our modeling
data showed that if they put all of their patients on regimens that included
at least 1 of the 5 new antiretrovirals, the annual cost savings would be
$100,000 plus or minus $17,000 in hospital admissions for AIDS-related
events, even though the drug costs were higher. And that old model actually
was the one that ADAP started using for predicting outcome
or ADAP tells you that they do not have the budget to put all 5 new
drugs on formulary. Give us some advice on which ones to use. How would you
differentiate among new drugs besides the drug cost?
Simpson: Viral load suppression. This includes time to viral
suppression as well as time to any subsequent virologic breakthrough. And
because different drugs have different levels of power in treatment-naïve
and treatment-experienced patients, we look at viral load suppression and
drug cost in both populations. The DES model could estimate the expected
cost and outcomes for the actual ADAP population distribution.
I assume you use the 48-week data from the drug’s registrational trials in
Simpson: Yes, I do, but I also “scrounge up” all the data I
can from anywhere and put this information into the model or use it for
validating the model’s predictions.
With data used from different studies, how do you account for differences
across the different studies? For example, differences in their study
populations and study methodologies?
Simpson: I gather all the data, but only data where the
populations are similar. I can’t use data from some studies. For example,
some comparative trial results show that at 96 weeks, one drug performs
equally as well as the other. But if 40% of the patients in one treatment
group dropped out by 24 weeks, the data are not really worth anything for
modeling. With the models I use, I do not have to use data beyond 24 weeks,
so I can take 24-week or 48-week data in the model to predict outcomes after
5 or 10 years of treatment under a reasonable set of assumptions.
Also, if a
trial’s methodology has what I call “informative dropout,” that is, if the
trial was designed in a such way that patients knew what drug regimen they
were receiving and could crossover to a rescue arm at a predetermined time
point, such as after 24 weeks, then I do not use this data because all of
the seriously ill patients have likely left the control group. The patients
who are left in the control group are going to look good, but they are only
the subjects who responded to the treatment. One advantage of using a model
is that you can examine the differences that would be expected at, say, 96
weeks with those actually published after 96 weeks of follow up. Thus,
modeling can give us a good idea of the bias from informative dropout,
something you often cannot do with the standard statistical analysis of
clinical trial data.
What kinds of published studies would you recommend for state ADAP directors
They should look at studies that evaluated overall cost-effectiveness and 5-
and 10-year budget impact, and the study should have results for all three
of these estimates because the overall cost-effectiveness shows whether the
treatment is a good investment for lifetime and the budget impact shows
whether you can afford it. There are published cost-effectiveness studies
for all types of drugs, including antiretrovirals, and these can be found
across all clinical journals. In states where budgets are inadequate and
waiting lists are long, they [GORDON: Again, who are they here?] may also
want to read some of the modeling studies that examine criteria that could
be considered for minimizing the health effects of this terrible problem. (Sax
PE, Losina E, Weinstein MC, et al. Cost-effectiveness of enfuvirtide in
treatment-experienced patients with advanced HIV disease.
J Acquir Immune Defic Syndr. 2005;39: 69–77; and
Linas BP. Zheng H. Losina E. et al.
resource allocation in United States AIDS
drug assistance programs. Clin Infect Dis. 2006;43:1357-1364.)
Actually, if third-party
payers would compare the results of HIV cost-effectiveness studies to those
evaluating drugs in other areas of medicine, they would see that the HIV
interventions that we have are often a much “better buy” in terms of
quality-adjusted life-years, than some heart disease medications and
definitely a better buy than many cancer treatments. With HIV medications,
you get a much bigger effect for your money because, if you suppress an
HIV-infected person’s viral load, then that person will likely live for long
time and have a very healthy life.
Given that it is becoming an issue of multidrug regimen costs—not
single-drug costs—how well does the DES model lend itself to not just
comparing one drug to another but comparing different regimens in terms of
costs and health outcomes?
The DES model is well suited to do
this, and it is specifically constructed to look at a key objective that has
to do with new drugs and what kind of resistance they generate. The DES
model is the type of model that
can take resistance data and
translate it into
clinical and economic outcomes.
So, the DES model can show not only the effect you get from the first drug
regimen, based on resistance
patterns, but it can also
provide information about how much damage there will be to the success of
subsequent drug regimens. The limitations for the DES model are not really
related to the model; its limits are mainly due to lack of data on what
actually happens to a subgroup of patients who are treated with specific
sequences of drugs. None of the models are magically able to predict
outcomes if no data are available to help with the prediction.
Let me understand
this better. Let’s say you are evaluating a new drug, such as the CCR5
coreceptor blocker maraviroc, and it is going to be combined with a protease
inhibitor plus two NRTIs. Also some patients will also get enfuvirtide and
some will not. So, with this rather unique regimen for which you cannot grab
a lot of clinical data, how do you use a model to make cost predictions and
comparisons to more common salvage regimens?
The problem with models like
this is that they have to be run on data; in other words, there has to be
some data available somewhere. If clinicians can’t point me to data on the
individual drugs, then the data requirements for the model cannot be met.
However, in recent clinical trial reports, a drug’s effects are reported
separately for each patient
pre-treatment subgroup in a very responsible and much more informative
manner than even 2 years ago. These types of reports often provide excellent
data for our models.
How are they reported differently now?
trials now routinely report the
actual responses rates for subgroups of patients not only by baseline CD4
cell count but also for the magnitude of viral load, such as above and below
100,000 copies/mL at study entry and,
resistance score based on the number
of active drugs in the regimen. Once you have this information, then you can
construct individual patient groups and put each individual
data in the model and run the model
for the combined subgroups.
You can do this as long as the clinical trial results are reported for valid
So you get 5-year and 10-year estimates of the cost and outcome of treatment
and compare these predictions to the cost and outcome of a different
Simpson: Yes, there are always two comparisons in the model
because you want to make a decision of which drug is the most
cost-effective. Doing modeling like this is like a puzzle. Let’s say there
are 20 different papers on atazanavir and 40 papers published on Kaletra.
All of these reports are searched to find the most valid pieces of
information to populate the model, and this is done for two drugs that have
not necessarily been compared head-to-head in a clinical trial. For example,
one drug might be compared to nelfinavir and the other drug might also have
been compared to nelfinavir in a different study. In this case, the model
would be adjusted for the response to nelfinavir. This is called an indirect
Can the model be used to compare two regimens that have a different number
of drugs, such as a 3-drug regimen versus a 5-drug regimen?
5-drug regimen has 5 drug costs and will have a certain percentage of
patients with viral load suppression below 400 copies/mL and a certain
percentage below 50. So the data for viral load below 400 and below 50 from
one arm are plugged into the model along with the cost of the 5 drugs, and
the corresponding data and drug costs from the 3-drug arm are also plugged
in, and the model is run. But it’s not possible to then remove or add a drug
and then run the model to see the effect. You could not even get this
information in an actual clinical trail because you have to look at each
individual patient’s resistance score and see which drugs they are sensitive
to. One of the things that these
models can do is actually show
clinicians what would be expected to happen clinically and costwise if they
were to stop treatment with an ineffective multidrug regimen that contains 5
or 6 or 7 drugs and instead use a simple regimen for 6 months (until one or
more new drugs become available).
In this case, such patients might be better off taking a very simple regimen
for 6 months that just keeps the virus suppressed in the CNS, until a new
drug becomes available. So the
modeling results may help inform clinicians’ discussion of which policies to
choose, although, individual treatment choices should be decided by the
patient and the doctor together.
Are there any studies on medical cost savings resulting in improved outcomes
that show early HIV treatment intervention is more cost-effective than later
Simpson: There is one study that was done by a group from
Boston University and Harvard. Ken Freedberg and Milt Weinstein were the
ones involved with that (Schackman
Weinstein MC. et al.
Cost-effectiveness implications of
the timing of antiretroviral therapy in HIV-infected adults..
Arch Intern Med.
They actually ran a model similar to the models I use, looking at starting
early with antiretroviral therapy versus starting late, and they found that
it was quite cost-effective to start early. My DES model can do the same
analysis; I just have not done it yet.
Thank you, Dr Simpson, for your time today and for continuing this important
research. We at the ADAP Fund are looking forward to your work on cost
models being adopted by state ADAP administrators and possibly bringing a
greater benefit to those persons with HIV/AIDS who need ADAP to obtain