Accurate model predicting sustained response at week 4 of therapy with pegylated

[Guest guest]

Accurate model predicting sustained response at week 4 of therapy

with pegylated interferon with ribavirin in patients with chronic

hepatitis C

Journal of Viral Hepatitis

Volume 13 Issue 10 Page 701 - October 2006

M. ot-Peignoux1, L. Comanor2, J. M. Minor3, M. P. Ripault1, B.-

N. Pham4, N. Boyer1, C. Castelnau1, N. Giuily1, D. Hendricks5 and

Marcellin1

“…., this study performed in routine patients showed that this

multivariate model allows early prediction of SVR and NR at baseline

as well as week 4 of therapy on naïve as well as in previously

treated patients…. we describe multivariate models that use the

commonly collected data [i.e. sex, age, genotype, serum alanine

aminotransferase (ALT) and HCV RNA levels, histologic

necroinflammation and fibrosis scores and whether previously treated

or naïve] to predict response to peginterferon plus ribavirin

therapy for patients in routine clinical practice….. The week 4

multivariate model included 26 terms, of which 3 were linear and 23

were combinatorial. The linear terms were the log of the viral load

decline by week 4, log of the baseline viral load and sex, which

ranked 1st, 3rd, and 20th, respectively, in relative importance to

the week 4 model….â€

Summary. Current models used to predict response to peginterferon

plus ribavirin treatment, based on viral decline during the first 12

weeks of therapy, have focused on creating an early stopping rule to

avoid unnecessary prolongation of therapy.

We developed a multivariate model that predicted sustained

virological response and nonresponse at baseline and during the first

12 weeks of therapy using collected data from 186 unselected patients

with chronic hepatitis C treated with peginterferon plus ribavirin.

This model employed ordinal regression with similarity least squares

technology to assign the probability of a given outcome.

Model variables include sex, age, prior treatment status, genotype,

baseline serum alanine aminotransferase levels, histologic

necroinflammation and fibrosis scores and serum hepatitis C virus RNA

concentration at baseline and weeks, 4, 8, and 12.

A multivariate model demonstrated high performance values at all time

points. At baseline, the model demonstrated a negative predictive

value (NPV) and a positive predictive value (PPV) of 91% and 95%,

respectively. At week 4, these values improved to 97% and 100%,

respectively, with 95% sensitivity, 89% specificity and 93% accuracy.

At week 4, the model was equally efficient for naïve or previously

treated patients. Internal validation demonstrated 90% PPV, 94% NPV,

95% sensitivity, 88% specificity and 92% accuracy.

A week 4 stopping rule for patients with chronic hepatitis C treated

with peginterferon with ribavirin might be proposed by using the

model developed in our study.

Introduction

Recently, there has been great emphasis on developing models for

early prediction of therapeutic outcome for patients with chronic

hepatitis C [1,2]. Although new formulations of peginterferon plus

ribavirin treatment have increased the rate of sustained virological

response (SVR) to 45-59% [2-4], moderate-to-severe side effects are

common and the cost of treatment is still considerable. Ideally

clinicians would like to assess the likelihood of SVR before

treatment or as early as possible during its course, not only to

discontinue treatment in those cases in which viral clearance likely

is not going to occur but also to encourage compliance in patients

likely to respond.

Currently, the most widely accepted models for the prediction of

outcome to peginterferon plus ribavirin therapy were developed from

data collected from two international registration trials and are

based on early virological response. The definition of early

virological response is a decline in serum hepatitis C virus (HCV)

RNA greater than or equal to 2-log10 units or clearance of virus by a

sensitive qualitative assay by week 12 of therapy. Univariate models

based on this definition provide the basis for early stopping rules

for peginterferon plus ribavirin therapy at week 12 [3,5]. These

univariate models maximize the negative predictive value (NPV) in

order to minimize loss of potential responders, as opposed to

optimizing the positive predictive value (PPV) that would yield a

higher rate of SVR among those continuing treatment but would exclude

some patients who might ultimately respond. Recently, investigators

have used multiple viral load measurements during the first month of

therapy in an attempt to create a week 4 stopping rule [6]. However,

as these univariate models are based only on virological response,

they provide good NPV but poorer PPV, and they have low specificity

and accuracy. In addition, they are unable to make baseline

predictions.

We conducted a prospective community-based study to evaluate the

relationship between viral kinetics and outcome to peginterferon plus

ribavirin therapy. During the course of this study, we developed

multivariate models to predict treatment outcome. Here, we describe

multivariate models that use the commonly collected data [i.e. sex,

age, genotype, serum alanine aminotransferase (ALT) and HCV RNA

levels, histologic necroinflammation and fibrosis scores and whether

previously treated or naïve] to predict response to peginterferon

plus ribavirin therapy for patients in routine clinical practice. We

also evaluated the efficacy of these multivariate models by internal

model validation.

Methods

Selection of patients

One hundred and eighty-six patients with chronic HCV infection (97

naïve and 89 previously treated) treated with peginterferon plus

ribavirin were prospectively included in this study. Of these, 73

(39%) received a reduced dosage of one or both drugs but finished the

treatment course; they were included in the study, as dose reduction

is common in clinical practice. All patients had elevated serum ALT

levels, detectable anti-HCV antibodies and HCV RNA in serum at

baseline and a liver biopsy performed within 12 months prior to the

initiation of peginterferon plus ribavirin treatment. Patients were

included for the purpose of model development only if they had

completed a full course of therapy and had blood sampled in our

centre for viral measurements at baseline and two or more time points

during therapy (week 4, 8 and/or 12) at the end of treatment (EOT)

and at the end of week 24 follow-up. The clinical characteristics of

these patients are shown in Table 1. Of the 89 previously treated

patients, 41 had received standard interferon monotherapy (3 MU, 3

times per week for 6 months), 3 peginterferon monotherapy (1.5

μg/kg/week) and 45 interferon and ribavirin (3 MU, 3 times per week

for 6 months plus ribavirin 1000-1200 mg/day according to weight) as

prior therapy. Sixty-three patients were previous nonresponders and

26 were previous relapsers to these prior therapies.

Treatment regimens and definition of response

Patients were treated with peginterferon α-2b (PEG-Intron; Schering

Plough Research Institute, Kenilworth, NJ, USA) at a dosage of 1.5

μg/kg body weight per week and ribavirin (REBETOL; Schering Plough

Research Institute) at a dosage of 800-1200 mg/day depending on body

weight, according to the consensus guidelines [7,8]. Naïve patients

infected with genotypes 1, 4 or 5 and all previously treated patients

were treated for 48 weeks; patients infected with genotypes 2 and 3

were treated for 24 weeks.

SVR was defined by undetectable serum HCV RNA after 24 weeks of

follow-up. NR was defined by detectable serum HCV RNA at the EOT. RR

was defined by undetectable serum HCV RNA at the EOT and by viral

breakthrough during follow-up.

Virological, biochemical and histological evaluation

Serum HCV RNA was quantified by the VERSANT® HCV RNA 3.0 (bDNA)

Assay (Bayer Healthcare LLC, Tarrytown, NY, USA) with a

quantification range of 615-7 690 000 IU/mL [9]. Serum samples below

615 IU/mL were evaluated with the VERSANT® HCV RNA Qualitative Assay

(HCV Qual TMA, Bayer Healthcare LLC) with a limit of detection of â‰

¤9.6 IU/mL [10]. For modelling purposes, samples with viral

measurements below 615 IU/mL but with detectable HCV RNA by HCV Qual

TMA were assigned a value of 385 IU/mL (2000 copies/mL), whereas

samples with undetectable HCV RNA by both assays were assigned a

value of 19 IU/mL (100 copies/mL). HCV genotypes were identified

using the VERSANT® HCV LiPA (Bayer Healthcare LLC) [11]. Serum HCV

RNA and ALT levels were determined at baseline, weeks 4, 8, 12, EOT

and follow-up week 24. Liver biopsies were graded using the METAVIR

scoring system [12].

Multivariate model development

A statistical data analysis network known as similarity least squares

(SMILES or SLSTM) [13,14] was used to develop multivariate models for

predicting likelihood of response to peginterferon plus ribavirin

therapy. These multivariate models were designed to distinguish 'two

outcomes'; SVR and nonsustained response at baseline and at weeks 4,

8 and 12. Whereas the baseline multivariate model used only variables

known at baseline, multivariate models for weeks 4, 8 and 12 used all

baseline variables plus viral load values at those time points.

Continuous variables, such as viral load and serum ALT, were used

directly. Nominal variables (i.e. those without inherent continuous

properties), such as genotype and sex, were converted to continuous

variables using ordinal logistic analysis to correlate nominal levels

to probability of outcome (JMP 5.0.1.2; SAS Institute, Cary, NC,

USA). Continuous variables were standardized to Z scores and entered

in the models as linear terms. As is the standard statistical

practice, all data were used for the univariate normalizations and

nominal conversions.

Then SMILES was used to create combinatorial terms that incorporate

combinations of linear terms into the model. A demonstration of the

model is available on the web site http://www.slsguy.com. A subset of

linear and combinatorial terms most important for prediction of

response (essential terms) was selected using best subset selection

techniques. In addition, a subset of patients necessary to produce a

complete model (critical patients) was identified using methods

similar to the principles of computational design of experiments

(http://www.echip.com). With SMILES technology, the number of

essential terms and critical patients required for each multivariate

model varies according to the amount of information available to the

model as well as the amount of predictive information required from

the model. For example, baseline models require more essential terms

and critical patients than models with additional information

obtained during therapy (e.g. viral kinetic data).

Finally, ordinal logistic regression was used to determine the best

linear combination of essential terms to predict the response of

critical patients and optimized to allow for approximately five

degrees of freedom from error.

where V is the SMILES-derived metric used to predict the outcome for

each patient where the collection {C} is a subset of clinical

variables, Sk is the similarity between the patient profile and the

profile of feature k where similarity can be any viable function of

profile proximity and the Greek symbol (μ,α,β and ) parameters to

be

estimated or optimized from the training data composed of critical

patients. V values of <-0.05 imply sustained response, values >0.1

imply NR and in-between values suggest that the patient is

unpredictable, that is, he/she is on the defining boundary of

interactions that divide sustained responders from nonresponders.

Assessment of model performance

The accuracy of the model was assessed by calculating the

sensitivity, specificity, PPV and NPV at each time point. Sensitivity

was calculated as the percentage of SVRs identified by the model

[(number of correctly predicted SVRs/number of true SVRS) × 100].

Specificity was calculated as the percentage of NRs identified by the

model [(number of correctly predicted NRs/number of true NRs) ×

100]. PPV was calculated as the percentage of SVRs predicted by the

model who are true SVRs [(number of correctly predicted SVRs/total

number of predicted SVRS) × 100]. NPV was calculated as the

percentage of NRs predicted by the model who are true NRs [(number of

correctly predicted NRs ÷ total number of predicted NRs) × 100].

Model validation

The SLS system performs its own validation since it trains on 20% of

the patients and subsequently predicts 80%. As an additional test of

robustness internal model validation was performed by creating a

multivariate model from a training set comprised of a random

selection of 80% of the patients, and then testing the performance of

this model in predicting the response of the remaining 20% of

patients. Week 4 was chosen as the time point for internal model

validation to assess the feasibility of creating a week 4 stopping

rule.

Results

Patient response to treatment

Baseline characteristics and response to treatment for the 186

patients are shown in Table 1. The overall sustained response rate

was 55%. The 48% sustained response rate among naïve patients was

not statistically different from the 62% rate observed in previously

treated patients. As expected, previously treated patients who did

not respond to prior treatment had a lower rate of SVR than those who

relapsed after prior treatment. Of the 63 patients who did not

respond to prior treatment, 39 did not respond and 24 achieved SVR

when subsequently treated with peginterferon plus ribavirin. Of the

26 patients who relapsed after prior treatment, 5 relapsed again, 2

did not respond and 19 achieved SVR after treatment with

peginterferon plus ribavirin. In addition, a greater SVR rate to

peginterferon plus ribavirin therapy was observed among patients who

previously had received standard interferon monotherapy (54%) as

compared with those previously treated with standard interferon plus

ribavirin (40%). All three patients who received previous treatment

with peginterferon monotherapy achieved SVR upon subsequent treatment

with peginterferon plus ribavirin.

Performance of the models

Model performances are shown in Table 2. Even with the limited

information available at baseline, the model demonstrated, 94% PPV,

91% NPV, 95% sensitivity, 89% specificity and 93% accuracy. At all

time points during therapy, multivariate models showed consistently

high sensitivity (98-100%), specificity (100%) and accuracy (99-

100%). The performances of the multivariate model for naïve patients

vs previously treated patients at baseline and week 4, the only time

points at which the performance of the multivariate models was less

than 100%, are shown in Table 3.

Relative contribution of clinical variables

To understand the relative contribution of the clinical variables in

the multivariate models, logistic regression analysis was performed

to rank the impact of individual clinical variables that are explicit

in the model (i.e. linear terms) and the impact of combinatorial

effects among all the clinical variables (i.e. combinatorial terms).

The baseline multivariate model included 48 terms, of which 3 were

linear and 45 were combinatorial. The linear terms were HCV genotype,

log of the baseline viral load, and fibrosis score, which ranked 5th,

16th and 28th, respectively, in relative importance to the baseline

model. The week 4 multivariate model included 26 terms, of which 3

were linear and 23 were combinatorial. The linear terms were the log

of the viral load decline by week 4, log of the baseline viral load

and sex, which ranked 1st, 3rd, and 20th, respectively, in relative

importance to the week 4 model.

Performance of validation model

The week 4 model validation, performed on 36 patients, demonstrated a

NPV of 94% (CI: 88-100%), a PPV of 90% (CI: 81-99%) with 95% (CI: 88-

100%) sensitivity, 88% (CI: 77-99%) specificity and 92% (CI: 86-98%)

accuracy. The overall high performances of the model (88-95%) upon

validation demonstrate its robustness. A retrospective analysis of

the 36 patients used to test the multivariate model revealed that the

model correctly predicted the response of 3 of 7 critical patients

and all 29 noncritical patients. In addition, the model correctly

predicted the response of 20 of 21 naïve patients and 13 of 15

previously treated patients. Of 36 patients used to test the

multivariate model, 12 had received reduced dosages of peginterferon

and/or ribavirin and the model correctly predicted 11 of these 12

patients.

Discussion

Previously developed univariate models for prediction of response to

peginterferon plus ribavirin were based on retrospective studies

using data from two international registration trials of that therapy

[3,4]. These registration trials admitted only naïve patients who

were capable of receiving an optimal treatment regimen. By contrast,

our multivariate model was based on data from a prospective community-

based study that included nearly equal numbers of naïve and

previously treated patients. The univariate models developed using

data from the registration trials were optimized for high NPV and

thus are well suited for identifying patients not likely to respond

to therapy [3,4]. However, these models have poor specificity and

therefore cannot be used with certainty to predict SVR.

Our multivariate model based on SMILES technology was optimized for

accuracy instead of NPV. At baseline, our multivariate model had

excellent values for all five performance attributes (89-95%),

allowing prediction of response using data available prior to

starting therapy. At week 4 of therapy, our model showed performance

values ranging from 97 to 100%. The robustness of this model is

indicated by its high performance values upon internal validation

ranging from 88 to 95%. Unlike univariate models, the predictive

ability of our multivariate models can be further improved by the

addition of more patient data.

The SMILES process allows us to model effectively a heterogenous

population using a data set based on 186 patients, whereas a

conventional logistic regression modelling approach for this

population would require more data. Our multivariate models use

patient profiles instead of individually weighted variables to

predict outcome, allowing relatively few critical patients to predict

the response of the remaining patients. The efficiency of the week 4

multivariate model is demonstrated by its ability to optimise all 26

model parameters based on a minimum of 30 patient profiles selected

by design of experiment techniques. Incorporation of these designs of

experiment concepts along with the ability to model combinatorial

effects make this efficiency possible. The limitation of building our

multivariate model with only 186 patients lies in not knowing if all

the important interactions have been accurately captured from the

available patient data.

Another advantage of this kind of multivariate model is its ability

to deal in part with missing data. Incorrect data is the worst kind

of missing data and the SMILES technology is useful in identifying

data errors during data preprocessing. In addition, in this type of

modelling, imputation can be used to calculate insertions for

specific values if the percentage of actual missing data is small.

Although these insertions do not create the missing information, they

do enable the effective use of the known data in a profile. The

impact of missing data depends on the importance to the model of

missing variables. Absence of a critical variable such as viral load

will make a large difference, whereas absence of a minor one will

have less impact. Nevertheless, even a minor variable can assume

importance in combination with another variable.

The variables used in our study have been shown to be independent

predictors of response in previous studies, [1,5,6,15-20]. They

reflect both host and viral factors such as genotype, age, sex, serum

ALT, fibrosis score and viral decline. Several of these variables

proved to be important as linear terms in our model. Because our

multivariate model also included prior treatment status as a

variable, it can be applied to both naïve and previously treated

patients and, in fact, performed well in predicting the response of

both patient groups.

Unlike univariate models, the predictive capability of our

multivariate model relies heavily on combinatorial terms. The

potential utility of combinatorial terms in prognostic models has

been suggested in an earlier study of patients treated with

interferon-based therapies that found a correlation between the

number of predictive factors and the response rate [21]. The more

positive predictive factors a patient had, the more likely he/she was

to achieve SVR. Conversely, response was synergistically impaired

when two or more negative predictive factors were present. While

these results indicate that interactions between two or more clinical

variables are important in predicting response, it is not practical

for the physician to weigh and evaluate the large number of possible

interactions between clinical variables in predicting response for a

given patient. The use of patient profiles instead of individually

weighted variables makes it possible for SMILES to correctly predict

difficult patients. For example, four of our correctly predicted

sustained responders were nonresponders to previous treatment,

infected with genotype 1, had above average baseline viral load and

less than a 2 log decline in viral load by week 4. However, the model

correctly captured the interactions of other variables such as

fibrosis and inflammatory scores and ALT, which enabled these

patients to respond.

Finally, the design of the multivariate model gives it inherent

flexibility that allows incorporation of new information, such as

genomic data, treatment compliance, physician monitored dose

modifications, body mass index, etc. Flexibility is an important

advantage for prognostic models, as evaluation, therapy and

monitoring of chronic hepatitis C patients is continuously evolving.

Although the presented models are specific to the data they were

trained on, with minor modifications they may be applied to slightly

different data. For example, the model could be tested on a new set

of patients taking a different regimen of interferon and an

extrapolation could be developed.

In conclusion, this study performed in routine patients showed that

this multivariate model allows early prediction of SVR and NR at

baseline as well as week 4 of therapy on naïve as well as in

previously treated patients. Prediction of SVR may provide patients

with an early incentive for treatment compliance, a critically

important factor during the first week of therapy. In this way, use

of the model could lead to increased sustained response. Prediction

of NR may allow discontinuation of therapy with confidence or may

suggest a modification of the current regimen.