| Abstract |
Introduction
Systemic lupus erythematosus (SLE) is a chronic autoimmune disease that mainly affects women of reproductive age. It is characterized by great clinical heterogeneity as it can affect almost any organ with varying degrees of severity. The disease has alternating periods of remission and relapses and can lead to irreversible organ damage as a result of chronic inflammation, corticosteroid use and comorbidities. Due to the wide range of disease manifestations, early diagnosis and consequently early treatment, poses a significant challenge, which is further complicated by the absence of diagnostic criteria. Instead, classification criteria have been developed, which are based on a constellation of clinical and laboratory / immunological findings. Although classification criteria were developed for the selection of homogeneous patient populations in the context of epidemiological or clinical studies, they are widely used in clinical practice to aid the diagnosis of the disease.
The most commonly used classification criteria are those of the American College of Rheumatology (ACR-1997). They consist of 11 clinical and immunological manifestations of the disease and require the presence of at least 4 of them in order to classify a patient as SLE. Despite their ease of use, they have low sensibility for early and severe forms of the disease, while they can classify as SLE patients with purely mild mucocutaneous manifestations. In addition, several manifestations are absent from certain organs / systems (e.g., nervous system, hematopoietic), resulting in delayed diagnosis. In 2012, the Systemic Lupus International Collaborating Clinics (SLICC) team proposed new classification criteria that require >4 of 17 manifestations, including at least one clinical and one immunological, or alternatively, biopsy-proven lupus nephritis with either positive antinuclear antibodies (ANA) or double-stranded DNA antibodies (anti-dsDNA). These criteria are advantageous as they include more clinical and immunological manifestations. Although they have higher sensitivity (92-97% vs. 77-92%), they are limited in terms of specificity (74-88% vs. 91-96%) as compared to the ACR criteria. These restrictions led to the recent (2019) cooperation of EULAR / ACR for the introduction of a new set of classification criteria. The new criteria presuppose the existence of a positive ANA titre as an entry criterion, in combination with a series of clinical and immunological items, which have different weight for the calculation of the total score that will classify the patient as SLE or not. Whether these new classification criteria allow for earlier classification of SLE patients, with superior sensitivity and specificity as compared to the ACR-1997 and SLICC-2012 criteria, allowing them to be used in clinical practice to aid diagnosis is not clear.
To this end, we first evaluated the comparative performance of the three SLE classification criteria in a large cohort of patients with early diagnosis of SLE or other rheumatological diseases, spanning from the community to tertiary care. We also examined which criteria enable earlier classification and whether this has implications in disease severity. Next, based on our sample characteristics, we proposed modifications to the existing classification algorithms to ensure the highest combination of sensitivity and specificity, thus allowing the timely classification and treatment of patients with potentially high disease burden.
Despite the high sensitivity and specificity of existing and modified criteria, the development of optimized classification (or diagnostic) criteria that will include all patients with clinically significant disease, while excluding healthy or with alternative diseases individuals, remains an important and unmet need. During this dissertation we tried to fill this gap by creating a clinically applicable algorithm for assessing the likelihood SLE diagnosis based on the frequent manifestations of the disease. More specifically, we applied artificial intelligence tools and specifically, machine learning techniques (ML) to create and validate a new predictive model for Systemic Lupus Erythematosus, which was based on the clinical characteristics of large, well-characterized patient datasets. The performance of the predictive algorithm was further evaluated in two external validation datasets and was compared to the classification criteria sets ACR-1997, SLICC-2012 and EULAR / ACR-2019. Four patient subgroups of special clinical interest from the external validation dataset, i.e., patients with early SLE, lupus nephritis, NPSLE and patients with severe manifestations, were separately used to compare the predictive model to the three criteria sets. We further examined the performance of the three classification criteria, as well as of the new SLE predictive ML-based algorithm, in a group of cases with undifferentiated disease, who were monitored prospectively during this dissertation.
Aims of the thesis
We first performed a comprehensive study of the existing classification criteria for SLE, exploiting a large group of patients with SLE or other rheumatic diseases in two Greek tertiary centers. Second, based on the characteristics of our cohort patients, we created a new prognostic model for SLE by applying machine learning techniques.
More specifically:
• we compared the sensitivity and specificity of the three sets of classification criteria (ACR-1997, SLICC-2012, EULAR/ACR-2019) in a well-defined cohort of patients with SLE, representative of the community, diagnosed according to the opinion of an experienced physician
• we examined which of the criteria sets allows earlier classification of the disease both in patients with established SLE and in a group of cases with undifferentiated disease who were monitored in the outpatient clinic every 6-12 months (prospective study)
• we analyzed the prognostic effects (development of organ damage) of the classification or non-classification of SLE patients with the three sets of classification criteria
• modifications and combinations of the algorithms of the SLICC-2012 and EULAR/ACR-2019 criteria were made to enhance their diagnostic accuracy for SLE
• we created a simplified assessment algorithm for the prediction of SLE using ML techniques, based on the characteristics of our cohorts
• we further evaluated the performance of the predictive algorithm in two external validation datasets and was compared to the classification criteria sets ACR-1997, SLICC-2012 and EULAR / ACR-2019.
Patients and methods
The study was conducted in two tertiary centres and included a retrospective and a prospective part. The SLE registries and the registries of patients with other rheumatic diseases of the Rheumatology Clinic of PAGNI and the 4th Internal Medicine Clinic of the Attikon Hospital of Athens were utilized.
• Inclusion criteria for the analysis of the sensitivity and specificity of the criteria and the modification of the existing criteria were consecutively registered patients aged > 15 years who were diagnosed with SLE by a specialist, during the period 1/2005-12/2016 and patients > 15 years with other rheumatic diseases randomly selected from the respective registries of patients with other rheumatic diseases of the 2 participating clinics.
• To create a diagnostic model with ML techniques, a randomized sample of 802 patients >15 years of age and diagnosis of SLE and other rheumatic diseases from 2005 onwards were included, which were randomly selected from the respective patient registries of the 2 participating clinics.
• For the prospective part of the study, individuals >15 years of age who do not meet any criteria for SLE or other rheumatic diseases, with positive autoantibodies and non-clinically significant manifestations or first-degree relatives (FDRs) of patients with SLE were included. This group constitutes the preclinical group of participants who are monitored every 6-12 months for possible transition to SLE, according to physician or the classification criteria.
From each study participant demographics, year of diagnosis of SLE or other diagnoses (according to physician opinion), presence and year of occurrence of each manifestation from the 3 sets of classification criteria (ACR-1997, SLICC-2012, EULAR / ACR-2019), the presence and the year of occurrence of selected additional clinical manifestations of the disease that are not included in the existing classification criteria (such as Raynaud's phenomenon, lymphadenopathy, etc.), disease severity, the development of irreversible organ damage (SLICC / ACR Damage Index) were collected. The RedCap online platform was used to collect and manage databases for the study.
Sensitivity of the criteria was assessed against physician diagnosis, both at the time of diagnosis and at last patient follow-up visit (overall sensitivity). Specificity was determined against patients with other rheumatological diseases. In separate analysis, we calculated the earliest date of fulfilment of each set of criteria and the time elapsed since the date of the earliest item. Hazard analysis was used to determine the median (95% CI) time-to-classification for each set of criteria. Modified classification algorithms were derived from a random 80% and validated in the remaining 20% of the dataset running multiple iterations. Between-groups comparisons were performed by the McNemar’s test or linear mixed model analysis for partially paired samples.
To create a simplified evaluation system for the prediction of SLE diagnosis we applied two ML models, Logistic Regression (LR) in combination with LASSO Feature Selection and Random Forest, in a number of clinical features from the classification criteria (ACR-1997, SLICC-2012, EULAR/ACR-2019) and not. The LR model with the highest accuracy was selected during the 10-fold cross-validation process and was verified in an independent sample of 513 SLE patients and 101 with other rheumatic diseases. Α simplified evaluation system for the prediction of SLE from the selected LR model was created.
Results
A total of 690 patients with SLE and 401 controls with a diagnosis between 1/2005-12/2016 were analysed to calculate the diagnostic value of the SLE classification criteria. This analysis showed a higher overall sensitivity (calculated throughout the monitoring period) of the EULAR/ACR-2019 and SLICC-2012 criteria compared to the ACR-1997 (88.6%, 91.3% and 85.7%, respectively), and higher specificity of the EULAR/ACR-2019 (97.3%, 93.8% and 93.0% respectively). However, at the time of clinician diagnosis the corresponding sensitivity of the criteria was clearly lower (74.4%, 73.5%, 69.5% respectively). Although both the EULAR/ACR and the SLICC criteria allowed the classification of SLE patients earlier than the ACR criteria, this classification was delayed by >3 months in 17.3-19.9% of cases. In addition, among patients with neuropsychiatric lupus (NPSLE), the delay in diagnosis was even more significant.
The analysis of patients with disease duration <3 years, showed a significantly increased overall sensitivity of the EULAR/ACR (87.3%) and SLICC (91.4%) compared to the ACR criteria (79.9%, p < 0.01 and p <0.001, respectively). In this group of patients, the median (95% CI) time elapsed between the appearance of the first manifestation of the disease and the fulfilment of the criteria was shorter for the EULAR/ACR (9.1 [6.5-11.8] months) and SLICC (9.1 [6.9-11.3] months) as compared to the ACR criteria (12.1 [9.6-14.7] months, p = 0.043 and p = 0.001, respectively), indicating that both the EULAR/ACR and the SLICC criteria have increased sensitivity among early SLE patients and allow classification earlier than the ACR criteria.
Overall, only 2.9% of SLE patients were not classified by any of the three criteria throughout the follow-up period, while only 76.7% of SLE patients met all three classification criteria, suggesting that each set of criteria classifies distinct disease phenotypes. Patients who did not meet the ACR criteria had a significantly higher prevalence of haematological and immunological manifestations. Patients who did not meet the EULAR / ACR had increased rates of mucocutaneous manifestations, while those who did not meet the SLICC criteria had predominantly skin and joint manifestations. Among patients who did not meet any of the three criteria, 15% had moderately severe BILAG manifestations and 45% had severe BILAG manifestations, while 50% of these patients developed irreversible organ damage during follow-up. Twenty percent of these patients had neurological manifestations, while the frequency of autoantibodies in these patients was lower.
In order to improve the accuracy of the classification criteria, we explored alternative classification algorithms based on the existing criteria. Specifically, a random sample of 80% of our total sample (derivation cohort) was extracted and using the characteristics of our cohort patients, the algorithms of the EULAR/ACR and SLICC criteria were modified. Low complement and/or positive antiphospholipid antibodies were included as an alternative entry criterion in the case of the EULAR/ACR, while in the case of SLICC criteria a modification was made to allow the classification of patients with fewer clinical features but with manifestations from multiple organs. We tested the modified algorithms of the criteria in the remaining 20% of the sample (validation cohort) and by performing 100 iterations of the analysis, to take into account the heterogeneity of the patients, we calculated the median sensitivity and specificity of the modified criteria. These modified EULAR/ACR and SLICC criteria showed increased sensitivity at diagnosis (median sensitivity 82.0% and 86.2% respectively) and increased overall sensitivity (median 93.7% and 97.1% respectively) with a slight decrease in their specificity (median specificity 94.9% and 90.2% respectively). Importantly, patients not classified by these modified criteria had a lower incidence of severe organ damage, use of immunosuppressive / biological therapies, and permanent organ damage.
Furthermore, for the development of a prognostic model for SLE, a total 40 clinically-selected panels of criteria and non-criteria features were subjected to machine learning algorithms yielding a 14-variable LASSO model (M31) with the best performance (CV10 accuracy 95.3%; AUC 98.4%). From the LR model we developed a simplified evaluation system based on 14 classical clinical and immunological characteristics, the SLE Risk Probability Index (SLERPI). Thrombocytopenia / haemolytic anaemia, acute cutaneous lupus erythematosus, proteinuria, low C3 / C4 levels, ANA and other autoantibodies (antiphospholipids, anti-DNA, anti-Sm) had the strongest prognostic value and, respectively, the highest weight factor in the model. The use of SLERPI allows on the one hand the classification of patients into prognostic categories (unlikely, probable, very probable and certain SLE), on the other hand the dichotomous diagnosis (SLE or not) with an accuracy of 94.8%, when a threshold of SLERPI>7 is applied. Also, SLERPI presents high sensitivity for SLE patients with early disease (93.7%), nephritis (97.9%) and severe disease that requires immunosuppressive-biological treatment (96.4%).
The prospective part of this dissertation is an ongoing study. After a follow-up of 14.3 ± 7.6 months (mean ± standard deviation), 127 participants (100% Caucasians, 93.7% females, mean age 38.2 years) were eligible for inclusion in the preliminary analysis. The majority of the participants (75.6%) were ANA positive at enrolment and the most frequent initial manifestations were arthritis (40.1%), photosensitivity (33.1%), malar rash (29.1%) and Raynaud’s phenomenon (29.1%). So far, 11 participants have transitioned to SLE and 10 participants to other diagnoses. Among cases who have transitioned to SLE so far, only one patient fulfilled the ACR-1997 criteria at enrolment, whereas 6 (54.5%) patients fulfilled the SLICC-2012 criteria and 2 (18.2%) patients fulfilled the EULAR/ACR-2019 criteria. At the time of SLE diagnosis by the physician, 5 (45.5%) patients were classified by the ACR-1997, 10 (90.9%) by the SLICC-2012 and 10 (90.9%) by the EULAR/ACR-2019 criteria. Among those patients who were diagnosed with alternative diseases 1 (10%) patient was misclassified as SLE by the ACR-1997 criteria, 2 (20%) by the SLICC-2012 criteria and 4 (40%) by the EULAR/ACR-2019 criteria, both at enrolment and at last follow-up. Among those patients who developed SLE (n=11), 4/11 had a SLERPI score >7 at enrolment and 10/11 at physician diagnosis. Among patients who were diagnosed with alternative diagnoses (n=10), 6/10 patients and 8/10 patients had a SLERPI score>7 at enrolment, and at last follow-up respectively.
Conclusions
Existing classification criteria may miss patients with potentially serious disease or delay classification, especially in cases of neurological SLE. Combination of all three sets of criteria may assure maximum capture of patients for inclusion in clinical trials. Modifying the classification threshold may further enhance the sensitivity of the new EULAR/ACR criteria, especially in the early stages of the disease. Our ML-based predictive model for SLE, which was developed based on frequent manifestations of the disease, could aid in the timely detection and treatment of the disease, including its severe forms.
Finally, our preliminary results of the prospective study suggest that among individuals with positive autoantibodies or FDRs with SLE, the short-term risk for transition into clinical SLE is low. The SLICC-2012 criteria were able to classify those cases who transitioned to SLE earlier, but at the time of physician diagnosis both the SLICC and the EULAR/ACR criteria captured the 90% of patients. The SLERPI algorithm showed high sensitivity at physician diagnosis, but misclassified as SLE most cases with alternative diseases. Following the prospective part of the study’s completion, clinical and lifestyle data will be combined with blood transcriptome to define a high-risk subgroup of individuals for progression into SLE.
Despite the high performance of classification criteria and the new SLE predictive algorithm SLERPI, the development of optimized classification (or diagnostic) criteria that will include all patients with clinically significant disease while excluding all mimickers, remains an important and unfulfilled goal.
|
Search
