Skip to main content

Effectiveness and safety of intravenous golimumab with and without concomitant methotrexate in patients with rheumatoid arthritis in the prospective, noninterventional AWARE study

Abstract

Background

Biologic therapies are often prescribed for patients with rheumatoid arthritis (RA) who have inadequate responses to or are intolerant of methotrexate (MTX) and patients with poor prognostic indicators. This post hoc analysis evaluated effectiveness and safety of intravenous golimumab + MTX vs golimumab without MTX in RA patients.

Methods

AWARE, a real-world, prospective and pragmatic, Phase 4 study, compared effectiveness and safety of golimumab and infliximab in biologic-naïve and biologic-experienced patients. All treatment decisions were at the discretion of the treating rheumatologist. Effectiveness was evaluated by mean change in CDAI scores at Months 6 and 12. Safety was monitored through approximately 1 year.

Results

Among 685 golimumab-treated patients, 420 (61%) received concomitant MTX during the study and 265 (39%) did not receive MTX after enrollment; 63% and 72%, respectively, discontinued the study. Relative to golimumab without MTX, golimumab + MTX patients had shorter mean disease duration (8.7 vs 10.0 years) and a lower proportion received prior biologics (60% vs 72%); mean ± standard deviation (SD) baseline CDAI scores were similar (30.8 ± 15.1 and 32.6 ± 15.4). Mean ± SD changes from baseline in CDAI scores at Months 6 and 12, respectively, were similar with golimumab + MTX (− 10.2 ± 14.2 and − 10.8 ± 13.8) and golimumab without MTX (− 9.6 ± 12.9 and − 9.9 ± 13.1). The incidence of adverse events/100 patient-years (PY) (95% confidence interval [CI]) was 155.6 (145.6, 166.1) for golimumab + MTX and 191.2 (176.2, 207.1) for golimumab without MTX; infections were the most common type. The incidence of infusion reactions/100PY (95% CI) was 2.1 (1.1, 3.6) for golimumab + MTX versus 5.1 (2.9, 8.3) for golimumab without MTX; none were considered serious. For golimumab + MTX versus golimumab without MTX, rates/100PY (95% CI) of serious infections, opportunistic infections, and malignancies were 2.6 (1.5, 4.3) versus 7.0 (4.4, 10.6), 0.9 (0.3, 2.0) versus 2.6 (1.1, 5.0), and 3.0 (1.7, 4.7) versus 1.0 (0.2, 2.8), respectively.

Conclusions

Mean change in CDAI score in the  golimumab without MTX group was generally similar to that of the golimumab + MTX group through 1 year, regardless of prior biologic therapy. Adverse events were consistent with the known IV golimumab safety profile. These results provide real world evidential data that may assist healthcare providers and patients with RA in making informed treatment decisions.

Trial registration: clinicaltrials.gov NCT02728934 05/04/2016.

Peer Review reports

Background

Rheumatoid arthritis (RA) is a chronic, systemic, inflammatory disease characterized by symmetrical joint synovitis of the hands and feet [1]. Extra-articular disorders such as ischemic heart disease, interstitial lung disease, pericarditis, pleural effusion, or bronchiectasis, are also common in patients with RA, even in those with only minimal articular involvement [1]. Furthermore, patients with RA are at greater risk for comorbidities across many organ systems (e.g., serious infection, osteoporosis, and malignancy) as well as a higher mortality rate than the general population [1].

In the mid-1980s, the conventional synthetic disease-modifying anti-rheumatic drug (csDMARD) methotrexate (MTX) emerged as the standard-of-care for RA patients [2]. Current guidelines from the American College of Rheumatology (ACR) and European Alliance of Associations for Rheumatology (EULAR) continue to recommend MTX as first-line therapy for DMARD-naïve patients with moderate-to-high disease activity [3, 4]. Escalation of therapy with the use of biologics, such as tumor necrosis factor inhibitors (TNFi), is generally recommended for patients with active disease despite MTX or for patients presenting with poor prognostic factors [3, 4]. TNFi have consistently demonstrated significantly greater efficacy in improving the signs and symptoms of RA, physical function, and health-related quality of life, either as monotherapy or in combination with MTX, compared with MTX alone [5,6,7,8,9,10]. Furthermore, TNFi can offer greater protection against radiographic damage compared with csDMARDs (e.g., MTX, leflunomide, and sulfasalazine) [11].

In some patients, significant toxicities attributed to MTX, including gastrointestinal adverse events (AEs), decreased blood cell counts, and abnormal liver function tests [2, 12], can lead to poor adherence. In addition, women of child-bearing age may be reluctant to use MTX, and healthcare providers may be reluctant to prescribe MTX to these patients due to the known teratogenic effects [13]. In a real-world study, approximately one-third of RA patients treated with a TNFi experienced dose reduction/discontinuation of concomitant MTX secondary to intolerance or AEs over a 2-year follow-up period [14]. Results from a previous observational study of MTX use in patients with RA, found that 26% reported nonadherence during the first 6 months of treatment, with 71% of those patients indicating that the nonadherence was intentional [15]. Among patients who provided a reason for nonadherence, 34% reported this was due to adverse effects of MTX, including nausea, headache, fatigue, and dizziness [15]. Many patients experience significant short-term side effects (e.g., gastrointestinal symptoms, fatigue, malaise) that are temporally related to MTX use, often occurring 1–2 days after weekly dosing, which may be extremely bothersome [16]. In an earlier literature review, Curtis et al. [17] found that adherence to MTX among patients with RA was highly variable. They also noted that persistence with MTX varied from 50 to 94% at 1 year and from 25 to 79% at 5 years, with intolerance being the most common reason for discontinuation.

The TNFi intravenous (IV) golimumab and infliximab are approved to treat adults with RA in combination with MTX [18, 19]. In the prospective and pragmatic, noninterventional, Phase 4 AWARE study, the safety and effectiveness of golimumab and infliximab were evaluated in patients with RA in a real-world setting [20]. Approximately 40% of patients in the AWARE study reported no concomitant MTX use; thus, exploratory, post hoc analyses were undertaken to further evaluate the effectiveness and safety of golimumab with and without concomitant MTX in RA patients.

Methods

Patients and study design

AWARE (NCT02728934) was an observational, prospective, noninterventional, real-world study conducted at 88 sites in the United States (US). Details of the patient inclusion criteria and the study design of AWARE have been previously reported [20]. Briefly, adults (≥ 18 years) with a confirmed diagnosis of RA were eligible for enrollment if they were initiating therapy with either IV golimumab or infliximab. Prior biologic therapy was permitted, with the exception of golimumab if enrolling in the golimumab group and infliximab if enrolling in the infliximab group. After enrollment, patients could switch from golimumab to infliximab (or infliximab biosimilar) or from infliximab to golimumab, provided they had not had previous exposure to the second therapy.

No study drug was provided by the sponsor, and all treatment decisions, including prescribed dose and dosing intervals of golimumab and infliximab as well as concomitant medications for RA or other conditions, were at the discretion of the treating rheumatologist. In the US, golimumab is approved at a dosage of 2 mg/kg (over 30 min) at Weeks 0 and 4, and every 8 weeks thereafter [18], and infliximab is approved at a dosage of 3 mg/kg (over 120 min) at Weeks 0, 2, and 6, and every 8 weeks thereafter with dose adjustments up to 10 mg/kg with administration as often as every 4 weeks [19]. According to the Food and Drug Administration (FDA)-approved indication for RA, both golimumab and infliximab are approved for use with concomitant MTX.

AWARE was planned as a 3-year study but was closed after 2 years when sufficient numbers of patients had been treated through Week 52 and it was determined that the primary and major secondary endpoints were achieved.

This study was conducted according to the Declaration of Helsinki and the International Committee on Harmonisation good clinical practices. The study protocol was reviewed by a centralized institutional review board (Copernicus Group, Approval QUI1-15-645); ethics committee review was not performed as there were no study sites outside the US. All patients provided written informed consent.

Assessments

Patient visits (and corresponding data collection) were completed according to usual clinical practice and could differ  across study sites. The Clinical Disease Activity Index (CDAI) [21] score was determined at baseline, Month 3, Month 6, and Month 12 to evaluate treatment effectiveness (remission [score ≤ 2.8], low disease activity [score > 2.8 to ≤ 10], moderate disease activity [score > 10 to ≤ 22], and high disease activity [score > 22]). Safety was assessed throughout the study. Infusion reactions were defined as any AE that occurred during an infusion or within 1 h after the infusion. AEs of special interest included serious infections, latent tuberculosis, opportunistic infections, and malignancies.

Statistical methods

In the post hoc analyses reported here, only data from patients who received golimumab were included; results are summarized by concomitant MTX use (i.e., with and without MTX) and prior biologic use (biologic-naïve vs biologic-experienced). Concomitant MTX use was defined as ≥ 1 administration at any time either at or post-Week 0. The use of MTX, including dosage, was not prespecified in the protocol and was at the discretion of the investigator. The primary endpoint (proportion of patients having an infusion reaction) and major secondary endpoints (mean change in CDAI at Months 6 and 12 in biologic-naïve patients) were assessed in the formal, preplanned, interim analysis conducted after approximately 66% of the patients completed Week 52 or permanently discontinued the study [20].

Baseline demographic and disease characteristics, prior biologic use, prior or concomitant therapy with csDMARDs and targeted synthetic (ts)DMARDs, smoking status, and selected comorbid conditions are summarized using descriptive statistics (counts and percentages; means and standard deviations [SDs]). Descriptive statistics are also reported for outcomes through the Week 52 database lock (February 1, 2019). Mean changes from baseline to Months 6 and 12 in CDAI score were determined using both imputed and observed data; all golimumab-treated patients with ≥ 1 post-baseline CDAI score were included in the analyses. In the analysis using imputation for missing data, missing CDAI scores at baseline were imputed using the mean score derived from non-missing baseline values, missing CDAI scores post-baseline were imputed utilizing last observed carried forward (LOCF), and for patients who discontinued due to lack of effectiveness (per the reason provided by the patient and/or the investigator), a change of 0 in CDAI was imputed for subsequent timepoints. In addition, inverse probability of treatment weighted (IPTW) propensity scores were utilized in the analysis of covariance model of change in CDAI to adjust for baseline characteristics of the golimumab + MTX and golimumab without MTX groups. Propensity scores were estimated using logistic regression analysis with concomitant MTX use as the dependent variable. The propensity score model included the following baseline covariates: age, sex, White race (yes/no), US geographic region (East/West), body mass index, weight, disease duration, CDAI score, biologic-naïve (yes/no), other medications (excluding MTX), number of prior biologics received, prior TNFi therapy (yes/no), selected comorbidities (diabetes mellitus, malignancy, cardiovascular, hyperlipidemia, psychiatric history), and smoking status [20]. The least square (LS) mean difference and 95% confidence interval (CI) were determined via analysis of covariance controlling for baseline CDAI. In addition, the proportions of patients in each CDAI disease activity category (remission, low disease activity, moderate disease activity, and high disease activity) at baseline and Months 3, 6, and 12 are summarized by treatment group and prior biologic use (all patients, biologic-naïve, and biologic-experienced).

AEs, serious AEs (SAEs), and AEs of special interest were summarized as counts, percentages, and number of events per 100 patient-years (PY) to adjust for variability in treatment exposure.

Results

Baseline characteristics and patient disposition

A total of 1270 patients were enrolled in the AWARE study; 685 received golimumab, and 585 received infliximab. Twenty-three (1.8%) patients switched therapies: 11 who initially received golimumab switched to infliximab and 12 switched from infliximab to golimumab. In addition, 4 patients switched from infliximab to a biosimilar infliximab.

Patient characteristics for the full study population have been previously detailed [20]. Among the 685 golimumab-treated patients, 420 (61%) reported receiving ≥ 1 dose of MTX during the study (golimumab + MTX group) and 265 (39%) reported no MTX use through the Week 52 database lock (golimumab without MTX group). Among the 420 patients categorized as receiving MTX, 3.8% reported concomitant MTX use during the study  while 96.2% reported prior and concomitant use. Prior MTX use was reported by 111 (42%) patients in the golimumab without MTX group.

Baseline characteristics were generally similar between patients in the golimumab + MTX and golimumab without MTX groups, with the exceptions that the former had a numerically higher proportion of females (87% vs 81%, respectively) and a shorter mean ± SD disease duration (8.7 ± 9.4 vs 10.0 ± 10.8 years, respectively) (Table 1). Mean ± SD baseline CDAI scores in the golimumab + MTX and golimumab without MTX groups indicated highly active disease (30.8 ± 15.1 and 32.6 ± 15.4, respectively); mean scores were similar for biologic-naïve and biologic-experienced patients regardless of concomitant MTX usage.

Table 1 Baseline demographic and disease characteristics, comorbidities, and prior and concomitant medications for RA

The proportions of patients reporting select prior or ongoing comorbidities were similar between the golimumab + MTX and golimumab without MTX groups Hyperlipidemia (27.9%) and diabetes mellitus (14.6%) were reported most often, while myocardial infarction (1.5%) and congestive heart failure (1.5%) were reported infrequently (Table 1).

At baseline, a lower proportion of patients in the golimumab + MTX group (60%) than in the golimumab without MTX group (72%) reported prior biologic use. Exposure to csDMARDs and tsDMARDs either prior to or during the study is summarized in Table 1. Prior/concomitant use of csDMARDs other than MTX was more common in the golimumab without MTX group (49%) than in the golimumab + MTX group (28%); patients in the golimumab without MTX group also more commonly reported having ever received ≥ 2 csDMARDs (34% vs 28%, respectively) (Table 1). Use of tsDMARDs was less common; 6% patients in the golimumab + MTX group and 10% of those in the golimumab without MTX group reported ever using tofacitinib and one patient (golimumab + MTX group) reported ever receiving baricitinib. In the golimumab + MTX group, the use of ≥ 2 csDMARDs was more common among biologic-experienced patients (30%) than in biologic-naïve (25%) patients; however, the opposite trend was observed in the golimumab without MTX group (biologic-experienced: 32%; biologic-naïve: 38%). Biologic-experienced patients more frequently reported ever receiving tofacitinib compared with biologic-naïve patients in both the golimumab + MTX (9% vs 1%, respectively) and golimumab without MTX (12% vs 4%, respectively) groups.

Overall, 454 (66%) golimumab patients discontinued the study prior to the Week-52 database lock (February 1, 2019): 263/420 (63%) in the golimumab + MTX group and 191/265 (72%) in the golimumab without MTX group. The primary reasons for study discontinuation included lack of effectiveness (27.1% for golimumab + MTX and 32.1% for golimumab without MTX) and AEs (9.3% for golimumab + MTX and 12.5% for golimumab without MTX). A total of 613 patients had missing data that required data imputation (LOCF).

Clinical effectiveness

As previously reported in the formal interim analysis of biologic-naïve patients, the mean changes from baseline in CDAI at 6 and 12 months in the golimumab group were − 9.5 and − 9.4, respectively, and were noninferior to those in infliximab patients (− 10.1 and − 10.1, respectively) [20].

In the current post hoc analysis of all enrolled golimumab patients with  ≥1 post-baseline CDAI score, the mean ± SD changes from baseline in CDAI score (imputation rule-based analysis) at Month 6 were − 10.2 ± 14.2 and − 9.6 ± 12.9 in the golimumab + MTX (N = 381) and golimumab without MTX (N = 232) groups, respectively (Fig. 1). Corresponding mean ± SD data at Month 12 were − 10.8 ± 13.8 and − 9.9 ± 13.1, respectively. IPTW mean changes in the  golimumab + MTX and golimumab without MTX groups, respectively, were − 10.6 and − 9.0 at Month 6 (LS mean difference: − 1.6 [95% CI − 3.60, 0.37]) and − 11.1 and − 9.2 at Month 12 (LS mean difference − 1.9 [95% CI − 3.89, 0.06]). Additionally, no apparent differences were observed for biologic-naïve and biologic-experienced patients, regardless of concomitant MTX usage. Similar trends were seen when using observed data (Additional file 1: Figure S1).

Fig. 1
figure 1

Mean change from baseline at Month 6 (a) and Month 12 (b) in CDAI score for IV golimumab with and without methotrexate (last observation carried forward for missing data) in biologic-naïve and biologic-experienced patients. CDAI Clinical Disease Activity Index, IV intravenous, SD standard deviation

At baseline, the proportions of patients in each CDAI disease activity category were similar between the golimumab + MTX and golimumab without MTX groups, with 22.0% and 21.7%, respectively, having moderate disease activity (CDAI > 10 to ≤ 22), and 69.7% and 71.5%, respectively, displaying high disease activity (> 22) (Fig. 2a). Response rates for achieving remission (CDAI ≤ 2.8) or low disease activity (CDAI > 2.8 to ≤ 10) were similar between the golimumab + MTX and golimumab without MTX groups over time, with 10% to 13% achieving remission and 31% to 34% achieving low disease activity at Month 12 (Fig. 2b–d).

Fig. 2
figure 2

Proportion of all patients in remission (CDAI ≤ 2.8) and with low disease activity (CDAI > 2.8 to ≤ 10), moderate disease activity (CDAI > 10 to ≤ 22), and high disease activity (CDAI > 22) at baseline (a) and Months 3 (b), 6 (c), and 12 (d) for IV golimumab with and without methotrexate. CDAI Clinical Disease Activity Index, IV intravenous

The proportions of patients in each CDAI disease activity category at baseline were generally similar within the treatment groups for biologic-naïve and biologic-experienced patients. However, response rates for achieving remission or low disease activity at Months 3, 6, and 12 tended to be numerically higher in biologic-naïve patients compared with biologic-experienced patients in both the golimumab + MTX and golimumab without MTX groups (Additional file 1: Figures S2 and S3). At Month 12, 35% and 45% of biologic-naïve patients in the golimumab + MTX group and the golimumab without MTX group, respectively, achieved low disease activity compared with 28% and 30% of biologic-experienced patients.

Safety

At the time of the Week-52 database lock (February 1, 2019), the mean ± SD durations of exposure were 61 ± 42 and 53 ± 42 weeks for the golimumab + MTX group and the golimumab without MTX groups, respectively (Table 2). The total patient-years of follow-up were 575 and 314, respectively.

Table 2 Week 52 database lock safety results

Approximately 55% of patients in both groups reported ≥ 1 AE (Table 2). AEs in the infections and infestations system organ class were the most common type, occurring in approximately 29% of patients regardless of concomitant MTX use. Common infections reported in both treatment groups were upper respiratory tract infection, urinary tract infection, sinusitis, and bronchitis. The frequency and types of AEs were similar among biologic-naïve and biologic-experienced patients in both the golimumab + MTX and golimumab without MTX groups (data not shown). The incidence of AEs/100PY (95% CI) was 155.6 (145.6, 166.1) in the golimumab + MTX group and 191.2 (176.2, 207.1) in the golimumab without MTX group.

Overall, 27 golimumab-treated patients had an infusion reaction (12 patients in the golimumab + MTX and 15 patients in the golimumab without MTX groups) (Table 2); the incidence of infusion reactions/100PY (95% CI) was 2.1 (1.1, 3.6) in the golimumab + MTX group and 5.1 (2.9, 8.3) in the golimumab without MTX group. The majority of infusion reactions were of mild intensity; none was considered serious. Among the 27 patients who had an infusion reaction, 7 patients (4/420 [1.0%] golimumab + MTX and 3/265 [1.1%] golimumab without MTX) discontinued the study due to an infusion reaction.

A total of 83 golimumab-treated patients had an SAE (Table 2). The incidence of SAEs/100 PY (95% CI) was 14.1 (11.2, 17.5) for golimumab + MTX and 17.8 (13.5, 23.1) for golimumab without MTX (Table 2). No SAEs related to hematologic or liver transaminase elevations were reported, and rates of gastrointestinal SAEs were similar in both the golimumab + MTX and the golimumab without MTX groups (1.4% vs 1.5%).

Serious infections occurred at rates  of 2.6 (1.5, 4.3)/100 PY for golimumab + MTX and 7.0 (4.4, 10.6)/100 PY for golimumab without MTX (Table 2). One (0.4%) patient was diagnosed with latent tuberculosis (golimumab without MTX group). Eleven golimumab-treated patients had an opportunistic infection. The incidence (95% CI) of opportunistic infections/100PY was 0.9 (0.3, 2.0) for golimumab + MTX and 2.6 (1.1, 5.0) for golimumab without MTX. A total of 18 golimumab-treated patients reported 20 malignancies; the incidence (95% CI) of malignancies/100PY was 3.0 (1.7, 4.7) for golimumab + MTX and 1.0 (0.2, 2.8) for golimumab without MTX.

Nine deaths occurred in the golimumab group (7 received golimumab + MTX and 2 received golimumab without MTX) (Table 2). The incidence (95% CI) of deaths/100PY was 1.2 (0.5, 2.5) for golimumab + MTX and 0.6 (0.1, 2.3) for golimumab without MTX.

Discussion

FDA-approved dosing for IV golimumab in patients with RA is 2 mg/kg infused over 30 min at Weeks 0 and 4, and every 8 weeks thereafter with concomitant MTX [18]. Previous results from the real-world AWARE study indicated that a majority of golimumab doses administered were consistent with the FDA-approved labeling; however, a substantial proportion of these patients did not receive concomitant MTX [20]. Therefore, we evaluated the effectiveness and safety of golimumab in patients who did and did not receive concomitant MTX.

The CDAI is a validated measure of disease activity (range 0–76, with higher scores indicating more severe disease) and is commonly used in clinical practice [21]. In this post hoc analysis of the AWARE study, patients had a mean baseline CDAI score of > 22, indicating high disease activity. Mean changes in CDAI score at Months 6 and 12 were generally similar between the golimumab + MTX and the golimumab without MTX groups when analyzed using LOCF imputation for missing data, IPTW adjustments, and observed data. Additionally, similar changes in CDAI scores were observed with and without concomitant MTX in both biologic-naïve and biologic-experienced patients. Inadequate response was the most common reason for discontinuation of golimumab; however, it should be noted that 65% of patients in the combined golimumab group had received  ≥1 prior TNFi therapy, suggesting that they may have had recalcitrant disease [22].

The Phase 3 GO-FURTHER study of IV golimumab in patients with RA evaluated only the combination of golimumab + MTX, precluding a comparison of golimumab without MTX vs combination therapy in that study population [9]. Golimumab was previously approved as a subcutaneous (SC) injection in combination with MTX for adults with RA [23]. In the Phase 3 studies of SC golimumab in patients with RA (GO-BEFORE [24] and GO-FORWARD [25]), ACR response rates were higher in the SC golimumab + MTX treatment group than in the SC golimumab without MTX group. In contrast, the results of this post hoc analysis of the AWARE study suggest that patients receiving IV golimumab may achieve improvements in RA disease activity either with or without concomitant MTX.

Unlike the FDA-approved indication for both IV and SC golimumab in RA, both IV and SC golimumab are approved as a monotherapy for adults with psoriatic arthritis (PsA) and ankylosing spondylitis (AS). The Phase 3 studies of IV (GO-VIBRANT) [26] and SC (GO-REVEAL) [27] golimumab in patients with PsA permitted concomitant MTX, and approximately 70% and 50% of patients, respectively, received MTX. Both studies reported similar ACR response rates in golimumab-treated patients who did and did not receive concomitant MTX [26, 27]. Concomitant MTX use was also permitted in the studies of IV (GO-ALIVE [28]) and SC (GO-RAISE [29]) golimumab in AS patients; however, few patients in these studies (14% and 21%, respectively) received concomitant MTX; therefore, similar analyses were not conducted.

Management of patients with RA is complex, with 3–20% of patients experiencing difficult-to-treat disease [30, 31]. Recently, a EULAR taskforce identified several characteristics of patients with difficult-to-treat RA: treatment failure history (i.e., failure of ≥ 2 biological DMARDs/tsDMARDs [with different mechanisms of action] after failing csDMARD therapy [unless contraindicated]); presence of active/progressive disease; and the perception by the clinician or patient that the management of the signs and/or symptoms of RA are problematic [32, 33]. In addition, comorbidities that may limit treatment options [34] and some patient characteristics (older age and obesity) have been associated with difficult-to-treat RA [31, 34, 35]. Although the AWARE study did not systematically identify study participants as having difficult-to-treat RA, golimumab-treated patients in AWARE were, on average, older than 60 years (ranging up to 89 years), and nearly 75% were overweight or obese. Furthermore, nearly half of AWARE participants in this post hoc analysis reported at least one comorbidity, with substantial proportions having hyperlipidemia (28%) and diabetes mellitus (15%). Mean baseline CDAI scores were consistent with highly active disease. In addition, nearly two-thirds of golimumab-treated patients were biologic-experienced, and of these, approximately half had received ≥ 2 prior biologic therapies. The totality of these results suggest that many patients enrolled in AWARE may represent difficult-to-treat RA.

Overall, the frequencies, types of AEs, and exposure-adjusted rates through approximately 1 year in these post hoc analyses were consistent with the known safety profile of golimumab [18]. Importantly, SAEs were uncommon (< 13% of patients and exposure-adjusted rate of 14.1 to 17.8/100 PY) regardless of concomitant MTX administration. In a recent pooled safety analysis of IV golimumab across the three aforementioned Phase 3 studies of patients with RA, PsA, and AS (N = 1248), concomitant MTX use was associated with a higher occurrence of elevated alanine transaminases (from ≥ 3 to < 5 times the upper limit of normal) compared with golimumab without MTX [36]. This is consistent with previous reports of hepatic toxicity following MTX therapy [2, 12]. No SAEs related to liver transaminase elevations were reported in AWARE; however, it should be noted that hematologic and blood chemistry evaluations were absent in this real-world study.

The incidence of infusion reactions was numerically higher in patients receiving golimumab without MTX than in those receiving golimumab + MTX. However, in the primary endpoint analysis of AWARE [20], the incidence of infusion reactions among all golimumab-treated patients was approximately five-fold lower vs infliximab (3.6% vs 17.6%, p < 0.001, IPTW-adjusted). In all clinical trials of infliximab across various indications (with varying use of concomitant MTX), 20% of patients experienced an infusion reaction [19]. Of clinical significance, the frequency of infusion reactions among all golimumab-treated patients in the AWARE study was low (4%) and consistent with rates observed in Phase 3 trials of golimumab in patients with rheumatic diseases [9, 10, 20, 26]. The requirement for concomitant MTX in the Phase 3 GO-FURTHER study precludes a similar analysis of infusion reactions and MTX use [9, 10].

Although immunogenicity assessments were not performed in the AWARE study, the reported incidence of antibodies to golimumab from the pooled Phase 3 IV golimumab trials (RA, PsA, and AS) [36] was assessed using a highly sensitive, drug-tolerant, enzyme immunoassay [37]. Overall, approximately 22% of patients were positive for antibodies to golimumab, and 6% developed neutralizing antibodies [36]. The occurrence of antibodies to golimumab, including neutralizing antibodies, was numerically lower in patients who received concomitant MTX compared with those who received golimumab without MTX [36]. Across the three IV golimumab Phase 3 studies, the incidence of infusion reactions was low regardless of antibodies-to-golimumab status [38,39,40]. However, these findings must be interpreted with caution due to the relatively small number of patients receiving concomitant MTX in the AS study and the lack of a golimumab without MTX population in the RA study.

These results from AWARE are limited by the open-label nature of the study. Additionally, these post hoc analyses were exploratory, the AWARE study was not powered to compare the effectiveness and safety of golimumab with and without MTX, and no adjustments were made for multiplicity of testing. Nevertheless, the sample size was large (265 patients receiving golimumab without MTX and  420 receiving golimumab + MTX), and mean changes from baseline in CDAI score were similar between the treatment groups. As this was an observational study, the post hoc analyses reported here are also limited by the relatively high discontinuation rate and potential imbalances (e.g., demographic, baseline disease characteristics, and prior/concomitant medications) between the golimumab + MTX and golimumab without MTX groups; however, similar results were seen with both imputed and observed data as well as with the IPTW analysis. In addition, results were not stratified by MTX dose nor by duration of MTX exposure as limited documentation about MTX therapy was collected. As specified in the protocol of this observational study, few restrictions were placed on prior RA therapies, and the use of any concomitant medication during the study was at the discretion of the treating rheumatologist. Exposure to other medications for RA, including MTX, was reported by the patients and investigators and may have been subject to recall bias or inconsistencies (e.g., partial information). Due to the manner in which medication use was recorded, we were not able to distinguish between prior and concomitant use of RA medications other than MTX, which may limit the generalizability of the results. Additionally, the reasons for discontinuation of prior or concomitant RA therapies were not collected. Finally, safety data were based on spontaneous reporting and may have led to underreporting of some AEs.

Conclusions

The totality of the results from the AWARE study provides real-world data on the use of golimumab with or without concomitant MTX in RA patients in the US. IV golimumab patients in this study had, on average, high disease activity and substantial prevalence of comorbidities, and thus may represent difficult-to-treat disease. Findings from these exploratory post hoc analyses of the AWARE study indicate that improvements in CDAI scores were similar for golimumab-treated patients, regardless of concomitant MTX use. Safety findings in patients receiving golimumab with and without concomitant MTX were consistent with the known safety profile of golimumab in patients with RA. The real-world data presented here may assist healthcare providers and patients with RA in making informed treatment decisions.

Availability of data and materials

Although these data are not currently publicly available for sharing, requests for sharing can be sent to the Corresponding Author and will be evaluated on an individual basis.

Abbreviations

AE:

Adverse events

ACR:

American College of Rheumatology

AS:

Ankylosing spondylitis

BMI:

Body mass index

CDAI:

Clinical disease activity index

CI:

Confidence interval

csDMARD:

Conventional synthetic disease-modifying anti-rheumatic drug

EULAR:

European Alliance of Associations for Rheumatology

FDA:

Food and Drug Administration

IPTW:

Inverse probably of treatment weighted

IQR:

Interquartile range

IV:

Intravenous

LOCF:

Last observation carried forward

LS:

Least square

MTX:

Methotrexate

PsA:

Psoriatic arthritis

PY:

Patient-year

RA:

Rheumatoid arthritis

SAE:

Serious adverse event

SC:

Subcutaneous

SD:

Standard deviation

tsDMARD:

Targeted synthetic disease-modifying anti-rheumatic drug

TNFi:

Tumor necrosis factor inhibitors

US:

United States

References

  1. Sparks JA. Rheumatoid arthritis. Ann Intern Med. 2019;170:ITC1–16.

    Article  PubMed  Google Scholar 

  2. Weinblatt ME. Methotrexate in rheumatoid arthritis: a quarter century of development. Trans Am Clin Climatol Assoc. 2013;124:16–25.

    PubMed  PubMed Central  Google Scholar 

  3. Smolen JS, Landewé RBM, Bijlsma JWJ, Burmester GR, Dougados M, Kerschbaumer A, et al. EULAR recommendations for the management of rheumatoid arthritis with synthetic and biological disease-modifying antirheumatic drugs: 2019 update. Ann Rheum Dis. 2020;79:685–99.

    Article  CAS  PubMed  Google Scholar 

  4. Fraenkel L, Bathon JM, England BR, St Clair EW, Arayssi T, Carandang K, et al. 2021 American College of Rheumatology guideline for the treatment of rheumatoid arthritis. Arthritis Care Res (Hoboken). 2021;73:924–39.

    Article  PubMed  Google Scholar 

  5. Breedveld FC, Weisman MH, Kavanaugh AF, Cohen SB, Pavelka K, van Vollenhoven R, et al. The PREMIER study: a multicenter, randomized, double-blind clinical trial of combination therapy with adalimumab plus methotrexate versus methotrexate alone or adalimumab alone in patients with early, aggressive rheumatoid arthritis who had not had previous methotrexate treatment. Arthritis Rheum. 2006;54:26–37.

    Article  CAS  PubMed  Google Scholar 

  6. Keystone E, van der Heijde D, Mason D Jr, Landewé R, van Vollenhoven R, Combe B, et al. Certolizumab pegol plus methotrexate is significantly more effective than placebo plus methotrexate in active rheumatoid arthritis: findings of a fifty-two-week, phase III, multicenter, randomized, double-blind, placebo-controlled, parallel-group study. Arthritis Rheum. 2008;58:3319–29.

    Article  CAS  PubMed  Google Scholar 

  7. Klareskog L, van der Heijde D, de Jager JP, Gough A, Kalden J, Malaise M, et al. Therapeutic effect of the combination of etanercept and methotrexate compared with each treatment alone in patients with rheumatoid arthritis: double-blind randomised controlled trial. Lancet. 2004;363:675–81.

    Article  CAS  PubMed  Google Scholar 

  8. St Clair EW, van der Heijde DMFM, Smolen JS, Maini RN, Bathon JM, Emery P, et al. Combination of infliximab and methotrexate therapy for early rheumatoid arthritis: a randomized, controlled trial. Arthritis Rheum. 2004;50:3432–43.

    Article  CAS  PubMed  Google Scholar 

  9. Weinblatt ME, Bingham CO 3rd, Mendelsohn AM, Kim L, Mack M, Lu J, et al. Intravenous golimumab is effective in patients with active rheumatoid arthritis despite methotrexate therapy with responses as early as week 2: results of the phase 3, randomised, multicentre, double-blind, placebo-controlled GO-FURTHER trial. Ann Rheum Dis. 2013;72:381–9.

    Article  CAS  PubMed  Google Scholar 

  10. Weinblatt ME, Westhovens R, Mendelsohn AM, Kim L, Lo KM, Sheng S, et al. Radiographic benefit and maintenance of clinical benefit with intravenous golimumab therapy in patients with active rheumatoid arthritis despite methotrexate therapy: results up to 1 year of the phase 3, randomised, multicentre, double blind, placebo controlled GO-FURTHER trial. Ann Rheum Dis. 2014;73:2152–9.

    Article  CAS  PubMed  Google Scholar 

  11. Kang EJ, Kavanaugh A. Psoriatic arthritis: latest treatments and their place in therapy. Ther Adv Chronic Dis. 2015;6:194–203.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Wang W, Zhou H, Liu L. Side effects of methotrexate therapy for rheumatoid arthritis: a systematic review. Eur J Med Chem. 2018;158:502–16.

    Article  CAS  PubMed  Google Scholar 

  13. Gerosa M, Schioppo T, Meroni PL. Challenges and treatment options for rheumatoid arthritis during pregnancy. Expert Opin Pharmacother. 2016;17:1539–47.

    Article  CAS  PubMed  Google Scholar 

  14. Favalli EG, Becciolini A, Biggioggero M, Bertoldi I, Crotti C, Raimondo MG, et al. The role of concomitant methotrexate dosage and maintenance over time in the therapy of rheumatoid arthritis patients treated with adalimumab or etanercept: retrospective analysis of a local registry. Drug Des Devel Ther. 2018;12:1421–9.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Hope HF, Hyrich KL, Anderson J, Bluett J, Sergeant JC, Barton A, et al. The predictors of and reasons for non-adherence in an observational cohort of patients with rheumatoid arthritis commencing methotrexate. Rheumatology (Oxford). 2020;59:213–23.

    CAS  PubMed  Google Scholar 

  16. Nowell WB, Karis E, Gavigan K, Stradford L, Zhao H, Chen L, et al. Patient-reported nausea and fatigue related to methotrexate: a prospective, self-controlled study in the ArthritisPower® Registry. Rheumatol Ther. 2022;9:207–21.

    Article  PubMed  Google Scholar 

  17. Curtis JR, Bykerk VP, Aassi M, Schiff M. Adherence and persistence with methotrexate in rheumatoid arthritis: a systematic review. J Rheumatol. 2016;43:1997–2009.

    Article  CAS  PubMed  Google Scholar 

  18. Simponi ARIA. Package insert. Horsham, PA: Janssen Biotech, Inc.; 2021.

  19. Remicade: Package insert. Horsham, PA: Janssen Biotech, Inc.; 2020.

  20. Curtis JR, Chakravarty SD, Black S, Kafka S, Xu S, Langholff W, et al. Incidence of infusion reactions and clinical effectiveness of intravenous golimumab versus infliximab in patients with rheumatoid arthritis: the real-world AWARE study. Rheumatol Ther. 2021;8:1551–63.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Smolen JS, Aletaha D. Scores for all seasons: SDAI and CDAI. Clin Exp Rheumatol. 2014;32(Suppl 85):S-75-9.

    PubMed  Google Scholar 

  22. Strand V, Miller P, Williams SA, Saunders K, Grant S, Kremer J. Discontinuation of biologic therapy in rheumatoid arthritis: analysis from the Corrona RA Registry. Rheumatol Ther. 2017;4:489–502.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Simponi: Package insert. Horsham, PA: Janssen Biotech, Inc., 2019.

  24. Emery P, Fleischmann RM, Moreland LW, Hsia EC, Strusberg I, Durez P, et al. Golimumab, a human anti-tumor necrosis factor α monoclonal antibody, injected subcutaneously every four weeks in methotrexate-naive patients with active rheumatoid arthritis: twenty-four-week results of a phase III, multicenter, randomized, double-blind, placebo-controlled study of golimumab before methotrexate as first-line therapy for early-onset rheumatoid arthritis. Arthritis Rheum. 2009;60:2272–83 (Erratum in: Arthritis Rheum. 2010;62:2812).

    Article  CAS  PubMed  Google Scholar 

  25. Keystone EC, Genovese MC, Klareskog L, Hsia EC, Hall ST, Miranda PC, et al. Golimumab, a human antibody to tumour necrosis factor α given by monthly subcutaneous injections, in active rheumatoid arthritis despite methotrexate therapy: the GO-FORWARD study. Ann Rheum Dis. 2009;68:789–96 (Erratum in: Ann Rheum Dis. 2011;70:238).

    Article  CAS  PubMed  Google Scholar 

  26. Kavanaugh A, Husni ME, Harrison DD, Kim L, Lo KH, Leu JH, et al. Safety and efficacy of intravenous golimumab in patients with active psoriatic arthritis: results through week twenty-four of the GO-VIBRANT study. Arthritis Rheumatol. 2017;69:2151–61.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  27. Kavanaugh A, McInnes I, Mease P, Krueger GG, Gladman D, Gomez-Reino J, et al. Golimumab, a new human tumor necrosis factor α antibody, administered every four weeks as a subcutaneous injection in psoriatic arthritis: twenty-four-week efficacy and safety results of a randomized, placebo-controlled study. Arthritis Rheum. 2009;60:976–86 (Erratum in: Arthritis Rheum. 2010;62:2555).

    Article  CAS  PubMed  Google Scholar 

  28. Deodhar A, Reveille JD, Harrison DD, Kim L, Lo KH, Leu JH, et al. Safety and efficacy of golimumab administered intravenously in adults with ankylosing spondylitis: results through Week 28 of the GO-ALIVE study. J Rheumatol. 2018;45:341–8.

    Article  CAS  PubMed  Google Scholar 

  29. Inman RD, Davis JC Jr, van der Heijde D, Diekman L, Sieper J, Kim SI, et al. Efficacy and safety of golimumab in patients with ankylosing spondylitis: results of a randomized, double-blind, placebo-controlled, phase III trial. Arthritis Rheum. 2008;58:3402–12.

    Article  CAS  PubMed  Google Scholar 

  30. de Hair MJH, Jacobs JWG, Schoneveld JLM, van Laar JM. Difficult-to-treat rheumatoid arthritis: an area of unmet clinical need. Rheumatology (Oxford). 2018;57:1135–44.

    PubMed  Google Scholar 

  31. Takanashi S, Kaneko Y, Takeuchi T. Characteristics of patients with difficult-to-treat rheumatoid arthritis in clinical practice. Rheumatology (Oxford). 2021;60:5247–56.

    Article  CAS  PubMed  Google Scholar 

  32. Nagy G, Roodenrijs NMT, Welsing PMJ, Kedves M, Hamar A, van der Goes MC, et al. EULAR definition of difficult-to-treat rheumatoid arthritis. Ann Rheum Dis. 2021;80:31–5.

    Article  PubMed  Google Scholar 

  33. Nagy G, Roodenrijs NMT, Welsing PMJ, Kedves M, Hamar A, van der Goes MC, et al. EULAR points to consider for the management of difficult-to-treat rheumatoid arthritis. Ann Rheum Dis. 2022;81:20–33.

    Article  CAS  PubMed  Google Scholar 

  34. Roodenrijs NMT, van der Goes MC, Welsing PMJ, Tekstra J, Lafeber FPJG, Jacobs JWG, et al. Difficult-to-treat rheumatoid arthritis: contributing factors and burden of disease. Rheumatology (Oxford). 2021;60:3778–88.

    Article  PubMed  Google Scholar 

  35. Messelink MA, Roodenrijs NMT, van Es B, Hulsbergen-Veelken CAR, Jong S, Overmars LM, et al. Identification and prediction of difficult-to-treat rheumatoid arthritis patients in structured and unstructured routine care data: results from a hackathon. Arthritis Res Ther. 2021;23:184.

    Article  PubMed  PubMed Central  Google Scholar 

  36. Husni ME, Deodhar A, Schwartzman S, Chakravarty SD, Hsia EC, Leu JH, et al. Pooled safety results across phase 3 randomized trials of intravenous golimumab in rheumatoid arthritis, psoriatic arthritis, and ankylosing spondylitis. Arthritis Res Ther. 2022;24:73.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Leu JH, Adedokun OJ, Gargano C, Hsia EC, Xu Z, Shankar G. Immunogenicity of golimumab and its clinical relevance in patients with rheumatoid arthritis, psoriatic arthritis and ankylosing spondylitis. Rheumatology (Oxford). 2019;58:441–6.

    Article  CAS  PubMed  Google Scholar 

  38. Bingham CO 3rd, Mendelsohn AM, Kim L, Xu Z, Leu J, Han C, et al. Maintenance of clinical and radiographic benefit with intravenous golimumab therapy in patients with active rheumatoid arthritis despite methotrexate therapy: week-112 efficacy and safety results of the open-label long-term extension of a phase III, double-blind, randomized, placebo-controlled trial. Arthritis Care Res (Hoboken). 2015;67:1627–36.

    Article  CAS  PubMed  Google Scholar 

  39. Husni ME, Kavanaugh A, Murphy F, Rekalov D, Harrison DD, Kim L, et al. Efficacy and safety of intravenous golimumab through one year in patients with active psoriatic arthritis. Arthritis Care Res (Hoboken). 2020;72:806–13.

    Article  CAS  PubMed  Google Scholar 

  40. Reveille JD, Deodhar A, Caldron PH, Dudek A, Harrison DD, Kim L, et al. Safety and efficacy of intravenous golimumab in adults with ankylosing spondylitis: results through 1 year of the GO ALIVE study. J Rheumatol. 2019;46:1277–83.

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

Medical writing support was provided by Teresa Tartaglione, PharmD, of Certara Synchrogenix under the direction of the authors in accordance with Good Publication Practice guidelines (Ann Intern Med 2022;175:1298–304) and was funded by Janssen Scientific Affairs, LLC. The authors also thank Michelle Perate, MS and Rebecca Clemente, Ph.D. of Janssen Scientific Affairs, LLC, and Cynthia Guzzo, MD (consultant funded by Janssen) for substantive manuscript review.

Funding

This work was supported by Janssen Scientific Affairs, LLC, Horsham, PA, including funding and guidance for the conduct of the research and the preparation of the article, including the collection, analysis, and interpretation of data; writing of the report; and the decision to submit the article for publication.

Author information

Authors and Affiliations

Authors

Contributions

AB, JS, DC, AK, NJS, SB, SX, WL, SS, and JRC contributed substantially to study conception and design, data collection, analysis, and interpretation. AB, JS, DC, AK, NJS, SB, SX, WL, SS, and JRC contributed to drafting the article or revising it critically for important intellectual content. All authors gave final approval for submission and publication.

Corresponding author

Correspondence to Aaron Broadwell.

Ethics declarations

Ethics approval and consent to participate

This study was conducted according to the Declaration of Helsinki and the International Committee on Harmonisation good clinical practices. The study protocol was reviewed by a centralized institutional review board (Copernicus Group, Approval QUI1-15-645); ethics committee review was not performed as there were no study sites outside the US. All patients provided written informed consent.

Consent for publication

Not applicable.

Competing interests

AB received consulting fees from Abbvie, Amgen, Aurinia, Celgene, Eli Lilly and Company, Janssen, Novartis, Pfizer, and Sandoz and speaker fees from Amgen, Abbvie, Eli Lilly and Company, Horizon, Janssen, Mallinckrodt, Novartis, Pfizer, Radius, Sanofi/Regeneron, and UCB. JS received speaker fees from AbbVie, Amgen, Boehringer Ingelheim, Bristol-Myers Squibb, ChemoCentryx, Eli Lilly and Company, Novartis, Mallinckrodt, Pfizer, Radius, Scipher, and UCB; and is on the scientific advisory board of Janssen. DC received consulting fees from Janssen and speaking fees from Janssen and Amgen. AK received stock options or bond holdings in a for-profit corporation or self-directed pension plan from Amgen, Gilead Sciences, Novartis, Pfizer, and Sanofi; received research grants from Janssen Research and Development; is an owner or partner for Amgen, Gilead Sciences, Inc, Novartis, Pfizer, and Sanofi; received consulting fees from AbbVie, Amgen, Boehringer Ingelheim, Flexion, Genzyme, Gilead Sciences, Janssen, Pfizer, Sanofi, SUN Pharma Advanced Research, and UCB; and speaker fees from AbbVie, Celgene, Eli Lilly and Company, Flexion, Genzyme, Horizon, Merck, Novartis, Pfizer, and Sanofi. SS received speaker fees from AbbVie, Eli Lilly and Company, Janssen, Novartis, Pfizer, Regeneron, Sanofi, and UCB and consultant fees from AbbVie, Eli Lilly and Company, Gilead, Janssen, Myriad, Novartis, Regeneron, Samsung, Sanofi, and UCB; owns stock in Amgen, Boston Scientific, Gilead, Medtronic, and Pfizer; and is a board member of National Psoriasis Foundation, and on the scientific advisory boards of Jubilant, Myriad, and Stelexis. JRC received grant/research support from AbbVie, Amgen, Bristol-Myers Squibb, CorEvitas, Eli Lilly and Company, Janssen, Myriad, Novartis, Pfizer, Sanofi, and UCB and consulting fees from AbbVie, Amgen, Bristol-Myers Squibb, CorEvitas, Eli Lilly and Company, Janssen, Myriad, Novartis, Pfizer, Sanofi, and UCB. NJS is an employee of Janssen Scientific Affairs, LLC, a wholly owned subsidiary of Johnson and Johnson, and owns stock in AbbVie, Gilead, Iovance, Jazz Pharmaceuticals, Johnson and Johnson, Novavax, and Viatris. SB, SX, and WL are employees of Janssen Research and Development, LLC, a wholly owned subsidiary of Johnson and Johnson, and own stock in Johnson and Johnson.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1

. Figure S1. Mean changes from baseline at Month 6 and Month 12 in CDAI score biologic-naïve and biologic-experienced patients receiving IV golimumab with and without methotrexate (observed data). Figure S2. Proportions of biologic-naïve patients in remission (CDAI ≤ 2.8) and with low disease activity (CDAI > 2.8 to ≤ 10), moderate disease activity (CDAI > 10 to ≤22), and high disease activity (CDAI > 22) at baseline and Months 3, 6, and 12 by treatment group (IV golimumab+methotrexate and IV golimumab without methotrexate). Figure S3. Proportions of biologic-experienced patients in remission (CDAI ≤ 2.8) and with low disease activity (CDAI > 2.8 to ≤ 10), moderate disease activity (CDAI > 10 to ≤ 22), and high disease activity (CDAI > 22) at baseline and Months 3, 6, and 12 by treatment group (IV golimumab+methotrexate and IV golimumab without methotrexate).

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Broadwell, A., Schechtman, J., Conaway, D. et al. Effectiveness and safety of intravenous golimumab with and without concomitant methotrexate in patients with rheumatoid arthritis in the prospective, noninterventional AWARE study. BMC Rheumatol 7, 5 (2023). https://doi.org/10.1186/s41927-023-00329-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s41927-023-00329-8

Keywords