Which patient reported outcome domains are important to the rheumatologists while assessing patients with rheumatoid arthritis?
BMC Rheumatology volume 3, Article number: 36 (2019)
Patient-reported outcomes (PROs) aid in rheumatoid arthritis (RA) management, but it is not well understood which measures would be most relevant to the rheumatologists for making treatment decisions.
We recruited rheumatologists nationally to participate in moderated structured group teleconference discussions using the nominal group technique. Participants in each group generated lists of the elements from patient’s history and signs that they use to make treatment recommendations for RA. Each participant then selected the three most important elements from the generated list. The results of each group were then combined and summarized.
Twenty-five rheumatologists participated in 4 groups (group size ranged from 4 to 8) and 150 available ranking votes across all groups. The statements generated across the 4 groups were categorized into 13 topics (including symptoms, physical function, comorbidities, social aspects, physical findings, response to treatment, treatment adherence, pain management, side effects, tests, access to care, contraception, and organ involvement), 10 of which received ranking votes. Symptoms received the highest ranking (46% of votes), followed by physical function (16%), and physical findings (13%). Among the unranked topics, social aspects had the highest number of statements (8 statements).
Rheumatologists highly valued patient-reported RA symptoms and physical function to inform their treatment decisions, even above objective data such as physical findings and test results. These results can guide the selection of validated PRO measures to assess these domains to inform the clinical care of patients with rheumatoid arthritis.
Determining the best therapeutic approach for patients with rheumatoid arthritis (RA) requires disease activity monitoring at each patient visit . Besides physician-assessed disease activity indices, patient-reported outcomes (PROs) should be incorporated to assess a patient’s health . It remains unclear which specific instrument would be helpful for the rheumatologist in making therapeutic recommendations for their patients with RA and in providing holistic care. This is despite having multiple options of PROs to choose from such as the Health Assessment Questionnaire (HAQ), Routine Assessment of Patient Index Data 3 (RAPID3), Rheumatoid Arthritis Disease Activity Index (RADAI), Rapid Assessment of Disease Activity in Rheumatology (RADAR), Patient Reported Outcomes Measurement Information System (PROMIS).
Unfortunately, too many choices can paralyze a clinician into indecision, and choosing too many may increase patient burden to provide data that is not useful for their care. Therefore, it is imperative to identify the truly meaningful PROs that physicians prioritize and can act upon once collected. By doing so, they are more likely to incorporate this information into their clinical practice. In this study, we aim to understand which clinical data rheumatologists consider most important to inform treatment recommendations for their patients with RA. These results will serve as a foundation to guide selection of PROs that are most relevant, timely, and actionable to measure the relevant data.
Physicians were recruited nationally by an email invitation to participate in one of 4 online nominal groups held in March and April of 2016. We sent email invitations to 325 rheumatologists who were members of the American College of Rheumatology (ACR). These members of the ACR are part of a manually curated list maintained by one of the authors (JRC). Physicians were eligible if they were rheumatologists and treated patients with RA. Invitees registered themselves to join the discussion. The investigators did not have a role in the selection of the participants for each group as invitees self-enrolled based on their availability and willingness to join the discussion. The maximum number of participants allowed to enroll per group were 12. We reached out to participants nationwide and those that participated were from the following states: Alabama, California, Florida, Montana, Pennsylvania, Puerto Rico, Tennessee, Texas, New Jersey, and New York. The University of Alabama at Birmingham Institutional Review Board approved this study. The need for consent was waived as the nominal groups were conducted online.
Nominal group sessions
We conducted online nominal group sessions, which utilizes semi-quantitative/qualitative methodology [3,4,5]. Nominal groups are moderated group discussions around a single question (described below). A trained moderator led each session, assisted by a scribe. Participants called into a conference call line and logged into a website designed to support nominal group sessions. The moderator described the purpose of the study, the nominal group procedure, read and displayed on the website question for discussion. The question was: “When seeing an established patient with RA in your office, from your personal perspective and professional practice, what elements of the patient’s history, signs, and symptoms are most helpful to you in making treatment recommendations for the management of RA?” This question was pilot-tested with three rheumatologists and modified based on their suggestions. We excluded these three rheumatologists from participating in the nominal groups.
The moderators for each of these nominal groups were the same (MSM and ROB). Each of the four groups then generated a list of statements in response to this question. Each participant first wrote down his or her responses to the question during a 5-min silent period. We asked them to contribute a single idea expressed as a phrase or brief sentence in a round-robin format. The contributions were captured verbatim by the scribe and displayed as part of a list of contributed statements on the participants’ screens. This process was repeated until the group had consensus that all significant ideas had been captured. We reviewed and discussed all listed statements among participants to ensure that they all had a shared understanding of the statements generated as part of the discussion. Once no new statements were generated, on each of the nominal groups, participants in each group were asked to rank the statements on the list in order of importance. Each participant could vote for the three most important statements on their group’s list from their own perspective, and this rank order was retained in the software. Each nominal group session lasted approximately 90 min.
The results of each group were then aggregated into common topics by the investigators (INM, MMS, and AJ). We grouped the topics using a consensus-based process. Each group’s results were then reviewed from the perspective of the topics derived from the aggregated data.
In order to evaluate the contribution of statements across broader topics and analyze the results across nominal groups, individual statements were categorized into topics by investigators. The topics grouping were based on concepts stated by the participants without a prior conceptual framework. We counted the number of nominal groups that discussed each topic, the number of statements in each topic and whether any statement within a topic received any ranking vote. We created a relative importance score for each statement generated during the nominal group session to summarize the findings. Each participant had a total of six votes for the question asked in the nominal group with 3 votes for the statement selected as the most important, 2 for the second most important, 1 for the third most important, and 0 for all other statements. The number of available votes per nominal group session was dependent on the number of participants per group (e.g. 10 participants = 60 votes; 5 participants = 30 votes). We summed the votes for each statement over the participants in each group divided by the number available votes both in in each group and across groups. This resulted in each statement receiving a weighted sum of ranking votes that could be combined across groups while accounting for the differential number of individuals in each group (percentage of total votes).
Twenty-five rheumatologists participated in four groups, with 4–8 participants per group. The total of available ranking votes across all 4 nominal groups was 150 votes. Since no new ideas emerged after four groups, there was no need for further recruitment of physicians. Forty-eight percent of the participants were older than 50 years of age, 64% were women, 76% were in private practices, and 24% were from academic centers.
“Symptoms” (defined as rheumatoid arthritis patient reported symptoms) was the topic with the highest number of votes (46%), followed by “physical function” (16%) (Fig. 1). Objective information, defined as physical findings (e.g. presence of joint swelling, tender joints) (13%) and tests (laboratory or X rays) (4%) captured 17% of the votes. Rheumatologists did not rank assessment of medication adherence highly (only 4% of votes). Among the three unranked topics (topics with statements that received 0 votes), “social aspects” (e.g. factors related to ability to work and provide for their family, family planning such as planning a pregnancy, or social support) had the highest number of statements (8 total) (See also Additional file 1: Figure S1). Table 1 shows the list of the statements that physicians made during the nominal groups and within each topic and possible PROs that can capture that information.
Of the 13 topics that emerged from the aggregated data, 10 received ranking votes with 9 of these topics been mentioned in either 3 or 4 of the groups (Table 1). Additional file 2: Table S1 shows the counts of 1st ranked, 2nd ranked, and 3rd ranked statements. The topics with the highest number of ranking votes within each group were “symptoms”. “Physical findings” received the second highest number of votes in group 1 and 3, and third highest in group 2 (Figs. 1 and 2). The percentage of votes in groups 2 and 4 were distributed across only 4 topics, with “symptoms” receiving a high proportion of the votes in each of these groups. Group 1’s priorities were distributed across 7 topics, and group 3’s across 9 topics, with “symptoms,” “physical findings”, and “physical function” receiving the most votes.
Patient-reported information most valued by the rheumatologists in our study were rheumatoid arthritis-related symptoms and physical function. Objective information such as physical examination findings was important as well. They enquired on other aspects of life such as family planning, social support, relationship with their spouse, and depression. Even though these “social aspects” did not receive ranking votes, it was the topic with the greatest number of statements among the unranked topics, suggesting that quantitative assessment of this topic may also be of value for practicing rheumatologists.
The most valued among symptoms, was patient-reported joint swelling, tenderness, and morning stiffness. These symptoms can be captured by using patient-reported instrument such as RADAI, which constitutes global disease activity in the last 6 months, current disease activity in terms of swollen and tender joins, pain, duration of morning stiffness and tender joints on a list . RADAR questionnaire contains global disease activity the past 6 months, current disease activity in terms of joint tenderness and swelling, pain, duration of morning stiffness, functional class and a tender joint list and can measure above mentioned statements as well . Morning stiffness duration is also a component of the RADAI and RADAR questionnaires and there is an ongoing effort to improve measurement of stiffness by OMERACT (Outcome Measures in Rheumatology) . Other elements of symptoms such as fatigue, sleep, and physical function can be quantified using PROMIS and HAQ, respectively. Additionally, inquiring about fatigue, depression, and sleep patterns is evidence of awareness among study physicians about these coexisting conditions. There is growing literature that RA is associated with comorbidities such as depression, anxiety, fatigue, and sleep disorders [19,20,21]. These problems may not be directly related to RA but are often associated with poor clinical outcomes and increased disease activity scores . Recognition of these comorbidities with PROs can help early management and improve patient’s quality of life.
Shared decision making between the patient and physician on the treatment of rheumatoid arthritis is advocated by the clinical practice guidelines . To accomplish the goals of shared-decision making, priorities from both, patients and physicians should be met. Previous studies show that there are differences between patients and physicians in terms of perspectives and priorities in management of RA. For example, discordance is noted between patient and physician global assessment of RA disease activity in approximately one third of cases . Physicians are more likely to rate RA disease activity lower than patients, possibly because they rely more on objective measures. However, patients have a more holistic view of their health that does not excessively partition RA disease activity distinctly from other related impact on their health and consider physical function, fatigue and health related quality of life as important factors [25, 26]. PROs describe a patient’s health status from their own perspective thereby providing valuable complementary information to an assessment focused only on RA and allow physicians to assess health in domains that patients consider important so as to provide more holistic care.
Despite the utilization of PROs in the research studies and trials, their incorporation into clinical practice is hampered by multiple challenges. A survey of 439 rheumatologists in the United States showed that they were not opposed to use quantitative measurement such as PROs while assessing patients with RA, but many clinicians felt that they lacked the time and electronic tools to do so efficiently . Patient-related factors such as health literacy, psychological stress, language proficiency may impact both the collection of and the utilization of PROs in real world settings . There is also a lack of understanding by many clinicians on how to effectively use the data over time, and to interpret changes in PROs. For example, analyses comparing RA treatment changes in response to moderate or high disease activity as measured by the RAPID3 vs. the clinical disease activity index (CDAI) suggests that the a high CDAI score prompt RA treatment changes more so than RAPID3 . The lack of psychometric assessment and validation of PROs in specific population (e.g. patients with high comorbidity burdens) is another limitation. Finally, while some PROs are commonly collected, such as the Patient global assessment (PGA), patients may feel that the PROs commonly measured in RA lack some relevance while more important health domains may be neglected (e.g ability to participate in meaningful social relationships; impairment in work-related activities etc.) . Our previous work showed that patients lost interest in completing PROs if the rheumatologist did not use the provided information . Thus, in the present study, we attempted to identify the information most valued by the rheumatologists to make treatment recommendations for RA so we can further determine which PROs could be utilized to measure it. We were able to generate a list of statements that we can now put together into a repository of PROs for electronic data collection. In this way, rheumatologists could utilize this data, as it is something they value and could also motivate patients to complete PROs because their physician is likely to act on it.
We will use the results of our hypothesis-generating study to devise a quantitative assessment of a representative sample of rheumatologists to confirm their priorities expressed by the sample in our study. Additionally, assessment of their preferences on how frequently and in what form (e.g. tablet, computer, paper, embedded in the electronic health record) they would prefer to collect this data will be needed. The strengths of this study were the use of the nominal group technique to collect semi-quantitative data. Studies suggest that this technique is suited for the development of multiple perspectives on an issue, elicitation of responses without strong opinions or personalities (a limitation of qualitative research such as focus groups), and prioritizing root causes of a problem [3, 32]. This method has demonstrated validity, and considers all participants’ views equally . A high degree of agreement on responses from the different groups in our study met our research needs and re ranking was not considered [34,35,36]. No new ideas emerged towards the end of discussion, which further contributed to the strength of the data. We recruited rheumatologists nationally, which reflects clinical practices throughout the nation and not only regionally.
The study limitations were that the nominal group technique does not allow development of full discussion and therefore the attitudes and beliefs of the participants were not fully elicited. It is possible that the distribution of physicians (and their preferences) could have affected the distribution of the topics. The characteristics of physicians were not identified until they had completed the session. We did not determine if either an academic physician, a private practitioner, early career, or senior career physician made particular statements. Therefore, we could not ascertain what factors may have influenced the voted topics as it was beyond the scope of nominal group session. The hypothesis generating stage of the research also precludes the generalizability to a larger population of rheumatologists, thus it is possible that a larger sample survey could result in different priorities. We note that this was not the goal of the research, lessening this concern, and that our study forms the foundation for future work in this important area.
Rheumatologists value certain aspects related to RA to inform their treatment recommendations that can be measured through PROs. The results of this study will be used to develop a survey for rheumatologists to assess the generalizability of the findings. This research will serve as a foundation for the development of a user-friendly PROs data collection platform for the physicians to aid treatment decisions in RA. Such tools can facilitate the data collection for RA registries, implementation of the treat to target guidelines, and optimization of clinical care for patients with RA.
Availability of data and materials
All data generated or analyzed during this study are included in this published article and its supplementary information files.
Activities of daily living
Anti-Cyclic Citrullinated Peptide
Clinical Disease Activity Index
Health assessment questionnaire
Interstitial lung disease
Multi-Dimensional Health Assessment Questionnaire
Non-steroidal anti-inflammatory drugs
Over the counter
Patient-Reported Outcomes Measurement Information System
Rheumatoid Arthritis Disease Activity Index
Rapid Assessment of Disease Activity in Rheumatology
Routine assessment of patient index data 3
Work productivity and Activity impairment questionnaire
Smolen JS, et al. EULAR recommendations for the management of rheumatoid arthritis with synthetic and biological disease-modifying antirheumatic drugs: 2016 update. Ann Rheum Dis. 2017;76(6):960–77.
Kirwan JR, Tugwell PS. Overview of the patient perspective at OMERACT 10--conceptualizing methods for developing patient-reported outcomes. J Rheumatol. 2011;38(8):1699–701.
Delbecq AL, Van de Ven AH. A group process model for problem identification and program planning. J Appl Behav Sci. 1971;7(4):466–92.
Jones J, Hunter D. Consensus methods for medical and health services research. BMJ. 1995;311(7001):376–80.
Delbecq AL, Van de Ven, A., A group process model for problem identification and program planning. 1971: Applied Behavioural Science.
Stucki G, et al. A self-administered rheumatoid arthritis disease activity index (RADAI) for epidemiologic research. Psychometric properties and correlation with parameters of disease activity. Arthritis Rheum. 1995;38(6):795–8.
Mason JH, et al. The rapid assessment of disease activity in rheumatology (radar) questionnaire. Validity and sensitivity to change of a patient self-report measure of joint count and clinical status. Arthritis Rheum. 1992;35(2):156–62.
Bacci ED, et al. Psychometric properties of morning joint stiffness duration and severity measures in patients with moderately to severely active rheumatoid arthritis. Health Qual Life Outcomes. 2017;15(1):239.
Bykerk VP, et al. Identifying flares in rheumatoid arthritis: reliability and construct validation of the OMERACT RA flare Core domain set. RMD Open. 2016;2(1):e000225.
(PROMIS), P.-R.O.M.I.S. Fatigue. 2019; Available from: http://www.healthmeasures.net/images/PROMIS/manuals/PROMIS_Fatigue_Scoring_Manual.pdf.
(PROMIS), P.-R.O.M.I.S. Sleep Disturbance. 2019; Available from: http://www.healthmeasures.net/images/PROMIS/manuals/PROMIS_Sleep_Disturbance_Scoring_Manual.pdf.
Joos E, et al. Reliability and reproducibility of visual analogue scale and numeric rating scale for therapeutic evaluation of pain in rheumatic patients. J Rheumatol. 1991;18(8):1269–70.
Fries JF, et al. Measurement of patient outcome in arthritis. Arthritis Rheum. 1980;23(2):137–45.
(PROMIS), P.-R.O.M.I.S. Physical Function. 2019; Available from: http://www.healthmeasures.net/images/PROMIS/manuals/PROMIS_Physical_Function_Scoring_Manual.pdf.
Pincus T, Swearingen C, Wolfe F. Toward a multidimensional health assessment questionnaire (MDHAQ): assessment of advanced activities of daily living and psychological status in the patient-friendly health assessment questionnaire format. Arthritis Rheum. 1999;42(10):2220–30.
(PROMIS), P.-R.O.M.I.S. Satisfaction with participation in discretionary social activities 2015; Available from: http://www.healthmeasures.net/images/promis/manuals/PROMIS_Satisfaction_with_Participation_in_Discretionary_Social_Activities_Scoring_Manual.pdf.
Reilly MC, Zbrozek AS, Dukes EM. The validity and reproducibility of a work productivity and activity impairment instrument. Pharmacoeconomics. 1993;4(5):353–65.
Craig ET, et al. Advancing stiffness measurement in rheumatic disease: report from the stiffness special interest group at OMERACT 2018. J Rheumatol. 2019;jrheum.181074.
Løppenthin K, et al. Sleep quality and correlates of poor sleep in patients with rheumatoid arthritis. Clin Rheumatol. 2015;34(12):2029–39.
Joharatnam N, et al. A cross-sectional study of pain sensitivity, disease-activity assessment, mental health, and fibromyalgia status in rheumatoid arthritis. Arthritis Res Ther. 2015;17:11.
Ranganath VK, et al. Comorbidities are associated with poorer outcomes in community patients with rheumatoid arthritis. Rheumatology (Oxford). 2013;52(10):1809–17.
Rathbun AM, Harrold LR, Reed GW. A prospective evaluation of the effects of prevalent depressive symptoms on disease activity in rheumatoid arthritis patients treated with biologic response modifiers. Clin Ther. 2016;38(7):1759–1772.e3.
Singh JA, et al. 2015 American College of Rheumatology Guideline for the treatment of rheumatoid arthritis. Arthritis Rheumatol. 2016;68(1):1–26.
Smolen JS, et al. Discordance between patient and physician assessments of global disease activity in rheumatoid arthritis and association with work productivity. Arthritis Res Ther. 2016;18(1):114.
Taylor PC, et al. A structured literature review of the burden of illness and unmet needs in patients with rheumatoid arthritis: a current perspective. Rheumatol Int. 2016;36(5):685–95.
Furu M, et al. Discordance and accordance between patient’s and physician’s assessments in rheumatoid arthritis. Scand J Rheumatol. 2014;43(4):291–5.
Curtis JR, et al. Routine use of quantitative disease activity measurements among US rheumatologists: implications for treat-to-target management strategies in rheumatoid arthritis. J Rheumatol. 2018;45(1):40–4.
Hirsh JM. The challenge and opportunity of capturing patient reported measures of rheumatoid arthritis disease activity in vulnerable populations with limited health literacy and limited English proficiency. Rheum Dis Clin N Am. 2016;42(2):347–62.
Yun H, Chen L, Xie F, Patel H, Boytsov N, Zhang X, Curtis JR. Do Patients with Moderate or High Disease Activity Escalate RA Therapy According to Treat-to-Target Principles? Results from the Acr’s RISE Registry [abstract]. Arthritis Rheumatol. 2018;70(suppl 10).
Ahlmén M, et al. Rheumatology outcomes: the patient's perspective. A multicentre focus group interview study of Swedish rheumatoid arthritis patients. Rheumatology (Oxford). 2005;44(1):105–10.
Navarro-Millan I, et al. Perspectives of rheumatoid arthritis patients on electronic communication and patient reported outcome data collection: a qualitative study. Arthritis Care Res. 2018;71(1):80–87.
Asmus CL, James K. Nominal group technique, social loafing, and group creative project quality. Creat Res J. 2005;17(4):349–54.
Harvey N, Holmes CA. Nominal group technique: an effective method for obtaining group consensus. Int J Nurs Pract. 2012;18(2):188–94.
Safford MM, et al. Reasons for not intensifying medications: differentiating “clinical inertia” from appropriate care. J Gen Intern Med. 2007;22(12):1648–55.
Singh JA, et al. Barriers to medication decision making in women with lupus nephritis: a formative study using nominal group technique. J Rheumatol. 2015;42(9):1616–23.
Levitan EB, et al. Barriers to Beta-blocker use and up-titration among patients with heart failure with reduced ejection fraction. Cardiovasc Drugs Ther. 2017;31(5–6):559–64.
We would like to acknowledge Anna Cornelius-Schecter for assisting preparing graphs and to Geyanne Lui for copy editing the manuscript. More importantly, we would like to thank the collaborators that assisted pilot testing the script for the nominal groups and the participants for their time and contribution to this study.
This project was funded by the Innovative Research Award from the Rheumatology Research Foundation (RRF). The RRF was not involved in the study design, data collection, analysis, or interpretation of the data; INM is funded by NIAMS K23AR068449; MMS was supported by K24 HL111154 from NHLBI. JC was supported by NIAMS P60 AR064172. JRC and MMS received salary support from Amgen.
Ethics approval and consent to participate
The need for consent was waived as the nominal groups were conducted online. The University of Alabama at Birmingham Institutional Review Board approved this study (IRB# 1707018365).
Consent for publication
Not applicable as data presented are summary data and no individuals are identifiable in this study.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Figure S1. Topics that Emerged During Physician Nominal Groups and the Respective Number of Participants that Voted for the Statements within Each Topic as the 1st, 2nd, and 3rd most important. (DOCX 42 kb)
Table S1. Statements with the respective distribution of votes as the 1st, 2nd, and 3rd most important. (DOCX 19 kb)
About this article
Cite this article
Jagpal, A., O’Beirne, R., Morris, M.S. et al. Which patient reported outcome domains are important to the rheumatologists while assessing patients with rheumatoid arthritis?. BMC Rheumatol 3, 36 (2019). https://doi.org/10.1186/s41927-019-0087-2