Statement of ethics
This study was granted exemption from full ethical review by the University College Dublin Human Research Ethics Committee—Sciences (LS-E-20-38-Dorris). Anonymous surveys were used and no identifiable data was collected. Digital informed consent was obtained as part of the survey (Additional file 2).
Public and patient involvement
PPI contributors were full partners in this study, involved from conception, throughout design and conduct of the study, and in the analysis, preparation and revision of the manuscript.
Governance and team
The initiative was developed by one of the authors (ED) after consultation with peers, patient insight partners and Arthritis Ireland, Irelands national and largest charity dedicated to supporting people with arthritis, rheumatic diseases, and musculoskeletal disorders. The Arthritis Ireland Research Working Group acted as the governance committee. This consisted of two senior members of Arthritis Ireland, a patient member of Arthritis Ireland, the Chairs of Rheumatology at two major Irish Universities, consultant rheumatologists, a research scientist, and a professor in physiotherapy and vice president for research at an Irish University.
The project team consisted of a research scientist with experience in RPS. There were three patient insight partners, all of whom had previous PPI experience, and all of whom had personal experience in different RMD areas and were located in different geographical locations within Ireland. The remaining team comprised two academic physiotherapists: a postdoctoral researcher with experience in mixed methods and a professor of physiotherapy with significant experience in research policy.
Process framework
Viergever et al.’s checklist for health research priority setting was used to plan the RPS process [5].
Situational analysis and development of scope
The scope of the research prioritisation was defined via situational analysis of the Irish research context. This consisted of an analysis of peer reviewed and grey literature related to how research decisions are currently made in Ireland; who makes research decisions in Ireland; Who are the influential actors in research decision making; What policy and procedural documents are in place for decision making in RMD research in Ireland; and whether there is scope for improvement in research decision making practices. We used online search engines including PubMed and Google Scholar, organizational databases including Lenus Irish Health Repository and the Houses of the Oireachtas Library Digital Collections.
We analysed if there was a need for research priority setting for arthritis and rheumatic diseases in Ireland, and if so, what was the best way to approach it. This refined the prioritisation into five key areas: 1. The problem: research to measure the size of the health problem associated with RMDs in Ireland; 2. The cause: research to understand the causal agents and determinants of RMD and RMD-related health issues; 3. Solutions: research into what new interventions, tools, medicines, therapeutics are needed; 4. Policy & Practice: research to translate new interventions into policy and practice and understanding the barriers to delivering known interventions; 5. Health Impact: research to monitor and evaluate the effectiveness or health impact of an interventions or programmes.
Design workshop
This phase was aimed at defining the selection and analysis criteria for RPS. In later steps we used surveys to gather information. We needed to know if (1) different stakeholders understood research prioritisation differently and (2) how to define if we had successfully captured enough information in these surveys.
We held a workshop with mixed stakeholder attendees including people living with rheumatic diseases, family members, junior doctors, consultant rheumatologists, allied health care professionals, researchers, and charity advocates.
The workshop used a banquet style layout with a facilitator and a note taker at each table. The tables consisted of mixed stakeholders. All facilitators were briefed in advance and had a facilitators guide to direct the workshop. The tables were covered in white paper tablecloths, which attendees were encouraged to write on should they wish to leave any additional thoughts or feedback.
The workshop used two main techniques: Mind Mapping and MoSCoW. Mind Mapping allowed attendees to deconstruct complex topics into a graphical representation of constituent subtopics and related themes. MoSCoW is an acronym derived from four prioritization categories (Must have, Should have, Could have, and Won't have). The workshop was designed to generate ideas using the mind map technique, and to use MoSCoW to narrow the scope. This combined method produced a strategy for the RPS, with all stakeholders agreeing on a single, clear set of deliverables for each stage of a project in a transparent way.
The design workshop facilitated better understanding of what was expected of the priority setting exercise from different stakeholder, which allowed better design of the survey to meet these expectations. The design workshop also informed the analysis phase. There were particular themes of research that the workshop anticipated should be reflected in the survey responses if the survey was conducted effectively and survey dissemination reached the intended diverse stakeholders. These acted as performance indicators for the survey. For example, under the key area “The Problem: Research to measure the size of the health problem in Ireland”; through the workshop we anticipated input on questions related not just to prevalence, but to a more diverse interpretation including social, psychological and economic costs of RMD. If we did not receive a diversity of interpretation in the survey data at the interim analysis, the design of the survey and its dissemination would be reviewed.
Identification of research topics: survey design
A survey was designed for people to submit what topics/areas they thought needed to be researched to improve quality of life for people living with RMD in Ireland. The survey can be found in supporting materials. The survey was completed anonymously. It stated explicitly on the survey that “by participating in this survey you consent to the use of your anonymous data”.
The survey was reviewed and revised for clarity by a communications specialist, two people for whom English was not their first language, representatives of the key audience for the survey including people living with RMD (n = 4), family members (n = 2), clinicians (n = 3), allied health care professionals (n = 2), charity advocates (n = 2), and researchers (n = 2).
The survey was divided into two sections. Section one collected demographic information and section two collected the research questions the respondent would you like to see answered by RMD research. The survey collected the following demographic data on respondents: stakeholder category, age band, sex, ethnic/cultural background, province of Ireland lived in, and community.
There were five subsections in section two, and respondents could submit up to three questions in each subsection. The survey was only available in English. There was a mechanism to request a paper version of the survey and request assistance to help complete the survey via phone, email or post. The eSurvey was conducted using the Survey Monkey platform.
The survey was publicly available for anyone to complete. It was launched on the 06 April 2020 and open for six weeks. It was available on the Arthritis Ireland (Irish National RMD charity) website. It was sent to Irish professional organizations for rheumatologists, physiotherapists, nurses and other health care professionals. It was sent out to researchers investigating different aspects of rheumatic diseases in Universities across Ireland by email. Two reminder emails were sent, to encourage participation. The survey link was included in each email. The survey was also advertised on social media. No incentives were offered for completion. Unique survey respondents were measured via IP address. Cookies were not used.
Survey analysis
An IP check was performed to identify potential duplicate entries. No duplicate entries were identified. All surveys, including incomplete surveys, were analysed.
There were 2185 research questions submitted by 545 respondents to the survey. Many people will ask a similar question in different ways. Thus, the submitted questions were analysed and grouped into themes.
Scope
Every submitted question was assessed to determine if it was within scope of the RSP. Respondents had been asked not to refer to specific RMDs in their responses. The responses should relate to arthritis and RMD research generally. If a respondent had simply answered with a condition that answer is out of scope.
-
Real Examples:
-
❖ “Fibromyalgia”
-
> Out of scope
-
❖ “Are symptoms attributed to fibromyalgia in people with Psoriatic Arthritis really a distinct entity or should they be considered part of the diagnosis of Psoriatic Arthritis?”
-
> In scope. Classed as Impact of co-morbidities on diagnosis.
Data quality expectations
The submitted questions were compared with the criteria identified during the design workshop to determine if the expected diversity of submissions was achieved. The demographic data was also assessed to determine if a diverse group of respondents had been reached and if different communities, geographies and stakeholder types had been reached.
Grouping
The aim is to group similar questions together to identify the unique questions/areas. This was done by independent researchers, not in the field of rheumatic disease, to reduce potential bias in interpretation. All topics were analysed by two separate researchers, who had to agree that the questions, or topics, should be grouped together. Any disparity between the two researchers were brought to a third researcher and discussed for consensus. A member of the project team reviewed the research themes and analysis. The project them as a whole then reviewed and agreed upon all the thematic grouping of the submitted questions. This reduced the list down to thirty-eight major research themes.
Research topic ranking (prioritisation) survey
A voluntary open survey hosted on the survey monkey platform was designed. This consisted of two pages. The first page gave information and instructions about the study. Page two consisted of two questions: Question one was “. Choose and rank your top ten most important research questions from the list below. With 1 being the most important, 2 the second most important and so on.” Question two was a free text space that asked, “Is there anything else you would like to tell us?”.
Two versions of the survey were used. Both were worded the same. However, feedback on the first survey, which used ranking dropdown as the answer method, was that it was cumbersome to complete on certain mobile devices. Thus, survey version two had a mobile-friendlier interface by using a matrix format, with the 38 questions to be ranked as row items and the ranking options of 1st Priority, 2nd Priority, 3rd Priority……., 10th Priority, NOT a top priority for me or N/A as the column choices. The matrix version, survey version two, did not automatically prevent more than one question being given a first priority rank.
The survey was publicly available for anyone to complete. It was launched on the 11 November 2020 and open for twelve weeks. It was disseminated via the same channels as the previous survey. No incentives were offered for completion. Unique survey respondents were measured via IP address. Cookies were not used. Demographic data was not collected for this survey.
Analysis of the ranking survey
Each rank was given a weighted score.
Priority rank | 1st | 2nd | 3rd | 4th | 5th | 6th | 7th | 8th | 9th | 10th |
---|
Weighted score | 10 | 9 | 8 | 7 | 6 | 5 | 4 | 3 | 2 | 1 |
When data from survey version two was being recoded with the weighted scores, cases where more than one question had been given the same ranking were adjusted. For example, if a respondent gave three questions a first priority, each of these questions would be given a score of 8, and subsequent rankings adjusted (the second priority a score of 7, the third a score of 6 and so on). No one respondent could give a combined score of > 55 in total. This prevented skewing of the results by any one individual. The scores for each of the thirty-eight questions were summed to determine a ranking of the research topics by priority.
Internal validity check between survey versions
There was n = 272 respondents to survey version one and n = 192 respondents to survey version two. An IP check was performed to identify potential duplicate entries. No duplicate entries were identified. All surveys, including incomplete surveys, were analysed. Surveys were checked for duplicate entries. None were observed.
The rankings of each version were compared to each other and to the final/overall ranking. If the format did not make an impact, we would expect there to be similar rankings. There was good internal consistency between the versions. Overall, the consistency between versions was very strong, particularly within the top 10 (Additional file 1: Fig. S1).
Top 20 identified research topics: analysis of the literature
A search of peer-reviewed original research, systematic review, and evidence synthesis literature was performed using online search engines including PubMed and Google Scholar. Grey literature was not included. Literature was searched for inclusion of Irish subjects or data within the study population. Most research is condition-specific, and therefore the search included articles related to “arthritis” “rheumatic disease” “RMD” “rheumatic and musculoskeletal Disease” and “rheumatology”. The search was not time-bound. Sample references are all within 15 years of the search. The aim was to identify recent literature (or lack thereof) to determine that the top 20 ranked research questions had indeed not yet been answered yet from 1. an Irish Perspective and 2. An international perspective. That could be a well cited, high impact publication in the field that says "more research is required" or a systematic review or similar (Fig. 1).