Search
An online search using the most popular international search engine (Google.com) was conducted in November 2019 using three different search terms (“Treatment of the mouth in scleroderma”; “Treatment of the mouth in systemic sclerosis”; “Treatment of the mouth in scleroderma/systemic sclerosis”). Although fewer than 25% of people search beyond the first page of a Google search, we included the initial 100 websites in the study to ensure a thorough evaluation of the available online information [13]. The first 100 websites of each term were assessed for duplications and screened for any non-operative link. The following exclusion criteria were then applied; scientific articles, book reviews, websites with non-related content, non-working links, non-English language links, membership-based websites, promotional product websites, discussion groups, video feeds and online medical dictionaries.
The remaining websites were categorised as defined by Ni Riordain and McCreary (2009), based upon affiliation (commercial, non-profit organisation, university/medical centre and government), specialisation (exclusively or partly related to treatment of the mouth in scleroderma/SSc), content type (medical facts, clinical trials, question and answers and human interest stories) and content presentation (image, video and audio).
Quality assessment
The quality of the online material was assessed independently by two reviewers (IA and RNR) using the DISCERN instrument [14], and the Journal of the American Medical Association (JAMA) benchmarks for website analysis [15]. Training was provided for each reviewer prior to data extraction and any disagreement was resolved by a third reviewer (SRP). The presence of the HON seal was also recorded.
The DISCERN instrument developed and validated to examine the reliability of online content and its specific information on treatment options and overall quality scoring. This instrument was originally developed in the University of Oxford and it consists of 16 items. Questions 1–8 explore reliability, questions 9–15 refer to specific details of information on treatment with an additional question to allow an overall rating of the quality of the material being evaluated. Each question is rated on a numerical scale from 1 to 5 (1 = very poor, 2 = poor, 3 = moderate, 4 = good, 5 = excellent) [14].
The JAMA benchmarks were used to analyse the quality of websites. These benchmarks include clarity of authorship of medical content including (authors, contributors, affiliations and relevant credentials), inclusion of attributions (references and sources), statements of disclosure (ownership, conflicts and interest) and indication of currency (dates of content posted and updates) [15]. Health on the Net (HON) is a non-profit organisation established in 1995 to guide in the evaluation of the reliability of online information and sources in the medical field. The HON seal can be displayed on websites that comply with eight elements ranging from the indication of authors’ qualifications to clearly distinguishing advertising from editorial content.
Readability assessment
Readability is defined as the determination by systematic formulae of the reading comprehension level a person must possess to understand written texts [16]. The readability assessment was undertaken by using 4 different measures: Flesch Reading Ease Scores (FRES), Flesch-Kincaid Grade Level (FKGL), the Simplified Measure of Gobbledygook (SMOG) Index and the Coleman-Liau index (CLI). The FRES is based upon a formula that incorporates the average sentence length and the average number of syllables per word and the outcome score is a number ranging from 0 to 100. The higher the score - the easier the passage is to read [17]. For example, scores above 90 are considered easily understandable by an average 5th-grade student while scores between 60 and 70 are supposed to be easily readable for 8th and 9th-grade students. Finally, scores less than 50 represented an academic grade level and considered as difficult level of readability.
The FKGL improves upon the FRES and is based on the average number of words per syllables and sentence. The SMOG index takes into account the number of polysyllabic words per sentence and estimates the years of education a person needs to understand a piece of writing. The CLI relies on characters instead of syllables per word and sentence length.
All FKGL, SMOG index and CLI output a U.S. school grade level; this indicate the average student in that grade level can read the text. For example, a score of 7.4 indicates that the text is understood by an average student in 7th grade. However, it has been recommended that FKGL and SMOG index should be 5 or below to be easily comprehended.
Readability was assessed independently by two reviewers (IA and NE). To indicate the textual comprehension difficulty of a text, the following automated formula was used through a website (www.readabilityformulas.com).
Statistical analyses
Standard descriptive statistical analysis was performed by using SPSS (version 25) and tabulated as mean ± standard deviation of the mean.