- Research article
- Open Access
- Open Peer Review
Readability of patient information and consent documents in rheumatological studies
© The Author(s). 2016
- Received: 18 January 2016
- Accepted: 22 June 2016
- Published: 16 July 2016
Before participation in medical research an informed consent must be obtained. This study investigates whether the readability of patient information and consent documents (PICDs) corresponds to the average educational level of participants in rheumatological studies in the Netherlands, Denmark, and Norway.
24 PICDs from studies were collected and readability was assessed independently using the Gunning’s Fog Index (FOG) and Simple Measure of Gobbledygook (SMOG) grading.
The mean score for the FOG and SMOG grades were 14.2 (9.0–19.0) and 14.2 (12–17) respectively. The mean FOG and SMOG grades were 12.7 and 13.3 in the Dutch studies, 15.0 and 14.9 in the Danish studies, and 14.6 and 14.3 in the Norwegian studies, respectively. Out of the 2865 participants, more than 57 % had a lower educational level than the highest readability score calculated in the individual study.
As the readability level of the PICDs did not match the participants’ educational level, consent may not have been valid, as the participants may have had a limited understanding of what they agreed to participate in. There should be more focus on the readability of PICDs. National guidelines for how to write clear and unambiguous PICDs in simple and easily understandable language could increase the focus on the readability of PICD.
- Informed consent document
- Gunning's Fog Index
- Simple Measure of Gobbledygook
This study examines the readability of written patient information and consent documents (PICDs) used in rheumatology research. Ethical guidelines for medical research on human beings, which includes PICDs, were developed by the World Medical Association in Helsinki in 1964 . The guidelines has been revised several times, the last time in 2013 . The guidelines have also been implemented by National Ethic Committees [3–5].
Participants in medical research must give their informed consent before they can participate in a study. The consent must be free and informed, which means that the person should not experience any kind of pressure and should have the necessary information about the research which is to be conducted. This comprises knowledge about the purpose of the research project, duration, methods and procedures, types of expected results, and planned dissemination. Additionally, participants should have knowledge about the study’s potential benefits and risks, or negative consequences such as discomfort. Finally, participants should also receive information about confidentiality and data handling, in addition to knowledge about, that they can withdraw from the study at any time without any explanation and without influence on their clinical care . The information should be written and easy to understand and carry as little risk as possible of misunderstanding. In medical research on humans, PICDs are key requirements, which must be approved by ethical committees [3–5]. In 2012, the European Commission described the literacy level in the population as important. Poor literacy is a hidden problem in most European societies, where one in five young people and adults lacks basic literacy skills . Health literacy means the ability to read and understand basic health information, make appropriate health decisions, and take appropriate actions accordingly [7, 8]. People with poor health literacy will probably not understand a PICD if the text is too complex. In addition to the complexity of the text, participants’ reading ability is an important factor in ensuring their understanding of the written information. This must be taken into account when developing written materials like PICDs to ensure that patients are provided understandable information about the research study. This will allow them to make an informed decision about study participation .
Readability describes the number of years of schooling necessary to read and understand a text and can be measured by standardized instruments. Gunning’s Fog Index (FOG)  and the Simple Measure of Gobbledygook (SMOG)  are among the most commonly used instruments . SMOG has a high correlation with FOG (0.95–0.99) [13, 14]. Scores from the two indexes reflect the average years of schooling a person needs to be able to read and understand a text. A SMOG or FOG score of 5 is equal to 5 years of schooling. In the health literature, the recommended readability level ranges from grades 5 to 9 [15–17].
Often PICDs have higher readability level than the study population or recommended readability level [15, 16, 18, 19]. In a study with 40 PICDs in anaesthesiology research, the readability score for the FOG was 11.9. This score was significantly higher than the average literacy level of the Australian and New Zealand populations, where 44–46 % people have a literacy level below the required minimum to meet the complex demands of everyday life . In a Croatian study a readability score of 13.25 was found in informed consent documents in diagnostic and therapeutic procedures where 80 % of the population older than 15 years had less education . Another study of patient information leaflets on physiotherapy showed that only 11 out of 33 leaflets (33 %) met the recommended readability level of grade 9 .
In addition to the readability level, the content of the text are essential for understanding. Studies on how participants understand medical consent documents showed four sources of uncertainty: language, information about risks and hazards, the nature of the procedure, and the documents’ composition and format . This is congruent with the perspective of research ethical committee members which concluded that language, structure and format of the patient information sheet should be improved .
A review of interventions used to improve participants’ understanding of informed consent in research showed that one-to-one discussion and participant feedback with a study team member improved the participant’s understanding of the PICD. Other interventions such as multi-media and enhanced PICDs showed mixed levels of improvement in participant understanding . Latest reviews from 2012–14 on improving participants understanding of PICDs, showed that enhanced PICDs and extended discussions were most effective . For older participants interactive multimedia and written materials which were easy-to-read, increased the patients understanding of the consenting procedure . For people with low literacy the most effective strategy was talking one-to-one with a study member, but this result is based on a single study only .
In the literature, it is recommended to conduct readability tests on patient materials, but this is not yet common practice . During the European League Against Rheumatism Congress in 2010 a workshop regarding readability stressed the importance of the topic and it was recommended to be followed up. This study aims to document whether the readability of PICDs corresponds with reported education or schooling level of the participants enrolled in rheumatological studies.
We identified relevant clinical studies in the rheumatology field from each of the authors’ home countries (the Netherlands, Denmark, and Norway). Studies were selected in the researchers’ own languages to ensure that at least one with the native language reviewed all consent documents in each language.
An initial search was performed in PubMed in July 2014: #1: rheumato* OR arthritis OR arthro* OR psoriatic OR ankylosing spondylitis OR Bechterew OR lupus OR fibromyal* OR (inflammatory AND joint) OR chronic widespread pain #2: (Norway OR Norwegian; Holland OR Dutch; Denmark OR Danish) #1 AND #2. Additionally, we used filters to identify clinical trials that were performed within the last 10 years on humans; we searched for trials whose full texts were available. To include enough Danish rheumatological studies, we repeated the search for Danish studies in Cumulative Index to Nursing and Allied Health Literature (CINAHL).
Inclusion criteria were studies published during the past 10 years (2004–2014), study population ≥ 18 years of age, and a patient population with a specific rheumatic disease. Exclusion criteria were register-based studies, literature reviews, or other studies where no patient information or consent forms were used, studies from other countries than the Netherlands, Denmark, and Norway, and studies where no information about educational level (i.e. years of education, literacy level, and other proxies) was reported.
The titles and abstracts of the identified studies were screened according to the admission criteria. If a study was not excluded based on the title or the abstract, the full text was retrieved and assessed according to the admission criteria. All authors screened the retrieved titles, abstracts, and full text from their own countries.
The first authors of the included studies were contacted through e-mail. We contacted the authors of the most recent studies first. They were informed about the aims of our study and were asked whether they would share the patient information and consent forms used in their study. If the first author did not respond within a week, we contacted the last author who was considered to have overall responsibility for the study. For one Danish and one Norwegian study, the authors were contacted for information about participants’ education level, which was not stated in the study, although the authors knew it was collected [27, 28].
As we expected each author could possibly reuse parts of the text from previous PICDs, we included only one study from each first author unless we knew that different people had been responsible for the development of the PICD. When data from the same study was reported in several papers by different first authors, only the author from the most recent paper was contacted. We aimed to be able to retrieve patient information and consent forms from 10 studies from each country and included studies that fulfilled the selection criteria consecutively.
To assess the readability of the material, we applied the FOG  and SMOG  to each set of PICD. FOG grading was calculated based on a sample of approximately 100 words. The average number of words per sentence was calculated by dividing the number of words by the number of sentences in the sample. Words with three or more syllables were considered hard words. The number of hard words were counted. If polysyllabic words were repeated, only the first instance was counted. Three-syllable words made up of a two-syllable word with endings like -e, −er, −es, and -ing were omitted. The summed number of words per sentence and hard words were then multiplied with 0.4, and this gave the final score for each sample of approximately 100 words. We used the first 100 words and last 100 words of the text for scoring FOG 1 and FOG 2. SMOG grading was performed on 10 consecutive sentences at the beginning, 10 in the middle, and 10 near the end of the text. The number of words with three or more syllables was counted (including repetitions). The square root of the number of polysyllabic words (of the nearest perfect square) was calculated and the number ‘3’ was added to reach the final score . In case the text did not contain 30 sentences for the SMOG grading, we assessed 10 sentences from the beginning of the text, the following 10 sentences, and finally 10 sentences counted backwards from the end of the text. Thus, some sentences were included twice.
In case abbreviations were defined, the abbreviated version was not counted as a hard word no matter the number of syllables. Headings and subheadings in the text were calculated as sentences only in case they consisted of at least two words.
Each PICD was graded independently by hand by two of the authors. Each author scored all the PICDs from her own country and half of the PICDs from each of the two other countries. Next, the grades were compared. In case of discrepancy between the calculated score, the third author scored the material as well, and a final grading was reached by comparison and discussion.
Gunning’s Fog Index (FOG) and Simple Measure of Gobbledygook (SMOG) grade levels of studies and participants’ educational levels
No of participants (n)
Mean age and sex
Participants years of schooling or education level
FOG grade 1
FOG grade 2
Participants with lower educational level than required to understand the PICD % (n)
Evaluation of self-management programs
21 years (n = 9) and 22 years (n = 10), female 84 %
5 % (1)
47 years, female 100 %
5 % (15)
Randomised controlled study
62 years, female 65 %
54 % (107)
Randomised controlled study
48 years, female 92 %
61 % (50)
Randomised controlled study
In five groups: 39 to 43 years, female 92–100 %
87 % (176)
Qualitative focus group study
57 years, female 75 %
75 % (15)
Randomised controlled study
47 years, female 94 %
Primary (mean 7 years) 8
Secondary (mean 12 years) 128
Tertiary (mean 17 years) 22
86 % (136)
Randomised crossover study
67 years, female 100 %
Elementary school 5
High school 0
<4 years at University level 9
4–6 years at University level 2
>6 years at University level 4
70 % (14)
A prospective study
55 years, female 77 %
54 % (169)
A qualitative interview study
50 years, female 75 %
No formal education 2
Minimal education 7
Secondary education 4
69 % (9)
A qualitative interview study
44 years, male 100 %
Minimal education 1
Secondary education 9
77 % (10)
Randomised controlled study
63 years, female 87 %
<11 years 232
≥11 years 55
81 % (232)
Qualitative focus group study
58 years, female 59 %
Unskilled 3 Vocational 18 Undergraduate 6 Graduate 5
84 % (27)
A qualitative interview study
38–89 years, female 85 %
<10 years 4 10–12 years 7 > 12 years 20
36 % (11)
Randomised controlled study
Group 1: 68 and Group 2: 69 years, female 56 %
≤12 years 24 > 12 years 33
42 % (24)
Randomised controlled study
58 years, female 69 %
≤12 years 88 > 12 years 53
63 % (88)
Randomised controlled study
49 years, female 35 %
≤12 years 58 > 12 years 37
61 % (58)
Group 1: 57 years, female 89 %. Group 2: 51 years, female 53 %
≤12 years 87 > 12 years 65
57 % (87)
Randomised controlled study
50 years, female 68 %
Elementary school 13 High school 36 College/university 19
72 % (49)
Randomised controlled study
44 years, all female
≤10 years 21 11–13 years 49 > 13 years 53 Unknown 12
52 % (70)
Follow up study
55 years, female 87 %
9 years 36 ≤ 12 years 43 > 12 years 55
59 % (79)
A longitudinal study
46 years, female 58 %
≤9 years 38 10–12 years 118 > 12 years 125
56 % (156)
Randomised controlled study
M-group 48 years, N-group 51ys, female 43 %
Mean years of education 13 years (M-group) 12 years (N-group)
Cannot be calculated
Randomised controlled study
54 years, female 79 %
≤12 years 36 > 12 years 37
49 % (36)
Participants in 23 studies
57 % (1619g)
Frequency distribution, means, and standard deviation (SD) were calculated for the FOG and SMOG grades in the PICDs.
Through our search strategy, we retrieved 721 Dutch studies, 256 Danish studies and 192 Norwegian studies. After the screening, we included 24 studies, of which seven were Dutch, seven were Danish, and ten were Norwegian. Five of the included studies were qualitative, and 19 were quantitative. The number of participants in the studies ranged from 13 to 333. A total of 2972 participants were included in the 24 studies. In 23 of these studies, 1619 participants out of 2865 participants (57 %) had a lower educational or schooling level than the highest scores required for the FOG or the SMOG. The percentage of participants with lower educational or schooling level than the required FOG and the SMOG scores ranged from 5 % to 84 % in the included studies (Table 1). In one of the 24 studies, it was not possible to calculate the proportion of participants with a lower educational level than that suggested in the FOG/SMOG scores, because the results were presented as mean values  (Table 1).
Education or schooling levels presented in the included studies were often graded in two or three levels. The highest educational level was often described to be more than 12 or 13 years of schooling or as ‘university level’.
All PICDs were written in standard templates without illustrations, figures or other initiatives that could improve the readability. During the analysis, there were minor discrepancies, which were discussed. The reasons were often words that were overlooked, or that the pronunciations were unknown for one of the authors and thus the number of syllables could be interpreted differently.
Gunning’s Fog Index (FOG) and Simple Measure of Gobbledygook (SMOG) mean scores in the PICDs in the Dutch, Danish, and Norwegian studies
Number of values
This study investigated whether the readability of the PICDs corresponds with the reported education or schooling level of the participants involved in rheumatological studies in the Netherlands, Denmark, and Norway.
The results show that in all 24 studies, the PICDs had a higher readability level than recommended for health literature in general [15–17], and that in all studies there were participants with a lower educational or schooling level than the required readability level in order to be able to read and understand the PICD used in the study. This corresponds with findings from a new study of 522 information sheets for research studies in the UK .
In total 57 % of the participants had less education or schooling than required to read and understand the PICD based on the readability scores for the FOG and SMOG. When participants have a lower level of education than the readability of the PICD, it can lead to a ‘readability gap’, meaning that participants will not fully understand the text in the PICD or what they are agreeing to . This may cause participants to refuse to participate in the study or imply that they are participating in the study without giving valid consent. Participants’ understanding of the text can be improved by oral information or discussion with a study team member about the content in the PICD . It is also possible to increase PICDs’ readability by using short words and phrases, headings, plenty of space, bulleted lists, illustrations, and large type [20, 32, 33].
An individual’s health literacy depends on more than education; for example, it can depend on how familiar the reader is to the health care system and the presented information . When readers possess such knowledge, they may understand the text in PCIDs even when their educational level is lower than the readability level of the PICD in question. Conversely, when readers lack this knowledge, they may fail to understand the text even when they their educational level corresponds to the readability level of the PCID.
The high readability level of the PICDs in this study can possibly be explained by the regulatory requirements in national guidelines, which give more attention to the content of the PICDs than their readability. In Denmark and Norway, there are no stated requirements or focus on the level of readability of the PICDs. In the Netherlands, the ethical committees recommend to write the material on a schooling level of lower secondary school, which is approximately at an age of 11–12 years. The literature has shown that the readability level of PICDs in USA, Australia, South Africa, India and several European countries does not fit requirements so it is plausible to suggest that it is a common and worldwide problem [16, 19, 21, 35–38]. Our findings and how we have conducted our study can therefore be relevant for health professionals working with PICDs.
For all the included studies, we received the PICD from the first author, and several authors commented that the current study was an important piece of work that they would gladly support. In this study, it was difficult to find 10 Dutch and 10 Danish studies in which education or years of schooling was reported as part of the participants’ demographic data. This indicate a lack of focus on the participants’ educational of schooling level, although social inequality is an important topic.
Patients’ educational level may affect how participants manage their disease and treatment . Educational level can also influence the risk of disease , disease severity , psychological distress , health , involvement in healthcare , and mortality . Therefore, it may be advantageous for studies to report the educational or schooling level of their participants.
This study’s strength is that it examines PICDs from studies in three different European countries. It enables us to obtain knowledge on the readability of PICDs in a larger context. Using two different tests for readability also strengthens the study. Average values for FOG and SMOG in the PICDs are quite similar in our study, but within individual PICDs, there is a wide variation in scores. This shows that the complexity of PICD texts varies and that it is difficult to obtain a single readability measure of the PICDs. Readability tests like FOG and SMOG have been criticized for focusing on the document alone, and for not capturing medical terms and difficult words . A weakness of a readability test is that it does not take into account the context in which the document is used .
For assessment of PICDs and other written patient information, there are several online tools which involve FOG, SMOG, and other readability tests. These electronic tools make it possible to quickly obtain the value of a text’s readability. Online tools are not exact and provide somewhat different results than tests scored by hand. One such tool calculates a total readability score based on several tests (http://www.readabilityformulas.com). Another tool shows the readability for each test and provides a list of which sentences should be rewritten to improve readability (http://www.online-utility.org/english/readability_test_and_improve.jsp). This tool is used in an example of PICD where the same text is written with high and low readability, see Additional file 1.
In our study, several of the PICDs were only available on paper. We also wanted to calculate the readability level as exactly as possible. Therefore, all PICDs were scored by hand.
The studies included in this study use different ways of describing the participants’ level of education. Some use the number of years of schooling, grouped or as average. Other studies use low-medium-high educational classifications, and some studies describe the professions or occupations of the participants. This makes it difficult to compare results. In most studies, there will be more participants who have lower educational levels than PICDs’ readability level (Table 1), but the way participants’ education is presented in the studies makes it impossible to perform a more accurate calculation. For example, when the number of participants in a study is presented with an education level of more than 12 years and the highest readability score is 19.0, then only a few in this group will have a schooling level equivalent to 19.0 years. A standardized way of presenting participants’ educational level is needed.
The FOG and the SMOG were developed to score texts written in English, which together with Dutch, Danish, and Norwegian, belong to the Germanic language family . All four languages have common language characteristics. This is why we assumed we could use the FOG and SMOG procedure in all three languages even though they are not yet validated in the three languages. The endings which are considered to be easy to understand varies between languages. We cannot ignore the fact that the scores may be influenced by the languages.
More than 57 % of the participants in the included studies had a lower educational or schooling level than the readability level of the PICDs. This raises concerns about the validity of the participants’ consent. It would be relevant to develop national recommendations for the readability of PICDs. In addition, the involvement of patient research partners may help ensure a highly necessary focus on participants’ reading level. Further research is needed in other medical fields and in other countries.
Our findings show that there should be more focus on the formulation and form of PICDs. The development of guidelines for writing PICDs would ensure that focus is placed on the formulation of a text that is clear and understandable. The use of such a text along with oral information will enable participants to understand a study’s contents and consequences and give valid consent. This will also ensure the best possible recruitment in the studies.
DK, Denmark; FOG, Gunning’s fog index; NL, Netherlands; NO, Norway; PICD, Patient information and consent document; SMOG, Simple measure of gobbledygook.
We thank patient research partner Carole Lindsell for her valuable comments. We also thank all the following authors for sharing their PICDs with us: Judy Ammerlaan, Ercolie R. Bossema, Daniël Bossen, Nadia Garnefski Saskia van Koulil, Henrik Rindel Gudbergsen, Louise Linde, Katrine Bjerre Loeppenthin, Mette Madsen, Tine Mechlenborg Kristiansen, Elise Broman Bukhave, Vigdis Bruun-Olsen, Kjersti Grønning, Ingvild Kjeken, Mari Klokkerud, Hege Svean Koksvik, Ólöf Birna Kristjánsdóttir, Ida Løchting, Maria Dahl Mjaavatten, Yndis Annette Staalesen Strumse, and Heidi Andersen Zangi.
We want to thank the Norwegian Nurses in Rheumatology and Medac Pharma, Norway for supporting our project meeting in Copenhagen. We also want to thank the Hospital for Rheumatic Diseases, Lillehammer, Norway for supporting our project.
Availability of data and materials
The data used to support the conclusions of this article is publicly available as cited in the manuscript. For two studies, the first author was contacted for information about participants’ education level. This information is available from the first authors of the two articles.
BH designed the study. JP performed the literature search. All authors obtained informed consent documents and performed the analyses. BH, JP and YEH drafted the manuscript and all authors read and approved the final manuscript.
The authors declare that they have no competing interests.
Consent for publication
Ethics approval and consent to participate
This study used the results of previously published studies. The first author of the studies sent the PICDs to us.
- WMA Declaration of Helsinki - Ethical Principles for Medical Research Involving Human Subjects. In.; 1964.Google Scholar
- WMA Declaration of Helsinki - Ethical Principles for Medical Research Involving Human Subjects. In.; 2013.Google Scholar
- Det etiske råd [http://www.etiskraad.dk/]
- The National Committee for Medical and Health Research Ethics (NEM) [https://www.etikkom.no/en/In-English/Committee-for-Medical-and-Health-Research]
- Central Committee on Research Involving Human Subjects [http://www.ccmo.nl/nl]
- Vassiliou A: For 1 in 5 europeans, the world is hard to read. In.: European Comission High Level Group of experts on Literacy, Bruxells/Nicosia; 2012.Google Scholar
- Swearingen CJ, McCollum L, Daltroy LH, Pincus T, DeWalt DA, Davis TC. Screening for low literacy in a rheumatology setting: more than 10 % of patients cannot read “cartilage,” “diagnosis,” “rheumatologist,” or “symptom”. J Clin Rheumatol. 2010;16(8):359–64.View ArticleGoogle Scholar
- Wangdahl JM, Martensson LI. Measuring health literacy - the Swedish Functional Health Literacy scale. Scand J Caring Sci. 2015;29(1):165–72.View ArticleGoogle Scholar
- Rudd RE, Rosenfeld L, Gall V. Health literacy and arthritis research and practice. Curr Opin Rheumatol. 2007;19(2):97–100.View ArticleGoogle Scholar
- Gunning R. The technique of clear writing. New York: McGraw-Hill; 1968.Google Scholar
- McLaughlin G. SMOG grading - a new readability formula. J Read. 1969;12:639–45.Google Scholar
- Luk A, Aslani P. Tools used to evaluate written medicine and health information: document and user perspectives. Health Educ Behav. 2011;38(4):389–403.View ArticleGoogle Scholar
- Meade CD, Smith CF. Readability formulas: Cautions and criteria. Patient Educ Couns. 1991;17(2):153–8.View ArticleGoogle Scholar
- McInnes N, Haglund BJ. Readability of online health information: implications for health literacy. Inform Health Soc Care. 2011;36(4):173–89.View ArticleGoogle Scholar
- Chapman JA, Langridge J. Physiotherapy Health Education Literature. Physiotherapy. 1997;83(8):406–12.View ArticleGoogle Scholar
- Taylor HE, Bramley DE. An analysis of the readability of patient information and consent forms used in research studies in anaesthesia in Australia and New Zealand. Anaesth Intensive Care. 2012;40(6):995–8.Google Scholar
- Adams J, Chapman J, Bradley S, Ryan SJ. Literacy levels required to complete routinely used patient-reported outcome measures in rheumatology. Rheumatology (Oxford). 2013;52(3):460–4.View ArticleGoogle Scholar
- Malik L, Kuo J, Yip D, Mejia A. How well informed is the informed consent for cancer clinical trials? Clin Trials. 2014;11(6):686–8.View ArticleGoogle Scholar
- Vucemilo L, Borovecki A. Readability and Content Assessment of Informed Consent Forms for Medical Procedures in Croatia. PLoS One. 2015;10(9):e0138017.View ArticleGoogle Scholar
- Donovan-Kicken E, Mackert M, Guinn TD, Tollison AC, Breckinridge B. Sources of patient uncertainty when reviewing medical disclosure and consent documentation. Patient Educ Couns. 2013;90(2):254–60.View ArticleGoogle Scholar
- Villamanan E, Ruano M, Fernandez-de Uzquiano E, Lavilla P, Gonzalez D, Freire M, Sobrino C, Herrero A. Informed consent in clinical research; Do patients understand what they have signed? Farm Hosp. 2016;40(n03):209–18.Google Scholar
- Flory J, Emanuel E. Interventions to improve research participants’ understanding in informed consent for research: a systematic review. JAMA. 2004;292(13):1593–601.View ArticleGoogle Scholar
- Nishimura A, Carey J, Erwin PJ, Tilburt JC, Murad MH, McCormick JB. Improving understanding in the research informed consent process: a systematic review of 54 interventions tested in randomized control trials. BMC Med Ethics. 2013;14:28.View ArticleGoogle Scholar
- Sherlock A, Brownie S. Patients’ recollection and understanding of informed consent: a literature review. ANZ J Surg. 2014;84(4):207–10.View ArticleGoogle Scholar
- Tamariz L, Palacio A, Robert M, Marcus EN. Improving the informed consent process for research subjects with low literacy: a systematic review. J Gen Intern Med. 2013;28(1):121–6.View ArticleGoogle Scholar
- Polit D, Beck C: Nursing Research, Generating and Assessing Evidence for Nursing Practice, 8th ed. edn: Lippincott Williams & Wilkins. 2008.Google Scholar
- Mjaavatten MD, van der Heijde DM, Uhlig T, Haugen AJ, Nygaard H, Bjorneboe O, Kvien TK. Should anti-citrullinated protein antibody and rheumatoid factor status be reassessed during the first year of followup in recent-onset arthritis? A longitudinal study. J Rheumatol. 2011;38(11):2336–41.View ArticleGoogle Scholar
- Bukhave EB, Huniche L. Activity problems in everyday life--patients’ perspectives of hand osteoarthritis: “try imagining what it would be like having no hands”. Disabil Rehabil. 2014;36(19):1636–43.View ArticleGoogle Scholar
- Staalesen Strumse YA, Nordvag BY, Stanghelle JK, Roisland M, Winther A, Pajunen PA, Garen T, Flato B. Efficacy of rehabilitation for patients with ankylosing spondylitis: comparison of a four-week rehabilitation programme in a Mediterranean and a Norwegian setting. J Rehabil Med. 2011;43(6):534–42.View ArticleGoogle Scholar
- Ennis L, Wykes T. Sense and readability: participant information sheets for research studies. Br J Psychiatry. 2016;208(2):189-94.Google Scholar
- Gribble JN. Informed consent documents for BRCA1 and BRCA2 screening: how large is the readability gap? Patient Educ Couns. 1999;38(3):175–83.View ArticleGoogle Scholar
- Miller JE: How to Write Low Literacy Materials. Journal of Extension. 2001;39(1). http://www.joe.org/joe/2001february/tt2/php.
- Villaire M, Mayer G. Low health literacy: the impact on chronic illness management. Prof Case Manag. 2007;12(4):213–6. quiz 217–218.View ArticleGoogle Scholar
- Badarudeen S, Sabharwal S. Assessing readability of patient education materials: current role in orthopaedics. Clin Orthop Relat Res. 2010;468(10):2572–80.View ArticleGoogle Scholar
- Paasche-Orlow MK, Brancati FL, Taylor HA, Jain S, Pandit A, Wolf MS. Readability of consent form templates: a second look. IRB. 2013;35(4):12–9.Google Scholar
- Paasche-Orlow MK, Taylor HA, Brancati FL. Readability standards for informed-consent forms as compared with actual readability. N Engl J Med. 2003;348(8):721–6.View ArticleGoogle Scholar
- Minnies D, Hawkridge T, Hanekom W, Ehrlich R, London L, Hussey G. Evaluation of the quality of informed consent in a vaccine field trial in a developing country setting. BMC Med Ethics. 2008;9:15.View ArticleGoogle Scholar
- Pandiya A. Readability and comprehensibility of informed consent forms for clinical trials. Perspect Clin Res. 2010;1(3):98–100.Google Scholar
- Lee SJ, Kavanaugh A. A need for greater reporting of socioeconomic status and race in clinical trials. Ann Rheum Dis. 2004;63(12):1700–1.View ArticleGoogle Scholar
- Bengtsson C, Nordmark B, Klareskog L, Lundberg I, Alfredsson L, Group ES. Socioeconomic status and the risk of developing rheumatoid arthritis: results from the Swedish EIRA study. Ann Rheum Dis. 2005;64(11):1588–94.View ArticleGoogle Scholar
- Fitzcharles MA, Rampakakis E, Ste-Marie PA, Sampalis JS, Shir Y. The association of socioeconomic status and symptom severity in persons with fibromyalgia. J Rheumatol. 2014;41(7):1398–404.View ArticleGoogle Scholar
- Evers AW, Kraaimaat FW, Geenen R, Jacobs JW, Bijlsma JW. Longterm predictors of anxiety and depressed mood in early rheumatoid arthritis: a 3 and 5 year followup. J Rheumatol. 2002;29(11):2327–36.Google Scholar
- Dagfinrud H, Mengshoel AM, Hagen KB, Loge JH, Kvien TK. Health status of patients with ankylosing spondylitis: a comparison with the general population. Ann Rheum Dis. 2004;63(12):1605–10.View ArticleGoogle Scholar
- Brekke M, Hjortdahl P, Kvien TK. Involvement and satisfaction: a Norwegian study of health care among 1,024 patients with rheumatoid arthritis and 1,509 patients with chronic noninflammatory musculoskeletal pain. Arthritis Rheum. 2001;45(1):8–15.View ArticleGoogle Scholar
- Pincus T, Keysor J, Sokka T, Krishnan E, Callahan LF. Patient questionnaires and formal education level as prospective predictors of mortality over 10 years in 97 % of 1416 patients with rheumatoid arthritis from 15 United States private practices. J Rheumatol. 2004;31(2):229–34.Google Scholar
- Encyclopedia: Germanic languages. In:.http://www.encyclopedia.com/doc/1E1-GermancLan.html The Columbia Encyclopedia, 6th ed. 2014.
- Ammerlaan J, van Os-Medendorp H, Scholtus L, de Vos A, Zwier M, Bijlsma H, Kruize AA. Feasibility of an online and a face-to-face version of a self-management program for young adults with a rheumatic disease: experiences of young adults and peer leaders. Pediatr Rheumatol Online J. 2014;12:10.View ArticleGoogle Scholar
- Bossema ER, van Middendorp H, Jacobs JW, Bijlsma JW, Geenen R. Influence of weather on daily symptoms of pain and fatigue in female patients with fibromyalgia: a multilevel regression analysis. Arthritis Care Res. 2013;65(7):1019–25.View ArticleGoogle Scholar
- Bossen D, Veenhof C, Van Beek KE, Spreeuwenberg PM, Dekker J, De Bakker DH. Effectiveness of a web-based physical activity intervention in patients with knee and/or hip osteoarthritis: randomized controlled trial. J Med Internet Res. 2013;15(11):e257.View ArticleGoogle Scholar
- Garnefski N, Kraaij V, Benoist M, Bout Z, Karels E, Smit A. Effect of a cognitive behavioral self-help intervention on depression, anxiety, and coping self-efficacy in people with rheumatic disease. Arthritis Care Res. 2013;65(7):1077–84.View ArticleGoogle Scholar
- van Eijk-Hustings Y, Kroese M, Tan F, Boonen A, Bessems-Beks M, Landewe R. Challenges in demonstrating the effectiveness of multidisciplinary treatment on quality of life, participation and health care utilisation in patients with fibromyalgia: a randomised controlled trial. Clin Rheumatol. 2013;32(2):199–209.View ArticleGoogle Scholar
- van Eijk-Hustings Y, Ammerlaan J, Voorneveld-Nieuwenhuis H, Maat B, Veldhuizen C, Repping-Wuts H. Patients’ needs and expectations with regard to rheumatology nursing care: results of multicentre focus group interviews. Ann Rheum Dis. 2013;72(6):831–5.View ArticleGoogle Scholar
- van Koulil S, van Lankveld W, Kraaimaat FW, van Helmond T, Vedder A, van Hoorn H, Donders R, de Jong AJ, Haverman JF, Korff KJ, et al. Tailored cognitive-behavioral therapy and exercise training for high-risk patients with fibromyalgia. Arthritis Care Res (Hoboken). 2010;62(10):1377–85.View ArticleGoogle Scholar
- Gudbergsen H, Bartels EM, Krusager P, Waehrens EE, Christensen R, Danneskiold-Samsoe B, Bliddal H. Test-retest of computerized health status questionnaires frequently used in the monitoring of knee osteoarthritis: a randomized crossover trial. BMC Musculoskelet Disord. 2011;12:190.View ArticleGoogle Scholar
- Linde L, Sorensen J, Ostergaard M, Hetland ML. Gain in quality-adjusted life-years in patients with rheumatoid arthritis during 1 year of biological therapy: a prospective study in clinical practice. J Rheumatol. 2013;40(9):1479–86.View ArticleGoogle Scholar
- Loeppenthin K, Esbensen B, Ostergaard M, Jennum P, Thomsen T, Midtgaard J. Physical activity maintenance in patients with rheumatoid arthritis: a qualitative study. Clin Rehabil. 2014;28(3):289–99.View ArticleGoogle Scholar
- Madsen M, Jensen KV, Esbensen BA: Men’s Experiences of Living with Ankylosing Spondylitis: A Qualitative Study. Musculoskeletal care. 2014.Google Scholar
- Primdahl J, Sorensen J, Horn HC, Petersen R, Horslev-Petersen K. Shared care or nursing consultations as an alternative to rheumatologist follow-up for rheumatoid arthritis outpatients with low disease activity--patient outcomes from a 2-year, randomised controlled trial. Ann Rheum Dis. 2014;73(2):357–64.View ArticleGoogle Scholar
- Kristiansen TM, Primdahl J, Antoft R, Horslev-Petersen K. Everyday life with rheumatoid arthritis and implications for patient education and clinical practice: a focus group study. Musculoskeletal Care. 2012;10(1):29–38.View ArticleGoogle Scholar
- Bruun-Olsen V, Heiberg KE, Wahl AK, Mengshoel AM. The immediate and long-term effects of a walking-skill program compared to usual physiotherapy care in patients who have undergone total knee arthroplasty (TKA): a randomized controlled trial. Disabil Rehabil. 2013;35(23):2008–15.View ArticleGoogle Scholar
- Gronning K, Skomsvoll JF, Rannestad T, Steinsbekk A. The effect of an educational programme consisting of group and individual arthritis education for patients with polyarthritis--a randomised controlled trial. Patient Educ Couns. 2012;88(1):113–20.View ArticleGoogle Scholar
- Kjeken I, Bo I, Ronningen A, Spada C, Mowinckel P, Hagen KB, Dagfinrud H. A three-week multidisciplinary in-patient rehabilitation programme had positive long-term effects in patients with ankylosing spondylitis: Randomized controlled trial. J Rehabil Med. 2013;45(3):260–7.View ArticleGoogle Scholar
- Klokkerud M, Hagen KB, Lochting I, Uhlig T, Kjeken I, Grotle M. Does the content really matter? A study comparing structure, process, and outcome of team rehabilitation for patients with inflammatory arthritis in two different clinical settings. Scand J Rheumatol. 2012;41(1):20–8.View ArticleGoogle Scholar
- Koksvik HS, Hagen KB, Rodevand E, Mowinckel P, Kvien TK, Zangi HA. Patient satisfaction with nursing consultations in a rheumatology outpatient clinic: a 21-month randomised controlled trial in patients with inflammatory arthritides. Ann Rheum Dis. 2013;72(6):836–43.View ArticleGoogle Scholar
- Kristjansdottir OB, Fors EA, Eide E, Finset A, Stensrud TL, van Dulmen S, Wigers SH, Eide H. A smartphone-based intervention with diaries and therapist feedback to reduce catastrophizing and increase functioning in women with chronic widespread pain. part 2: 11-month follow-up results of a randomized trial. J Med Internet Res. 2013;15(3):e72.View ArticleGoogle Scholar
- Lochting I, Fjerstad E, Garratt AM. Illness perceptions in patients receiving rheumatology rehabilitation: association with health and outcomes at 12 months. BMC Musculoskelet Disord. 2013;14:28.View ArticleGoogle Scholar
- Zangi HA, Mowinckel P, Finset A, Eriksson LR, Hoystad TO, Lunde AK, Hagen KB. A mindfulness-based group intervention to reduce psychological distress and fatigue in patients with inflammatory rheumatic joint diseases: a randomised controlled trial. Ann Rheum Dis. 2012;71(6):911–7.View ArticleGoogle Scholar