Table 3 Results of coding validation

Questionnaires with completed response:

n = 13

All coding in questionnaire correct:

4/13 (31%)

Some coding in questionnaire incorrect:

9/13 (69%)

“Position taken by publication” chart coding correct:

8/13 (61%)

“Position taken by publication” chart coding incorrect*:

4/13 (31%)

“Position taken by publication” chart not filled out:

1/13 (8%)

Overall sum of quotations coded (in 13 questionnaires):

n = 98

 Quotations coding correct

77/98 (79%)

 Quotations coding incorrect**

21/98 (21%)

  Coding corrected by author (vs. sum coding incorrect):

19/21 (90%)

  Coding not corrected by author (vs. sum coding incorrect):

2/21 (10%)

Quotations coding incorrect: Type*** (vs. sum coding incorrect):

 Broad type of reason:

5/21 (24%)

 Narrow type of reason:

5/21 (24%)

 For ensuring PTA/against ensuring PTA:

9/21 (42%)

 Position taken by author:

2/21 (10%)

Countercheck “Position taken by publication” (vs. quotations):

 Verified correctness of “Position taken by publication” coding:

10/13 (77%)

 Verified incorrectness of “Position taken by publication” coding:

2/13 (15%)

 (“Position taken by publication” chart not filled out):

1/13 (8%)

Countercheck coding:

 Quotations coding correct (vs. overall sum of quotations):

84/98 (86%)

 Quotations coding incorrect (vs. overall sum of quotations):

14/98 (14%)

 Verified incorrectness of coding (vs. sum coding incorrect):

14/21 (67%)

  Broad type of reason

2/21 (10%)

  Narrow type of reason

1/21 (5%)

  For ensuring PTA/against ensuring PTA

9/21 (43%)

  Position taken by author

2/21 (10%)

 Verified correctness of coding**** (vs. sum coding incorrect):

7/21 (33%)