Skip to main content

Development of a core outcome set for informed consent for therapy: An international key stakeholder consensus study



300 million operations and procedures are performed annually across the world, all of which require a patient’s informed consent. No standardised measure of the consent process exists in current clinical practice. We aimed to define a core outcome set for informed consent for therapy.


The core outcome set was developed in accordance with a predefined research protocol and the Core OutcoMes in Effectiveness Trials (COMET) methodology comprising systematic review, qualitative semi structured interviews, a modified Delphi process and consensus webinars to ratify outcomes for inclusion in the final core outcome set. (Registration— Participants from all key stakeholder groups took part in the process, including patients and the public, healthcare practitioners and consent researchers.


36 outcome domains were synthesised through systematic review and organised into a consent taxonomy. 41 semi-structured interviews were performed with all consent stakeholders groups. 164 participants from all stakeholder groups across 8 countries completed Delphi Round 1 and 125 completed Round 2. 11 outcomes met the ‘consensus in’ criteria. 6 met ‘consensus in’ all stakeholder groups and were included directly in the final core outcome set. 5 remaining outcomes meeting ‘consensus in’ were ratified over two consensus webinars. 9 core outcomes were included in the final core outcome set: Satisfaction with the quality and amount of information, Patient feeling that there was a choice, Patient feeling that the decision to consent was their own, Confidence in the decision made, Satisfaction with communication, Trust in the clinician, Patient satisfaction with the consent process, Patient rated adequacy of time and opportunity to ask questions.


This international mixed-methods qualitative study is the first of its kind to define a core outcome set for informed consent for intervention. It defines what outcomes are of importance to key stakeholders in the consent process and is a forward step towards standardising future consent research.

Peer Review reports


Over 300 million operations and procedures take place around the world annually. All these procedures require the patient to give informed consent [1]. Consent, alongside shared decision making are cornerstones of Good Medical Practice as outlined by the General Medical Council (GMC) [2]. Consent is an integral part of medical and public health ethics and international law. Failings in the informed consent process can lead to dissolution of the clinician-patient relationship, complaints and occasionally litigation [3]. In the United Kingdom alone, National Health Service (NHS) England compensated patients £134.5 million between 2013 and 2018 in cases relating directly to deficiencies in the informed consent process [4].

Numerous studies exist evaluating the effects of various techniques (e.g. audio-visual/multimedia assisted consent) designed to improve informed consent. However, systematic review of this evidence has highlighted the heterogeneous nature of data in terms of study design and the choice of outcome measures which ultimately limits the generation of consensus on which interventions are most effective. Only one study, at high risk of bias, has attempted to measure informed consent as a unified concept [5]. It has been highlighted that trialists should recognise the complexity of the informed consent process by considering the overall patterns of outcomes and not simply use a measure of knowledge that has often been the case previously. Therefore, there has been a call for greater consensus on appropriate, validated and reliable tools for assessing the effects of interventions for the consent process to facilitate comparison between studies and to enable the meaningful synthesis of results [6,7,8,9]. In addition, it is unclear as to whether the outcomes that have been measured in previous consent research are the things that are of importance to key stakeholders during the consent process.

Core outcome sets (COS) aim to define a minimum set of outcomes that should be considered essential in the evaluation and reporting of studies of a particular intervention or condition [10]. There are well-defined guidelines with a growing evidence base to support the use of COS and the methodology employed to develop them [10,11,12,13,14,15]. Increasingly, researchers are inviting different stakeholder groups to identify the important outcomes for future evaluations of interventions in a variety of health areas such as cancer, rheumatology and otorhinolaryngology [16,17,18]. These activities have demonstrated that each stakeholder group may rate the importance of outcomes differently, reflecting their own priorities [15, 19]. Additionally, these priorities may not always align with the priorities of researchers who have traditionally been in control of the outcomes being investigated.

Knowledge and with that patient understanding have been the predominantly measured primary outcomes in consent research however, there are a range of other issues that may matter to stakeholders in the process.

The development of a Core Outcome Set (COS) may help researchers select and measure the most relevant outcomes that are most important to stakeholders involved in the process [20]. The primary benefit of using a COS allows the most important outcomes to be consistently measured and reported, thus allowing; comparisons between studies, the synthesis of data in meta-analyses and a reduction in reporting bias [10].

The aim of this study was to define a COS to evaluate interventions to improve consent for surgery and other invasive procedures, in adult patients (over 18 years) with adequate mental capacity to make their own consent decisions.


This study was developed in accordance with the guidance published in the Core Outcome Measures in Effectiveness Trials (COMET) Handbook and the Core Outcome Set-Standards for Development (COS-STAD) statement [10, 21]. The reporting of the study methods and findings has been undertaken in accordance with the Core Outcome Set-Standards for Reporting (COS-STAR) [13]. The protocol for the development of this COS was registered prospectively on the COMET database and published in full before work on the consensus building components of the project were undertaken [22, 23]. Prospective ethical approval for the study was obtained from the Office of Research Ethics Northern Ireland (RECA 17/NI/0234) and the Research and Development Office of the South Eastern Health and Social Care Trust (SET.17.36_SEHSCT). All methods were carried out in accordance with the ethical principles of the Declaration of Helsinki.

Study advisory group (SAG) and patient and public involvement (PPI)

The study advisory group was formed from members of the authorship list (LMC, SMcC, SJK, MC, WJC) as well as an experienced patient participant who has worked with the Research & Development department of the South Eastern Health and Social Care Trust on a wide range of clinical trials and other clinical research for several years.

The GRIPP2-Short Form Checklist (Table 1) outlines the Patient Involvement in Research in this study [24]; Our methods are reflective of some of the learning points from previous studies defining and evaluating novel procedures for involving patients and the public in COS research [25]. We defined “Public involvement”, “Public participation” and “Public engagement” according to those definitions from the INVOLVE advisory group members document [26].

Table 1 The guidance for reporting involvement of patients and the public (GRIPP2-short form) checklist

Recruitment of participants

This study captured the views of four stakeholder groups of which the future uptake of the COS is dependent upon. (Patients, Clinicians, Researchers who have conducted previous consent research and Academics working in bioethics). Solicitors and barristers who practice medical negligence law were also included in the process to provide a legal perspective and offer additional potential validity to the core outcome set.

Semi-structured interviews and Delphi consensus

  1. 1.

    Patients were recruited from a database of patients involved in qualitative research previously conducted by our group investigating the question ‘What is important to patients in the consent process?’ who had indicated that they would participate in future research. These patients had undergone emergency or elective surgery for a wide range of conditions, including day surgery and in-patient surgery for benign and malignant conditions. In addition, an advertisement was posted on the NIHR Peoples in Research website to recruit other willing patient participants.

  2. 2.

    Non-patient groups were approached through the research group’s professional networks, through social media promotion and email contact facilitated by professional bodies such as the Association of Surgeons of Great Britain and Ireland (ASGBI), the Association of Surgeons in Training (ASiT), the Royal College of Surgeons England (RCSEng), the American Society of Bioethics and Humanities and the Department of Legal Services Department of Health and Social Services Northern Ireland among others.

All stakeholder groups were invited to participate via email. A social media page was created by a member of the SAG on twitter© (@IconsStudy) to create direct engagement with professional patient organisations nationally and internationally.

Generating the survey information

The list of outcomes chosen for prioritisation in this Delphi survey were developed through a series of initial steps. Firstly, a systematic review of outcome reporting in existing trials was conducted to determine which outcomes had been measured previously [7]. Secondly, a systematic review of trial protocols of ‘as yet unpublished’ studies was undertaken to determine if outcome reporting in future trials was likely to be significantly different to that in existing studies. A systematic review of qualitative studies examining patients and clinicians’ attitudes to the consent process determined what mattered most to these stakeholders in the consent process and identified novel outcomes that could be measured. The outcomes identified from these reviews were organised into a consent outcomes taxonomy [27]. Additionally, semi-structured interviews with 41 stakeholders, including patients (n = 12), clinicians (n = 9), consent researchers (n = 10) and medico-legal lawyers (n = 10), explored the opinions on how the quality of the informed consent process should be determined and identified additional outcomes that had not been included in existing research. Semi-structured interviews were conducted by a single member of the SAG (LJC) and a reflective diary was kept. The list of outcomes was reviewed and organised into categories by the study advisory group. Duplicate outcomes were removed and outcomes with different names, but which captured the same phenomenon were amalgamated. A small number of outcomes that were not relevant to the consent process generally were removed from the prioritisation process. The final list of 36 potential outcomes was organised into six categories for prioritisation. These categories (domains) were knowledge, decision making, communication, trust, process, and patient characteristics. The wording of each outcome and explanatory text to describe the meaning of each outcome was developed with the help of the study advisory group’s patient and public representative. To ensure that the items included in the Delphi survey would be understood by all participants and, by patients, four “think out loud” cognitive interviews with lay participants were conducted. These cognitive interviews were undertaken in accordance with recognised methodology in this field [29]. Participants were asked to read aloud the outcome and explanatory text and to describe what they believed the outcome meant. Participants were observed while they read the outcomes to assess for physical cues that might indicate that they did not understand or were unsure such as, grimacing or appearing confused. Where a participant was not clear on the meaning of an outcome, it was discussed with them contemporaneously, and an alternative wording was developed. The wording of the items and their explanatory text was revised following each cognitive interview until no further amendments were deemed necessary. This process was designed to ensure items included in the Delphi would be understood by all participants, particularly patients. The full list of outcomes and accompanying explanatory text is included in Table 2.

Table 2 Final list of outcome domains and accompanying explanatory text for Round 1 of Delphi survey

The consensus process

An anonymous online Delphi survey was chosen for the consensus process (January-April 2019). The short timeline for completion between rounds aimed to maximise interest and engagement whilst minimising attrition. Summary videos for the Delphi Process ( were developed with consultation with the COMET POPPiE Group to provide an audio-visual summary of the study, help participants understand the reason for this research and to optimise recruitment. Participants were asked to rate the importance of each outcome measure in determining the quality of the informed consent process for surgery or another invasive procedure. Outcomes were rated using the Grading of Recommendations, Assessment, Development and Evaluations scale of 1 to 9. In the Delphi exercise, the scale was presented as 1–3 labelled ‘not important’, 4–6 labelled ‘important but not critical’ and 7–9 labelled ‘critical’ [30].

Participants were given the opportunity to suggest additional outcomes not included in the survey at the end of Round 1. These outcomes were reviewed by the study advisory group and duplicate recommendations were removed. Suggested outcomes that were like existing outcomes were excluded and suggested outcomes that were like each other were amalgamated. Participants who suggested an outcome were contacted by email to explain the fate of their suggestion and the reasons for the associated decision, to ensure their ideas had not been misinterpreted and to afford them a right of reply. The included additional outcomes were incorporated into Round 2 of the survey. All outcomes, despite their score, were carried forward to Round 2 to ensure that participants had the opportunity to review their scores for each outcome considering feedback from other participants. Additionally, the number of outcomes identified for prioritisation meant that carrying all outcomes forward to Round 2 would not be unduly onerous. Taking part in Round 1 was a pre-requisite for completion of Round 2. A further video was created to remind participants about the rationale for the survey and explain how Round 2 differed from Round 1. ( Round 2 provided graphical feedback of the distribution of each stakeholder group’s responses to all participants and reminded them of their previous scores. Presenting participants with feedback from all stakeholders appears to; improve consensus, reduce the variability of responses and improve agreement on those items to keep at the conclusion of the process [31]. Graphical feedback demonstrating the entire distribution of scores was chosen for this study as it was deemed the most easily interpreted form of feedback to demonstrate the spread of scores for each stakeholder group. No specific evidence currently exists to support one form of feedback over another [10].

The distribution of scores for each outcome was calculated as a percentage of total responses. Consensus that an outcome should be considered for inclusion in the COS was defined as 70% or more of total respondents rating it as critical by giving a score in the 7–9 range and no more than 15% rating it is as unimportant by giving it a score of 1–3. Conversely, an outcome would be considered for exclusion if 70% or more of respondents rated it as unimportant and no more than 15% rated it as critical. Additionally, if an outcome met the ‘consensus in’ criteria in three of the four stakeholder groups, but did not reach these criteria overall it was considered for inclusion in the COS. All other measures were thought to be equivocal [11, 32].

Patient focus group and consensus webinars

Consensus meetings for patient and other stakeholders were conducted separately. All stakeholders who completed both rounds of the Delphi process were eligible for the consensus webinars and were invited by email. However, many of the patients who had completed the online Delphi survey were not interested in attending such a meeting. For those patients who were prepared to attend a face-to-face meeting, it was not possible to find a mutually convenient date and time to obtain a critical mass. Additionally, the language and non-verbal communication used in such meetings can undermine or exclude patient participants [10]. Indeed, some COS developers recommend that professional and patient consensus meetings should always be conducted separately to allow patients to speak freely and to prevent contamination of their ideas [14]. As such, a patient focus group session was organised with patient participants from the Royal College of Surgeons England (RCSEng) Patient and Lay Group (PLG). The focus group discussion was convened at the Royal College of Surgeons, London on 4 April 2019 over a 1-h period. The PLG was established in 1999 and aims to ensure patient voices are adequately represented in the standards and policies of the RCSEng, raises areas of patient concern to the RCSEng, and advise the RCSEng about the optimal manner to engage patients. This meeting explored patients’ thoughts and perceptions regarding all outcomes prioritised in the Delphi process. Patients were not asked to vote on whether to include outcomes in the final COS. The aim, rather, was to determine patients’ views regarding each of these outcomes meeting consensus in the Delphi process and to use that feedback to inform the discussion during the consensus webinars. Participants were provided with a brief overview of the research and the rationale for the focus group session one week before the session. The patient focus group meeting opened with a brief presentation on the history and development of informed consent for surgery. The results of the online Delphi survey were briefly summarised before discussion was opened to the floor on the outcomes identified for discussion in the consensus meetings. The discussion was semi-structured and participants were asked to indicate their thoughts on all outcomes reaching unanimous ‘consensus in” across all stakeholder groups as well as each of the five borderline outcomes in turn and to highlight any outcomes that they believed were important but not included among the 11 prioritised outcomes.

Two separate consensus webinars were conducted. Participants who completed both rounds of the Delphi survey were invited by e-mail to attend the webinars to produce a consensus panel. Webinars were chosen over an exclusively face-to-face meeting to facilitate participation from a wide geographic area, without the time and financial constraints that international travel would have imposed. These were held on two separate days (one of the webinars was conducted in the morning (UK time) and the other in the afternoon) to facilitate participation of experts from different time-zones and to maximise international attendance and determine the concordance between the panels. An anonymised online voting system was used, and the results were broadcast immediately. In cases where there was no clear consensus result, a discussion was held and a revote was taken. Participants in webinar 2 were not advised how participants in webinar 1 had voted. The decision to include or exclude any outcome was determined by a simple majority across both consensus meetings. Each consensus webinar was recorded using the Adobe Connect software package which recorded the audio, video, online presentation, and online chat generated from the meeting. Voting from each round and salient points of discussion was noted contemporaneously.

Statistical analysis

DelphiManager Version 3.0 (University of Liverpool) was used to build and manage the Delphi survey. Descriptive statistics and the distribution of scores for each outcome were assessed using SPSS for Windows, Version 24. (IBM Inc., Armonk, NY, USA). Cohen’s kappa scores were calculated to assess the level of agreement between each Delphi round for all outcomes meeting the consensus criteria at the end of Round 2. This was to examine whether consensus might be overestimated because participants with minority opinions do not complete Round 2. Mean and standard deviations were also calculated for these outcomes and an independent t-test was performed to detect a difference in the mean scores entered by completers and non-completers between the rounds to evaluate the level of this attrition bias. Graphs for feedback to participants were produced using the R statistical package (R Foundation for Statistical Computing, Vienna, Austria).


A total of 164 participants completed all elements of Round 1, and a further 5 participants provided usable partial responses: with 125 (76.2%) complete responders in Round 2 and a further 3 participants providing partial responses to that round. Participants completing all elements of the survey came from eight countries and all four key stakeholder groups (Table 3). This included 53 (42.4%) patient participants who completed both rounds. Most respondents from all stakeholder groups originated from the UK.

Table 3 Demographics of Delphi participants completing all rounds

During Round 1, participants suggested 29 additional outcomes that they believed were not represented in the original survey. These are included in Additional file 1. Review and discussion among the study advisory group resulted in four of these additional outcomes being added to Round 2 (Table 4).

Table 4 Additional outcomes suggested by respondents of Round 1 and included in Round 2

Table 5 displays all outcomes scored in the Delphi process and shows which met the ‘consensus in’ criteria per stakeholder group in both rounds. In most cases, outcomes that met consensus criteria did so in at least two stakeholder groups, but consent technique, diagnosis, shared language of communication, trust in the hospital and trust in medicine were prioritised by patients only. Solicitors alone prioritised measured and self-rated patient knowledge in both rounds of the survey while consent researchers and bioethicists were the only group to prioritise whether the consent process had been conducted in an emergency or elective setting in Round 2, which the lawyer group had rated as critical in Round 1.

Table 5 All outcomes scored during the Delphi process

When the responses of participants completing both rounds of the survey were analysed, there were 527 of 4392 opportunities for change instances where a participant moved score categories between the two rounds. Five clinicians and four patients upgraded their rating of an outcome from unimportant to critical between rounds. Conversely, three clinicians and two patients changed their ratings from critical in Round 1 to unimportant in Round 2.

At the end of the Delphi process, 11 of 40 (27.5%) outcomes met the consensus criteria (Table 6). Of these, 6 outcomes met the “consensus in” criteria in each of the four stakeholder groups that took part in the survey, in addition to meeting the criteria overall.

Table 6 Outcomes meeting inclusion criteria following two rounds of an online Delphi survey and Cohen’s kappa coefficients to show the degree of agreement between rounds

These outcomes were:

  1. 1.

    Patient rated satisfaction with the quality and amount of information disclosed during the consent process.

  2. 2.

    Patient believing that there was a choice in the consent process.

  3. 3.

    Patient rated perception that the decision to consent or not to the procedure was their own.

  4. 4.

    Trust in the clinician.

  5. 5.

    Patient satisfaction with the consent process.

  6. 6.

    Opportunity to ask questions.

    There were a further 4 outcomes that while achieving the “consensus in” criteria overall, did not achieve this level of support in all four stakeholder groups. These were:

  7. 7.

    Confidence in the decision made.

  8. 8.

    Satisfaction with communication.

  9. 9.

    Trust in the hospital.

  10. 10.

    Adequacy of time for consent.

Additionally, one outcome, namely, patient desire for extra information after the consent discussion, met “consensus in” criteria in three of the four stakeholder groups but was just short of meeting the criteria when results were analysed overall (69% critical ratings versus 70% required for “consensus in” criteria).

Following discussion among the study advisory group it was determined that the 6 outcomes achieving unanimous consensus should be included in the final COS without the need for prolonged discussion at a consensus meeting. It was also agreed that the other 5 outcomes would be taken forward to the consensus meeting to determine if they would be included in the final COS.

There was substantial agreement in the responses of participants between Round 1 and Round 2 for most outcomes, as indicated by the kappa value > 0.6. The outcomes: “Adequacy of time”, “Patient feeling that there was a choice” and “Patient feeling that the decision to consent was their own” demonstrated moderate agreement between rounds. The proportion of respondents rating these measures as critical increased between rounds, accounting for the lower levels of agreement between rounds but increased agreement between stakeholders.

Comparing mean ratings for outcomes meeting the consensus criteria between participants completing both rounds and those completing Round 1 only does show evidence of attrition bias in the case of “Patient satisfaction with the quality and volume of information” (Table 7). In this case, non-responders rated this outcome significantly lower than responders (7.55 v. 7.98 p = 0.03). However, it should be noted that this outcome very clearly made inclusion criteria in both Rounds (Round 1: Critical = 82.8%, Unimportant = 0% and Round 2: Critical = 91.3% Unimportant = 0%) and the difference is small.

Table 7 Assessment of attrition bias between completers and non-completers of Round 2

Patient focus group and consensus webinars

The focus group meeting with the RCSEng Patient Liaison (PLG) Group comprised 20 patient representatives from throughout the United Kingdom (8 female and 12 male). Participants endorsed the inclusion of the six outcomes that had reached ‘consensus in’ criteria among all stakeholder groups during the Delphi process in the final COS.. The focus group discussions regarding the remaining 5 borderline outcome were presented to participants in the consensus webinars. The PLG reported that clear verbal communication, avoiding medical jargon and checking for understanding were integral to the consent process. Also, the use of good quality information leaflets would augment the consent discussion. Participants valued the patient rating on the adequacy of time and felt this was a better metric than simply an arbitrary time taken to obtain consent. While members of the group suggested that the need for additional information after the consent discussion and the level of trust patients have in the hospital were important variables, they believed these measures reflected a patient’s personality rather than the quality of the informed consent process. Overall, participants from the RCSEng PLG welcomed a change in the discourse around informed consent and were pleased to see that patient voices were adequately represented in the development of this COS.

The consensus webinars were hosted from the Ulster Hospital Dundonald, Northern Ireland on 6th June 2019, starting at 09:00 BST and 17th June 2019, starting at 14:30 BST. Participants across the two meetings included 12 clinicians, 2 medico-legal lawyers and 3 consent researchers. (Table 8) Most participants originated from the UK with 2 participants from the USA and one each from Australia and Canada.

Table 8 Participant characteristics in Consensus Webinars 1 and 2

Participants endorsed the six outcomes that had met unanimous ‘consensus in’ criteria during the Delphi process without further discussion. A summary of the outcome scoring from both webinars is included in Table 9. At the conclusion of both meetings, three of the five outcomes exceeded the 50% threshold to be included in the COS. These were; confidence in the decision made, satisfaction with communication and adequacy of time. This resulted in a final COS consisting of 9 core outcomes (Table 10).

Table 9 Outcome voting in both Consensus Webinars
Table 10 Final COS to evaluate interventions designed to improve the informed consent process for surgery


This is the first study that has attempted to standardise the important outcomes that should be measured in the informed consent process. It has captured the attitudes of patients, clinicians, lawyers and academics in the field of informed consent internationally. There was a high level of patient involvement throughout the process across various qualitative elements. Patients provided as many complete responses (n = 53 (42.4%) in the Delphi Survey as the clinician group. Collectively, the patients have experience of both emergency and elective surgery for a wide range of conditions, including minor day surgery and major surgery for benign and malignant conditions ensuring a diverse group of patients in terms of age, surgical procedures, and clinical outcomes because of surgery. Furthermore, the consent researcher / bioethicist and clinician groups came from a variety of geographic areas and diverse professional backgrounds and practices.

The prioritisation of outcomes using a Delphi survey and consensus meetings has defined a list of nine outcomes that reflect “what matters” most in the consent process. However, this COS does not preclude the measurement of other outcomes in future consent trials. Despite most of the research to date focusing on knowledge, recall and comprehension as primary outcomes this has not been reflected in the final COS. Measurement of patient knowledge either by objective or subjective means was only rated as critical by the lawyer stakeholder group. Stakeholders preferred a patient satisfaction rating on the quality and volume of information they were provided, as opposed to an attempt to prove whether the information was remembered or understood. The COS aligns with ethical, legal and professional standards and public opinion [33] of seeking to understand what matters to each patient, building trust, communicating effectively, promoting autonomous choice and allowing appropriate time for patients and clinician to make an informed shared decision about treatment. These elements combined highlight that achieving valid consent more likely an reflects a “process” rather than simply satisfying the standards of signing a consent form at a particular point in time.

5 of the outcomes in the final COS have not been reported in any randomised trial of interventions designed to improve the consent process. These are patient rated adequacy of time, opportunity to ask questions, patients feeling like they had a choice in the consent process, that the decision the patient made was their own and trust in the clinician. The fact that these outcome measures, despite being of critical importance to stakeholders in the consent process have not been reported in existing consent research may be because no validated tools exist for measuring some of them, researchers may have considered them but did not think them important to stakeholders, or they have not been considered at all. The remaining outcomes have been measured in existing trials.

This study adds to a paucity of literature on the development of a COS for the evaluation of communication interventions [34]. Additionally this study adds to a limited number of examples where COS developers have used simple statistics to demonstrate stability between rounds and to assess the level of attrition bias. The analysis in this study demonstrated evidence of attrition bias for only one of the eleven outcomes rated as ‘consensus in’. The protocol for the development of this COS was registered prospectively on the COMET database, the full protocol was published prospectively and the parameters for determining consensus were established a priori and mirror the standards used by other COS developers [14, 18, 35]. No deviations from the published protocol were necessary.

Overall, a key question of “what” should be measured in future trials to improve the informed consent process has been satisfied which paves the way to identifying “how” and “when” the outcomes should be measured. This work will follow the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) to identify a complementary core measurement set [36, 37].


Despite participation from stakeholders in multiple continents most participants included in this research originate from the UK with a significant proportion of the patient participants coming from a single NHS trust in Northern Ireland. As such, it is possible that these findings may not be generalisable to consent practices in other countries. While several clinicians, bioethicists and consent researchers involved in the studies included live and work in countries outside of the UK, the findings presented may have limited applicability in other settings. As a group of surgeons undertaking this work, it is possible that our professional background may have subjected the design and findings of the study to some unconscious bias. As is the case with all Delphi surveys, limitations of this study could be considered to be responder bias, reduced accountability of views on account of the anonymity afforded to participants and the potential for attrition bias. The outcome “trust in hospital” whilst meeting consensus in both round for patients did not receive any votes at the consensus webinar. This may be reflected by their absence from the consensus webinar component of the study. However, all stages of this research have been subject to clinical, methodological, and patient and public representative oversight. Throughout the qualitative components of this work, great efforts were made to ensure that participants’ voices were being fairly and accurately represented by repeating back our interpretation of what participants had said and agreeing the identification of themes from primary findings among the wider research group. Furthermore, as the study progressed it became clear that the themes identified in a relatively small number of participants during the semi-structured interviews were strongly reinforced by a larger cohort during the Delphi process.

Implications for clinical practice

Consent to undergo intervention applies to all patients in all specialities of medicine and surgery and therefore has the broadest potential application of any developed core outcome set. The COS from this study has the potential to influence consent practices on a global scale. Many of the outcomes included in the final COS have not been reported in existing consent trials yet these appear to be the aspects that are most important to the key stakeholders in the process. This presents an opportunity to redefine the direction of consent-based research. Regulatory bodies and guideline development groups e.g., National institute of clinical excellence (NICE) endorse the use of core outcome sets and investment to ensure future consent researchers adopt the COS when undertaking and reporting their research is necessary. This would facilitate comparisons between interventions and the synthesis of data while reducing the level of reporting bias. There remain several fields of consent and Shared decision making (SDM) research that may be able to adopt this COS for their own purposes including the trialling of novel communication interventions, Shared decision-making tools and Core Information Sets [8, 38].


We propose that this COS represents the minimum number of outcomes to report in all future studies of interventions designed to improve the quality of informed consent for invasive procedures. Future work is required to identify the best mechanism of assessment of each core outcome.

Availability of data and materials

All data generated or analysed during this study are included in this published article.



Association of Surgeons of Great Britain and Ireland


Association of Surgeons in Training


Core outcomes in effectiveness trials


Core outcome set


Core outcome set-standards for development


Core outcome set-standards for reporting


General Medical Council


INVOLVE national advisory group


National Health Service


National institute of clinical excellence


Patient liaison group


People and patient participation, involvement and engagement


Office of research ethics Northern Ireland


Royal College of Surgeons England


Study advisory group


Shared decision making


South Eastern Health and Social care trust


  1. Meara JG, Leather AJM, Hagander L, Alkire BC, Alonso N, Ameh EA, et al. Global Surgery 2030: evidence and solutions for achieving health, welfare, and economic development. The Lancet. 2015;386(9993):569–624.

    Article  Google Scholar 

  2. Good medical practice. General Medical Council. 2013.

  3. Department of H. Reference guide to consent for examination or treatment. Department of Health UK. 2012:11.

  4. Digital NHS. Hospital Admitted Patient Care Activity, 2016-17—NHS Digital. 2017.

  5. Friedlander JA, Loeben GS, Finnegan PK, Puma AE, Zhang X, de Zoeten EF, et al. A novel method to enhance informed consent: a prospective and randomised trial of form-based versus electronic assisted informed consent in paediatric endoscopy. J Med Ethics. 2011;37(4):194–200.

    Article  Google Scholar 

  6. Kinnersley P, Phillips K, Savage K, Farrell E, Morgan B, Stephens BL, et al. Interventions to promote informed consent for patients undergoing surgical and other invasive healthcare procedures. Cochrane Database Syst Rev. 2013;2013(7):1–247.

    Article  Google Scholar 

  7. McCain R. Informed consent for general surgery. Belfast: Queens University Belfast; 2018.

    Google Scholar 

  8. de Mik SML, Stubenrouch FE, Balm R, Ubbink DT. Systematic review of shared decision-making in surgery. Br J Surg. 2018;105(13):1721–30.

    Article  Google Scholar 

  9. Légaré F, Adekpedjou R, Stacey D, Turcotte S, Kryworuchko J, Graham ID, et al. Interventions for increasing the use of shared decision making by healthcare professionals. Cochrane Database of Syst Rev. 2018.

    Article  Google Scholar 

  10. Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, et al. The COMET handbook: version 1.0. Trials. 2017;18(Suppl 3):1–50.

    Article  Google Scholar 

  11. Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, et al. Developing core outcome sets for clinical trials: Issues to consider. Trials. 2012;13:132.

    Article  Google Scholar 

  12. Gillies K, Entwistle V, Treweek SP, Fraser C, Williamson PR, Campbell MK. Evaluation of interventions for informed consent for randomised controlled trials (ELICIT): protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey. Trials. 2015;16:484.

    Article  Google Scholar 

  13. Kirkham JJ, Gorst S, Altman DG, Blazeby J, Clarke M, Devane D, et al. COS-STAR: a reporting guideline for studies developing core outcome sets (protocol). Trials. 2015;16(1):373.

    Article  Google Scholar 

  14. Potter S, Holcombe C, Ward JA, Blazeby JM. Development of a core outcome set for research and audit studies in reconstructive breast surgery. Br J Surg. 2015;102(11):1360–71.

    Article  Google Scholar 

  15. Harman NL, Bruce IA, Kirkham JJ, Tierney S, Callery P, O’Brien K, et al. The importance of integration of stakeholder views in core outcome set development: otitis media with effusion in children with cleft palate. PLoS ONE. 2015;10(6):e0129514-e.

    Article  Google Scholar 

  16. Kirwan JR, Minnock P, Adebajo A, Bresnihan B, Choy E, Wit MD, et al. Patient perspective: fatigue as a recommended patient centered outcome measure in rheumatoid arthritis. J Rheumatol. 2014;34(5):1174–7.

    Article  Google Scholar 

  17. Harman NL, Bruce IA, Callery P, Tierney S, Sharif MO, O’Brien K, et al. MOMENT—Management of otitis media with effusion in cleft palate: protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey. Trials. 2013;14(1):70.

    Article  Google Scholar 

  18. Avery KNL, Katy Chalmers ÃA, Sara Brookes ÃT, Natalie Blencowe ÃS, Karen Coulman Ã, Katie Whale Ã, et al. Development of a core outcome set for clinical effectiveness trials in esophageal cancer resection surgery. Ann Surg. 2018;267(4):700–10.

    Article  Google Scholar 

  19. Keeley T, Williamson P, Callery P, Jones LL, Mathers J, Jones J, et al. The use of qualitative methods to inform Delphi surveys in core outcome set development. Trials. 2016;17(1):230.

    Article  Google Scholar 

  20. Clarke M. Standardising outcomes for clinical trials and systematic reviews. Trials. 2007;8(1):39.

    Article  Google Scholar 

  21. Kirkham JJ, Davis K, Altman DG, Blazeby JM, Clarke M, Tunis S, et al. Core outcome Set-STAndards for development: the COS-STAD recommendations. PLoS Med. 2017;14(11):1–10.

    Article  Google Scholar 

  22. COMET. Core Outcome Measures in Effectiveness Trials Initiative (COMET). Accessed 18th July 2022.

  23. Convie LJ, McCain S, Campbell J, Kirk SJ, Clarke M. Evaluating interventions for informed consent for surgery (ICONS): protocol for the development of a core outcome set. Trials. 2018;19(1):609.

    Article  Google Scholar 

  24. Staniszewska S, Brett J, Simera I, Seers K, Mockford C, Goodlad S, et al. GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. BMJ. 2017;358:j3453.

    Article  Google Scholar 

  25. Smith H, Horobin A, Fackrell K, Colley V, Thacker B, Hall DA, et al. Defining and evaluating novel procedures for involving patients in core outcome set research: creating a meaningful long list of candidate outcome domains. Res Involv Engagem. 2018;4(1):8.

    Article  Google Scholar 

  26. Tarpey M. What is public involvement in research? Public involvement/ participation/ engagement in research. National Istitute for Health and Care Research, Accessed 18th July 2022.

  27. Convie LJ, Carson E, McCusker D, McCain RS, McKinley N, Campbell WJ, et al. The patient and clinician experience of informed consent for surgery: a systematic review of the qualitative evidence. BMC Med Ethics. 2020;21(1):58.

    Article  Google Scholar 

  28. Katiri R, Hall DA, Hoare DJ, Fackrell K, Horobin A, Buggy N, et al. Redesigning a web-based stakeholder consensus meeting about core outcomes for clinical trials: formative feedback study. JMIR Form Res. 2021;5(8):e28878.

    Article  Google Scholar 

  29. Beatty PC, Willis GB. Research synthesis: the practice of cognitive interviewing. Public Opin Q. 2007;71(2):287–311.

    Article  Google Scholar 

  30. Guyatt GH, Oxman AD, Kunz R, Atkins D. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin. 2011;64(4):395–400.

    Article  Google Scholar 

  31. Brookes ST, Macefield RC, Williamson PR. Three nested randomized controlled trials of peer-only or multiple stakeholder group feedback within Delphi surveys during core outcome and information set. Trials. 2016;17(1):409.

    Article  Google Scholar 

  32. Wylde V, MacKichan F, Bruce J, Gooberman-Hill R. Assessment of chronic post-surgical pain after knee replacement: development of a core outcome set. Eur J Pain. 2015;19(5):611–20.

    Article  Google Scholar 

  33. IPSOS. Attitudes towards consent and decision making - Prepared for the General Medical Council. General Medical Council. 2018.

  34. Kaufman J, Ryan R, Hill S. Qualitative focus groups with stakeholders identify new potential outcomes related to vaccination communication. PLoS ONE. 2018;13(8):e0201145.

    Article  Google Scholar 

  35. O’Donnell CM, Black N, McCourt KC, McBrien ME, Clarke M, Patterson CC, et al. Development of a core outcome set for studies evaluating the effects of anaesthesia on perioperative morbidity and mortality following hip fracture surgery. Br J Anaesth. 2019;122(1):120–30.

    Article  Google Scholar 

  36. Prinsen CAC, Vohra S, Rose MR, Boers M, Tugwell P, Clarke M, et al. How to select outcome measurement instruments for outcomes included in a “Core Outcome Set”–a practical guideline. Trials. 2016.

    Article  Google Scholar 

  37. Prinsen CAC, Mokkink LB, Bouter LM, Alonso J, Patrick DL, de Vet HCW, et al. COSMIN guideline for systematic reviews of patient-reported outcome measures. Qual Life Res. 2018;27(5):1147–57.

    Article  Google Scholar 

  38. Main BG, McNair AGK, Huxtable R, Donovan JL, Thomas SJ, Kinnersley P, et al. Core information sets for informed consent to surgical interventions: baseline information of importance to patients and clinicians. BMC Med Ethics. 2017;18(1):29.

    Article  Google Scholar 

Download references


We thank, Richard Fallis for his assistance with search strategy development and Elaine Carson, Darren McCusker and Nichola McKinley for independent data extraction and synthesis during the systematic review process. Additionally, we would like to thank Laura Moore, Dr Patrick Donnelly, and our patient research partner Mr Tony Samworth from the Innovation, Research and Development team at the Ulster hospital for all their support. The core outcome set is registered on the COMET website The systematic review is registered on PROSPERO—CRD42018088878 Available from:


Not applicable.

Author information

Authors and Affiliations



LJC, SJK, MC, JC conceived the informed consent COS. LJC, SMcC, JC, SJK, MC contributed to the development of the protocol and systematic review. LJC recruited participants, conducted the semi-structured interviews, led the Delphi exercise, patient focus groups and consensus meetings. JMC wrote the first manuscript draft and all authors critically reviewed the paper to create the final manuscript.

Corresponding author

Correspondence to Joshua M. Clements.

Ethics declarations

Ethical approval and consent to participate

Prospective ethical approval for the study was obtained. All experimental protocols were approved by the Office of Research Ethics Northern Ireland (ORECNI) (Ethical approval reference—RECA 17/NI/0234) and the Research and Development Office of the South Eastern Health and Social Care Trust (SET.17.36_SEHSCT) aligning with relevant guidelines and regulations and ethical principles of the Declaration of Helsinki. Informed consent was obtained from all study participants.

Consent for publication

Not applicable.

Competing interests


Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1

. Additional outcomes suggested by Delphi participants during Round 1.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Convie, L.J., Clements, J.M., McCain, S. et al. Development of a core outcome set for informed consent for therapy: An international key stakeholder consensus study. BMC Med Ethics 23, 79 (2022).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: