Skip to main content

Table 2 Commonly used tools for ethical evaluation in HTA

From: Steps toward improving ethical evaluation in health technology assessment: a proposed framework

Tool Description Strengths Challenges References
A. Ethics literature review and appraisal
Methodologies for the search and retrieval of information on ethical issues in HTA Methodological approaches for the systematic retrieval of ethical information are discussed in two articles. These articles provide recommendations for good practice in selection of sources of ethical information, designing and executing ethics-specific search strategies, quality check of search results, and reporting information retrieval process. Encourages a separate literature search relevant to ethical questions, using the common retrieval framework for effectiveness assessments. The proposed search terms or strategies might not be sufficient for retrieval of all relevant ethical issues. Additional targeted searches might be necessary. [31, 32]
Tools for critical appraisal of empirical ethics research An article by Strech discusses the appropriate criteria for appraisal of empirical research required for ethical reasoning. He suggests four appraisal criteria related to the relevance of study questions, selected outcomes and measure, study design and generalizability of study results. Addresses some important challenges of considering empirical data in ethical analysis. No detailed guidelines or case studies are provided for how to apply the appraisal criteria. [33]
Mertz et al propose a set of structured quality criteria which can be used as a checklist to guide empirical ethics researchers and appraisers in the following four domains: research methodology, scientific and social relevance of the research project, interdisciplinary research practice, and research ethics. Designed based on an in-depth analysis of existing empirical ethics research and the opinion and experience of experts in the field of medical ethics. The practicality of the criteria is not tested in real life empirical ethics research practice. [34]
A tool for critical appraisal of normative medical ethics literature McCullough et al offer a tool to help clinicians (particularly obstetrician/gynecologists) in critical appraisal of normative bioethics literature. The tool incudes four questions about the focus of the study, validity and soundness of the study results, as well as their implication and usefulness in clinical practice. Designed based on the standards of critical appraisal of argument-based ethics and evidence-based medicine. Judgment about the validity and quality of ethical analyses and arguments requires some level of knowledge about ethical reasoning. This might not be an easy task for the target audience of the tool, i.e., physicians. [35]
Guidelines for systematic reviews of ethical evidence Strech e t al propose a 7-step approach for systematic reviews of empirical bioethics literature, The stepwise process involves definition of review questions, development and execution of search strategies, assessment of relevance and quality of identified studies, and analysis and presentation of data. Practical recommendations are provided for each step.
The application of the proposed approach is illustrated with an example.
The proposed search algorithms are not definitive and might need some modifications depending on the context and review questions.
Data analysis and presentation may require some level of knowledge and skills in synthesis of qualitative data.
[36]
Strech and Sofaer also offer a methodology for systematic reviews of non-empirical reason-based bioethics literature. Their model provides instructions for formulation of review questions and study selection criteria, identifying eligible literature, data extraction and synthesis, as well as presentation of the review results. Structured based on the common steps of a systematic review process.
Provides a detailed description of operational steps, and examples of how to apply the model in practice.
Performing a “systematic” review based on this model might be time-consuming.
This type of review requires some level of knowledge about ethical reasoning. be time-consuming and
[37]
B. Stakeholder analysis
Stakeholder Power/Interest grid This tool is a four quadrant matrix that classifies stakeholders in relation to the power that they hold and their level of interest in the technology. Power classification can be based on the ability of stakeholders to define or influence health care systems and services, change the way services are provided, or guide the public opinion. Highlights the importance of actors and interest groups in the technology The stakeholders interests, perceptions positions, and influence are subject to change [38]
Stakeholder SWOT A SWOT analysis (Strengths, Weaknesses, Opportunities, and Threats) can help in understanding the interests of key stakeholders, the actions they can take to support and the risks that they pose to implementation of the technology. Can be used to stimulate and organize thoughts and discussions in stakeholder analysis. Procedures for performing a SWOT-analysis are not clearly defined.
The analysis is prone to subjective biases of the assessors.
[39]
C. Public/stakeholder engagement
Exploring public values and preferences A methodology document published by the National Coordination Centre for Health Technology Assessment (UK) presents the results of a systematic review of qualitative and quantitative approaches to involving the public in in HTA. The document identifies and describes details of the techniques that can be used to obtain public preferences and makes recommendations regarding the use of different techniques. Some of the commonly used methods identified in this document are as follows:
− Quantitative techniques, including ranking (e.g., simple ranking, qualitative discriminant process, and conjoint analysis) rating (e.g., visual analogue scale) and choice-based (e.g., standard gamble, time-trade-off, discrete choice conjoint analysis and willingness to pay) methods.
− Qualitative techniques, including individual interviews, focus group discussions, Delphi technique, citizen’s juries, consensus panels, and nominal group techniques.
Summarizes and compares various techniques in a single document.
Uses pre-defined sets of criteria to evaluate methodological issues of different techniques (e.g., validity, reliability/reproducibility, generalizability, acceptability to respondents, or cost) identified methodologies.
Provides examples of how the techniques have been used in research practice.
No single best technique or group of techniques for public engagement is recommended by this document.
Users of the tool may require background knowledge and specific skills that enable them to choose and conduct an appropriate public engagement technique.
[40]
D. Identification and analysis of ethical issues
The Socratic approach (Hofmann’s guiding questions) This approach consists of 6 steps, whereof one step covers 7 main questions and 33 explanatory and guiding questions. This checklist is designed t for identification of and reflecting on ethical data throughout the HTA process, and for reflexive dialogue with stakeholders. Takes into account several ethical perspectives and analytical approaches.
Can be used by HTA practitioners who may be less familiar with ethical analysis.
Facilitates ethical analysis.
Users of the tool may require some level of ethical knowledge in order to use appropriate approaches to answer the questions. [41]
HTA core model’s assessment element cards (AECs) AECs describe the details of the information that is outlined by the basic units of the HTA Core Model (assessment elements). Each AEC provides information on the element, its importance and transferability for different applications (diagnostic, surgical, pharmaceutical or screening technologies), and appropriate sources of information and research methodologies to address the question defined by the element.
The ethical domain of the Core Model includes 19 elements related to the 19 ethical issues on the topics of beneficence/non maleficence (4 AECs), autonomy (4 AECs), respect for persons (3 AECs), justice and equity (3 AECs), legislation (2 AECs), and ethical consequences of HTA (3 AECs).
Designed to provide structured information required for answering the generic question defined by each assessment element.
Useful when producing HTA reports based on the HTA Core Model.
The way in which AECs should be used as a part of the assessment is not fully addressed in the model. [5]
Ethical matrix Ethical matrix is an analytical tool to aid ethical analysis of technological options The matrix uses a tabular format to identify ethical impact of a particular technology on different stakeholders. The table lists a set of prima facie moral principles, typically the four Beauchamp and Childress’s moral principles (autonomy, beneficence, non-maleficence, and justice), along one axis and different stakeholder groups along the other axis. Relevant facts and values are usually listed in each cell of the ethical matrix. Ethical matrix can be used either to identify ethical considerations around the technology or to quantify and compare the impact of the technology on different principles using semi-quantitative scores (e.g., ranging from -2 to +2). Facilitates ethical analysis by simplifying and structuring ethical discussion
Raises awareness of a wide range of ethical concerns
Helps researchers and decision-makers to avoid bias towards a specific moral principle.
Can be used in both expert-led and participatory/deliberative ethical evaluation processes.
May become large, complex and difficult to manage, when too many moral principles are listed or diverse groups of stakeholders are identified. [42]
Consequences table A summary table of consequences of using and not using a particular healthcare technology is recommended in the HTA core model as an open framework for performing ethical analysis. This table summarizes key benefits and adverse impacts of implementing of the technology or otherwise on various stakeholder groups.
A consequences table summarizing positive and negative impacts of the technology on all domains of HTA, along with references to the quality of their evidentiary sources, is also proposed as a part of the HTA core model’s reporting template.
Allows for highlighting key impacts of a particular technology on various domains of HTA.
Can be used by decision-makers to compare anticipated ethical issues around alternative technologies in relation to other domains.
Cannot be used as a substitute for careful ethical reflection [5]
E. Computerized support tools for aiding ethical analysis
EthXpert EthXpert is a computer program designed to help the user in summarizing and structuring ethical problems, describing potential inter-relations between the interests of different stakeholders, and analyzing the impact of alternative technologies on various stakeholders’ interests. Does not focus on a specific audience or any specific contexts. Therefore, can be applied to ethical evaluation in HTA. In some cases, the use of the these computer programs can be difficult and time consuming, especially when one needs to include all details about complex ethical problems, or too many different perspectives.
The use of the software may require investment in resources.
[29]
ETHOS Ethos is a computer program that provides a framework for organizing, storing and analyzing ethical information needed for problem solving or decision-making. The program allows for ethical analyses using different ethical theories and approaches. Illustrates the flow of data collection and analysis in a map format.
Enables the user to add or remove information through an iterative process.
[30]