Skip to main content

Performance of IRBs in China: a survey on IRB employees and researchers’ experiences and perceptions

Abstract

Background

Performance evaluation is vital for IRB operations. As the number of IRBs and their responsibilities in reviewing and supervising clinical research grow in China, there is a significant need to evaluate their performances. To date, little research has examined IRB performance within China. The aim of this study was to ascertain the perspectives and experiences of IRB employees and researchers to (1) understand the current status of IRBs; (2) compare collected results with those of other countries; and (3) identify shortcomings to improve IRB performance.

Methods

This study was conducted in China from October 2020 to September 2021, using an online survey with the IRB-researcher assessment tool-Chinese version.

Results

757 respondents were included in the analysis and classified into IRB employees, researchers, or those who are both IRB employees and researchers. Overall, the score for an ideal IRB was significantly higher than that of an actual IRB. Compared to the US National Validation study, Chinese participants and American participants both agree and differ in their perspectives on the most and least important ideal items.

Conclusion

This investigation provides a benchmark of the perceived performance of actual IRBs in China. IRBs in China can be precisely adjusted by targeting identified areas of weakness to improve their performances.

Peer Review reports

Background

All research (i.e., biomedical, behavioral, social science, and epidemiological research) that involves human subjects should be submitted to an institutional review board (IRB) for periodic review, including prior to conducting a study [1]. An IRB is an independent body that operates to ensure the protection of the rights, safety, and well-being of human subjects [2]. While the functions of IRBs around the world and across institutions may differ slightly, their general responsibilities in ethical review and procedures are the same [3, 4]. In general, ethical reviews are legally mandated and must be obtained ahead of the conduct of human subjects research [5].

Performance evaluation can be an effective tool in identifying areas of vulnerability in an IRB to subsequently improve its quality and efficiency [6]. Evaluating the performance of an IRB though remains a challenging task [7]. Perspectives from IRB members and researchers are fundamental then to understanding IRB performance, and several studies have targeted this concept [8]. To date, IRB performance has been examined in more than 14 countries worldwide such as in Europe and the Americas [9,10,11,12,13,14,15], Africa [16,17,18,19,20,21,22], and Asia [23,24,25,26]. One study conducted in low to middle-income countries (LMICs) focused on IRB chairpersons to evaluate whether the IRB’s functions complied with recognized international standards [27]. Another study in Myanmar chose IRB representatives to assess the structures and processes of Research Ethics Committees (RECs) at medical institutions [24]. Investigators in Jordan evaluated the awareness and attitudes of healthcare investigators toward the structure and importance of IRBs in that region [28]. Research conducted in Singapore targeted biomedical researchers and support staff to gain a general understanding of the perceptions surrounding IRB functions and characteristics [29]. Lastly, recent research in China has indicated that IRBs routinely face issues related to performance, including the absence of supervision, unclear review criteria, the limited competency of ethics committees (including inadequate knowledge of ethics), and poor follow-up of reviews [30].

Before 2019, China’s IRBs operated and conducted ethical reviews based on regulations and guiding documents issued by the National Health Commission (NHC) and National Medical Product Administration (NMPA)—formerly the China Food and Drug Administration (CFDA) [31, 32]. Since 2019, IRBs in China have been confirmed with legislative statuses, resulting in the strict enforcement of ethical reviews of clinical research. Subsequently, IRBs have gradually gained their legal status through various laws, such as The Civil Code which states that whenever clinical studies are necessary for the development of new drugs, medical devices, or new prevention and treatment methods, they should be approved by the IRB [33]. Additionally, according to the Chinese Biosecurity Law, clinical research on new biomedical technologies should also be approved by ethical review [34]. The Law of the People’s Republic of China on Basic Medical and Health Care and Health Promotion requires clinical trials of drugs and medical devices and other medical research to be conducted in accordance with standards in medical ethics and approved by ethical reviews [35]. China’s Drug Administration Law further requires that clinical trials of drugs be implemented in accordance with ethical principles, with study proposals formulated for examination and approval by an IRB [36]. Finally, the Physician Law of the People’s Republic of China states that physicians who carry out clinical trials of drugs and medical devices and other types of clinical studies must abide by medical ethics and seek approval by an IRB following the law [37].

In 2018 and 2019, NMPA and NHC changed previous qualifications for the certification of clinical trials for medical devices and drugs, respectively, into a registration policy. Specifically, the basic condition that institutions conducting clinical trials should meet for registration is having an IRB [38, 39]. As a result, many IRBs were established in newly registered research institutes. As of February 2, 2022, 1,214 and 1,108 institutes in drug and medical device trials, respectively, have been registered [40]. Performance evaluations can standardize and strengthen IRBs and their review processes [41]. Therefore, it is imperative to evaluate IRBs, so that IRBs can be improved based on empirical data. Our study included IRB employees (including IRB members and staff) and investigators since IRB employees and researchers have the most direct perspectives on IRB performance, making this study distinct from past research. For instance, Keith-Spiegel et al. sampled researchers almost exclusively [42], while Markus K. Labude et al. investigated Singaporean biomedical researchers/research support staff [29]. Jonathan C. Reeser’s study population included research coordinators, IRB members, as well as investigators [43]. Daniel E. Hall’s study population consisted of principal investigators/project coordinators and IRB members/staff [44]. While Tiffany Chenneville’s study in India included IRB members and faculty investigators [45], similar to our study, her study population mainly drew from medical colleges, whereas our participants came from both medical and health institutions, with a small number from medical schools. Due to differences between studies, including the separate aims and purposes, it can be difficult to make a horizontal comparison. However, the general perspectives on IRBs’ performance can help to understand and distinguish the gaps between different countries [44].

Currently, there are no empirical studies in China evaluating these groups’ perspectives and experiences within IRBs. Our study aims to use the IRB-Researcher Assessment Tool Chinese Version (IRB-RAT-CV) to investigate IRB employees and researchers in China. In addition to understanding perspectives on the characteristics and functions of IRBs, this study also facilitates the comparison of these findings with studies conducted in other countries, so as to comprehend identified differences and improve IRB operations.

Methods

Study design

From October 2020 to September 2021, a national cross-sectional study was conducted to assess the performance of IRBs in China. Participants were recruited using WeChat through convenience sampling. In China, residents primarily communicate through this free messaging and calling app. It is the most popular communication app in China and has approximately 549 million active users per month [46]. For this study, we targeted WeChat groups, meaning conversations between more than two users sharing several common characteristics or goals; in this case, we aimed for WeChat groups consisting of IRB members and researchers [47]. Every year, China holds nationwide academic exchange and training activities for IRBs and researchers. IRB staff and researchers from different regions of the country participate in the training and establish WeChat groups from these activities. Thus, several groups exist on WeChat that consist of IRB staff and researchers with different professional backgrounds from various regions of China. An online questionnaire, accessible through a link, was distributed to these WeChat groups. This method allowed us to recruit IRB-related staff and researchers from different areas across China.

The questionnaire was divided into three parts: informed consent, a basic demographic sheet, and the IRB-RAT-CV. The first part included the informed consent form, which described the purpose as well as the risks and benefits of the study. If participants clicked “Agree”, the web page would directly jump to the next portion with the demographic sheet and IRB-RAT-CV questionnaire.

The participants of this study were IRB employees and researchers from medical colleges and hospitals nationwide, with IRB employees including chairpersons, vice chairs, members, and staff, and researchers including those who have undertaken at least one biomedical research project involving human participants. Participants could designate their role as one or more of the positions mentioned above, and selections were categorized into either “IRB Employee”, “Researcher” or both. As a result, three roles were created for analysis: IRB employee, researcher, and those who are both an IRB employee and researcher. Participation in the survey was voluntary and anonymous.

Sampling method

The Cochrane formula (n = z2pq/e2) was used for the sample size calculation, with a confidence interval of 0.95 and a margin of error of 5%. The resulting minimal sample size was 385. Considering the questionnaire may be completed incorrectly or there may be a significant amount of missing data, the total number of questionnaires was increased by 20%. Therefore, the final sample size of the study was estimated to be 462. A total of 757 respondents were ultimately included in our study.

The IRB-RAT-CV instrument

The instrument IRB-RAT has been used in the United States [42, 43], Singapore [29], India [45], and Peru [48]. It is a self-reported measure of IRB performance that consists of 45 items describing a variety of IRB activities and functions [42]. The questionnaire assesses the relative importance of these items to participants across eight themes. The eight themes include: procedural justice (how the decision-making process is carried out); absence of bias (a feature of procedural justice); pro-science sensitivity and commitment; interactional justice (interpersonal sensitivity and justification); formalities (an IRB’s formal functioning, structure, and composition); upholding the rights of human research participants; IRB outreach (offering services beyond those mandated); and competence (how competently the IRB performs its functions) [42]. Respondents in this study were asked to give two Likert Scale ratings on each item, to indicate both the importance of that item within their conception of an ideal IRB and how closely the item describes the actual IRB they work within. In our study, an ideal IRB is understood as an IRB that displays features that are most central in enabling the study participants to achieve their best work while an actual IRB is understood as relating to the IRB features those participants feel their IRB displays. Specifically, the survey asks, “As an investigator, how important is each item to you in your work? First, rate, along a seven-point continuum, how important each item would be to you in performing your best work, with 7 = ‘Absolutely essential’ to 1 = ‘Not important’. Next, for the same item, along another seven-point scale, rate how well that item describes your actual IRB, with 7 = ‘Highly descriptive’ to 1 = ‘Not at all descriptive.’” [31].

The reliability and validity of the IRB-RAT-CV (See the Appendix for summaries of the 45 items). were confirmed previously by our research team [31]. The Cronbach’s alpha coefficients for the ideal IRB and actual IRB were 0.989 and 0.992, while the Spearman-Brown coefficients were 0.964 and 0.968, respectively. Item-total correlation values ranged from 0.631 to 0.886, and 0.743 to 0.910. Confirmatory factor analysis for the ideal IRB yielded χ2/df = 2.811 l; root mean square error of approximation (RMSEA) = 0.062; normed fit index (NFI) = 0.904; Tucker-Lewis Index (TLI) = 0.931; and comparative fit index (CFI) = 0.936. Confirmatory factor analysis for the actual IRB yielded χ2/df = 2.967; root mean square error of approximation (RMSEA) = 0.065; normed fit index (NFI) = 0.914; Tucker-Lewis Index (TLI) = 0.936; and comparative fit index (CFI) = 0.941. This is consistent with values indicating adequate reliability and validity.

Data analysis

Analysis was conducted using SPSS 26.0 (IBM, Chicago, IL, USA). Both descriptive and inferential statistics were used. The categorical variables are described in terms of frequency and percentage, and the median of the inter-quartile range (IQR) was used for the continuous variable. Nonparametric Mann–Whitney U test and the Kruskal–Wallis (KW) test were used to assess the association between actual and ideal scores with the characteristics of respondents, with p values less than or equal to 0.05 considered statistically significant.

Results

Participant characteristics

A total of 757 respondents were included in the analysis, of which 44.6% were IRB employees, 50.5% were researchers, and 4.9% were both IRB employees and researchers. The demographic characteristics of the participants are shown in Table 1. Among the three age groups, most participants were over 41 years old, and the majority (62.1%) were female. The proportion of those with a master’s degree (43.3%) was higher than those with a bachelor’s degree and below (28.0%) and those with a Ph.D degree (28.7%). Most participants (84.3%) reported having received ethics training in the previous three years. Most participants (86.5%) also worked in hospitals.

Table 1 Characteristics of participants

The effect of different characteristics on the overall ideal score and actual score

Table 2 compares the effects of participants’ characteristics on scoring. In general, participants’ rating for an ideal IRB was significantly higher than that for their actual IRB. Participants who received ethics training in the last three years had higher scores for their ideal IRB and actual IRB than those who did not receive ethics training in the last three years (P < 0.05). Participants with an independent IRB office at their IRB had statistically significant differences in ratings with greater requirements for the ideal IRB and satisfaction with actual IRB services compared to those with no independent office (P < 0.05). Similarly, participants in tertiary and secondary hospitals rated actual IRBs higher than participants in medical schools (P < 0.05). There were no statistically significant associations between ideal and actual scores and other demographic variables (role, sex, age, study abroad experience, length of employment, job title, and education) (see Table 2).

Table 2 The effect of demographic factors on ideal IRB and actual IRB

Perspectives of different roles within IRBs on the most important and least important ideal items

In addition to highlighting the ideal items which respondents in our study regarded as the most and least important, Table 3 compares the perspectives of the Chinese sample with the US National Validation (USNV) sample since the author Keith-Spiegel et al. originally developed the IRB-RAT and used it as a tool to evaluate IRB performance. Overall, Chinese participants and American participants seem to both agree and differ on which ideal items are the most and least important (see the Appendix for descriptions of the 45 items). Specifically, researchers in both cohorts agree that an IRB that reviews protocols in a timely fashion (item 1) is one of the most important items, falling within the top 5. In particular, among the 45 items, American researchers considered item 1 (6.43 ± 0.80) as the most important ideal item [42]. In terms of the ranking of the least important ideal features, Chinese and American researchers nearly hold the same opinions, in perceiving an IRB offering consultation during the development of protocols and applications, an IRB providing editorial suggestions for documents and protocols, an IRB having a diverse membership, and an IRB being composed of more than one public member (item 32, item 34, item 40, and item 41, respectively) as the least important.

Table 3 Comparison of top-ranking ideal items between the Chinese sample and USNV sample by mean scores

Within the Chinese sample, participants with different roles in the IRB also ranked ideal items differently. The three roles (IRB employee, researcher, and those who are both an IRB employee and researcher) all agreed that IRB members should be required to abstain from a review when there are conflicts-of-interest (item 15), and such protocols are essential to the development of IRBs, ranking this item among their top 5 most important ideal items. Specifically, those who are both IRB employees and researchers ranked item 15 as the most important (6.46 ± 0.89). On the other hand, IRB employees and researchers regarded an IRB’s primary function as protecting human participants (item 42) (6.58 ± 0.69) and an IRB that takes appropriate action when there is scientific misconduct (item 43) (6.30 ± 0.79), respectively, as the most important item. All three roles also agreed that an IRB that applies flexible standards for informed consent requirements (item 45) belongs as one of the least important ideal items, of which, IRB employees considered this item to be the least important.

The difference in participants’ ratings for each theme

Table 4 shows the mean scores of actual IRB and ideal IRB across eight themes. In terms of the actual overall mean score, researchers (5.80 ± 0.91) scored the highest, and for the ideal overall mean score, IRB employees (6.27 ± 0.61) scored the highest. In evaluating the actual IRB, scores for researchers on procedural justice, interactional justice, IRB competence, IRB outreach, IRB formal functioning, structure, and composition, and upholding the rights of human research participants were higher than the other two groups. Regarding the absence of bias and pro-science sensitivity, IRB employees scored higher than the other groups.

Table 4 Comparison, by role, of mean factor domain scores awarded to actual IRB and the ideal IRB

In terms of the scores for the ideal IRB, IRB employees provided higher scores on procedural justice, absence of bias, pro-science sensitivity, IRB competence, IRB formal functioning, structure, and composition, and upholding the rights of human research participants compared to the other two groups. Researchers meanwhile scored higher on IRB outreach than the other two groups.

Discussion

In our study, both roles (IRB employee and researcher) had higher ideal means than actual means on 45 items, and this difference was statistically significant (P < 0.05). Those who are both IRB employees and researchers had higher ideal means than actual means on 44 items (except for item 41, an IRB that is composed of more than one public member, in which the ideal mean is equal to the actual mean), and the difference was statistically significant (P < 0.05). This indicates that those who are both IRB employees and researchers believe that no improvements are necessary for IRB membership, as their actual IRBs are most likely composed of multiple public members.

In addition, the highest and lowest rated items in our sample differed from those in the USNV sample. The distinct cultures between China and the US may have resulted in these differences in participants’ values and ethical consciousness [45]. Compared with the USNV sample’s score for an ideal IRB (5.19 ± 1.50), Chinese researchers (6.21 ± 0.67) had higher expectations for their IRBs. This is perhaps due to how Chinese IRBs operate largely under policies set at the national level in China in recent years, so the demands placed on IRBs are considerable and increasing in this region.

Participant characteristics and their effect on ideal and actual scores

In terms of the characteristics of the participants in our study, 43.3% of respondents were over 41 years old, and the majority (62.1%) were female; this is consistent with how females in general make up 65% of IRBs in China. Contrastingly, Chenneville’s study revealed that 73% and 83% of IRB members at the two IRBs of interest, respectively, were male [45]. On the surface, these percentages may seem high, but they are likely consistent with the male-dominated ethics committees in India. It is interesting to note that Chenneville’s figures also mimic the typical gender distribution of IRBs in the US. In a systematic review of the empirical literature evaluating IRBs in the US, Abbott and Grady found that the composition was predominantly male [49].

Most participants (77.8%) reported that the IRB they worked for had an independent IRB office to coordinate administrative tasks which indicates that the development of IRBs in China is becoming professionalized and specialized. A majority of participants (84.3%) also reported having received ethics training in the previous three years. This finding demonstrates that participants have a high enthusiasm to engage in ethical training, which is conducive to the cultivation and promotion of the subjects’ ethical awareness in their work. Such a finding can be significant for IRBs in China, as some studies have shown that training even a small number of individuals can change policies and practices in research ethics [50]. In addition, most participants (86.5%) worked in hospitals, reflecting how hospitals in China serve as both medical and health institutions, with established IRBs.

The influence of participants’ characteristics on the overall ideal and actual scoring

In our study, the ideal score exceeded the actual score across all eight themes, indicating that participants’ ideal IRB functioning was greater than actual IRB functions. With these data, IRBs in China can be precisely adjusted according to the IRB-RAT-CV results to improve areas with low scores, thus improving the quality of IRBs [45]. As the discrepancies between actual and desired IRB functions are likely to be common, they can be utilized as positive indicators for high ideals that may contribute to interest in seeking significant improvements in IRBs [45]. In general, the score for ideal IRB is significantly higher than that for actual IRB, indicating that participants have higher expectations and requirements for IRBs.

Moreover, the vast majority of participants’ ratings on the ideal performance of an IRB were significantly higher than the USNV sample in Keith-Spiegel et al.’s study (except item 1, an IRB that reviews protocols in a timely fashion) [42] and exceeds the ideal score of the India-based study on most items (30 items) [45]. The ideal score of the Chinese sample was found to be more consistent with that of the Singaporean one [29]. In terms of actual scores, the ones in the Chinese study are higher than the actual scores of the Indian and Singaporean samples across the 45 items, with more than half of the items (23 items) being one point higher.

Perspectives of different roles on the most important and least important ideal items

The differences between the three roles in their views of the most and least important ideal themes are reflected in the following aspects. Firstly, regarding the most important items, in our study, IRB employees thought that an IRB whose primary function is to protect human participants (item 42) to be the most important ideal theme (6.58 ± 0.69). This is very similar to studies in which researchers in the US and Singapore placed a high value on preserving the rights of participants [29, 42]. In contrast, researchers, and those who are both IRB employees and researchers considered an IRB that reviews protocols in a timely fashion (item 1) as one of the most important ideal items. These results were consistent with the USNV sample [42]. Ranking the timeliness of IRB review processes as an important IRB characteristic reflects researchers’ desire to avoid unnecessary delays in carrying out their studies. Untimely reviews are not simply a matter of inconvenience for researchers; such delays may also have detrimental impacts on interpersonal relationships within IRBs which may consist of IRB members and members of the research community. At present, most protocols in China must undergo ethical review before they can be submitted to the national human genetic resources office, to meet requirements for sample and data collection approval and subsequently be implemented in clinical trials [51]. Based on the ideal items deemed the most important in this study, an IRB can continue to improve in these areas to better its services and IRB members’ and researchers’ satisfaction.

Regarding the least important items, IRB employees, and those who are both IRB employees and researchers thought that an IRB that works with investigators to resolve disagreements and an IRB that is composed of members regarded as competent investigators (item 9 and item 29) belong among the least important ideal items. This may indicate that collaboration between researchers and IRBs is not an important factor in IRB operations for these two groups. In contrast, respondents in the Singapore and USNV studies valued item 9, demonstrating how these regions view such collaborations as essential to the timely approval of their research projects (this item was the 3rd highest ideal score in the Singaporean study and the 10th highest ideal score in the USNV sample) [29].

In addition, researchers in the Chinese and USNV samples seemed to agree on which ideal items were the least important; both samples agreed that an IRB that offers consultation during the development of protocols and applications, an IRB that offers editorial suggestions for documents and protocols, an IRB that has a diverse membership, and an IRB composed of more than one public member (item 32, item 34, item 40, and item 41, respectively) are the least important. Perhaps for these researchers, IRB membership is not as significant and has little relevance to their interests. In regards to item 41, the findings in the Singapore and USNV studies were similar to ours: having diverse IRBs with multiple lay members was considered less important than other items [42]. Researchers in the US may perceive laypersons as being unqualified to evaluate research proposals [52]. Compared to the Singaporean survey, item 34 was also a low-ranked ideal item in our study. Given that IRBs are tasked with evaluating ethical and regulatory aspects of a study, correcting typographical and grammatical errors may seem trivial or irrelevant to their functioning, even though these errors do impact the accuracy, readability, and comprehension of proposals and reports [29]. Similarly, both the Singaporean cohort and the USNV sample viewed lay membership, diversity of membership, as well as editorial suggestions from the IRB as some of the least important features [29].

The difference in participants’ ratings for each theme

This study sheds light on both researcher and IRB employee perceptions of IRBs in China. Overall, there was little difference between the ideal and actual scores of these two groups, and their views were relatively consistent (P > 0.05).

Across the eight themes, the difference between the Chinese sample and the samples from India and Singapore in the ideal score is small, but the actual score of the Chinese sample is higher than that of the other samples. In particular, on procedural justice, interactional justice, pro-science sensitivity, and IRB outreach, the actual score of the Chinese sample was much higher than that of the Indian and Singaporean samples (the difference was greater than one point), indicating that Chinese participants were more satisfied with the services provided by their actual IRBs in these four aspects. Similarly, the ideal score of the Chinese sample was significantly higher than that of the USNV sample in all eight themes especially in three themes (IRB competence, IRB outreach, and IRB formal functioning, structure, and composition), with the difference being greater than one point.

Conclusions

To our knowledge, this is the first study evaluating IRB performance in China, in a general manner using the IRB-RAT-CV tool. Overall, the experiences and perceptions of IRB employees and researchers as well as specific strengths and weaknesses of IRBs in China were identified. In addition, the IRB-RAT-CV facilitated comparisons with evaluations conducted in other countries and pinpointed areas for improvement within Chinese IRBs. There are at least three key insights that can be gleaned from our study.

First, the IRB-RAT-CV demonstrated Chinese IRBs’ strong performances in several areas, including respecting investigators (item 10), maintaining accurate records (item 36), having diverse memberships (item 40), and preserving the protection of human participants as its primary function (item 42). However, the importance of these items differed across the three roles in our study. For instance, an IRB whose primary function is to protect human participants (item 42) was among the most important ideal characteristics for IRB employees and those who are both IRB employees and researchers but not for researchers, whereas, an IRB that has diverse membership (item 40) was one of the least important for researchers but not for the other two groups. Understanding these differences, gleaned from IRBs’ current performances as well as members’ and researchers’ perceptions, is critical in constructing and sustaining infrastructures that support and protect research participants.

Utilizing the perspectives of the three roles in the study, to identify differences between the ideal and actual scores, can highlight to IRBs and institutions areas in which more time and effort should be devoted to improving IRB performance. In particular, our results revealed that IRBs in China can improve in the following items: reviewing protocols in a timely manner (item 1), conducting a comprehensive review of protocols (item 2), seeking outside assistance (item 7), having members who are knowledgeable about IRB procedures and national policy (item 23), providing training programs for new members (item 30), and having sufficient resources to carry out IRB functions (item 37). Items 1 and 30 were among the top 5 most important ideal characteristics for either IRB employees, researchers, or those who are both IRB employees and researchers, emphasizing the significance of these aspects to these participants and in strengthening IRBs.

Lastly, by comparing our results with other IRB evaluations, we can learn from various countries’ experiences to improve aspects in which IRBs in China are lagging. As an example, in our study, all three roles were aware of the importance of IRB members in avoiding conflicts of interest (COI) (item 15), with this item being listed among the top 5 for two of our studied roles but was not among the most important for the USNV sample. Despite these differences in awareness and significance, there are currently no specific regulations for COIs in clinical research in China [32]. Thus, we suggest the following recommendation by Gregor Scherzinger and Monika Bobbert, to be adopted in IRBs across China: IRBs should maintain an up-to-date, publicly available registry in which members can declare any relationships that may lead to conflicts of interest; in addition, IRBs should include rules that specify when members should withdraw from ethics review [53]. Comprehending similarities and differences, as well as the contexts in which they occur, between IRBs across regions can not only identify areas for improvement but also allow China to draw from empirical data as well as other countries’ experiences to strengthen IRB performance.

One limitation of this study was that the inclusion criteria for our sample were limited to IRB employees and researchers. However, other staff and individuals involved in research and IRB operations may have distinct, valuable experiences and perceptions toward IRBs in China. As these views were not included in the present study, the current comparisons and recommendations may be limited. Lastly, because the questionnaire was completed by participants who were more familiar with the operations of an IRB, IRB performance may have been overestimated. Although convenience sampling can result in an overestimation or underestimation of the studied outcome, this may not be a significant limitation in our study because a large proportion of adults in China use WeChat.

In conclusion, we gained a general understanding of China’s current IRB situation, with the collected data providing a benchmark of the perceived performance of actual IRBs in China. Moreover, the data were expanded upon by comparing them with those of other countries, to highlight differences between IRBs in various regions. These results serve to identify areas for improvement and provide empirical data and important references which may be useful in guiding institutions as well as in aiding China to formulate policies in IRB evaluation.

Availability of data and materials

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

Abbreviations

IRB:

Institutional review board

IRB-RAT-CV:

Institutional review board-researcher assessment tool- Chinese version

LMIC:

Low to middle-income country

REC:

Research ethics committees

NHC:

National health commission

NMPA:

National medical product administration

CFDA:

China food and drug administration

RMSEA:

Root mean square error of approximation

NFI:

Normed fit index

TLI:

Tucker Lewis index

CFI:

Comparative fit index

SPSS:

Statistical product and service solutions

IQR:

Inter-quartile range

KW:

Kruskal–Wallis

USNV:

US national validation

References

  1. World Health Organization. Standards and operational guidance for ethics review of health-related research with human participants. World Health Organization. 2011. https://apps.who.int/iris/handle/10665/44783. Accessed 21 Feb 2022.

  2. ICH Harmonised Tripartite Guideline for Good Clinical Practice E6(R1). International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. 2016. http://www.ich.org/products/guidelines/efficacy/article/efficacy-guidelines.html. Accessed 21 Feb 2022.

  3. Enfield KB, Truwit JD. The Purpose, Composition, and Function of an Institutional Review Board: Balancing Priorities, vol. 53. Respiratory care. Dallas, TX: Daedalus Enterprises;2008, p. 1330–6.

  4. Huanhuan C, Li M, Wang M, Roder D, Olver I. Challenges for ethics committees in biomedical research governance: illustrations from China and Australia. J Med Ethics History Med. 2021;2022;14:25–25.

  5. Heimer CA, Petty J. Bureaucratic ethics: IRBs and the legal regulation of human subjects research. Annu Rev Law Soc Sci. 2010;6(1):601–26.

    Article  Google Scholar 

  6. Tsan M. Measuring the quality and performance of institutional review boards. J Empir Res Hum Res Ethics. 2019;14(3):187–9.

    Article  Google Scholar 

  7. Sleem H, Abdelhai RA, Al-Abdallat I, Al-Naif M, Gabr HM, Kehil E, et al. Development of an accessible self-assessment tool for research ethics committees in developing countries. J Empir Res Hum Res Ethics. 2010;5(3):85–96.

    Article  Google Scholar 

  8. Klitzman R. From anonymity to “open doors”: IRB responses to tensions with researchers. BMC Res Notes. 2012;5(1):347–347.

    Article  Google Scholar 

  9. Tsan M, Ling B, Feske U, Zickmund S, Stone R, Sonel A, et al. Assessing the quality and performance of institutional review boards: levels of initial reviews. J Empir Res Hum Res Ethics. 2020;15(5):407–14.

    Article  Google Scholar 

  10. Fontanesi J, Magit A, Ford JJ, Nguyen H, Firestein GS. Systems approach to assessing and improving local human research institutional review board performance. Clin Transl Sci. 2018;2(2):103–9.

    Article  Google Scholar 

  11. Grady C. The contribution of ethics review to protection of human participants: comment on measuring the quality and performance of institutional review boards. J Empir Res Hum Res Ethics. 2019;14(3):197–9.

    Article  Google Scholar 

  12. Tsan M. Improving the quality and performance of institutional review boards in the USA through performance measurements. J Empir Res Hum Res Ethics. 2021;16(5):479–84.

    Article  Google Scholar 

  13. Feldman JA, Rebholz CM. Anonymous self-evaluation of performance by ethics board members: a pilot study. J Empir Res Hum Res Ethics. 2009;4(1):63–9.

    Article  Google Scholar 

  14. Czarkowski M, Rózanowski K. Polish research ethics committees in the European union system of assessing medical experiments. Sci Eng Ethics. 2009;15(2):201–12.

    Article  Google Scholar 

  15. Halila R. Evaluation of the work of hospital districts’ research ethics committees in Finland. J Med Ethics. 2014;40(12):866–8.

    Article  Google Scholar 

  16. Jaoko W, Bukusi E, Davis AM. An evaluation of the middle east research training initiative tool in assessing effective functioning of research ethics committees. J Empir Res Hum Res Ethics. 2016;11(4):357–63.

    Article  Google Scholar 

  17. Elsayed DE, Kass NE. Assessment of the ethical review process in Sudan. Dev World Bioeth. 2007;7(3):143–8.

    Article  Google Scholar 

  18. Hyder AA, Zafar W, Ali J, Ssekubugu R, Ndebele P, Kass N. Evaluating institutional capacity for research ethics in Africa: a case study from Botswana. BMC Med Ethics. 2013;14(1):1–14.

    Article  Google Scholar 

  19. Owusu SA, Addison G, Redman B, Kearns L, Amuna P, Laar A. Assessment of the operational characteristics of research ethics committees in Ghana. J Empir Res Hum Res Ethics. 2022;17(1–2):114–28.

    Article  Google Scholar 

  20. Matar A, Silverman H. Perspectives of Egyptian research ethics committees regarding their effective functioning. J Empir Res Hum Res Ethics. 2013;8(1):32–44.

    Article  Google Scholar 

  21. Sleem H, El-Kamary SS, Silverman HJ. Identifying structures, processes, resources and needs of research ethics committees in Egypt. BMC Med Ethics. 2010;11(1):1–8.

    Article  Google Scholar 

  22. Ikingura JK, Kruger M, Zeleke W. Health research ethics review and needs of institutional ethics committees in Tanzania. Tanzan Health Res Bull. 2007;9(3):154–8.

    Google Scholar 

  23. Adams P, Kaewkungwal J, Limphattharacharoen C, Prakobtham S, Pengsaa K, Khusmith S. Is your ethics committee efficient? Using “IRB metrics” as a self-assessment tool for continuous improvement at the faculty of tropical medicine, Mahidol university, Thailand. PLoS ONE. 2014;9(11): e113356.

    Article  Google Scholar 

  24. Oo ZZ, Wun M, Oo YTN, Mya KS, Silverman HJ. Assessing research ethics committees in Myanmar: results of a self-assessment tool. Asian Bioeth Rev. 2020;12(1):37–49.

    Article  Google Scholar 

  25. Ahsin S, Naghma Saeed GE. Self-evaluation of ethical review committees functioning at foundation university medical college (FUMC) through structured constitution-practice-outcome (CPO) assessment model. J Pak Med Assoc. 2017;67(1):42–8.

    Google Scholar 

  26. Janakiram C, Venkitachalam R, Joseph J. Profile of institutional ethics committees in dental teaching institutions in Kerala. India Acc Res. 2016;23(4):219–29.

    Article  Google Scholar 

  27. Silverman H, Sleem H, Moodley K, Kumar N, Naidoo S, Subramanian T, et al. Results of a self-assessment tool to assess the operational characteristics of research ethics committees in low- and middle-income countries. J Med Ethics. 2015;41(4):332–7.

    Article  Google Scholar 

  28. Rababa’h AM, Alzoubi KH, Ababneh M, Khabour OF. Awareness of jordanian investigators about the importance of ethics review committees: a pilot study. Sci Eng Ethics. 2020;26(2):821–31.

    Article  Google Scholar 

  29. Labude MK, Shen L, Zhu Y, Schaefer GO, Ong C, Xafis V. Perspectives of Singaporean biomedical researchers and research support staff on actual and ideal IRB review functions and characteristics: a quantitative analysis. PLoS ONE. 2020;15(12): e0241783.

    Article  Google Scholar 

  30. Wang ZH, Zhou GH, Sun LP, Gang J. Challenges in the ethics review process of clinical scientific research projects in China. J Int Med Res. 2019;47(10):4636–43.

    Article  Google Scholar 

  31. Liu X, Wu Y, Yang M, Li Y, Hahne J, Khoshnood K, et al. Cross-cultural validation of the IRB researcher assessment tool Chinese version. BMC Med Ethics. 2021;22(1):133–41.

    Article  Google Scholar 

  32. Wang X, Hahne J, Li L, Khoshnood K, Yang G, Yuan H, Liu X. Developing quality and efficiency of institutional review board review under a human research protection program at a leading hospital in central southern China: a descriptive analysis of the first three years. J Empir Res Hum Res Ethics. 2021;16(3):280–9.

    Article  Google Scholar 

  33. The National People’s Congress (NPC) of the People’s Republic of China. The Civil Code. 2020. http://www.npc.gov.cn/npc/c30834/202006/75ba6483b8344591abd07917e1d25cc8.shtml. Accessed 21 Feb 2022.

  34. The Central People’s Government of the People’s Republic of China. Biosecurity Law of the People’s Republic of China. 2020. http://www.gov.cn/xinwen/2020-10/18/content_5552108.htm. Accessed 21 Feb 2022.

  35. The Central People’s Government of the People’s Republic of China. the Law of the People’s Republic of China on Basic Medical and Health Care and and Health Promotion. 2019. http://www.gov.cn/xinwen/2019-12/29/content_5464861.htm. Accessed 21 Feb 2022.

  36. The Central People’s Government of the People’s Republic of China. Drug Administration Law of the People's Republic of China. 2019. http://www.gov.cn/xinwen/2019-08/26/content_5424780.htm. Accessed 21 Feb 2022.

  37. The National People’s Congress (NPC) of the People’s Republic of China. Physician Law of the People's Republic of China. 2021. http://www.npc.gov.cn/npc/c30834/202108/d954d9fa0af7458aa862182dc50a0d63.shtml. Accessed 21 Feb 2022.

  38. National Medical Products Administration. Announcement of the National Medical Products Administration and the National Health and Family Planning Commission on Issuing the Conditions for Medical Device Clinical Trial Institutions and Administrative Measures for Filing. 2017. https://www.nmpa.gov.cn/ylqx/ylqxggtg/ylqxqtgg/20171124123401301.html. Accessed 21 Feb 2022.

  39. The Central People’s Government of the People’s Republic of China. Announcement of the National Medical Products Administration and the National Health Commission of the People’s Republic of China on Issuing the Regulations on the Administration of Drug Clinical Trial Institutions. 2019. http://www.gov.cn/gongbao/content/2020/content_5496785.htm. Accessed 21 Feb 2022.

  40. Drug and Medical Device Clinical Trial Institution Filing Management Information System: The register/Log in. https://beian.cfdi.org.cn/CTMDS/apps/pub/public.jsp. Accessed 21 Feb 2022.

  41. Nicholls SG, Hayes TP, Brehaut JC, McDonald M, Weijer C, Saginur R, Fergusson D. A scoping review of empirical research relating to quality and effectiveness of research ethics review. PLoS ONE. 2015;10(7):e0133639.

    Article  Google Scholar 

  42. Keith-Spiegel P, Tabachnick B. What scientists want from their research ethics committee. J Empir Res Hum Res Ethics. 2006;1(1):67–82.

    Article  Google Scholar 

  43. Reeser JC, Austin DM, Jaros LM, Mukesh BN, McCarty CA. Investigating perceived institutional review board quality and function using the IRB researcher assessment tool. J Empir Res Hum Res Ethics. 2008;3(1):25–34.

    Article  Google Scholar 

  44. Hall DE, Hanusa BH, Ling BS, Stone RA, Switzer GE, Fine MJ, Arnold RM. Using the IRB researcher assessment tool to guide quality improvement. J Empir Res Hum Res Ethics. 2015;10(5):460–9.

    Article  Google Scholar 

  45. Chenneville T, Menezes L, Bylsma LM, Mann A, Kosambiya J, Baxi R. Assessing institutional ethics committees in India using the IRB-RAT. J Empir Res Hum Res Ethics. 2014;9(4):50–9.

    Article  Google Scholar 

  46. Dong Y, Wang P, Dai Z, et al. Increased self-care activities and glycemic control rate in relation to health education via Wechat among diabetes patients: a randomized clinical trial. Medicine (Baltimore). 2018;97:e13632–e13632.

    Article  Google Scholar 

  47. Liang D, Li S, Ji B, et al. Research on Matrix Multiplication Based WeChat Group Tagging Technology. In: 2018 IEEE First International Conference on Artificial Intelligence and Knowledge Engineering AIKE. 2018;2018:41–45.

  48. Roque-Henriquez JC, Bazo-Alvarez JC, Toia AMDC, Mormontoy-Laurel W. Adaptación y validación al español del institutional review board researcher assessment tool (IRB-RAT) [Spanish adaptation and validation of the institutional review board researcher assessment tool (IRB-RAT)]. Rev Peru Med Exp Salud Publica. 2018;35(3):456–64.

    Article  Google Scholar 

  49. Abbott L, Grady C. A systematic review of the empirical literature evaluating IRBs: what we know and what we still need to learn. J Empir Res Hum Res Ethics. 2011;6(1):3–19.

    Article  Google Scholar 

  50. Kass NE, Hyder AA, Ajuwon A, et al. The structure and function of research ethics committees in Africa: a case study. PLoS Med. 2007;4(1): e3.

    Article  Google Scholar 

  51. The Central People’s Government of the People’s Republic of China. Regulation of the People's Republic of China on the Administration of Human Genetic Resources. 2019. http://www.gov.cn/zhengce/content/2019-06/10/content_5398829.htm. Accessed 21 Feb 2022.

  52. Klitzman R. Institutional review board community members: who are they, what do they do, and whom do they represent? Acad Med. 2012;87(7):975–81.

    Article  Google Scholar 

  53. Scherzinger G, Bobbert M. Evaluation of research ethics committees: criteria for the ethical quality of the review process. Account Res. 2017;24(3):152–76.

    Article  Google Scholar 

Download references

Acknowledgements

We are very grateful to the participants in this study.

Funding

This work was supported by Major Scientific and Technological Projects for Collaborative Prevention and Control of Birth Defects in Hunan Province (2019SK1010); and by the NIH Fogarty International Center Masters Level Bioethics Program at Central South University in Changsha, China, under award number R25 TW007700.

Author information

Authors and Affiliations

Authors

Contributions

XL and YW conducted the literature review, participated in distributing the questionnaire and writing the first draft, MY and YW conducted the statistical analysis. YL checked the references. KK, EL and LL helped to revise the article. XMW designed and conceived this research. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Xiaomin Wang.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the IRB of the Third Xiangya Hospital of Central South University in Changsha, China. The study was in accordance with the Declaration of Helsinki. Informed consent was obtained from all the participants.

Consent for publication

Not applicable.

Competing interests

The authors declare no potential conflicts of interest in the research, authorship, and/or publication of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix: IRB-RAT 45 item content, with descriptions and summaries.

Appendix: IRB-RAT 45 item content, with descriptions and summaries.

IRB-RAT item Summarized item
1. An IRB that reviews protocols in a timely fashion Reviews protocols in a timely fashion
2. An IRB that conducts a conscientious and complete review of protocols Comprehensively reviews protocols
3. An IRB that gives a complete rationale for any required changes to or disapprovals of protocols Provides complete rationale for changes regarding protocols
4. An IRB that includes a complete rationale when it denies or mandates changes in a protocol based on criteria that are more stringent than or different from national research policy Provides complete rationale for decisions based on stringent criteria
5. An IRB that is open to reversing its earlier decisions Open to reversing earlier decisions
6. An IRB that invites investigators to present their position whenever a question or concern about a research protocol arises Invites investigators to present their positions when questions arise
7. An IRB that recognizes when it lacks sufficient expertise to evaluate a protocol and seeks outside experts Recognizes when it lacks expertise and seeks outside assistance
8. An IRB that responds in a timely manner to investigators’ inquiries about its processes and decisions Responds promptly to investigators’ inquiries
9. An IRB that works with investigators to find mutually satisfying solutions whenever disagreements exist Works with investigators to resolve disagreements
10. An IRB that treats investigators with respect Treats investigators with respect
11. An IRB that acknowledges full responsibility for its errors or delays in processing protocols and attempts to correct them as expeditiously as possible Takes full responsibility and action in resolving errors
12. An IRB that is open and pleasant in its interactions with investigators Pleasantly interacts with investigators
13. An IRB with members who do not allow personal biases to affect their evaluation of protocols Members do not allow personal biases to affect their work
14. An IRB whose members hold no preconceived biases against particular research topics Members hold no biases towards particular research topics
15. An IRB that requires members to abstain from evaluating protocols whenever a real or apparent conflict-of interest arises Requires members to abstain from a review when there is conflict-of-interest
16. An IRB whose members hold no preconceived biases against particular research techniques Members hold no biases towards particular research techniques
17. An IRB that is open to innovative approaches to conducting research Open to innovations in research
18. An IRB that does a good job of upholding participants’ rights while, at the same time, facilitating the conduct of research Upholds participants’ rights while facilitating the conduct of research
19. An IRB that does not use its power to suppress research that is otherwise methodologically sound and in compliance with national policy whenever it perceives potential criticism from outside the scientific community Does not use its power to suppress research and avoid criticism
20. An IRB that views itself as an investigator’s ally rather than as a hurdle to clear Views itself as ally to investigators
21. An IRB that shows considerable evidence that the advancement of science is part of its mission Demonstrates that its mission is to advance science
22. An IRB that shows empathy with the difficulties that can present themselves during the design and conduct of research Shows empathy over the difficulties present in research
23. An IRB with members who are very knowledgeable about IRB procedures and national policy Members are knowledgeable about IRB procedures and national policy
24. An IRB that conducts a conscientious analysis of potential benefits weighed against potential risks before making decisions Weighs potential benefits against risks before coming to a decision
25. An IRB that can competently distinguish exempt from nonexempt research Can distinguish exempt from nonexempt research
26. An IRB that ensures that at least one member is knowledgeable about the content domain of submitted protocols At least one member knows the content of submitted proposals
27. An IRB whose members arrive at meetings well-prepared Members come to meetings well-prepared
28. An IRB with a Research Compliance Officer (or staff member in charge of IRB functions) who has a research background Has Research Compliance Officer with research background
29. An IRB that is composed primarily of members regarded as highly competent investigators Composed of members regarded as competent investigators
30. An IRB that provides a comprehensive training program for its new members Provides a training program for new members
31. An IRB that offers information to improve the chances of gaining IRB approval Offers information to improve chances of approval
32. An IRB that offers consultation during the development of research protocols and grant applications Offers consultation during the development of protocols and applications
33. An IRB that offers investigators opportunities to be educated about national research policy Offers opportunities to educate investigators about national research policy
34. An IRB that offers editorial suggestions regarding consent documents and protocols Offers editorial suggestions for documents and protocols
35. An IRB whose members fully understand and act within the scope of their function Members understand and act within the scope of their functions
36. An IRB that maintains accurate records Maintains accurate records
37. An IRB that is allocated sufficient resources to carry out its functions Has sufficient resources to carry out its functions
38. An IRB that requires that its Chair be an experienced investigator Requires Chair to be an experienced investigator
39. An IRB that monitors the progress of each approved research project in line with national policy Monitors approved research projects according to national policy
40. An IRB that has a diverse membership (i.e., includes women, minorities and junior and senior members of the institution) Has a diverse membership
41. An IRB that is composed of more than one public member Composed of more than one public member
42. An IRB that views protection of human participants as its primary function Primary function is to protect human participants
43. An IRB that takes timely and appropriate action whenever scientific misconduct is alleged Takes appropriate action when there is scientific misconduct
44. An IRB that takes timely action when an investigator has violated its decisions Acts promptly when its decisions are violated by an investigator
45. An IRB that applies appropriately flexible standards regarding voluntary and informed consent requirements (e.g., required wording is not as demanding for minimal risk research as it is for more risky research) Applies flexible standards for informed consent requirements

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Liu, X., Wu, Y., Yang, M. et al. Performance of IRBs in China: a survey on IRB employees and researchers’ experiences and perceptions. BMC Med Ethics 23, 89 (2022). https://doi.org/10.1186/s12910-022-00826-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12910-022-00826-4

Keywords

  • China
  • Institutional review board
  • Performance