Skip to main content

How should communities be meaningfully engaged (if at all) when setting priorities for biomedical research? Perspectives from the biomedical research community

Abstract

Background

There is now rising consensus that community engagement is ethically and scientifically essential for all types of health research. Yet debate continues about the moral aims, methods and appropriate timing in the research cycle for community engagement to occur, and whether the answer should vary between different types of health research. Co-design and collaborative partnership approaches that involve engagement during priority-setting, for example, are common in many forms of applied health research but are not regular practice in biomedical research. In this study, we empirically examine the normative question: should communities be engaged when setting priorities for biomedical research projects, and, if so, how and for what purpose?

Methods

We conducted in-depth interviews with 31 members of the biomedical research community from the UK, Australia, and African countries who had engaged communities in their work. Interview data were thematically analysed.

Results

Our study shows that biomedical researchers and community engagement experts strongly support engagement in biomedical research priority-setting, except under certain circumstances where it may be harmful to communities. However, they gave two distinct responses on what ethical purpose it should serve—either empowerment or instrumental goals—and their perspectives on how it should achieve those goals also varied. Three engagement approaches were suggested: community-initiated, synergistic, and consultative. Pre-engagement essentials and barriers to meaningful engagement in biomedical research priority-setting are also reported.

Conclusions

This study offers initial evidence that meaningful engagement in priority-setting should potentially be defined slightly differently for biomedical research relative to certain types of applied health research and that engagement practice in biomedical research should not be dominated by instrumental goals and approaches, as is presently the case.

Peer Review reports

Introduction

Community engagement is a core feature of participatory health research, and there is growing agreement that it is essential for all health research, from biomedical to applied forms [1]. As Nunn et al. affirm, it is estimated that by 2025, nearly two billion people worldwide will have had their DNA sequenced and this creates a global imperative for engagement in genomic research [2]. Community engagement is now encouraged, or even mandated, as a key element of all health research, including ‘traditional’ non-participatory health research, by research institutions and funding bodies [3,4,5]. For example, it was a required component of applications to the second and final round of funding for the H3Africa Consortium, which supports genomics research in Africa [6]. Many high-profile genomics research initiatives have made public statements about the importance of involving the community [2]. Community engagement is also increasingly required by international biomedical research ethics guidelines [5,6,7].

Arguments have been made that it is ethically essential to undertake some form of engagement in health research because it is central to showing respect for communities and the traditions and norms that they share, increases the chances that research will improve health outcomes, builds public trust, enhances prospects for justice, and facilitates better stewardship of resources [1]. Where engagement is undertaken as shared decision-making throughout the research process, it facilitates self-determination because those who are significantly affected by the selection of health research priorities and the translation of the evidence generated by their investigation are included in discussions and decision‐making about them [8]. Such engagement also promotes cognitive and epistemic justice and maximizes the social knowledge generated to identify and solve complex problems that impede health and well‐being [8]. It is seen as a key means to ensure that research projects ask the ‘right’ questions—namely, those that are responsive to urgent community-identified needs—and create ‘better’ knowledge that uses and reflects a diversity of knowledge systems and is shared beyond peer-reviewed journals and academic conferences [9].

Yet debate continues about when, how, and for what purpose engagement should be performed in health research, and whether the answer should vary between different types of health research. Several types of ethical goals have been attributed to community engagement in health research: instrumental, intrinsic, and transformative. Engagement can advance intrinsic goals like building a sense of inclusion or demonstrating respect [3]. Engagement activities can further purely instrumental goals such as facilitating smooth research operations, augmenting the efficiency of study recruitment, or gaining community ‘buy-in’ [4, 10]. It also has “the potential to redress past harms; compensate for or resolve existing differences in power, privilege, and positionality; [and] allow for marginalized voices and experiences to be represented in the production of scientific knowledge” [4, p. 257].

Who is engaged can vary considerably based on how the “community” is defined. A community can be defined based on geography; shared experiences, characteristics, or ethnicity; or special interests or goals [3]. In health research, communities are often related to the nature of the research activity, e.g., the geographical area or illness group a given study involves. Engagement activities then frequently include patients, carers, and/or the broader public. However, depending on how the relevant community is defined, ministries of health, ethics committees, policymakers, international organizations, the media, and universities may be engaged as well [3].

What constitutes community engagement in health research practice also varies dramatically along a spectrum from shallow and tokenistic engagement to deeper and more meaningful engagement. Engagement takes different forms, ranging from raising community awareness of research projects, to consultation on certain parts of research projects, to community representation throughout the entire research cycle, to long-term and authentic partnerships [11]. Where these forms of engagement fall on the spectrum largely relates to the stage of the research cycle at which engagement occurs, and the level of participation they afford to those engaged. Goulet contends that the earlier ‘non-elites’ enter the process, the higher the quality of their participation. In the health research project context, this means entry during grant writing and priority-setting comprises deeper participation than, for instance, when communities enter during data collection or analysis [12].

Even so, the quality of communities’ participation is not exclusively determined by when they enter into the research process. A range of levels of participation exist, with some more “active, deliberative, and influential” than others [13, 14]. Informing refers to generating awareness and understanding within host communities about what research is and about already defined research projects that are being undertaken with them. The outputs of decision‐making in research projects are shared with community members; they do not give their input and are not involved in making the decisions. Consulting is a particularly common form of engagement in health research [3, 15]. Community members are asked to give their input (e.g., feedback, suggestions, critiques) on aspects of research projects but with no guarantee that those who decide will use or consider the information they provide. There is broad consensus that they can potentially be consulted about a wide variety of health research activities, including protocol development, research conduct, access to data and samples, and the dissemination or publication of research findings [16]. Often, their consultation occurs through specifically established bodies such as community advisory boards.

Collaborative partnership, on the other hand, means involving community members in decision-making throughout health research projects. This is consistent with growing emphasis on practicing engagement in health research as the co‐construction of knowledge or co-design. Taking such approaches means researchers jointly construct knowledge with communities: all parties design and conduct research together in ways that achieve the purposes of both sets of actors [17]. Community members are part of assessing what local health problems should be the focus of the research; planning, conducting and overseeing the research; and integrating the research into the health care system [18]. These approaches thus share many similarities with community-based participatory action methodologies. A key point of difference between them and other forms of engagement is the balance of power. Consulting and informing do not entail community members’ participation in decision‐making, whereas collaboration does.

A key matter to investigate is whether the type of health research is ethically significant in specifying what engagement should entail—namely, when, how, and for what moral purpose engagement should be performed. A spectrum of health research disciplines exists, ranging from basic science, clinical, and genomics research to more applied types like public health, health services, and health policy research. It has been suggested that types of applied health research place a greater emphasis on meaningful engagement with communities [5]. Co-design and participatory action methods are not regular practice in genomics or other types of biomedical research [2]. Questions have been raised by researchers as to whether co-design methods are even possible in biomedical research, which, unlike public health or health services research, requires more specialised and technical knowledge [19]. A scoping review investigating public involvement in human genomics projects found their involvement was highest at the stage of “implementation and management” (19/32), while the stages of engagement with the lowest number of initiatives reporting involvement were “funding” (1/32) and “identifying topics” and “prioritization” (4/32)” [2]. Similarly, a recent realist review showed that the collaborative partnership thread of community engagement is less common in biomedical research. Instead, where engagement with instrumental goals and approaches dominates [20]. Thus, biomedical research potentially seems less suited and/or inclined to adopt more meaningful forms of engagement, but this does not necessarily mean they aren’t nonetheless ethically ideal.

This paper investigates what form of community engagement (if any) ideally belongs in biomedical research priority-setting. We empirically examine the question: should communities be engaged when setting priorities for biomedical research projects, and, if so, how and for what purpose? Biomedical research is defined as encompassing basic science, clinical, and genomics research. Basic science and clinical research have traditionally fallen within the biomedical research category. Over the past few decades, genomics has also become a central and cohesive discipline of biomedical research [21]. Research priority-setting refers to defining the research topic and study questions for individual health research projects or programs. It does not encompass defining a set of global, national, or institutional research topics that should receive priority funding and implementation. This study’s focus on priority-setting reflects the moral importance of engagement from the beginning of research projects. Communities’ early engagement equates to deeper participation and enhances the responsiveness of research priorities. It means community members are part of making a greater number of decisions about a given study, including those that determine the direction of the entire research project. Without engagement from priority-setting, researchers may ultimately miss the needs deemed of high import and urgency by communities [22]. Research has shown that patients, based on their lived-experience, prioritize different topics than experts [23]. Despite this, in practice, communities concerningly often do not participate in the priority-setting stage of biomedical research projects [2]. Our study’s focus, however, is not meant to suggest it is sufficient to engage communities in the priority-setting stage of health research projects alone.

We conducted in-depth interviews with 31 members of the biomedical research community from the UK, Australia, and African countries who had engaged communities in their work. Both genomics and clinical research have become increasingly globalised since the 1990s [24, 25]. As such, the recruitment of interviewees spanned both high-income countries and low- and middle-income countries. African countries were selected to provide a low- and middle-income country perspective due to the growth of clinical and genomics research initiatives requiring community engagement on the continent. The UK and Australia were selected to provide a high-income country perspective because engagement in health research is established in both countries. Since “patient and public involvement” in health research has been a feature of UK policy for longer and is widely adopted by UK research funders [26], it was also thought that UK interviewees might have different ideas and experiences related to what comprises ideal engagement in biomedical research priority-setting than Australian or African interviewees. We analysed interview data thematically and report four main themes in the paper: the value of engagement, pre-engagement essentials, ideal goals and models of engagement, and barriers to engagement in biomedical research priority-setting. We conclude by considering whether our findings suggest community engagement in priority-setting should occur but look different for biomedical research relative to applied health research, should occur and look the same, or should not occur.

Methods

Study methods and sample

We performed 31 semi-structured interviews with basic science, clinical, and genomics researchers who had engaged communities in their work (27 interviewees) and community engagement experts who were embedded in biomedical research (4 interviewees).In-depth interviews were chosen as the primary method to explore the research question because they allow for the rich details of key informants’ experiences and perspectives to be gathered. All procedures were performed in accordance with the National Health and Medical Research Council of Australia’s National Statement on Ethical Conduct of Human Research.

The sampling strategy used a mix of purposive and snowball methods. To recruit Australian interviewees, we identified potential candidates systematically through a structured search, targeting five universities in Australia with high-calibre biomedical/genomic institutes. We approached sixteen Australian researchers via email and interviewed eight. Five did not respond, and three did not believe their experience to be relevant to the research question. Another three interviewees were identified via snowball sampling.

To identify UK and African interviewees, we initially employed a similar strategy targeting UK and African universities, but it was very difficult to tell from academic profiles in those countries whether individuals had experience with community engagement in biomedical research. Across the UK and Africa, not many academics advertised that they did community engagement. As such, we revised the sampling strategy to target biomedical consortia and institutions that we knew were supportive of and/or funded by entities that require community engagement and that funded/employed UK and African biomedical researchers: the Sanger Institute (UK), the US National Institutes of Health, and the H3Africa Consortium. The Sanger Institute was chosen because its researchers spanned basic science, clinical, and genomics research and it has links to Wellcome Trust, which have funding requirements for community engagement. H3Africa was selected because it is a large consortium of genomics researchers spanning the African continent and its funding made community engagement a required component of applications [6, 27]. The NIH was chosen because its researchers were likely to have done community engagement to secure fundingFootnote 1 and it was easy to find UK and African scientists through their website. We identified the websites of NIH centres for biomedical research and searched their staff directories for UK and African professors/group leaders. At all three institutions, senior researchers were approached, with the rationale being that they could refer on junior research team members as potential interviewees. In total, we approached 195 UK researchers and interviewed seven. Another thirteen were identified via snowball sampling, of whom three were interviewed. We approached 63 African researchers and interviewed three. Another 24 were identified via snowball sampling, of whom seven were interviewed. The low success rates reflect that many potential interviewees declined due to being overburdened with clinical responsibilities during the Covid-19 pandemic.

Study inclusion and exclusion criteria are described below (Box 1). All interviews were conducted remotely. Sampling continued until no new information emerged and saturation was achieved.

Box 1 Interview inclusion and exclusion criteria

In total, our 31 interviewees came from the UK, Australia, Kenya, South Africa, Malawi, Nigeria, Uganda, and Burkina Faso. Eighteen women and thirteen men were interviewed. Interviewees’ work spanned basic science (4), clinical (13) and genomics (17) research. Three interviewees had experience with both clinical and genomics research. Interviews’ duration was approximately 40–75 min. 22 interviewees had experience with community engagement in priority-setting. Of those, nine were from Australia, three were from Africa, and ten were from the UK.

Data collection and analysis

We conducted semi-structured interviews according to the technique of thick description [28]. Thick description means that questions for interviewees are open-ended and attempt to draw out interviewees’ experiences and views. In keeping with in-depth interviewing techniques, explanatory probes (such as asking “why” and “could you tell me more about that”) were also used to elicit richer details and clarificatory probes were employed to generate a better understanding of interviewees’ comments [29]. The interview guide used in this study is provided in Box 2 below.

Box 2 Interview guide

Interviews were transcribed verbatim and thematic analysis of interview data was undertaken in the following five phases: initial coding framework creation, coding, intercoder reliability and agreement assessment, coding framework modification, and final coding of entire dataset [30, 31]. Two coders (NE and JB) independently examined six transcripts and identified categories and subcategories. They then developed an initial coding framework together and discussed that framework with BP. Using the initial coding framework, NE and JB next undertook an iterative process of coding a transcript, assessing intercoder reliability and agreement, and modifying the coding framework [30]. Here, a “negotiated agreement approach” was adopted, whereby NE and JB separately coded a transcript, compared their codings, and then discussed their disagreements in an effort to reconcile them and arrive at a final version in which as many discrepancies as possible were resolved [31]. Six transcripts were co-coded. Across the six transcripts, coders ultimately agreed with proposed inclusion/exclusion of codes 100% of the time. Where a coder identified codes that the other had not, agreement to include JB’s codes occurred 92% of the time and agreement to exclude occurred 8% of the time, and agreement to include NE’s codes occurred 89% of the time and agreement to exclude occurred 11% of the time. During this process, the coding framework was modified, discussed with BP, and finalised. NE then applied the final coding framework to recode the African interviewee transcripts, and JB applied it to recode the UK and Australian interviewee transcripts. All data was coded using NVivo software. From this analysis, four main themes emerged: the value of engagement, pre-engagement essentials, ideal goals and models of engagement, and barriers to engagement in biomedical research priority-setting.

Results

The value of engagement

Most interviewees strongly affirmed there is value for community engagement when determining priorities for biomedical research projects: “communities should be engaged in the conceptualisation [of research]…at the time of planning, communities at the table bring in their own perspective.” – African interviewee (06). Interviewees described four reasons for why engagement should occur in priority-setting– epistemic value, community ownership, equity, and responsiveness (see Table 1). Interviewees across all geographical contexts (Africa, the UK, Australia) strongly affirmed that the epistemic knowledge held by community members is a critical resource for any priority-setting exercise in biomedical research. According to one UK interviewee (01), “… the people who have the best knowledge of those problems are the people that live with particular conditions or challenges that the research is trying to benefit.”

Table 1 Reasons for community engagement in biomedical research priority-setting

However, a minority of interviewees expressed concern about carrying out community engagement during biomedical research priority-setting under specific unsafe circumstances. Some interviewees worried that community engagement in particular countries or generally could put individuals from certain patient communities at risk of harm. According to an African interviewee (07),

“ [I] think the reason why we decided to leave them out it’s because homosexuality is illegal in the country, right, it’s not accepted and so sometimes if you actually involve such individuals in the [HIV] research study sometimes you put them at risk. The same with sexual, we call them sex workers yeah."

Where a sub-group within a patient community experiences stigma or a patient community experiences a stigmatised illness, it may be ethically appropriate to instead engage family members, or individuals who are close to them, or to not engage them at all. An interviewee from Africa (07) provided an example,

"I’ve been involved in some psychiatric genomic research as well, we know that psychiatric patients in Africa are completely marginalised right…How do you engaged those people right? Who carriers their voice? Now you can do beautiful community engagement around psychiatric illness, and we’ve tried right, we’ve even engaged patients with schizophrenia… but to get them to really help you set research priorities is, is you know what’s quite a different kettle of fish and so what often happens is you engage their family members…"

Interviewees further highlighted that, in other contexts, it may not be culturally safe for communities to engage with researchers. As one Australian interviewee (08) described,

“…what I am seeing is this reflex to, really rapid, I would say, quick and dirty relationship being formed, and then expected that a letter of support will be written, without the research being properly discussed with anyone beyond the leadership of a particular organisation. So, what that means is, there has often not been sufficient community consultation”.—Australian interviewee (08).

Where a “loose threads” or tokenistic approach to engagement is taken, there is insufficient planning, inadequate time for engagement, and limited engagement occurs, often random, one-on-one communication. For example, one member of a research team speaks to one person within an Aboriginal organisation. This is not representative of a community and can reinforce unequal power dynamics within a community. Such practices may cause more harm to communities than benefit.

They are often a product of a research economy, where funders require evidence of community engagement, but pay little attention to the depth of engagement. Where there is no funding for community engagement in priority-setting, and/or no requirement (or expectation) from funders to meaningfully engage communities when determining research priorities.

Pre-engagement essentials

Interviewees identified two components that comprised pre-engagement essentials: defining the community and building foundations. These should be achieved before attempting to engage a community in biomedical research priority-setting.

Defining the community

First, defining the community with whom to engage was thought to be essential in biomedical research priority-setting. Interviewees described two overlapping ways of defining the community that are common to biomedical research: communities of people who live with a certain disease and communities defined by geographical boundaries. The latter was more commonly described by African interviewees and the former by Australian and UK interviewees. Some interviewees also explained that they used geographical boundaries to define a patient community, as some diseases or genomic conditions are more prevalent in certain areas.

Building foundations

Four main types of foundations were thought to be essential to have in place before meaningful engagement can occur in biomedical research priority-setting: environmental, relational, collective and individual (see Table 2). Of these, none were specific to biomedical research relative to health research broadly, with the exception of genomic literacy. Genomic literacy means those engaged have a base level of understanding of genomics concepts and terms. Interviewees believed such literacy is critical to the success of genomics research priority-setting discussions involving community members. In instances where there is not an existing level of genomic literacy, efforts should be made to educate the community.

Table 2 Foundations for meaningful engagement

When the community of interest is defined by geographical boundaries or is marginalised, being embedded in the community is an especially important relational foundation. Identifying and obtaining the support of community gatekeepers (i.e., individuals or groups who have significant influence in the community) is a key first step to build relational foundations with such communities. Gatekeepers can facilitate access to members of geographical communities through different stakeholder groups, including local leaders and community health workers, civil society organisations, local government health committees, and/or community advisory boards. Identifying gatekeepers or community leaders is also important because they may have a “a bigger understanding of the health needs at a national level, or they can be able to advise what the implications are at, at the national level and if it fits within their planning at that level”—African interviewee (04).

Ideal goals and models of engagement in biomedical research priority setting

Interviewees described two models of engagement in setting biomedical research priorities that correspond to different ethical goals: empowerment and instrumental. Interviewees cautioned against tokenistic engagement at all costs. Some interviewees believed that if engagement is not evident from research conceptualisation, it can be tokenistic:

“You know, having a fully developed and finalised document that they wave under a community advisory group’s nose. That really just doesn’t cut it. If it’s not involvement from the start, it’s tokenistic and it’s insulting”—Australian interviewee (05).

Having empowerment goals means community engagement assists with breaking down power disparities between community members and biomedical researchers. Several interviewees believed the ideal engagement model to achieve this goal is where communities’ initiate engagement and set research priorities, as this action can facilitate power sharing in the relationship. According to African interviewee (01), “An ideal would be the communities reaching out to researchers to tell them what they want researchers to do. It wouldn’t be researchers engaging communities about the research researchers want to do.” The interviewee further noted that, while the community-initiated model is “ideal”, self-aware and mobilising communities are rare.

Other interviewees felt empowerment goals are best furthered when research priorities arise out of synergistic relationships between communities and researchers. The synergistic model is embodied in an “approach of action-based research where we would engage people at the start, and co-design research programs” (Australian interviewee 08). In this model, community members were commonly described as “partners” and as critical members of the research team. Priority setting comprises a two-way process that requires both researchers and community members to identify research priorities. Where communities alone set priorities and design research, the potential for research to actually address those priorities is constrained, as the methods to address their priorities is limited to the community’s experience or imagination. Similarly, where researchers set priorities completely in isolation from those who have lived experience with a disease, they may favour inappropriate priorities and methods. Several interviewees reflected that community engagement is most powerful when it fosters an exchange of perspectives and knowledge.

Having instrumental goals means community engagement helps ensure that research reflecting priorities identified by researchers is more feasible. Here, researchers initiate engagement and approach a community with a research idea before they develop a research proposal. Engagement either functions to assess whether the community believes the research idea addresses a priority and, if so, to refine the idea, or it primarily functions to refine a researcher-initiated idea with the community. The former approach is described by an interviewee:

“Before the researcher develops their research protocol, they should go to the communities and find out whether that particular research idea will address the priority that the communities that will be involved have. The researcher should have input from the communities at that point and then from there then they, the researcher, can draw up the research proposal and then you know, submit it for review and all that.”—African interviewee (02).

In contrast, another interviewee suggests the latter approach

“We’ve identified this problem in the community, for example, from preliminary reports we are seeing that there is a high level of maybe of schistosomiasis, it’s a common disease that is prevalent in communities here that really along the shores of the lakes … we tell them what kind of plan we have as the design as of the study in our mind that this is what we want to do … And then ask them for their opinions, is it feasible; do you think it’s going to be beneficial to the community”—African interviewee (04).

Generally, instrumental goals were more commonly described as achieved through consultative models of engagement, where community members provide “input” and “information” and are “asked for opinions”.

Irrespective of whether interviewees believed that meaningful engagement should facilitate empowerment or instrumental goals, they collectively thought consideration of diversity, bringing out voices, and the nature of the engagement space are necessary in biomedical research priority-setting. Engagement of patient and geographic communities should ensure diversity in terms of what demographics are represented and, in geographic communities, what community roles are represented. Interviewees highlighted that ensuring there is ethnic diversity in biomedical research is important to capture issues affecting unique genomics populations.

Engagement should also be designed to draw out different voices, with a particular focus on marginalised voices. Interviewees describing several strategies to do so when engaging in person such as providing those engaged with background materials and pre-readings in advance; breaking into smaller groups of people with similar life-experiences; using deliberative, individualised communication; and making the engagement mirror an informal meeting. The latter reflects that “formal and structured” engagement processes can make community members uneasy, an observation more commonly reported by African interviewees.

In some circumstances, online engagement methods were recommended. The diaspora of a patient community can be reached through online engagement methods (i.e., online discussions). When considering online engagement methods, interviewees highlighted that it is important to be aware that this can disadvantage some individuals from being able to participate as they may have limited access to the necessary tools to participate (i.e., computer, internet access).

In relation to space, interviewees felt that the space chosen for engagement can influence which voices are the loudest, and which voices are missing from priority setting process. They believed that the researchers should ideally go to the community and use safe community spaces that are not imbued with exclusionary norms. Interviewees from Africa more commonly reported on the impact of cultural norms in spaces for engagement.

Barriers to and challenges within engagement

Interviewees identified environmental, relational and individual barriers to engaging communities in setting priorities for biomedical research projects (see Table 3) and challenges that arise when doing so. Certain challenges—undefined communities, literacy, bias, and sidelining—were unique or more likely to occur in relation to engagement in genomics or biomedical research than applied health research. Interviewees reflected that one of the biggest challenges for exploratory genomics work can be the identification of the community:

“The biggest challenge with genomics research is that it’s still a developing area and some terms you can’t identify a community or who you need to engage with kind of until you start understanding what’s going on”.—Australian interviewee (11).

Table 3 Barriers to meaningful engagement

Efforts to build genomics literacy also come with challenges

“So one of our challenges in genomics is of course that we don’t have a vocabulary and a language for genomics in many of our African settings. So that has been interesting in itself and as part of this bigger project that I’m involved in…people are actually looking at words and concepts…and how they can try and make it more accessible and also more culturally sensitive”—African interviewee (05).

Biases can arise in biomedical research priority-setting due to disease-based lobbying from different stakeholder groups and government bodies. This can result in priorities being selected for funding that are not always priorities that the community deems most important. Rare diseases can be sidelined in biomedical research priority-setting in favour of diseases that are slightly more common within a geographic community in order to address the needs of a greater number of people. But, as one interviewee affirms, “I think you also can’t always go well if it’s common you have to call it a priority and if it’s rare you ignore it.”—African interviewee (09). Priority-setting for rare diseases is also affected by the number of different rare disease groups competing for money. In some instances, the “really emotive, or effective advocacy groups” (Australian interviewee 01), may get more funding. This was not identified as a “good or a bad thing”, but the interviewee cautioned it is important to be aware of these factors. Other interviewees highlighted strategies to manage bias such as (1) undertaking fair processes that achieve transparency and accountability and (2) understanding a given community and ensuring a diversity of participants from it. These strategies were not discussed in relation to disease-based lobbying but may be relevant to dealing with it.

Certain barriers varied by location or country context. Australian and African interviewees more commonly described barriers created by unsupportive local funding structures. In contrast, achieving adequate community engagement is often required and supported by UK funding agencies. Barriers related to unfair power dynamics, especially those grounded in coloniality, were reported more often by interviewees from Africa. They commented that an impact of biomedical research funding coming from the global North to the global South means the global North holds more power over the global South, and therefore controls the narrative and elevates voices of “those they want to hear”. This can be hugely disempowering.

Stigma associated with different diseases was also commonly reported by interviewees from Africa as affecting the scope of community engagement in biomedical research. Living with certain illnesses can be hugely stigmatising for community members, and they are, therefore, often excluded from engagement generally and during priority-setting processes.

Discussion

Whether the type of health research is ethically significant in specifying what engagement should entail is a key question to investigate. This study gathered empirical evidence that can help inform an answer. We interviewed biomedical (largely clinical and genomics) researchers and community engagement experts embedded in biomedical research from Australia, the UK, and several African countries to obtain their perspectives on whether community engagement should occur in priority-setting for biomedical research projects and, if so, how and for what purpose.

The researchers we interviewed strongly affirmed that engagement should occur as early as the priority setting phase of biomedical research projects, except under circumstances where engagement puts individuals or communities at risk of harm, and the risk cannot be mitigated. Their endorsement of early engagement is consistent not only with literature on engagement in applied and participatory health research [8, 32,33,34,35] but also with literature on engagement in genomics research [36, 37], in biomedical research [20, 38], and international biomedical research ethics guidelines [5, 39]. For instance, Ogurin et al. propose a four-stage model for community engagement in genomics research, where the first two stages include the process of conceptualising and defining the research question. Their model was developed based on interviews and focus groups with biomedical researchers, community rulers, opinion leaders, community health workers, and prospective research participants in Nigeria [36].

Study participants’ reasons for endorsing engagement during biomedical research priority-setting (see Table 1) also align with the findings of prior studies that speak to the value of early engagement in health research—namely, responsiveness, epistemic benefits, and community/local ownership or self-determination [8, 32, 40,41,42]. However, these and other studies identify reasons to value engagement that were not voiced by study participants here, including building relationships, opening doors, making those engaged feel valued, and giving them competence and confidence [32, 41]. This perhaps reflects the fact that our interviewees did not include those who had been engaged in health research.

Although most interviewees felt strongly that engagement should occur in biomedical research priority-setting, they gave two distinct responses on what ethical purpose it should serve. Some proposed engagement should have transformative goals, whereas others suggested instrumental goals. Both ethical goals have previously been described in the research ethics literature on engagement [4, 43] and debate continues over whether or not transformative goals should apply to engagement in health research. In this study, interviewees proposed two overlapping models of engagement as the ideal means to achieve empowerment goals: community-initiated and synergistic. The community-initiated model aligns more with Arnstein’s “citizen control” level of participation, whereas the synergistic model corresponds more closely with collaborative partnership and co-design approaches [14, 18]. Thus, the two models call for different levels of participation for those engaged.

In accordance with these findings, existing literature on engagement in genomics research also supports its being empowering and synergistic. May et al. purport that the techniques of community-based participatory research, which emphasise true partnership, should be applied in genomic science [44]. They affirm that such techniques can empower communities and can provide meaningful strategies to build trust, especially where underrepresented groups are engaged [44]. Watson et al.’s conceptual model for engagement in genomics research calls for an approach of “collaborative decision-making, facilitating dialogue, balancing power” that encompasses the priority-setting phase [37, p. 1]. Similarly, literature on participation [45,46,47], engagement in health research priority setting [19, 41], community-based participatory research [32], and co-design in health research [35, 48] all purport that transformative goals, collaborative partnership, and shared-decision-making are ideal or necessary to achieve more meaningful engagement.

In contrast, other interviewees in our study felt that engagement in biomedical research priority-setting should seek to achieve instrumental goals using a consultative model. Their views are consistent with some of the existing ethics literature on what engagement should look like in biomedical research [15, 49, 50], though that literature does not discuss the priority-setting phase specifically. In relation to genomics research, including priority-setting, Ogurin et al. also argue for meaningful community engagement as a way to ensure the success of a research program [36].

Although no consensus existed amongst interviewees on what the ethical goal(s) and model of community engagement should be in biomedical research priority-setting, identifying instrumental goals and consultative models as ideal is perhaps less common than in many forms of applied health research, which tends to associate meaningful engagement with empowerment goals and co-design/synergistic models. Thus, this study offers initial evidence that meaningful engagement in priority-setting should potentially be defined slightly differently for biomedical research relative to applied health research. Empowerment and instrumental goals achieved by community-initiated, synergistic, or consultative models may each comprise meaningful engagement in biomedical research priority-setting, though more research is needed to further assess this, both conceptually and empirically (as discussed further below in relation to study limitations). Future work should determine whether robust ethical or philosophical arguments can be made for defining meaningful engagement more broadly in biomedical research. It should also consider whether the different goals and approaches should apply under different circumstances of biomedical research priority-setting. If they should vary by context, it is necessary to determine when/where different goals and models should be used, i.e., what contextual factors demand certain goals and models of engagement. Future research could usefully further investigate the rationale for why the community-initiated or synergistic model should be used to advance empowerment goals in biomedical research priority-setting and which (if either) is better at achieving them in practice. In this study, for example, epistemic arguments were made by biomedical researchers for relying on the synergistic model.

It is also important to note that our interviewees’ endorsement of empowerment goals and models contrasts with much current biomedical research practice, where the collaborative partnership thread of community engagement is less common and instrumental goals and approaches dominate [20]. Thus, this study offers initial evidence that engagement in biomedical research priority-setting, and more broadly, should not be dominated by instrumental goals and approaches. A different balance may be ethically appropriate than what is currently practiced.

Our study identified several individual and collective qualities of researchers and community members, as well as relational and environmental essentials to build if biomedical researchers want to engage meaningfully with communities in priority-setting. Our study also identified numerous personal, relational, and environmental barriers to assess for (and address) before commencing and/or during engagement. Many of these foundations and barriers are not specific to engagement in biomedical research or during the priority-setting phase. They have been identified in previous work on engagement in health research priority-setting [51, 52] and in the wider literature on participatory development and participatory health research [32, 53,54,55,56,57]. However, certain barriers are likely to be more common in certain country contexts. This study suggests funding structures that are unsupportive towards engagement in priority-setting are more common in African countries and Australia relative to the UK. Unequal power dynamics and stigma were emphasised more strongly by African interviewees as barriers to engagement, which is unsurprising given the context in which much health research occurs and has historically occurred in Africa (with the global North providing funding to the global South and controlling the agenda).

Some of the challenges identified in this study are unique to engagement in genomics research: identifying the community and genomic literacy. This result is support by the findings of Manafo et al. and Stauton et al., who affirm that “defining ‘community’ is challenging and can depend on the particular social, cultural and geographical context in which the [genomics] research takes place.” ([58, 59], p. 2).

It is critical to acknowledge the main limitations of this study. First, this study did not solicit the voices of research participants or their communities on whether, how and for what purpose they should be engaged in biomedical research priority-setting. As a matter of epistemic justice and democratizing knowledge within the ethics field, this is a key group to focus upon in subsequent research on this topic. Second, while interviewees spanned basic science, clinical, and genomics research, few respondents to our recruitment efforts were basic scientists. This perhaps reflects the fact that community engagement in such research is less common and thus there were not as many researchers who met our inclusion criteria. Future ethics research should also solicit the views of basic science researchers on the topic of community engagement in biomedical research priority-setting. Such research could also potentially seek out the perspectives of biomedical researchers who have not engaged communities in their work. Although this study only sought the views of those researchers with some community engagement experience, other biomedical researchers would still have views about whether community engagement should happen in priority-setting or not, though they would not be grounded in actual engagement experience. Third, interviewees were recruited from Australia, several African countries, and the UK only. While engagement in biomedical research is increasingly common in these countries, there are other countries where engagement is frequently occurring in biomedical research, including non-English speaking countries and countries in other regions. Future research should capture their views as well.

Conclusions

Our study shows that members of the biomedical research community support engagement in biomedical research priority-setting. However, interviewees did not demonstrate consensus on what ethical purpose it should serve or how it should be done. Some conveyed support for engagement to promote empowerment via co-design and community-initiated approaches, which are more common in forms of applied health research. Others endorsed a more instrumental consultative approach that is consistent with current biomedical research engagement practice. This finding suggests that how meaningful engagement in biomedical research priority-setting is defined should potentially look different to engagement in applied research priority-setting and that engagement should be undertaken differently to current practice. Going forward, there is still much more ethics research to do to further explore whether community engagement in priority-setting should look different in biomedical research relative to applied health research.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Notes

  1. The NIH is supportive of community engagement overall and has formalised it into their strategy for improving research translation. See https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5582944/; https://www.niehs.nih.gov/news/assets/docs_a_e/community_engagement_efforts_at_nih_examining_best_practices_to_bridge_community_and_research_agendas_508.pdf; https://ncats.nih.gov/engagement; https://datascience.nih.gov/community-engagement.

Abbreviations

CIOMS:

Council for International Organizations of Medical Sciences

NIH:

National Institutes of Health

UK:

United Kingdom

UNAIDS:

Joint United Nations Programme on HIV/AIDS

References

  1. Solomon MZ, Gusmano MK, Maschke KJ. The ethical imperative and moral challenges of engaging patients and the public with evidence. Health Aff (Millwood). 2016;35(4):583–9.

    Article  Google Scholar 

  2. Nunn JS, Tiller J, Fransquet P, Lacaze P. Public involvement in global genomics research: a scoping review. Front Public Health. 2019. https://doi.org/10.3389/fpubh.2019.00079.

    Article  Google Scholar 

  3. Participants in the Community Engagement and Consent Workshop. Consent and community engagement in diverse research contexts. J Empir Res Hum Res Ethics. 2013;8(4):1–18.

    Article  Google Scholar 

  4. Reynolds L, Sariola S. The ethics and politics of community engagement in global health research. Crit Public Health. 2018;28(3):257–68.

    Article  Google Scholar 

  5. Council for International Organizations of Medical Sciences. International ethical guidelines for health-related research involving humans. Geneva: CIOMS; 2017. https://cioms.ch/wp-content/uploads/2017/01/WEB-CIOMS-EthicalGuidelines.pdf

  6. National Institutes of Health. Human heredity and heath in Africa (H3Africa): research projects (U01). 2016. http://grants.nih.gov/grants/guide/rfa-files/RFA-RM-16-015.html

  7. UNAIDS. Ethical considerations in biomedical HIV prevention trials. Geneva–Switzerland 2007. https://www.unaids.org/en/resources/documents/2012/20120701_jc1399_ethical_considerations

  8. Pratt B. Engagement as co-constructing knowledge: a moral necessity in public health research. Bioethics. 2019;33(7):805–13.

    Article  Google Scholar 

  9. Hall P, Smith J, Kay A, et al. Introduction: learning from the social economy community-university research partnerships. In: Hall PV, MacPherson I, editors., et al., Community-university research partnerships: reflections on the Canadian social economy experience. Victoria: University of Victoria; 2011. p. 1–27.

    Google Scholar 

  10. MacQueen KM, Bhan A, Frohlich J, Holzer J, Sugarman J, the Ethics Working Group of the HIVPTN. Evaluating community engagement in global health research: the need for metrics. BMC Med Ethics. 2015;16(1):44.

    Article  Google Scholar 

  11. Fregonese F. Community involvement in biomedical research conducted in the global health context; what can be done to make it really matter? BMC Med Ethics. 2018;19(1):44.

    Article  Google Scholar 

  12. Goulet D. Development ethics: a guide to theory and practice. New York: Apex Press; 1995.

    Google Scholar 

  13. Crocker DA. Ethics of global development: agency, capability, and deliberative democracy. Cambridge: Cambridge University Press; 2008.

    Book  Google Scholar 

  14. Arnstein SR. A ladder of citizen participation. J Am Inst Plan. 1969;35(4):216–24.

    Article  Google Scholar 

  15. Dickert N, Sugarman J. Ethical goals of community consultation in research. Am J Public Health. 2005;95(7):1123–7.

    Article  Google Scholar 

  16. Marsh V, Kamuya D, Rowa Y, Gikonyo C, Molyneux S. Beginning community engagement at a busy biomedical research programme: experiences from the KEMRI CGMRC-Wellcome Trust Research Programme, Kilifi. Kenya Soc Sci Med. 2008;67(5):721–33.

    Article  Google Scholar 

  17. Hall B, Tandon R, Lepore W, Singh W, Easby A, Tremblay C. Theoretical pedagogical framework for community-based research. In: Tandon R, Hall B, Lepore W, Singh W, editors. Knowledge and engagement: building capacity for the next generation of community-based researchers. Canada: Participatory Research in Asia (PRIA); 2016. p. 7–39.

    Google Scholar 

  18. Tindana PO, Singh JA, Tracy CS, Upshur REG, Daar AS, Singer PA, et al. Grand challenges in global health: community engagement in research in developing countries. PLoS Med. 2007;4(9):e273.

    Article  Google Scholar 

  19. Pratt B. Inclusion of marginalized groups and communities in global health research priority-setting. J Empir Res Hum Res Ethics. 2019;14(2):169–81.

    Article  Google Scholar 

  20. Vincent R, Adhikari B, Duddy C, Richardson E, Wong G, Lavery J, et al. ‘Working relationships’ across difference—a realist review of community engagement with malaria research [version 1; peer review: 1 approved]. Wellcome Open Res. 2022;7:13.

    Article  Google Scholar 

  21. Collins FS, Green ED, Guttmacher AE, Guyer MS, USNHGRI. A vision for the future of genomics research. Nature. 2003;422(6934):835–47.

    Article  Google Scholar 

  22. Clavisi O, Bragge P, Tavender E, Turner T, Gruen RL. Effective stakeholder participation in setting research priorities using a global evidence mapping approach. J Clin Epidemiol. 2013;66(5):496-502.e2.

    Article  Google Scholar 

  23. Groot B, Haveman A, Buree M, Rv Z, Jv Z, Abma T. What patients prioritize for research to improve their lives and how their priorities get dismissed again. Int J Environ Res Public Health. 2022;19(4):1927.

    Article  Google Scholar 

  24. Petryna A. When experiments travel: clinical trials and the global search for human subjects. Princeton: Princeton University Press; 2009.

    Book  Google Scholar 

  25. Gurdasani D, Carstensen T, Tekola-Ayele F, Pagani L, Tachmazidou I, Hatzikotoulas K, et al. The African genome variation project shapes medical genetics in Africa. Nature. 2015;517(7534):327–32.

    Article  Google Scholar 

  26. van Bekkum JE, Hilton S. UK research funding bodies’ views towards public participation in health-related research decisions: an exploratory study. BMC Health Serv Res. 2014;14(1):318.

    Article  Google Scholar 

  27. National Institutes of Health. Human heredity and health in Africa (H3 Africa): Collaborative Centers (U54) 2016. http://grants.nih.gov/grants/guide/rfa-files/RFA-RM-16-016.html

  28. Geertz C. Thick description: toward an interpretative theory of cultures: selected essays. New York: Basic Books; 1973.

    Google Scholar 

  29. Legard R, Keegan J, Ward K. In-depth interviews. In: Ritchie JL, editor. Qualitative research practice: a guide for social science students and researchers. London: SAGE Publications; 2003.

    Google Scholar 

  30. Hruschka DJ, Schwartz D, John DC, Picone-Decaro E, Jenkins RA, Carey JW. Reliability in coding open-ended data: lessons learned from HIV behavioral research. Field Methods. 2004;16(3):307–31.

    Article  Google Scholar 

  31. Campbell JL, Quincy C, Osserman J, Pedersen OK. coding in-depth semi structured interviews: problems of unitization and intercoder reliability and agreement. Sociol Methods Res. 2013;42(3):294–320.

    Article  Google Scholar 

  32. Cargo M, Mercer SL. The value and challenges of participatory research: strengthening its practice. Annu Rev Public Health. 2008;29:325–50.

    Article  Google Scholar 

  33. Oswald K. Interrogating an engaged excellence approach to research. Brighton: Institute of Development Studies; 2016.

    Google Scholar 

  34. Tembo D, Hickey G, Montenegro C, Chandler D, Nelson E, Porter K, et al. Effective engagement and involvement with community stakeholders in the co-production of global health research. BMJ. 2021;372:n178.

    Article  Google Scholar 

  35. Egid BR, Roura M, Aktar B, Amegee Quach J, Chumo I, Dias S, et al. “You want to deal with power while riding on power”: global perspectives on power in participatory health research and co-production approaches. BMJ Glob Health. 2021. https://doi.org/10.1136/bmjgh-2021-006978.

    Article  Google Scholar 

  36. Ogunrin O, Gabbay M, Woolfall K, Frith L. Community engagement in genomic research: Proposing a strategic model for effective participation of indigenous communities. Dev World Bioeth. 2021. https://doi.org/10.1111/dewb.12329.

    Article  Google Scholar 

  37. Watson KS, Cohn EG, Fair A, Menon U, Szalacha LA, Carpenter SM, et al. Adapting a conceptual framework to engage diverse stakeholders in genomic/precision medicine research. Health Expect. 2022;25(4):1478–85.

    Article  Google Scholar 

  38. Woolf SH, Zimmerman E, Haley A, Krist AH. Authentic engagement of stakeholders in assessing local health problems; planning, conducting patients and communities can transform research, practice, and policy. Health Aff. 2016;35(4):590–4.

    Article  Google Scholar 

  39. H3Africa. H3Africa Guidelines for Community Engagement (Version Two). Developed by the H3Africa Community Engagement Working Group for the Human Heredity and Health (H3Africa) Consortium. 2017. https://h3africa.org/wp-content/uploads/2018/05/CE%20Revised%20Guidelines_Final_September%202017%20(1).pdf

  40. Ellis LE, Kass NE. How are PCORI-funded researchers engaging patients in research and what are the ethical implications? AJOB Empir Bioeth. 2017;8(1):1–10.

    Article  Google Scholar 

  41. Pratt B. What should engagement in health research look like? Perspectives from people with lived experience, members of the public, and engagement managers. Camb Q Healthc Ethics. 2022;31(2):263–74.

    Article  Google Scholar 

  42. Oswald K, Gaventa J, Leach M. Introduction: interrogating engaged excellence in research. IDS Bull. 2016;47(6):1–18.

    Article  Google Scholar 

  43. Ahmed SM, Palermo AG. Community engagement in research: frameworks for education and peer review. Am J Public Health. 2010;100(8):1380–7.

    Article  Google Scholar 

  44. May T, Bogar S, Spellecy R, Kabasenche W, Craig J, Dick D. Community-based participatory research and its potential role in supporting diversity in genomic science. J Health Care Poor Underserved. 2021;32(3):1208–24.

    Article  Google Scholar 

  45. Hickey S, Mohan G. Towards participation as transformation: critical themes and challenges. In: Hickey S, Mohan G, editors. Participation from tyranny to transformation. London: Zed Books; 2004. p. 2–24.

    Google Scholar 

  46. Gaventa J. Finding the spaces for change: a power analysis. IDS Bull. 2006;37(6):23–33.

    Article  Google Scholar 

  47. Crocker DA. Forward. In: Goulet D, editor. Development ethics at work: explorations—1960–2002. New York: Routledge; 2006. p. 20.

    Google Scholar 

  48. Mannell J, Amaama SA, Boodoosingh R, Brown L, Calderon M, Cowley-Malcolm E, et al. Decolonising violence against women research: a study design for co-developing violence prevention interventions with communities in low and middle income countries (LMICs). BMC Public Health. 2021;21(1):1147.

    Article  Google Scholar 

  49. Weijer C, Emanuel EJ. Protecting communities in biomedical research. Science. 2000;289(5482):1142–4.

    Article  Google Scholar 

  50. King KF, Kolopack P, Merritt MW, Lavery JV. Community engagement and the human infrastructure of global health research. BMC Med Ethics. 2014;15(1):84.

    Article  Google Scholar 

  51. Pratt B. Towards inclusive priority-setting for global health research projects: recommendations for sharing power with communities. Health Policy Plan. 2019;34(5):346–57.

    Article  Google Scholar 

  52. Pratt B. Achieving inclusive research priority-setting: what do people with lived experience and the public think is essential? BMC Med Ethics. 2021;22(1):117.

    Article  Google Scholar 

  53. Cornwall A. Whose voices? Whose choices? Reflections on gender and participatory development. In: Cornwall A, editor. The participation reader. New York: Zed Books; 2011. p. 203–23.

    Google Scholar 

  54. Muhammad M, Wallerstein N, Sussman AL, Avila M, Belone L, Duran B. Reflections on researcher identity and power: the impact of positionality on community based participatory research (CBPR) processes and outcomes. Crit Sociol (Eugene). 2015;41(7–8):1045–63.

    Article  Google Scholar 

  55. Ponic P, Reid C, Frisby W. Cultivating the power of partnerships in feminist participatory action research in women’s health. Nurs Inq. 2010;17(4):324–35.

    Article  Google Scholar 

  56. Wallerstein N, Duran B. The theoretical, historical, and practical roots of CBPR. In: Wallerstein N, Minkler M, editors. Community-based participatory research for health: from process to outcomes. San Franciso: John Wiley & Sons; 2010. p. 25–46.

    Google Scholar 

  57. Leach M, Scoones, I. Science and citizenship in a global context. IDS Working Paper. Brighton: Institute of Development Studies; 2003.

  58. Manafò E, Petermann L, Vandall-Walker V, Mason-Lai P. Patient and public engagement in priority setting: a systematic rapid review of the literature. PLoS ONE. 2018;13(3):e0193579.

    Article  Google Scholar 

  59. Staunton C, Tindana P, Hendricks M, Moodley K. Rules of engagement: perspectives on stakeholder engagement for genomic biobanking research in South Africa. BMC Med Ethics. 2018;19(1):13.

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank all interviewees for taking the time to participate in this study.

Funding

BP was supported by an Australian Research Council Discovery Early Career Researcher Award (DE170100414).

Author information

Authors and Affiliations

Authors

Contributions

BP conceived of the study: its topic, aims, and methods. JB and NE were primarily responsible for recruiting interviewees, conducting interviews, and undertaking thematic analysis, though BP oversaw and contributed to both data collection and analysis. BP and JB were responsible for writing the first draft of the paper and revising the work critically for intellectual content. NE revised the paper critically for intellectual content. All authors gave final approval of the version to be published.

Corresponding author

Correspondence to Bridget Pratt.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the Melbourne School of Population and Global Health (MSPGH) Human Ethics Advisory Group (HEAG) at the University of Melbourne (Ethics ID: 1749720.5). Written informed consent was obtained from all interviewees. All procedures were performed in accordance with the National Health and Medical Research Council of Australia’s National Statement on Ethical Conduct of Human Research.

Consent for publication

Not applicable.

Competing interests

BP is a member of the editorial board (Associate Editor) of BMC Medical Ethics. NE and JB have no competing interests to declare.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Borthwick, J., Evertsz, N. & Pratt, B. How should communities be meaningfully engaged (if at all) when setting priorities for biomedical research? Perspectives from the biomedical research community. BMC Med Ethics 24, 6 (2023). https://doi.org/10.1186/s12910-022-00879-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12910-022-00879-5

Keywords