Article Text
PDF
Abstract
Introduction Indigenous communities worldwide are leading calls for all research involving Indigenous people to be underpinned by values and principles articulated by them. Many researchers are explicitly adopting these principles to guide what, where, how and when research is undertaken with Indigenous people. With critical reflection to support the implementation of such principles largely absent from published literature, this paper explores both the implementation of, and the outcomes from a set of guiding principles used in a large-scale Australian research collaboration to improve Indigenous health.
Methods In this inductive qualitative study, we adopted a principles-focused evaluation approach. Based on interviews with 35 actors in the collaboration and a review of project documents, we generated themes that were then iteratively discussed, refined and categorised into (1) ‘strategies’—activities by which implementation of our guiding principles were recognised; (2) ‘outcomes’—results seen from implementing the principles and (3) ‘conditions’—aspects of the context that facilitated and constrained implementation of the principles.
Results Respondents found it difficult to articulate how the guiding principles were actually implemented, and frequently referred to them as part of the fabric of the collaboration. They viewed the set of principles as mutually reinforcing, and as providing a rudder for navigating complexity and conflict. Implementation of the principles occurred through five strategies—honouring the principles; being dynamic and adaptable; sharing and dispersing leadership; collaborating purposefully and adopting a culture of mutual learning. Outcomes included increased Indigenous leadership and participation; the ability to attract principled and values-driven researchers and stakeholders, and the development of trusting and respectful relationships. The conditions that facilitated the implementation of the principles were collaborating over time; an increasing number of Indigenous researchers and taking an ‘innovation platform’ approach.
Conclusion Our findings show that principles guiding collaborations are valuable in providing a focus, direction and a way of working together when they are collaboratively developed, hold genuine meaning for all members and are implemented within a culture of continuous critical reflection, learning and adaptation, with ongoing reinterpretation of the principles over time.
- health services research
- health systems
- health systems evaluation
- health policy
- qualitative study
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Key questions
What is already known?
Researchers are beginning to adopt principles articulated by (or in collaboration with) Indigenous people to guide what, where, how and when Indigenous health research is undertaken in Indigenous communities.
To date, there have been limited publications that critically reflect on how, to what end and even whether, such principles are being implemented.
What are the new findings?
Respondents found it difficult to articulate how each principle worked individually; instead, they discussed the principles as operating as a mutually reinforcing set.
The principles were implemented through five strategies—honouring the principles; being dynamic and adaptable; sharing and dispersing leadership; collaborating purposefully and adopting a culture of mutual learning.
Respondents identified that implementing the principles led to increased Indigenous leadership and participation, the ability to attract principled and values-driven researchers and stakeholders, and the development of trusting and respectful relationships.
The conditions that facilitated the implementation of the principles were collaborating over time, an increasing number of Indigenous researchers and taking an ‘innovation platform’ approach.
Key questions
What do the new findings imply?
Principles that are developed collaboratively, hold genuine meaning for all members, and are implemented within a culture of continuous critical reflection, learning, adaptation, and ongoing re-interpretation provide research collaborations with a focus, direction and a way of working together.
Evaluating principles involves more than simply measuring implementation, but rather adopting a systems thinking perspective that asks how are principles apparent in the work, and in what ways do they influence the dynamics of the network as a whole.
Introduction
There remain concerns that Indigenous populations continue to be over-researched without corresponding improvements in their health.1 2 This has led Indigenous communities around the world to call for more ethical processes in the conduct of research and evaluation.1–4 In Australia, national documents guiding research with Aboriginal and Torres Strait Islander people (Aboriginal and Torres Strait Islander peoples are hereafter referred to respectfully as Indigenous Australians, acknowledging cultural and historical diversity) specify that all research must be underpinned by values and principles that are important to, and defined by Indigenous people.5 6 Despite this, the available literature provides limited critical reflection and evaluation of how such research principles are meaningfully implemented.7
Practical examples of how principles are implemented and evaluated could help to strengthen implementation and learning across settings.7 8 To this end, a principles-focused evaluation approach has recently emerged which examines ‘(1) whether principles are clear, meaningful and actionable, and if so, (2) whether they are actually being followed and, if so, (3) whether they are leading to the desired results.’9 (p.3) Patton explains that principles-focused evaluation is an approach rather than a methodology, with an essential component that examines evidence to ask ‘how does the principle work and with what results, if any?9 (p.6)
The terms ‘principles’ and ‘values’ are at times conflated or used interchangeably. Figure 1 shows our perspective on the relationship between these two concepts and how they relate to ‘practices’. Values describe what is important in the life of an individual or community (eg, what is valued by an Indigenous population). Values rarely change. However, principles are based on norms, values, beliefs, experience and knowledge, and provide direction as a basis for action.9 They guide us in how to think and behave. While values are enduring, principles can evolve and change based on circumstances. As such, they are continuously contextualised and situationally interpreted7 (eg, to guide the way health research is done with Indigenous people). Practices are how principles are expressed and actioned (ie, applied in research practice).
Relationship between values, principles and practices.
The study draws on a principles-focused evaluative approach to explore the processes by which a set of guiding principles were implemented, and the outcomes of this implementation, in the context of a large-scale Indigenous health research collaboration—the Centre for Research Excellence in Integrated Quality Improvement (CRE-IQI).10 11 We address the question: How was the set of principles we developed to guide the research collaboration implemented, and, what were the outcomes? Addressing this question enabled the CRE-IQI to be more cognisant of the process and outcomes of applying our principles, thereby learning from self-reflection. The findings provide an opportunity for others to learn from our experience with this relatively under-reported evaluation approach.
Methods
Study setting
Indigenous Australians have demonstrated extraordinary cultural strength, adaptability and resilience across time. Yet they continue to experience worse health outcomes and shorter life expectancies than other Australians.12 These inequalities result from a pervasive legacy of colonisation, land dispossession, displacement, disempowerment, social and economic exclusion and ongoing racial discrimination.13
With the vision of improving Indigenous health outcomes by strengthening primary healthcare (PHC) systems through continuous quality improvement (CQI) efforts, Australia’s National Health and Medical Research Council funded the CRE-IQI from 2015 to 2019.10 14 15 Building on more than two decades of participatory CQI research and development with Indigenous communities, and operating as an ‘innovation platform,’10 the CRE-IQI purposefully brought together PHC centres (both Aboriginal community-controlled and Government-managed), research institutions, government health departments and key regional support organisations (such as health councils) to work on ways of strengthening system-wide CQI.10 16 17 Innovation platforms are characterised by collective problem solving, the exchange of ideas and the sharing of expertise to generate knowledge and change. Continuous reflection, adaptation and learning are also key elements.11
The CRE-IQI was implemented as an ‘open collaboration’ in which new members were welcomed to join the collaboration. Within the scope of ‘IQI’17 research priorities were collaboratively developed and refined to address the needs of key stakeholders. This approach enabled PHC practitioners and policy-makers to articulate the knowledge gaps, and to work with researchers and health sector stakeholders on relevant research topics.10 16 New collaborations were encouraged through sharing information, open seed-funding calls for the development of projects and the promotion of collaborative research. Stakeholders participated in biannual face-to-face meetings that provided opportunities to progress project development and research translation, hear about project methodologies, findings and outcomes, share ideas and build relationships. Masterclasses were offered around each bi-annual meeting to increase the skills and knowledge of CRE–IQI members. Further details about the aims and functions of the research collaboration are published elsewhere.10 11 14
Development of the guiding principles for the research collaboration
CRE-IQI investigators and members brought considerable experience in Indigenous PHC and CQI research to the collaboration, including experience in articulating and applying principles for guiding their work.18 Many members were familiar with the national guidelines on ethics and values in Indigenous health research and were committed to putting values into practice through principled research. The CRE-IQI drew on the principles developed by a closely associated Indigenous-led national research collaboration (DISCOVER-TT)19 as a basis for their research principles. The DISCOVER-TT research principles were refined, with permission, through consultation with the CRE-IQI members. The outcome of this process was the collective development of eight guiding principles to encompass and lead the work of the research collaboration (box 1).
Principles of the research collaboration
Respect the past and present experiences of Indigenous people.
Work in partnership.
Ensure Indigenous leadership and direction of research—in all stages of the process.
Conduct ethical research.
Get the research question right.
Design research that will be feasible, produce outcomes and build capacity.
Identify and provide the right resources and training
Establish systems and practices to support the application of evidence to improve Indigenous primary healthcare and health outcomes
Study design
Interviews were conducted with a purposive sample of 35 respondents in the CRE-IQI research collaboration. The study used an inductive qualitative approach deemed appropriate both for Indigenous settings and for questions where there is little prior research and participant voices have previously been rendered invisible.3 20 We adopted a constructivist perspective, which assumes that neither data nor theories are discovered but rather are constructed based on the shared experiences of researchers and respondents.3 20 21 Design and reporting of our study were guided by the Consolidated criteria for Reporting Qualitative research guidelines22 (online supplemental additional file 1).
Supplemental material
Participants
Purposive sampling techniques were used to ensure that we captured a diversity of relevant views from members in the CRE-IQI,23 with the aim of including wide representation from Indigenous people, organisations and roles. Study respondents met one or more of the following criteria:
Be an investigator or a member of the management committee listed on the original research grant.
Hold a past or currently funded position in the research collaboration.
Be a chief investigator of research that was aligned with the research collaboration.
Be a student or early career researcher aligned with the research collaboration.
Be a member of the Indigenous Advisory Committee or Research Advisory Committee listed on the original research grant.
Patient and public involvement
No patients or members of the public were involved in the design, analysis or reporting of this study.
Data collection
Individual interviews
We developed an interview guide, published elsewhere,11 which used open-ended questions to explore perceptions on how the CRE-IQI principles were developed; the importance and meaning of the principles to respondents and the collaboration; how implementation of the principles occurred; critical points in the collaboration where the principles were modified or elevated in response to emerging issues or contexts and the outcomes of implementing the principles.
The first two authors, JB and AFL, conducted interviews in the final year of the research collaboration (April to May 2019) using videoconferencing or by telephone. As interviewers, we encouraged respondents to tell stories and provide examples of their experience. We also critically reflected on our own assumptions to promote a heightened awareness of listening to stories as openly as possible. Respondents were reminded of the principles of the research collaboration prior to the interview to allow them time for reflection. Interviews averaged 39 min (range 21–75 min), were audiorecorded and professionally transcribed.
Document review: administrative project records
Data sources from administrative project records included minutes from meetings; lists of publications, presentations, students and grants; preliminary findings from a network analysis and impact assessment; and results from the developmental evaluation of the CRE-IQI published in year 2 and year 4.11 14
Data analysis
To assist the qualitative analysis, a reflective summary was generated after each interview. The first two authors (JB and AFL) met regularly to identify and discuss emerging themes and the direction of subsequent interviews—with the analysis commencing during the interview process. It became apparent early in our analysis that respondents referred to the principles as a set, rather than individually. This realisation led us to focus subsequent interviews and analysis on the principles in a more general and complementary sense, rather than on each specific principle.
Interviews, documents and reflective summaries were loaded in NVivo qualitative data management software for coding, searching and organising data. The lead author (JB) read all the interview transcripts multiple times, making reflective notes in the process. JB open coded the data, with AFL independently coding 10 interviews, followed by joint review and discussion of the initial coding to ensure consistency in the analytic process.24 Following this, there was then an iterative process of more focused coding,25 with the refinement of codes being discussed with AFL, KPC, KH, VM and senior author RGB—with a focus on exploring interconnections in an iterative process of analysis.
Through this process of coding and discussions themes were identified, refined and categorised into three higher order categories including: (1) ‘strategies’—specific activities by which implementation of the principles were recognised; (2) ‘outcomes’—results seen from implementing the principles and (3) ‘conditions’—aspects of context that facilitated or constrained implementation of the principles. ‘Outcomes’ sometimes operated as strategies, or as conditions. The distinction was not always clear. We categorised strategies/conditions/outcomes according to their predominant ‘influence’ or ‘function’ as identified through interviews and the iterative analysis process, and as reflected in the frequency and relative importance that each theme was associated with each category.
To deepen our understanding of the interview findings, project documents were analysed concurrently, using an iterative process to identify major decisions and developments in the collaboration, and to clarify key issues, dates and events. The robustness of the findings were enhanced by: comparing, contrasting and seeking consensus of findings between coauthors; presenting early findings at meetings of the research collaboration as a way to member check findings; and triangulation with findings from document reviews.
Results
We approached 52 people via email to participate in the study, of whom 35 (67%) agreed to be interviewed. The majority of respondents were researchers, although many had dual roles, for example, clinician and researcher. Eight respondents identified as Indigenous and a further seven were employed in organisations established to support Indigenous health services (table 1).
Individual interview respondent characteristics by organisation type, position type, jurisdiction and gender
Respondents initially found it difficult to articulate how the principles were implemented and many referred to them as being intrinsic to the collaboration, as illustrated by the following quote:
In terms of [a] practical application of them, I find that hard to comment on because most of them are… just kind of non-negotiables that you really couldn’t be [working] in this space if you didn't adhere to them. So, I would say that applies to a lot of people in this CRE… it’s so much ingrained in what you do… they’re just very much woven into the fabric of everything, every activity. (Researcher, Non-Indigenous, University / Research Institute)
The principles were viewed as a guide to a way of working, of providing a direction rather than being prescriptive. Importantly, the principles were viewed by most respondents as a package comprising integral parts that reinforced one another and were not to ‘be unpicked and separated’. Given this consistently expressed view, we discuss implementation of the principles as a set rather than individually.
The specific activities undertaken to implement the principles (‘strategies’), the results seen from implementing these principles (‘outcomes’), and the aspects of context that facilitated or constrained implementation of the principles (‘conditions’) are depicted in figure 2.
Strategies, outcomes and conditions related to how principles were implemented in the research collaboration.
Strategies
The implementation of the principles within the CRE-IQI occurred through the five overlapping strategies of honouring the principles; being dynamic and adaptable; sharing and dispersing leadership; collaborating purposefully; and adopting a culture of mutual learning.
Honouring the principles
There was strong agreement about the importance of having the principles in place ‘from the outset’, and explicitly honouring them in the work of the collaboration. This was done by displaying and discussing them at the start of biannual meetings; requiring that they be addressed in all applications for seed funding; and listing them in the front of the majority of CRE-IQI reports.
I think it’s been really good to be continually reminded and to have [the principles] articulate very clearly what we’re about, and to refer back to that again and again… when we’re thinking about new pieces of work… (Program Manager, Non-Indigenous, Indigenous Community-Controlled Sector Support Organisation)
Further examples included instigating research coleadership arrangements between Indigenous and non-Indigenous members; prioritising funding support to Indigenous people and/or organisations; and encouraging Indigenous people and organisations to present and lead sessions whenever members met.
Our values and principled approach meant a deepening, over time, in the relationship between the collaboration and the Indigenous Elder who regularly provided the Welcome to Country at face-to-face biannual meetings. The implementation of the principles led to her feeling comfortable to remain for part of the meeting, and to listen, share stories and provide insights on our discussion, thus joining the collaboration itself.
Respondents new to working in Indigenous health particularly valued having the principles articulated and discussed at the biannual meetings of stakeholders and in applications for funding to develop research. The principles were seen as ‘a kind of scaffolding’ that enabled newcomers to the collaboration to identify with, and participate in, our way of working. Some did suggest, however, that they would have benefited from further discussions of what the principles meant and how to implement them appropriately.
Being dynamic and adaptable
The principles were observed as being dynamic, with their implementation evolving over time, based on reflection, discussion and feedback. Continual reflection on the principles was viewed as a positive strategy for refining and implementing them, keeping them valid and relevant, and enabling them to evolve alongside the CRE-IQI.
I think they’re a good thing to have from my perspective, but you have to operationalise them in some way… you’ve done that. The other thing, as you have done, is just reemphasising that these are not a static sort of set of principles, they have to be continually evolving as the work and people and new issues emerge. (Researcher, Indigenous, University / Research Institute)
Having the principles clearly articulated and regularly reflected on offered collaboration members an accountability lens through which they could question whether or not the collaboration was adhering to its principles—for example, indicators were developed and reported on over time to enhance efforts to ensuring adherence to the principle of increasing Indigenous participation and leadership.
Sharing and dispersing leadership
Many respondents described how, over time, leadership roles became increasingly shared and dispersed. This process was perceived as an example of implementing the principles through different forms of interactive leadership.
The CRE-IQI research capacity strengthening programme focused on facilitating Indigenous leadership and participation at all levels.26 Coleadership arrangements were established between Indigenous and non-Indigenous researchers in many of the CRE-IQI projects, which in turn created bonds of interdependence, respect and trust.
So, when we have our weekly meetings, while [XX] is away they’re led by [two Aboriginal academics], and even before that there was that real respect between the co-leaders, and… ensuring that it wasn’t tokenism, that there was true decision-making. (Researcher, Non-Indigenous, University/ Research Institute)
Coleadership arrangements were consistently highlighted as a strategy through which to implement the principles, but they were also perceived to be a tangible outcome of operationalising the principles through increased Indigenous leadership and participation. This is an example of the overlap between outcomes and strategies in which a strategy was implemented, and the outcome was the successful realisation of that strategy—namely, an increase in Indigenous leadership and participation.
Most respondents expressed enthusiasm for the concept of co-leadership, with some noting that use of the term coleadership ‘breathes new life’ into the pursuit of not-so-new arrangements. However, tensions relating both to coleadership arrangements and to Indigenous and non-Indigenous ways of doing business were also identified. Funding applications for competitive grants, for example, may benefit from using a conventional hierarchical structure for investigators and showing traditional academic track records for improving chances of success within current assessment processes. Some respondents spoke about the need to keep pushing boundaries and to disrupt traditional views of research leadership. The importance of not overwhelming the perceived ‘relatively small pool’ of Indigenous researchers was also identified as a point of tension, as expressed here:
There is a tension of ensuring leadership and participation but also ensuring people aren’t overloaded. (Researcher, Indigenous, University / Research Institute)
A few respondents sensed that some non-Indigenous people felt threatened by the concept of Indigenous-led and co-leadership arrangements, and what this could mean for their jobs.
The risk of losing something means that people will keep the status quo, because they haven’t done the thinking through… that losing means gaining. But when you create a void, something new comes in. (Researcher, Indigenous, University / Research Institute)
Several project managers with experience of co-leadership arrangements spoke about having to be deliberate in their actions to make space and to ‘take a step back’, neither of which was necessarily an easy thing to do. There needed to be a negotiation of roles and expectations, and finding new ways of working. Having the principles in place helped, because there was a shared commitment to making arrangements work and to prioritising these arrangements.
Collaborating purposefully
The concept of ‘collaborating purposefully’ was reflected in the effort and resources allocated to increasing the participation of both Indigenous people and of representatives from Indigenous organisations at CRE-IQI meetings, which were generally perceived as safe spaces to voice concerns and discuss issues openly.
I felt that the implementation of the principles was deliberate, it was certainly done with absolute intention. (Researcher, Non-Indigenous, University/ Research Institute)
Purposeful collaboration brought people together in respectful ways to operationalise the principles by encouraging participation and creating mechanisms for further collaboration and increased Indigenous leadership. In turn, the meetings provided opportunities to plan further implementation of the principles and programmes of work.
Many respondents identified a specific bi-annual meeting midway through the collaboration life-cycle as an important turning point for implementing the principles. The Indigenous Advisory Committee had not been operating as conceived (because of overcommitment by Committee members), and at this biannual meeting there were deliberate discussions about how to uphold the principles and to bring more Indigenous people into the collaboration to ensure Indigenous leadership and governance. Several respondents recalled the tension in the ensuing discussions, and how the principles helped to guide the CRE-IQI as it navigated tension.
I think there was a little bit of conflict, that might not be the right word, but I guess it is close enough. I think … we all were probably a bit threatened by the whole conversation because there was an element of frustration from our Indigenous colleagues in feeling that they don’t always have equal voices. Whenever anyone gives you any pushback it does make you feel a bit awkward, a bit uncomfortable because your genuine intent is to have really good engagement. … The principles helped us navigate this tension. (Program Manager, Non-Indigenous, Indigenous Community-Controlled Sector Support Organisation)
Respondents also recognised the importance of being able to bring different perspectives safely to the surface.
It’s OK to have the tensions and disagreements … because they bring out a level of understanding and appreciation for where priorities are for different people and organisations, but there has to be a safe space for this to occur. Like in the CRE where … the principles helped to guide this tension related to Indigenous leadership and participation. (Researcher, Indigenous, University/ Research Institute)
Adopting a culture of mutual learning
Recognising the need to nurture a culture of mutual learning, the collaboration established a funded position and programme for research capacity strengthening.
The capacity strengthening program, this is where the rubber meets the road in terms of implementation of principles. It was about actually putting it into practice… in a more collaborative way, more deliberative dialogue, without it being a tokenistic thing, genuinely having Aboriginal co-leadership, valuing mutual learning through an ‘All teach, all learn’ approach. (Program Manager, Non-Indigenous, Indigenous Community-Controlled Sector Support Organisation)
The capacity strengthening programme created a strategy of ‘All teach, all learn’, which placed value on mutual learning among all members of the collaboration - Indigenous and non-Indigenous researchers, research users, and communities—in the collaboration.26 Importantly, the research capacity strengthening programme and its ethos, which embodies the value of both Indigenous and Western knowledge, was seen to put the principles into action.
I think we see [the application of Principles]… in the bi-directional learning and that whole sort of expert–learner dyad, and the dynamic around that. And the certain creation of a safe space for people to move between those two poles… plays out throughout those few days when everyone’s together for the bi-annual meetings and masterclasses. (Researcher, Non-Indigenous, University/ Research Institute)
Outcomes
The implementation of the principles enabled trusting and respectful relationships, attracted principled and values-driven people, and increased Indigenous leadership and participation.
Enabled trusting and respectful relationships
Many respondents described the principles as working like a ‘code of conduct’, which assisted members in building relationships embodying the principles and supporting their implementation. The principles were seen to enable trustful and respectful relationships by providing guidance on navigating conflict, developing shared values and communicating respectfully. Having clearly articulated principles facilitated our understanding of how to work together to find common ground.
There have been points of conflict, points where agendas didn’t quite meet or where we felt threatened or uncomfortable by certain directions, but we’ve been able to have discussions with… [the] chief investigators and… senior researchers, and have nearly always been able to come to a happy balance, to accommodate each other’s needs. The principles worked as a code of conduct to help us navigate tensions and develop trusting relationships. (Program Manager, Non-Indigenous, Indigenous Community-Controlled Sector Support Organisation)
In the historical context of research not providing benefits for many Indigenous communities, several respondents hypothesised that trust was a necessary precursor to enabling this successful collaboration.
Attracted principled and values-driven people
As an open collaboration, new individuals and organisations were encouraged both to join and to feel able to leave as circumstances required. Openly articulating the principles attracted people whose values aligned with those of the collaboration.
I feel like I trust all those [long-standing researchers] that have been around. I trust their values—that you want to be involved with researchers who have similar values. (Health Service Manager, non-Indigenous, Aboriginal Community-Controlled Health Service)
Increased Indigenous leadership and participation
Implementing the principles was perceived to increase the number of Indigenous people attending meetings and events over time and to firmly establish Indigenous leadership.
I actually looked around the room and thought ‘Wow, look how much this group’s changed’. There were just lots more Indigenous people in the room, and in leadership roles. (Researcher, Non-Indigenous, University / Research Institute)
The application of the principles also shaped the successful funding twice daily for a new CRE in Strengthening Systems for Indigenous Health Care Equity (CRE-STRIDE).27 The leadership structure of this next phase of collaborative research has seen a significant increase in Indigenous representation—with the collaboration led by an Indigenous researcher and with 50% of the other investigators also Indigenous—an outcome most respondents perceived as resulting from the implementation of the principles.
[A] practical example of our application of principles is the new CRE-STRIDE. The fact that we have got an Indigenous Chief Investigator and at least 50 percent Indigenous investigators is a signal of the application of the principles. (Researcher, Indigenous, University/ Research Institute)
Conditions
The three conditions that facilitated the implementation of the principles were collaborating over time, an increasing number of Indigenous researchers and taking an innovation platform approach.
Collaborating over time
As many of the CRE-IQI members had been working together for nearly two decades, they had a shared purpose and understanding of applied CQI research, similar experiences in Indigenous health, and knowledge of each other’s strengths and weaknesses. This close collaboration over time helped to build trust among members, which ultimately facilitated the implementation of the principles.
Articulating principles for the collaboration was not a new concept, as a set of principles had been developed to promote research values and guide researchers in the early years of community-based CQI research.18 However, the ‘bedrock of trust’ that had been built over two decades provided new collaborators with a strong foundation and a model for working with others to put the CRE-IQI principles into practice:
There’s new people that have come in, but it’s coming to a base of a strong network already. Being a new person as well, I think there is a sense of security in that. It’s very clear from the network analysis that [the collaboration is] quite robust and it’s fairly stable in terms of its capacity to bring people in and integrate them into that network as it’s so strong already. (Researcher, Non-Indigenous, University/Research Institute)
Increasing numbers of Indigenous researchers
Another enabling condition relates to what one respondent referred to as the ‘rising tide of Aboriginal researchers’ in the research environment generally, with the increase in numbers facilitating the implementation of Indigenous research principles. For the CRE-IQI, increased interest in CQI research from Indigenous researchers was coupled with a greater expectation on growing the leadership and participation of Indigenous researchers within the collaboration. However, respondents highlighted a tension associated with this aspect of Australian health research funding, whereby the nation’s major health research funder demands strong academic track records from a limited pool of researchers with the requisite background and experience. An example offered by respondents was the CRE-IQIs original concept of having an Indigenous Advisory Committee in its governance structure. Attempts to operationalise this committee did not progress as planned, however, because those initially identified as members held ‘high-profile’ positions in Indigenous health, which meant that they had multiple, often competing demands on their time. As noted by one respondent who was listed as a member of the original Committee:
… while you can have Indigenous people on your grant applications with good track records and high profiles… there needs to be a discussion with some of those people about what kinds of roles are actually feasible for them to take on, and what the best way of using their time is. Just in terms of the demands, [there are] so many multiple demands on people’s time. (Researcher, Indigenous, University/Research Institute)
Despite this, the increase in the number of Indigenous researchers involved in the collaboration was almost certainly in part because of the existing positive relationships between researchers, the development of new and diverse research projects, collaboration expansion, and the previously described efforts to promote Indigenous leadership and governance of the CRE-IQI.
Taking an innovation platform approach
The third condition that enabled implementation of the principles was having the collaboration operate as an innovation platform.10 An innovation platform is a collaborative arrangement in which diverse members representing different parts of a system are brought together to promote mutual learning, identify problems collectively, and achieve shared solutions. The strategies respondents identified as being directly responsible for how the principles were implemented in the research collaboration—that is, sharing and dispersing leadership; collaborating purposefully; adopting a culture of mutual learning; and being dynamic and adaptable—were similar to those strategies generally required for the successful operation of an innovation platform. Thus, the innovation platform approach worked synergistically to enable the outcomes of implementing principles in the research collaboration.
Discussion
In this qualitative study, we identified strategies and conditions by which a set of principles were implemented in the context of a large-scale Indigenous health research collaboration, and the outcomes of implementing these principles. Implementation occurred through five strategies: honouring the principles; being dynamic and adaptable; sharing and dispersing leadership; collaborating purposefully; and adopting a culture of mutual learning. These strategies resulted in increased Indigenous leadership and participation and enabled trusting and respectful relationships between principled and values-driven people with a shared commitment to improving Indigenous health outcomes. Conditions within the research collaboration were important—not least because collaboration over time between many members had engendered trust, there was an increasing number of Indigenous researchers and taking an innovation platform approach facilitated the implementation of the principles.
The principles were viewed as an integrated whole, with overlap between individual principles. As respondents saw the principles interactively reinforcing and complementing each other, there was no corresponding ordering of the principles in relation to priority, with all determined to be important and interdependent. Collectively, the principles became the rudder for navigating complexity and conflict. Fluidity between the strategies, conditions and outcomes, and was also evident. For example, the outcome ‘increased Indigenous leadership and participation’ at times was a deliberate strategy.
The difficulty respondents had in articulating how the principles were actually implemented, and the common reference to them as part of the fabric of the collaboration, reveals congruency between ‘tacit knowledge’ and principles. This is in keeping with Polanyi’s28 famous expression—‘we know more than we can tell.’ Tacit knowledge, like the principles in our collaboration, is embedded within the context of work and, as such, is not easily communicated.29 30 Like tacit knowledges, the principles functioned as ‘situation-specific’ and ‘subconsciously understood’ ‘knowledge-in-practice’ which is ‘developed from experience and action’, and is ‘difficult to articulate’.31
We identified several ways in which the principles were implemented, although sometimes tacitly. For instance, consider the ‘purposeful collaboration’ example in which concerns were raised about the low levels of Indigenous leadership. In this example, we observe how the principles were present not only in guiding a process towards generating solutions and a marked turning point, but also in enabling respondents to identify and articulate the specific concern in the first place. Without explicit principles that prioritised respect for Indigenous peoples (ie, working in partnership, ensuring Indigenous leadership, etc) being articulated and agreed on by the collaboration, concerns about Indigenous participation and leadership might not have been raised or, critically, not given high priority.
The principles functioned like a compass, like setting a direction. Travelling in the desired direction requires ‘implementation’ of the principles. This involves (1) interpretation—and reinterpretation—of what achieving the principle looks like under current conditions; (2) taking stock of the current position in relation to the intended direction; and (3) identifying and taking steps to course-correct considering conditions and outcomes. In our example, the principles helped respondents identify when they were ‘drifting’ from the desired course, engage in discussion to reinterpret what the principles meant in practice, and course-correct through new specific actions towards a shared direction. Such events, although described as ‘tense’ or ‘uncomfortable’, may be necessary for moving towards a deeper understanding and fuller implementation of principles in practice. Thus, evaluating principles involves adopting a different perspective—shifting away from measuring implementation to asking: how have principles been articulated, how are principles apparent and in what ways do they lead to desired outcomes?
A focus on principles may lead to isomorphic mimicry, as principles may be written down and processes established by imitation, to falsely appear as if they are being followed, but not in actual practice.32 33 However, such imitation may be necessary to initiate a process of real change. Our research implies that principles are not static, because their meaning and implementation must change as our understanding of them deepens. Establishing a process for ongoing reflection and reinterpretation reflects ‘triple-loop’ learning, in which we learn how to learn, and how to make decisions about what is right.34–36 ‘Triple-loop’ learning can involve evaluator roles, and an openness both to unfavourable evaluation findings (eg, that principles are not being adequately followed), and to making change as a result of such findings (eg, from top-down collaborative practices to adopting equity and justice principles).37 Evaluators who engage with such collaborations must, when necessary, be willing to speak truth to those in powerful roles about the necessity of upholding principles and advancing goals, and to manage the consequences of doing so.37
Strengths and limitations
The strengths of our analysis include the use of a multi-methods approach, diversity of respondents and long-term repeated engagement with collaboration members who contributed to interviews, data analysis and interpretation. In addition, we had Indigenous representation and input, with Indigenous researchers and collaboration members having a role in the design, data analysis, and interpretation of this study.
Patton9 posits that it is useful to explore the extent to which principles are meaningful and evaluable. However, as the principles in our research collaboration were developed based on extensive consultation and discussion, we did not feel it was appropriate to assess how meaningful they were. In the context of Indigenous-related research, our experience points to the importance of having a collaborative process for developing principles, one that is driven by Indigenous stakeholders and does not impose, or focus on, principles based on their ‘evaluability’, a concern echoed by others.38 39 As articulated by Maori researchers, Were et al,39 there are important future discussions to be had about how Indigenous knowledges and approaches might intersect with principles-focused evaluation.
The findings from this study must be considered within the limitation that most of the authors are either embedded researchers or investigators within the collaboration, and that the respondents could also be viewed as having a vested interest in presenting the study in a positive light. However, factors that enhance the credibility of our findings included having two researchers undertake the interviews, the participatory nature of the research process, our familiarity with the setting and the topic, and the systematic comparisons between our data, themes and categories. Additional information about authors is available in online online supplemental file 1, with their roles detailed in the author contributions.
While Patton’s principles-focused evaluation9 approach is well cited, we have only been able to identify one peer-reviewed publication on this approach to date,37 although other reports may exist in the grey literature. Likewise, little has been published on the role of principles in the design and implementation of innovation platforms,10 11 16 even though these collaborative network forms have been gaining traction as a way of bringing people together to learn, share ideas and solve problems. Given that the focus of innovation platforms is on empowering local actors to solve problems collaboratively, and as lessons on their use accumulate,40 41 the findings from our study suggest that there is scope to be more explicit about the principles governing them and to embed, constantly monitor and reflect on their role within innovation platforms.
Conclusion
The principles in our collaboration—implemented through a set of five overlapping and mutually reinforcing strategies that interactively reinforced and complemented each other—became the compass we relied on to navigate complexity and conflict. Given the challenges inherent in evaluating complex collaborations, our experience may be useful for future principles-focused evaluation of multistakeholder collaborations (such as innovation platforms). Like tacit knowledge, how principles are enacted in practice might never be completely revealed, and the conditions and outcomes of their implementation may be nuanced. However, our experience points to the importance of having a collaborative process for developing and implementing principles within a culture of continuous reflection, learning and adaptation, and a process for ongoing recalibration as the context and the interpretation of the principles meaning changes over time.
Acknowledgments
The development of this manuscript would not have been possible without the active support, enthusiasm and commitment of staff in participating primary health care services, and members of the Centre for Research Excellence in Integrated Quality Improvement. We would like thank Jane Yule for her editing and proofreading support.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Handling editor Valery Ridde
Twitter @JodieBailie1, @AlisonLaycock2, @kpconte, @DrVMatthews, @davidpeiris, @RossBailie, @seyeabimbola, @CunninghamF_C, @DrRoxBainbridge
Contributors JB designed the study with guidance from RGB, AFL and KC. JB and AFL undertook interviews, and KH the document review. JB drafted the initial manuscript, with close support from RGB, AFL and KPC. All authors provided feedback on drafts of the manuscript, and read and approved the final manuscript. JB was the embedded developmental evaluator from January 2017. RSB was the principal investigator of the CRE-IQI.
Funding The National Health and Medical Research Council (www.nhmrc.gov.au) funded the Centre for Research Excellence in Integrated Quality Improvement (#1078927) and the Centre of Research Excellence in Strengthening Systems for Indigenous Healthcare Equity (#1170882). Jodie Bailie has been supported by a University of Sydney Postgraduate Award (#SC0649). In-kind support has been provided by a range of community-controlled and government agencies. Megan Passey is supported by a Career Development Fellowship from the NHMRC (GNT 1159601).
Competing interests The authors declare that this research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest. SA is the Editor in Chief of BMJ Global Health, but was not involved in the evaluation or peer-review process of this article.
Patient consent for publication Not required.
Ethics approval The University of Sydney Human Research Ethics Committee (Project 2018/206) and the Human Research Ethics Committee of the Northern Territory Department of Health and Menzies School of Health Research (Project 2018-3105).
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available on request. Research involving Aboriginal and Torres Strait Islander peoples is strictly governed in Australia by the Australian Code for the Responsible Conduct of Research and by Ethical Conduct in Research with Aboriginal and Torres Strait Islander Peoples and Communities: Guidelines for Researchers and Stakeholders. Data requests addressing these criteria may be sent to the CRE-IQI Management Committee care of RSB (ross.bailie@sydney.edu.au).
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.