Systematic review example nursing

The inclusion criteria were as follows:

• Published from 1990 onwards
• Population: The review was focused on those delivering adult nursing [i.e. licensed or registered] in health services [both in hospital and community] in developed economies [according to the definition of the International Monetary Fund [25]]
• Intervention: The review examined any type of service, management or human resources activity aimed at reducing rates of adult nurse turnover
• Comparison: Any comparators used within the included reviews
• Outcomes: Any outcome examined within the included reviews
• Review design: Any form of literature review which had been peer-reviewed, contained a statement of review, reported its search strategy and/or inclusion/exclusion criteria, reported either empirical findings or a list of included primary studies and included a methodological quality assessment of its included primary studies, that is a review containing key aspects of a well-conducted systematic review

Exclusion criteria were as follows: Reports from any types of primary studies; reviews published in language other than English; reviews that did not evaluate adult nursing turnover as described in the inclusion criteria or presented data on nurses working across settings that could include the care of children or in specific mental health settings; reviews that did not report empirical findings; reviews published only in abstract form; any form of literature review using informal and subjective methods to collect and interpret evidence; commentaries and non peer-reviewed reviews; any review in which the majority of included articles were non-peer reviewed publications and reviews that did not report an appraisal of the quality of the studies they included.

We searched the Cochrane Database of Systematic Reviews, MEDLINE [Ovid], EMBASE [Ovid], Applied Social Sciences Index and Abstracts ASSIA, CINAHL plus [EBSCO] and SCOPUS V.4 [Elsevier] from 1990 to 2015 [searches conducted January 2015]. Search strategies were guided by a systematic approach to the research questions [26] and a Medline search strategy was developed [Table 1] and converted or modified to run on other databases [Supplementary file 1]. We identified additional studies by searching on PubMed by using the related citations algorithm and screening the reference lists of included studies on for other reviews [27].

The results of the electronic search were downloaded into an Excel spreadsheet After removing duplicate articles, relevant reviews were selected according to eligibility criteria using a two-step screening process:

• Title and abstract screening: Two authors [FP and MH] reviewed in parallel the titles and abstracts of all the articles resulted to ascertain their eligibility for full text retrieval. Disagreements were resolved by peer discussion and a third view from the project lead [VMD] if required.
• Full-text screening: Two reviewers [FP and CB] read in parallel all the selected full-text articles to analyse whether they met the inclusion/exclusion criteria. Any discrepancies between the two reviewers were resolved in discussion with the third reviewer [MH].

Data were extracted to excel spreadsheets [FP and CB] using a predefined extraction form and spreadsheet on: general characteristics of the reviewe.g. author[s], year, geographical scope, research area, and authors aims/ research question[s]; descriptive characteristicse.g. type of review [design], selection criteria to include primary studies, number and study designs of articles incorporated in the reviews and outcome measures; results- effectiveness of all interventions reported in the included reviews, the direction of findings against the outcome measure and the references for the primary studies; main conclusions, using the review authors words and limitations, as noted by the review authors. Discrepancies were resolved through discussion among the data extractors.

The primary studies included in each review were also listed and compared across the reviews to assess the degree of overlap in the reviews we included.

The 11-point Assessment of Multiple Systematic Reviews [AMSTAR] checklist [28] was used to assess the quality of each included review. This tool has been widely used in previous similar overview of reviews and it is considered to be a valid and reliable instrument [29]. Using the AMSTAR scale two authors appraised each included paper. Any review that scored eight or higher was considered at low risk of bias, between five and seven at moderate risk of bias and four or less at high risk of bias.

Because of the heterogeneous nature of the focus, inclusion criteria and outcome measures of the included studies data were analysed thematically. Following the detailed reading involved for data extraction, the resultant spreadsheet was examined and a thematic index of interventions developed. The thematic index [supplementary data file 2] was applied to each data extraction and four main groupings of interventions [individual, leadership, group and organisational levels] were used to analyse across reviews, using Microsoft Excel 2010 to record the decisions applied for all reviews considered. A narrative account of the findings from the reviews containing an assessment of the methodological quality of included primary studies has been structured using the risk of bias in the review as the primary grouping level and the thematic content analysis as the second level, also drawing on the number and quality of the included primary studies. In this way we aim to describe the findings by weight of evidence. [30]

The systematic review protocol was registered with PROSPERO 2015: CRD42015017535 [31].

The flow chart representing study selection, including reasons for exclusion, is summarised in Fig. [1]. A total of seven reviews met the inclusion criteria and were included in the review. Supplementary File 3 provides a list of citations for the excluded studies in the final stage of the selection process.

Details of the seven included reviews are provided in Tables [2, 3 and 4]. The included reviews were conducted between 2008 and 2014. All were published in English and originated from Canada [32-35], the United States [36, 37] and Taiwan [38]. Five reviews focused on the effectiveness of retention strategies targeted at registered nurses [33, 38] or newly graduated nurses [34-36]. One review aimed to examine the relationship between managers leadership practices and staff nurses intent to stay in their current position [32], and another focused on a single intervention: sabbaticals as strategy to enhance nursing retention and revitalisation to generate positive outcomes [37].

The majority of the reviews limited their searches to the English language, with the exception of two reviews [33, 36]. The number of included primary studies in each review ranged from five to forty-seven Table [3]. Three systematic reviews included quasi-experimental study designs [33, 34, 38]; however, observational study designs dominated, and no review reported any randomised controlled trials. Only two reviews included qualitative or mix-methods primary studies [32, 34]. Of the 164 primary studies in the seven included reviews, 14 were included in at least two reviews, and of these only four papers of primary studies [39-42] were included in three reviews Table [4].

The AMSTAR scores for the reviews ranged from three to seven. Of the reviews, six [32-36, 38] were judged of moderate quality and one [37] of low quality. The assessment of each review against the AMSTAR criteria is presented in Fig. [2].

All reviews noted that the methods of the included studies were different, and that overall quality of the primary study ranged from high to low. The tools used to assess the quality of included papers in the included studies are shown in Table 3.

No reviews received an AMSTAR score of greater than seven, therefore none have been judged to have presented the strongest category of evidence on interventions to reduce turnover. Six of the included reviews were assessed as being of moderate quality, although three [34, 35, 38] received an AMSTAR score of seven at the top of the range in this category. Only one review [35] was failed to be assessed in the strong evidence category on the basis of one AMSTAR quality criteria. These authors comprehensively described the literature sources used but did not report the keywords or search strategy.

At the individual level, interventions were heavily but not exclusively focused on newly qualified/graduated nurses [NGNs], and on supportive programmes of transition or development. Preceptorship - one-to-one guidance through clinical experience - was a component of the majority, alongside a range of programme components [for example, classroom learning or group discussion] and support systems [for example, the programme director or clinical educator]. These programmes were variously named, including the terms residency, internship and orientation as well as mentoring and preceptorship itself.

Residency received positive support in four reviews [34-36, 38] reporting several studies, with some overlapping reviews. One of these reviews did not provide the description of the study designs, although the authors noted that the excluded studies appraised to be of low quality [36], and another described its three included studies as one pre test-post test and two experimental case study designs [35]. The included studies measure the turnover outcome in the experimental group and compare it to general pre-published reports locally or nationally.

Internships also received some emphasis as positive for retention, supported in two reviews [35, 36], drawing on six studies of variable design but including one with a controlled pre test-post test design. Orientation focus programmes for transition were highlighted in three reviews [34-36] as positively impacting on turnover in four primary studies.

One to one mentorship programmes of three months duration were reported as essential to retaining newly registered nurses, reducing turnover in two quasi experimental pretest-posttest studies [from the USA and Taiwan] in one review [38]. For preceptorship itself there was evidence in one review [35], from ten studies from the USA, all of which were experimental in design, although six were one group case studies, two were one group pretest-posttest and only two included a control group [nonrandomised] pretest-posttest design. Half of these studies were also reported in the review to have included a focus on supporting RNs to work in the preceptor capacity through educational training and/or monetary incentives [35].

Evidence for the positive impact of externships [preceptored and employment experiences of the student nurse the year before graduation from a basic RN education programme] was limited to one study reported in one review [35]. Needs-based training or specialty training programmes [designed to develop skills for specific clinical areas and including classroom instruction, observational experience, journaling, case study, coaching, and computer-based training] was also reported to increase retention, although only one of the primary studies [where the design was clearly reported by the review authors [35]] was robust, with a control arm.

These reviews vary in how actual turnover rates are reported, making comparison or synthesis difficult. For example, while the rates of turnover compare favourably with average ranges for turnover or retention of new graduates, few studies had designs with the degree of control necessary to rule out competing explanations[34]. With the same caveats about strength of evidence, longer transition programmes [up to one year] appeared to achieve better results regarding turnover [34, 35].

Other interventions identified at the individual level in the moderate quality reviews were bicultural training which was positively associated with turnover in one study in one review [35], and degree of fit to the job/lower work abilities which was negatively associated with turnover in one study in another review [33].

In addition, one review of poor quality/high risk of bias [37], suggested that a clinical practice sabbatical [a leave of absence for an identified purpose] for nurses in acute care settings was a viable option as a strategy to increase retention. Descriptions of the 19 included articles in this review are unclear, but it appears that only five addressed nursing sabbaticals and retention and all were at best descriptive studies and at worst anecdotal accounts; the review authors conclude that the evidence is limited.

Two reviews addressed interventions at the leadership level. Management training in leadership behaviour featured in one study in one review [33], and supervision support in seven primary studies in another review [32], as significantly related to intent to stay.

Two reviews considered interventions at the group or organisational level. One of these reviews [32] discussed nine primary studies where group cohesion was reported as significantly associated with intention to stay. However, no detail was given about the nature of the interventions and variation in the reliability and validity of the measurement tools used was highlighted. Another review also described two studies demonstrating a positive impact of one year team oriented interventions [one of team discussion groups, the other undefined] on turnover [33]. Nursing practice models, for example nurse-managed units and unit-level self-management widely used by hospitals with Magnet accreditation in North America [43-45] were reported to have mixed evidence of effect [33].

Tables [3 and 4] present the interventions reported in the reviews, alongside a summary of evidence of effectiveness. In summary, the tables highlight that the specific or multiple interventions reviewed that may have an effect on retention or intention to stay are orientation programmes [including preceptorship, internships, residencies, and structured orientation programmes [34, 36] and mentorship [38]] for new graduates; transformational or relational leadership [32, 33]; and team work. [33] Retention is reported as highest when multiple interventions are used [33]. Clinical sabbaticals received some support although the review quality was low [37].

Seven reviews of interventions to reduce nurse turnover were found which had undertaken a quality appraisal of their included studies. These reviews provided consistent and important messages about what might work to increase retention or at least intention to stay. Firstly, they reported positive impact of transition programmes for newly qualified nurses. [34-36, 38]. These programmes were variously named, including the terms residency, internship and orientation as well as mentoring and preceptorship itself. Questions still remain as to the effectiveness, efficiency, and costs of different methods, frequency and duration of preceptorship programmes for newly qualified nurses who are in different types of clinical specialities and are themselves at a different life stages.

Secondly, the reviews also offered evidence of the positive impact of nurse manager leadership styles that were perceived as transformative or encouragment of work group cohesion in reducing turnover or increasing retention [used interchangeably in this review] [32, 33]. Questions remain as to the extent these types of nurse leadership or management styles influence nurse turnover rates in different types of clinical services and in different types of organisational and job market contexts. In addition there are questions as to the effective [including judgements of cost] mechanisms for developing, maintaining and enacting these nurse leadership styles as judged by the primary outcome [rates of nurse turnover] and secondary outcomes such as described above.

Reviews of multiple interventions suggest that this approach is more effective than single interventions [33].

There was little overlap in the included primary studies in the reviews we analysed, due to their different foci, apart from two reviews of supportive programmes for newly graduated nurses [35, 36]. This finding highlights the importance of providing a summary of evidence from more than one systematic review on an important topic [19], as the separate reviews focus on different interventions or different sub-populations of adult nurses.

The quality of the reviews was in the AMSTAR category of moderate, although half of them were borderline with being classified as strong reviews. These included the quality appraisal of their included primary studies in weighting their discussion and drawing conclusions [32, 34, 35].

The reviews primary studies variously measured the intention to leave, turnover and retention, and the original reviews authors commented on the reduction in the strength of conclusions they could infer due to the weak study designs and variable validity and reliability of the measurement of these outcomes [32, 33]. Examples of weak designs commented on were the absence of control data in many of the primary studies [34, 36]. Other limitations noted by the original reviews authors included a predominantly North American focus [33, 35, 38], and a lack of focus on the retention of experienced nurses [33]. The absence of meta analysis in each review due to heterogeneity of studies was also a limitation [32].

In view of the critique of systematic reviews in terms of the ease of conducting poor quality ones that are given high esteem in publication and read by many who cannot easily differentiate their quality [46], we suggest that this summary has an important role to play in highlighting what is already known alongside where a lack of research studies leave space for uncertainty in the field of retention strategies for adult nursing.

Our overview is limited by design. Systematic reviews themselves are open to criticism as destructive of reading, writing, thinking, interpreting, arguing and justifying [47]. As an overview [systematic review of systematic reviews] we have also relied upon the review authors reporting and interpretation of the primary studies and have made some assumptions about quality based on descriptions of research design if critical appraisal of each primary study was not clearly described in the reviews. We suggest that this limitation is mitigated by only including reviews that have reported a quality appraisal of their included studies. We have also assessed the quality of the included reviews using a widely recognised tool for this task [28]. We have therefore provided an account of what should be the highest quality reviews available although we note that there are no reviews which offer strong evidence, and the AMSTAR tool authors themselves promote further testing of the tool [28]. We have accepted review authors descriptions of heterogeneity as limiting opportunities for meta analysis without carrying out any formal analysis to consider whether the diversity has implications for the interpretation of findings. We have also treated the terms turnover and retention as direct opposites noting that the included reviews do not comment on their differences, although there are some authors who argue that though interrelated, they are conceptually different and the reduction of one does not necessarily lead to savings in the other in economic terms at least [48].

We initiated our investigation of the evidence on interventions to reduce adult nurse turnover following our early reading and subsequent interlinked systematic overview of the determinants of consequences of such turnover [8]. The finding we report there of a myriad of determinants of turnover, albeit a body of literature widely open to critique of its quality, which might suggest a myriad of linked interventions have been studied. In contrast, when we applied criteria based upon guidance for the good conduct of systematic reviews [15] we have found a fairly narrow range of interventions tested and reported. Moreover, interventions were tested with a narrow range of nurses, in studies which the reviews authors consider to lack rigour, particularly in the measurement of the primary outcome of interest, that is, the rate of turnover. In the context of the ongoing international phenomenon of nursing turnover and projected supply-demand concerns [6] this is a somewhat surprising finding.

It was also a surprising finding that otherwise well conducted reviews conflated evidence from self-reported determinants of turnover [i.e. what reports groups of nurses say are the causes of their intent or action to leave] with the review authors opinions as to strategies to decrease turnover [including in a very recent overview [49]]. While the reviews we overviewed make it clear that there remains much to be done to improve the strength of evidence, for example there are no controlled trials and very few attempts to control observational studies, we argue such misapplied conflation has restricted the development of true intervention studies. In this we mean studies based on an intervention hypothesis for testing and framed by questions of interest to nurse and human resource managers which have the primary outcome of reduction of turnover rates [effectiveness] rates but secondary outcomes linked to accepted quality dimension criteria [50] such as of acceptability, patient safety and experience, staff well-being and cost consequences.

These findings are particularly pertinent when strategic guidance exists on good practice in staff retention [10, 51]. A comparison of the available guidance on retention of nursing in England and the evidence from our literature review [52] suggests there is some evidence that the current guidance offered to retain adult nurses is supported in part by the research literature regarding the determinants of turnover, [8] within the limitations of this evidence being of moderate strength. The guidance is supported in part by the research evidence on interventions with regard to developing nurse leaders and line managers, investing in the workforce and developing and continuing staff engagement [51] when we draw on the findings of two reviews [32, 33]. The strategys focus on newly qualified nurses is supported more widely supported by the evidence. [32, 33, 35, 36, 38] It would appear therefore that this is the most likely intervention to have a positive impact on retention. However this statement needs to be seen in the context that most of these studies were not true intervention studies and we are unable therefore to conclude that the strategies in the guidance would have the desired impact on turnover, according to the available research evidence.