1.1 Background

Practice activities refer to what individuals do in their roles, revealing the knowledge and skills required (Fraser & Greenhalgh, 2001). For effective workforce planning and human resource management, it is essential to identify the various activities of different groups of nurses. The Strong Model of Advanced Practice was developed in the United States during the 1990s (Ackerman et al., 1996) to identify APN activities and domains of practice and create a usable framework to guide practice and role development. The model described the blended role of NP and CNS that was in place at the time. A continuum of experience from novice to expert was identified within the recognized domains of practice of Direct Comprehensive Care, Support of Systems, Education, Research, and Publication and Professional Leadership (Ackerman et al., 1996). In recent years, the Strong Model of Advanced Practice and its modifications (e.g. Advanced Practice Role Delineation [APRD] tool) have been used around the world to differentiate and examine nursing and APN roles (Jokiniemi, Tervo-Heikkinen, et al., 2022). Although the use of the Strong Model of Advanced Practice and APRD tool has increased internationally in the past few years, there is little research-driven information on the validation of the content and construct of the original and modified tool.

To identify and describe the usage and validation of the original Strong Model of Advanced Practice and modified APRD tool, we conducted a comprehensive review of the literature with the help of an academic health sciences librarian in February 2021. Our search was conducted on CINAHL, Scopus and Google Scholar to gather relevant information between January 2000 and December 2020. The keywords for searching the databases included ‘advanced (nursing) practice’, ‘strong model’, ‘strong APN model’, ‘role delineation’ and ‘APRD tool’. To ensure a thorough search, we conducted two separate searches on CINAHL and SCOPUS. The following inclusion criteria were used: (a) peer-reviewed articles between 2000 and 2020 published in English; (b) focus on the Strong Model of Advanced Practice OR APRD tool; and (c) use or validation of either one of the tools.

After removal of duplicates, 37 articles were identified from the database search (CINAHL and Scopus) and 166 from the Google Scholar search. The literature selection from CINAHL and Scopus was conducted independently by two review authors (N.N, N.N), first on the title and then on the abstract and full-text levels. The title and abstract level selection on Google Scholar were conducted by N. N, after which the full-text selection was conducted independently by two research authors (N.N, N.N). After the selection process, 15 articles were identified for the review and rated for their validity. (Figure 1).

The identified studies (n = 15) were conducted in six countries: the United States (n = 1), Singapore (n = 1), Spain (n = 2), Australia (n = 8), New Zealand (n = 2) and Canada (n = 1). (Table 1) The original Strong Model of Advanced Practice and its modification, APRD tool, have been used to differentiate nursing and APN roles (Duffield et al., 2019; Gardner et al., 2013; Mick & Ackerman, 2000), to provide an overview of the practice patterns and profiles (Sevilla, Miranda Salmerón, & Zabalegui, 2018; Spooner et al., 2019; Woo et al., 2019) and to describe the time spent on APN activities (Spooner et al., 2019; Wilkinson et al., 2018; Woo et al., 2019). Several articles also scrutinized the variability of different titles used in nursing workforce positions (Carryer et al., 2018; Duffield et al., 2019; Gardner et al., 2016; Gardner et al., 2017). In addition, the APRD tool has been used to survey the relationship between the level of education and domain practice scores (Duffield, Gardner, Doubrovsky, & Adams, 2020; Wilkinson et al., 2018) and to test the hypothesized model of the relationship among educational structural empowerment, NP role competence, patient safety competence and psychological empowerment (Duff, 2019).

Three articles described the validation process of the Strong Model of Advanced Practice, whereby the tool was validated in terms of the content and construct in Australia (Chang et al., 2010, 2012) and Spain (Sevilla, Vilarasau, et al., 2018). Within these studies, the content of the Strong Model of Advanced Practice was slightly modified to clarify wordings. The modified tool was called the APRD tool. The construct of the APRD tool was found consistent with the Strong Model of Advanced Practice within Australia. However, variation existed in the sample from Spain, where the results of an exploratory factor analysis displayed a different theoretical underlying factor structure (Sevilla, Vilarasau, et al., 2018).

Based on the literature review, we conclude that the Strong Model of Advanced Practice and APRD tools have been used to describe and differentiate nursing and APN roles and education; however, there is limited research on the content and construct validation of the tools. Although previous research has examined the content of the Strong Model of Advanced Practice (Chang et al., 2010; Sevilla, Vilarasau, et al., 2018), no major modification to its content (APN activity items) has been carried out after its development in the 1990s. Because of the changes in health-care services during the last two decades, a content validation of the APRD tool was needed prior to the use of the tool in the Finnish context.

Finland is among the countries that introduced the APN role around the turn of the century. Over the past 20 years, there has been a national effort to conceptualize advanced-level nursing roles (Jokiniemi et al., 2019), but there is little information on the variability of generalist, specialist and advanced nursing activities. None of the APN roles are regulated, although the role of CNS is well recognized, while the NP role is more unclear (Jokiniemi, Heikkilä, et al., 2022). To investigate nursing and APN roles, the APRD tool was considered a purposeful tool with the potential to guide the development and evaluation of these roles. The lack of validation research combined with the increasing number of the APN roles highlights the importance of this study examining the content validity of the APRD tool in the Finnish context. The process described can be replicated in other countries utilizing and validating the Strong Model of Advanced Practice or APRD tools. Furthermore, suggested new items can be added to the APRD tool in future examinations of the nursing and APN roles.

2.1 Aims

The aim of this study was to describe the process and findings of a content validation study conducted on the APRD tool in a Finnish context.

2.2 Methodology

A content validation study was performed to assess a 41-item APRD tool. In content validity assessment, the representativeness of the content of an instrument is determined with a predetermined process (Lynn, 1986). We invited APN experts to judge the content of the APRD tool with the specific objectives of (a) examining the content of the APRD tool items by evaluating each item's relevancy on a 4-point, Likert-type scale (0 = very irrelevant to 3 = extremely relevant); (b) identifying any missing items; and (c) examining and clarifying the wording of the translated APRD tool. A meticulous step-by-step process of development and judgement quantification was carried out to establish and quantify the instrument's content validity (Lynn, 1986).

2.3 Sampling and participants

For this study, the research team utilized purposive sampling as described by Panacek and Thompson, 2007. The experts with a high level of knowledge or skill relating to APN were identified by the research team knowledgeable about the APN expert positions in Finland. To reach wide expertise, we invited APNs, nurse educators and experts in workforce planning who were familiar with national and/or international APN roles and research. Participants were asked to evaluate their own expertise in APN roles and health-care practice environments using a scale that ranged from 0 (no expertise) to 3 (high expertise). According to the method described by Lynn (1986), between 5 and 10 experts provide a sufficient level of control for change agreement; therefore, a total of 12 experts were invited in August 2019 to participate in the study.

2.4 Validity and rigour

2.5 Data collection

All data were collected over three iterative data collection rounds using a Web-based self-report survey between September 2019 and October 2019 (n = 9, n = 8, n = 5). The survey consisted of closed and open-ended questions. During the first round, the participants were asked to independently evaluate the 41 APRD items on a Likert scale in terms of their relevancy (0 = irrelevant to 3 = extremely relevant) and identify any issues with clarity of wordings in the translated tool. If the participant had rated an item as ‘very irrelevant’ or ‘irrelevant’, they were requested to give the reasoning behind their rating. The participants could also clarify items and evaluate the structure of the APRD subscales. In the second round, the same expert panellists who had participated in the first round were asked to evaluate any items requiring clarification or that had received low Item Content Validity Index (I-CVI) scores in the first round. In addition, the expert panellists were asked to provide additional activity items if needed. In the third round, new items identified during the second round were rated.

2.6 Data analysis

In the data analysis, both qualitative and quantitative analysis methods were used. Qualitative content analysis (Graneheim & Lundman, 2004) was used to analyse the data from the open-ended questions to condense, organize and group the responses. Quantitative data were analysed by using descriptive statistics to summarize the demographic characteristics of participants. From the participant relevancy ratings, we measured I-CVI. To obtain the I-CVI scores, we divided the number of ratings 2 and 3 by the total number of respondents (n = 9). For more than six raters, the I-CVI should be over 0.78. The Scale CVI Average (S-CVI/Ave) was calculated by adding up all I-CVIs and dividing the sum by the total number of items. To meet the standard criterion for scale content acceptability (Lynn, 1986), S-CVI/Ave should be above 0.90.

2.7 Ethical considerations

The research methods employed in this study are both ethical and sustainable. The University of Eastern Finland's Committee on Research Ethics provided an ethical statement (Statement no. 22/2018). Each participant was approached individually and presented with an information sheet that contained a link to the survey. Furthermore, they were provided with an information sheet about the study, which consisted of a link to the survey. Completion of the e-survey was regarded as implied consent to participate, of which the participants were informed. Voluntary participation was reinforced with the possibility of ceasing participation at any point in time.

3.1 Participant demographics

Twelve experts were invited to participate, and nine experts consented to participate, yielding a response rate of 75. The respondents' (n = 9) mean age was 46 years. Over half (67%, n = 6) worked in a specialist medical health-care context (specialist nurse [n = 1], NP [n = 2], CNS [n = 2] and nurse consultant [n = 1]), one was from an educational organization (head of education [n = 1]), and two were from professional health-care organizations (consultant position [n = 2]). The participants self-evaluated their experience (on a scale of 0 = no expertise to 3 = high expertise) in specialist nursing roles as 1.8, in CNS roles as 2.9, in NP roles as 1.9 and in nursing practice environments as 2.3.

3.2 APRD tool content validation

In the first round, a panel of nine experts (with a response rate of 75%) evaluated 41 activities in relation to Finnish health care on a scale of 0 (very irrelevant) to 3 (extremely relevant). Based on the first-round analysis, 38 out of 41 items received an Item Content Validity Index (I-CVI) above the cut-off point of 0.78. However, three items did not meet the cut-off point: ‘4.3 Contributes to the identification of potential funding sources for the development and implementation of clinical projects/programmes’ (I-CVI 0.67), ‘1.4 Identify and initiate required diagnostic tests and procedures’ (I-CVI 0.67), and ‘1.3 Make a medical diagnosis within specialty scope of practice and practice guidelines’ (I-CVI 0.44). The participants did not request any changes to the domains of the subscale items; however, two major and four minor wording clarifications were suggested. (Table 2).

In the second round, a group of eight participants (response rate 89%) provided feedback on four items that had been previously rated in the first round. Based on their feedback, two items with an I-CVI value of 0.67 were rephrased, whereas two items underwent major changes. In addition, two items were amended with major wording changes suggested by the participants. The item ‘Make a medical diagnosis within specialty scope of practice and practice guidelines’ (I-CVI 0.44) was not subjected to further rating as the participants indicated during the first round that medical diagnosing is not within the scope of nursing practice in Finland, as physicians exclusively perform it in accordance with the law. As a result of the revisions, all four items that were reassessed during the second round received an I-CVI above the cut-off point. (Table 2).

During the second round, the participants were also asked to identify any missing items in the APRD tool. In response, they recognized a total of five new items. Three of these items belonged to the domain of support of systems: ‘integrate evidence-informed practice procedures’, ‘promote innovation activity within own practice area’, and ‘facilitate the implementation of changes’. The other two were associated with the education domain: ‘develop and conduct staff education programmes’ and ‘evaluate nursing staff competence’.

In the third, the participants (n = 5, 67% response rate) who took part in the second round were asked to rate five new items identified during the second round. All five new items received an I-CVI of 1 (Table 2). Overall, the I-CVIs of the 45-items varied between 0.88 and 1.00, reflecting excellent item validity. Finally, the S-CVI/Ave for the entire scale was counted. A S-CVI/Ave after the first round was 0.95. With the changes suggested by the expert panel during the three rounds, the 45-item APRD tool S-CVI/Ave was 0.97.

4.1 Limitations

The study provides a contemporary content validation of the APRD tool. The study was completed in one country, where APN roles have been developing over the past 20 years. Although the sample size was sufficient for the validation of the competency criteria by experts, the short history of the introduction of APN roles may limit the knowledge of the panel members. Therefore, the 45-item version of the APRD tool has been validated for use in the Finnish context, and the generalizability of the results may be limited in other countries; however, the methods may be replicated to modify the APRD tool in other contexts and circumstances. Further research is needed to validate the content and construct of the modified tool across countries.

AUTHORSHIP STATEMENT

KJ, LH and AMP made contribution to the design, acquisition, and analysis of the empirical study phase. KJ, LH, AMP and MA contributed substantially to the literature review design, data collection, analysis and interpretation of the empirical and literature review data. All authors were involved in drafting the manuscript and revising it critically and gave final approval of the version to be published. Each author participated sufficiently in the work to take public responsibility for appropriate portions of the content and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work were appropriately investigated and resolved.