Caregivers’ Perceptions of Telehealth for Child Developmental Screening in the Community

Citation

Keary, G., Corcoran, N., Heavin, C., O’Connor, Y. (2023). Caregivers’ perceptions of telehealth for child developmental screening in the community. Online Journal of Nursing Informatics (OJNI), 26(3), https://www.himss.org/resources/online-journal-nursing-informatics

Abstract

Background: The onset of the Covid-19 pandemic impacted on the timely delivery of essential child health screening programs by Public Health Nurses (PHNs) in the community. Although research indicates diverse experiences of telehealth across health services and settings, there remains a dearth of research investigating the delivery of child developmental screening, particularly focused on the perceptions of caregivers.

Aim: The aim of this study is to explore caregivers’ social and technical perspectives of the usability of the Attend Anywhere platform, when engaging with PHNs for the delivery of a developmental screening service.

Method: This mixed-methods study combined qualitative, semi-structured interviews with caregivers, and the Systems Usability Scale (SUS) to establish a usability rating of the telehealth platform.

Findings: Analysis indicates that caregivers were largely positive about their experiences, and patient outcomes were similar to face-to-face consultations. Considerable benefit was found by caregivers through convenience of use, time saving, cost saving, and flexibility. The SUS results found Attend Anywhere to be an ‘excellent’ system in the context, and technological barriers to use were not significant. The acceptance of the telehealth system as a replacement for face-to-face consultations may be attributed to a general acceptance of pandemic-enforced online alternatives to physical meetings, coupled with the caregivers falling into the ‘digital natives’ age category.

Conclusion: This research points to telehealth as a viable option for caregivers accessing developmental screening services for young children beyond the pandemic and will be of interest to practitioners and researchers who are investigating the use of telehealth in paediatrics and other contexts.

Introduction

The Covid-19 pandemic has brought about an acceleration in the use of digital health services, including telehealth (Meyer, 2020). Routine healthcare services have been upended globally, and health systems have had to attempt to continue to deliver care for patients with chronic medical conditions (March et al., 2021). However, many non-essential healthcare services were paused, postponed, or significantly altered (O’Leary et al., 2021). According to Spaulding and Smith (2021), the pandemic has made telehealth a necessity and presented an opportunity for patients, providers, and researchers to utilize and understand these services for future generations. Greenhalgh et al. (2020) identified video consultations as an opportunity to scale up innovations and learn lessons for those seeking to make it more accessible across a wider range of health services. However, existing research acknowledges several challenges associated with telehealth adoption, and such innovations should not merely be installed but must ensure sustainability of change in complex systems.

One of the public health services that was affected by the pandemic in Ireland was the National Healthy Childhood Programme (NHCP), a service provided by the Health Service Executive (HSE) to children from birth to their first year of second level school (circa 12 years). This service is delivered through General Practitioners (GPs), dentists, schools, and public health nurses (PHNs). The PHNs provide developmental screening at the ages of three months, nine to 11 months, 21 to 24 months, and 46 to 48 months. During the early wave of the pandemic, several aspects of the NHCP were maintained, such as infant visiting and all essential clinical care. However, developmental screening for older children (21 to 24 months and 46 to 48 months) by PHNs was suspended. Not providing developmental screening at these markers will inevitably lead to delayed referrals for children requiring further interventions, such as for speech and language, and developmental delay, placing increased pressure on care systems.

Telehealth platforms are now being adopted as an alternative mode of healthcare delivery by professionals in primary care, including physiotherapy, respiratory integrated care, diabetic services, mental health, psychology, and dietetics. The objective of this study was to explore caregivers’ perspectives of the usability of telehealth when engaging with PHNs for the delivery of a developmental screening service to children aged between 46 and 48 months. Developmental screening is the practice of systematically looking for, and monitoring, signs that a young child may be delayed in one or more areas of development. Screening is not meant to establish a diagnosis for the child, but rather to help professionals determine if more in-depth assessment is required. In most cases, screening rules out the likelihood that further assessment is needed (Radecki et al., 2011), but children who participate in developmental screening programmes are more likely to be referred to early intervention services in a timely fashion (Guevara et al., 2013). This research specifically examined the impact of social and technical factors on the intentions of caregivers to engage with the developmental screening service delivered via telehealth, and the usability of the system.

There are several studies on the use of telehealth systems with children across jurisdictions, mostly related to specific conditions such as, autism spectrum disorder (Dahiya et al., 2020; Smith et al., 2017; Talbott et al., 2020), speech and language (Ciccia et al., 2011; Wales et al., 2017), and audiology (McCarthy et al., 2019), while others focus on geographical (Antezana et al., 2017), and socio-economic contexts (Obeid et al., 2019). There are no comparable studies on the use of telehealth to carry out the developmental screening of children. Furthermore, limited research has been conducted on the usability of telehealth systems in general in Ireland. This study attempts to fill this gap by using qualitative interviews and the System Usability Scale (SUS) (Brooke, 1996) to examine the usability of telehealth for child developmental screening in an Irish context.

Methods

The telehealth platform Attend Anywhere was selected by the HSE in Ireland as a platform of choice for video consultations and promoted as a solution to allow remote, clinical consultations using videoconferencing. A pilot study using Attend Anywhere, during July and August 2020, found that older children (46 to 48 months) engaged and could be screened more effectively than younger children (21 to 24 months), whose attention was difficult to maintain. Therefore, the older cohort of children was selected for this study.
This study employed a mixed methods approach to the research design, the collection and analysis of data, and presentation of results. This approach is important in health services research as it can provide a comprehensive understanding of more complex interventions, such as telehealth, rather than employing a single-method study (Caffery et al., 2017). The study design follows a convergent parallel approach, where qualitative and quantitative data are gathered independently and then analyzed to answer the same research questions. This approach is the most common type of mixed methods research (Caffery et al., 2017), and has been used in several telehealth studies (e.g. Floresca et al., 2020; Williams et al., 2021).
Purposeful sampling was used to select the participants from a group of caregivers who had experience using the Attend Anywhere platform. Thirty-two caregivers were part of a wider pilot program using the platform between December 2020 and April 2021. During consultations over a one-week period in April 2021, every caregiver was asked to participate in the study. Ten agreed and six were purposefully selected for interviews based on language capability and viable internet connectivity. Based on their age, these six female caregivers could be characterized as ‘digital natives’ from a technology use perspective (Prensky, 2001). A semi-structured interview approach was used, so that the same questions were asked of all interview subjects, but with some flexibility to develop lines of enquiry that might be particularly relevant to exploring emergent themes. The semi-structured interviews were conducted online, audio recorded, transcribed and thematically analysed to identify common patterns and themes (Braun & Clarke, 2006).

The System Usability Scale (SUS) was developed in the mid-1990s in response to a need in the usability community for a tool that could easily and quickly collect users’ subjective ratings of a system’s usability, and is a highly robust and versatile tool for usability professionals (Brooke, 1996, Bangor et al., 2009). According to Peres et al. (2013), it was developed so that perceived usability could be measured in a quantitative manner. The scale consists of ten items which are simple statements such as “I thought the system was easy to use” and these are answered on a five-point Likert scale. The final score is on a scale of 1 to 100 and applying SUS to different systems allows their usability to be compared to each other. The odd-numbered questions have a positive orientation, and the even ones a negative orientation. For each of the odd numbered questions, one is subtracted from the score, and for each of the even numbered questions, the score is subtracted from five. The new values are added up to a total score and then multiplied by 2.5 to give an overall score out of 100. According to Bangor et al. (2009), the median score for usability is 68, and they applied a seven-point adjective rating scale to SUS, ranging from “worst imaginable” to “best imaginable”. SUS was selected because it can produce reliable results with small sample sizes and can effectively differentiate between usable and unusable systems (Sauro, 2011). Lewis (2018) recommended that when researchers and practitioners need a measure of perceived usability, they should consider using the SUS.

Results

A thematic analysis of the transcribed interviews identified several social and technical factors that influenced the use of the Attend Anywhere system (Braun & Clarke, 2006). Social factors reflect characteristics of an individual person or their situation, whereas technical factors are more direct indicators of characteristics of the technology (Hester, 2010).

Social Factors

Several social themes emerged from the data. Screening clinics are held on set days which can be inconvenient and restrictive for caregivers. The convenience of using the platform emerged as a social factor, with caregivers not having to travel for appointments and not having to arrange childcare for other children, cited as a key advantage. The technology presents a more flexible solution that better suits the needs of caregivers who have other children, jobs, and additional commitments. Caregivers were offered some flexibility around online appointment times and so could schedule their screening appointment around work hours. The time saved in not having to travel to clinics was highlighted as a significant benefit. One participant captured these feelings by saying she did not “have to travel, or bring the other child, or take time off work” and could “schedule when it suited”.
Caregivers were generally satisfied with the usability of the platform and found it to be appropriate for the age of the children being screened (46 to 48 months), as it was possible to keep them onscreen and engaged with the clinician for the required length of time. Although not expressly asked, some caregivers believed that the platform would not be appropriate for screening of their younger children. The perception of the screening process in terms of clinical outcomes indicates that the use of telehealth produces similar outcomes to face-to-face appointments. This was evidenced by the level of engagement of the children during the appointment. Caregivers noted that children were more engaged in the non-clinical, home environment as it was more familiar, with one participant stating that the consultation was not “as daunting because they are not in a cold clinical environment where they don’t know anyone.” Comfort emerged as a social factor with caregivers able to stay at home for screening and children having more comfort in their own surroundings. Some caregivers suggested that they got more from the experience due to the child having a good rapport with the clinician in a familiar environment, leading to better outcomes in terms of representation of the child’s true development milestones being checked. The use of telehealth meant that no Personal Protective Equipment (PPE) was required for clinicians, caregivers, or children, leading to a less daunting experience. One participant observed that “being able to see your face and being friendly” was important as “children may not find it so easy to engage with the masks and all the gear.”

Technical Factors

Technical factors that contributed to the user experience include broadband connectivity and speed, quality of the user’s computer equipment, connecting via the weblink in a browser, and the reliability of the system. An information leaflet and video link were sent to caregivers prior to appointments. This included instructions to access the site and to test their devices for sound, video quality, and broadband speed prior to the appointment. There were some issues with connectivity and “freezing screens”, most of which were quickly overcome using refresh options, and all the appointments were conducted successfully. One participant commented that “we did get cut off, but we called back.… we’re well able to get around those things.” Although some of the participants were initially unsure and afraid of the system, they quickly came to terms with using it. Every participant indicated that they would have no hesitation in using it again for similar screening appointments. This was also reflected across the wider sample population of 32 individual screening appointments. One participant experienced a very unreliable internet connection and had to switch to a 3G connection on a smartphone. Although this impinged on the user experience, the participant still found the experience beneficial, stating “I definitely see an advantage for myself. I’d like to have a better connection but that's something we all want”, and the objectives of the screening were met.

System Usability Scale (SUS)

The users found no complexity with the system and did not require the support of a technical person to set up the appointment. The experience using the platform was not cumbersome and there was no extra learning required for set up. Strong agreement was found across the areas exploring using the platform. All respondents indicated it was easy to use and that the instructions were well integrated and easy to follow, building confidence in using Attend Anywhere. The results of the SUS calculations are presented in Table 1. The score is a percentile rather than a percentage and the average score at the 50th percentile is 68. The percentile score of 85.4% equates to an ‘excellent’ on the associated adjective rating scale and means that the system is acceptable. There is also correlation between the SUS and the Net Promotor Score (NPS), and the system rates as a ‘promotor’ meaning that users are likely to recommend its use to others (Lewis, 2018).

Table 1: SUS Results

Discussion

This study illustrates that caregivers accept and, in some instances prefer, telehealth, video-enabled developmental screening for their children aged 46 to 48 months. Based on this small sample size of caregivers of well children, there are several social factors that positively influenced their perception of telehealth for accessing the public health service. These factors included travel, childcare, interaction, age, familiarity, comfort, outcomes, engagement, convenience, and the screening process itself.

Caregivers reported a similar level of care with clinic screening, with all areas checked, and similar outcomes for the children. The lack of need to be concerned with PPE was seen as a factor in improving the quality of the consultation as it reduced perceived anxiety in the children. One participant commented that her child “didn’t see a massive difference between in person and virtual because you were not wearing PPE.” The convenience of not having to leave home with small children was frequently cited by participants as a positive aspect of a telehealth consultation. Although monetary costs are not mentioned specifically, the cost of childcare was alluded to on several occasions, and not having to make alternative arrangements for other children could also be interpreted as a cost saving to caregivers. Additional convenience was identified as appointments were made for a time when caregivers were not at work, so they did not have to leave work to attend the appointment. Furthermore, the experiences of the video consultations were shaped by their efficiency and comfort factors for both caregivers and their children. Caregivers were provided with a list of requirements prior to the assessment, including blank pages, black crayon/marker, coloured blocks, a favourite book, and a toy. When asked to perform a task, it was observed that the children were familiar with their own toys, and this led to high levels of engagement.

Several technical factors were found to impact the caregivers use of telehealth technologies, including broadband speed, the ease in which they could access the site via the links provided, the usability of the platform, and familiarity with similar types of platforms. One participant was initially fearful of the system, and “was afraid I’d blow it up - I am more familiar with Zoom.” Some participants cited connectivity and device problems, but these were all overcome, and the consultations were completed satisfactorily. The proliferation of use of video platforms, such as Zoom and Microsoft Teams, during the pandemic, resetting or re-joining meetings and groups has become commonplace, and this group of participants were already proficient in the use of this type of technology. They were using it in other areas of their lives, falling into the category of what Prensky (2001) describes as ‘digital natives’. One caregiver made a distinct comparison with the child feeling like “he was like his daddy at work and felt he too was important on his own call”. In general, Attend Anywhere was perceived as just another way of engaging with the screening service, suggesting that it is a mature system. One of the key indicators for a mature system is ease of use for both professionals and non-experts, clinicians, and caregivers in this instance. The technology itself should fade into the background so that the purpose for which it is being used becomes the primary focus. If problems with the technology are encountered, then the experience becomes about the system and not the service.

The SUS mean score of 85.4% identified Attend Anywhere as an ‘excellent’ system with few usability challenges. The platform was seen as easy to use, would be used again, and the instructions given were appropriate and clear to users. The SUS scores demonstrated a correlation with the qualitative findings in setup and ease of access. Caregivers talked about it working perfectly when engaged with the children, and the links being easy to navigate. They all agreed or strongly agreed with affirming questions relating to ease of use, simplicity, their confidence in using the system, and speed of adoption. Birkhoff et al. (2021) also used a mixed methods study leveraging the SUS to examine virtual nurse visits, but only achieved a mean score of 66% as the study population was an older age cohort. In this study, the users are young parents familiar with using technology, and the SUS score is comparable to Cox et al. (2013) who found that video conferencing was an appropriate delivery mechanism for assessments for cystic fibrosis patients of a similar age to the caregivers, with a SUS score of 85.63%. This suggests that telehealth may not be suitable for all types of consultations, across all age ranges and medical conditions, and corresponds with findings from other studies (e.g. Murphy et al., 2022).

The findings suggested that the use of telehealth may serve to empower caregivers through greater transfer and assimilation of knowledge about the developmental markers for their children compared with face-to-face contexts. This is because the telehealth screening process requires a greater level of involvement from the caregiver who must act in some ways as an assistant to the clinician to achieve the cooperation of their children during the consultation.

Limitations

This is a pilot study of caregivers’ perceptions of using the Attend Anywhere video platform. The children that were screened represent a subset of the children who are offered routine screening as part of the NHCP. The caregivers were all female with similar demographics. There was no gender or racial diversity in this group of participants. The study did not capture social factors such as socioeconomic background, education status, and housing situation. Future studies should involve a more diverse caregiver population. Some participants used phones for the screening appointment whilst others used an iPad or laptops, although this information was not explicitly captured during the study. Only the Attend Anywhere platform was used to virtually connect with the caregivers for the child development screening check, so the findings are limited to this technology. Using another video-conferencing platform may have resulted in different outcomes. The participants mostly live in one small area of the 36,000-population covered by the collective PHN team. One participant lived just outside the urban area and their broadband connectivity was not stable.

Conclusion

This pilot was the first time that Attend Anywhere was used in the context of screening children. Social factors such as convenience of use, time saving, cost saving, and flexibility emerged as key benefits of telehealth. Technical factors, including broadband connectivity and speed, quality of computer equipment, connecting via the weblink in a browser, and reliability of the system were highlighted as being key to a successful telehealth consultation among the user group. The platform was found to be an acceptable and a viable alternative to face-to-face screening for the 46-to-48-month age group among this group of ‘digital native’ caregivers.

Telehealth is being used in Ireland for other pediatric services such as speech and language therapy and the delivery of physiotherapy consultations; to the best of our knowledge little published research has emerged. The findings from this study suggest that telehealth may be a useful adjunct model for other areas of primary care involving ‘digital natives’ as patients or caregivers. Future research should include other application areas such as chronic disease management where travel for patients can be difficult or harmful. Within public health nursing, follow-up visits to ambulatory, older clients may also be considered. As this is such a new area in telehealth in the primary care setting in Ireland, increasing the use of Attend Anywhere and similar platforms, needs to be continued across the spectrum of populations, and the potential for patient empowerment should not be overlooked. There is further work to be done in promoting the need for telehealth adoption among the wider nursing community and other health and social care professionals.

A more extensive study of the use of telehealth in providing screening services by PHNs to children would be useful to further validate and explore telehealth adoption. This would extend the body of knowledge in this field and allow comparisons with similar studies in other jurisdictions. Developing this evidence base would help to convince the medical and patient communities of the viability, reliability, and usefulness of telehealth applications. There are also significant opportunities to use telehealth for the delivery of healthcare to rural communities or patients from disadvantaged socio-economic backgrounds. However, it may be that telehealth applications are not suitable for certain age groups, or medical conditions, and it is important to establish these limitations through research, so that negative experiences of telehealth do not adversely impact its adoption across the health service.