Identifying Linguistic Cues; Towards Developing Robots With Empathy in Autism Interventions

The last decade autism research has been enhanced by the use of social robots and several studies suggest that children with Autism Spectrum Disorders could benefit from robot-assisted interventions. In order to design and implement therapeutic interventions with robots without interrupting the intervention’s flow, one should take into account possible technical issues that could arise. The main objective of this study was to gather information from experts in the field of autism and develop linguistic cues to which the robot would respond automatically. A qualitative approach was used to explore specialists’ preferences. Online surveys were completed by 33 professionals from different backgrounds to select the vocabulary more often used in psychosocial interventions with autism spectrum disorders children, in specific situations. Six linguistic cues were identified and specific phrases were used so as to accordingly program the robot to show empathy and respond, when a crisis emerges. The session’s flow in robot-enhanced interventions could benefit by controlling robot’s behaviour with linguistic cues phrased by the therapist. The implications of these findings are discussed in relation to pilot implementation. This work consists of a qualitative study aiming at strengthening the application of a larger research intervention protocol to explore the interaction of children with autism spectrum disorder with a social robot.


Introduction
There is a growing recognition of the potential value of robot-assisted therapy in children with Autism Spectrum Disorders (ASD) [1][2][3][4][5]. Despite the promising results from prior studies [6][7][8], we are far away from conducting a psychological intervention in which a robot will have the main role of a therapist [9]. In order to achieve this level of autonomy the robot's emotional intelligence should reach a level where the child's emotions, either expressed verbally or non-verbally, would be properly identified and addressed with an appropriate response.
During psychosocial interventions, the recognition and understanding of the child's emotions is crucial for establishing the therapeutic relationship [10]. Human therapists also use specific clinical skills and promote empathy to enhance the quality of interaction; they use their tone of voice, their facial expression, their body language and choose carefully their words to reflect, to paraphrase and to empower the child. These skills would also be effective for controlling human-robot interaction. Dialogue is the main joint process of communication in humanrobot interaction. Thus, in order to improve the quality of such an interaction, it is essential for social robots to be capable of guiding a dialogue towards specific directions, so as to serve specific therapeutic intervention purposes. The latter is considered of great importance due to the current robots' perception (e.g. speech processing) limits. A meta-analytic review concluded that empathy accounts for between 7-10% of the variance in therapy outcome studies [11]). It is therefore very important to support robots' communication skills with empathetic cues.
A dominant approach towards the implementation of therapeutic interventions incorporates supervised autonomous robotic systems [12]. Until now the usual technological architecture in therapeutic scenarios includes the "pressing buttons procedure" or using the laptop to move forward from one scenario to another [13]. Most of the times in studies involving social robots, a professional is required to appraise the completion of the task and at the same time to manually guide the robot in order to include it in the process and keep the session flowing [9]. However, the challenges that derive when working with ASD children, due to the variety of their symptoms, make the previously described method difficult to implement without interrupting the session's flow. Challenging behaviours such as ignoring requests, physical aggression towards self or others, temper tantrums and extreme emotions are major barriers to robot's autonomy.
One of the biggest challenges in designing a robot-assisted intervention is to program the robot in order to be able to respond accordingly when the child is showing signs of distress. This needs to be done without manually controlling the robot from the laptop or by pressing embedded buttons, since this will demand a shift of the therapist's attention to the robot instead of the child.
The current study is part of an ongoing larger project that aims at involving the robot as a co-therapist in a psychosocial intervention targeting the autism core symptom of impaired social communication, challenging behaviours in children with ASD. In order to address the above mentioned challenges and develop a robust protocol design, a multidisciplinary focus group was put together to identify the intervention's requirements. After several focus group sessions, one of the main challenges was to identify how the robot will behave when a "crisis" appears relevant either to the child or to the robot itself (robot's malfunction).
To overcome the potential hurdles and minimize the children's discomfort, it was decided to conduct a qualitative study that would explore the vocabulary used by therapists, educators, and medical doctors when a problem arises during the interaction with the child. The results of the study would subsequently guide the IT team to program the robot in a way to be able to regulate a conversation, guide the therapeutic activities and thus, enrich the robot's communication with the child. Trigger words were selected based on the frequency the professionals are using them so as to guide the robot's behaviour when something unexpected is happening. We aim at programming the robot to react accordingly to linguistic cues phrased by the therapist facilitating the intervention's flow. Based on the results of the aforementioned qualitative study, the robot's vocabulary was adjusted so as to include phrases that express empathy, suitable for a variety of situations from minor to great difficulty.

Material and methods
The main objective of this study was to systematically transform information from experts in the field of autism into linguistic cues to which the robot would respond automatically by modifying its behaviour (movement or verbal communication). A qualitative approach was chosen to quantify the occurrence of words and phrases expressing empathy that are being used during therapeutic interventions with ASD children.

Research questions
The following research questions were formed to address the purpose of this study: • What are the characteristic phrases that reveal empathy and strengthen the therapeutic relationship in a crisis?
• What are the most frequent words used by specialists under specific circumstances in autism interventions?

Participants
Purposive sampling was used [14]. The sample consisted of 33 people in total; 31 female and 2 male specialists (15 psychologists, 6 special educators, 6 speech therapists, 4 occupational therapists and 2 pediatricians). The sample consisted of employees in public hospitals (n=8), in private practice settings (22) or both (3) with at least four years of working experience each in the field of developmental disorders. Thirty percent of the participants (n=10) were aged between 26-29y, 45,5% (n=15) between 30-45y and 24,2% (n=8) between 46-59y. All the participants gave their informed consent prior to their inclusion in the study.

Data collection and analysis
An online survey with open-ended questions was the selected method in order to allow the access to a broader participant pool. The questions were developed by the researchers in line with the themes proposed by the focus group.
A set of open-ended questions was used to identify the main research questions. Participants remained anonymous and used the online survey tool at a time and place of their preference.
Survey topics focused on the difficult situations potentially experienced during an intervention with an ASD-child and on how therapists empower children when needed.
The survey was conducted during a three months timerange, between June and August 2019 in Thessaloniki, Greece. Qualitative analysis was conducted using content analysis techniques [15][16][17]. Participants were given a numerical code for analysis purposes. Prior to responding to the open-ended questions, participants had to read the protocol rationale, provide their consent and some demographic data such as their age, years of experience and professional status. The completion of the survey required approximately 30 minutes.
The answers of the respondents were scrutinized one by one in a qualitative way. The analyst examined each patient's response and selected specific data according to the aim of the present study. Upon targeting data, the researcher focused on further analyzing the answers looking for potential correlations among the respondents. Furthermore, the aim was to discern specific topics associated with the current study's objectives. Table 1 presents categorised themes regarding situationspecific conditions usually experience during a therapeutic or educational session when working with ASD children.  Table 1 Themes

Results
Furthermore, data analysis offered an interpretation of the communication strategies employed by professionals during their interaction with autistic children. Table 2 includes the communication and interaction strategies in specific contexts. The codes for the condensed meaning units were detected during the data analysis process while a combined coding template was agreed upon. The latent content was also assessed and included. Some participants' quotes were coded into multiple themes as appropriate. Seventeen categories of communication and interaction strategies were coded in total, and are summarized in Table 3 along with their frequency of occurrence between each professional.
According to the findings the most frequently reported communication technique regarding Theme 1 was "Positive Reinforcement" (N=58) followed by "Specific cues" (N=30). For Theme 2 and Theme 6 the most frequent proposed technique was "Setting Boundaries" (N=8) and (N=17), respectively. For Theme 3 the majority of specialists proposed "Specific cues" (N=11) and for Theme 4 "Encouraging active participation" (N=8). For Theme 5 they preferred "Time out" (N=13) and for Theme 7 "Acceptance" (N=20). Finally for Theme 8 the respondents selected "Reflecting feelings" (N=6) as their main technique.
Following that, frequent used single word cues were identified from the examples that the experts gave representing each category and were tested at the IT lab for adaptability by the robot. This process resulted in six linguistic cues presented in Table 4 that were used to control the robot's behaviour, when the normal flow has to be changed. The trigger words-recognition implemented algorithm was based on Interval Numbers (INs) [18]. An IN is an established mathematical object that may represent either a fuzzy interval or a distribution of numbers. A classification scheme was adopted to recognize the trigger words. This method was especially designed to be a computationally inexpensive tool used in conjunction with the robots' build-in speech recognition engine.

Discussion
The purpose of this study was to explore patterns that shape empathetic rapport in clinical and educational settings that may subsequently aim to the development of robot-based therapeutic protocols for children with autism.
Firstly researchers have selected single words to be applied as trigger words based on the frequency they have been used by the professionals. Following that, phrases have been chosen to match the techniques revealed from the experts' reports. Lastly, the IT experts programmed the robot accordingly and tested it in the lab.
As verbal communication in humans does not come isolated from non-verbal signs and head nods and gestures are useful to provide positive feedback [19], researchers used the capabilities of the robot to produce some non-verbal behaviour to strengthen this rapport.
The first theme required that specialists report on how they encourage the child to perform specific tasks. From their responses it became apparent that they are mostly using simple Reflecting feelings Participant 7: "I reflect his/her emotional reactions." 11 Acceptance Participant 10: "Some days are bad and that's ok, for example when a child is sleepy or tired, we need to be flexible and understanding."  words, familiar to the children. Two words have been selected as triggering linguistic cues. To mimic the human facial expression that usually accompanies words of encouragement, the robot will have his eyes changing colour.

Theme 1
Well done: the robot applauds and empowers the child saying "you did it, wow" having his eyes changing colour (when the child has completed a task successfully).
Again: the robot utters an interjection ("Hmmm") showing uncertainty and says "I am not sure, let's try one more time".
The next theme is referring to one of the most common symptoms in autism: stereotypes. Participants proposed setting boundaries acceptance and distancing techniques to overcome the situation. The trigger word that has been selected could be used to situations relevant to Theme 3 as well.

Theme 2/Theme 3
Break: robot stops whatever it's doing and says "we need some time to relax". This phrase could be used when the child looks tired, or performs stereotypical movements without stopping or does not participate in the scheduled tasks, or lacks eye contact for a significant amount of time or does not respond to the therapist's cues.
Sometimes the therapist will need to change the flow of the session adjusting to the child's mood and preferences. In that case the robot should adjust as well.

Theme 3/Theme 4
Change: robot stops whatever it is doing and says: "let's do something different". This phrase would apply as a transition signal to the next task.
When safety is an issue, there is a need to resolve the crisis as quickly as possible and ensure that the child won't get hurt.

Theme 5/Theme 6
Attention: robot says: "this is not safe, I am going" and moves to the relaxation space. This phrase will be used by the therapist when the child moves aggressively towards the robot and the therapist perceives a possible danger or when the child abuses toys or other therapeutic elements.
Meltdowns are very different from temper tantrums. They refer to an uncontrollable emotional overload possible due to overstimulation [20]. Temper tantrums are learned behaviours that a child uses to fulfill his/her desires. Both situations demand the full attention from the therapist, therefore in situations like this the robot needs to withdraw so the therapist could focus on the child and use strategies to minimize the stress of the child. Upon listening to the word "stop" by the therapist the robot withdraws but still communicates to the child that the situation in upsetting.

Theme 7/Theme 8
Stop: Robot pauses and says "I am upset; I need some time to relax". This phrase is used when the child meltdowns, has a tantrum, is aggressive or dysphoric.
Based on the same rationale if by any chance the robot's battery runs out or other technical problems arise, (that the robot recognizes itself) the robot informs the child: "I like your company a lot, but I have to rest". If the therapist realizes the existence of other technical problems that the robot itself cannot recognize, he/she says "Over" and the robot enters the sleep mode.
Robot's don't have all the capabilities a therapist has, therefore is extremely difficult to autonomously handle the session's flow. A therapist should always, at least at the current state, be responsible for the intervention, but it is possible to include the robot even when something unexpected arises.

Limitations
There is gender bias since only two males participated in the study. This difference though is similar in the representation of males in humanistic professions. Contingent Worker Supplement (CWS) estimates that 75% to 80% of students studying psychology or participate in health care training programs are women [21].
Given the heterogeneity of the autism spectrum disorders the strategies each specialist chose may vary. Professionals gave their answers based on their own experiences and not based on a specific case study.

Implications
The results of this study aim to facilitate the procedures involved to maintain the sessions flow when a crisis arises and to make robots more empathetic during robot-assisted therapy.