An Unsupervised Approach to Extract Life-Events from Personal Narratives in the Mental Health Domain
p. 394-398
Résumé
Personal Narratives are an important source of knowledge in the mental health domain. Over an extended period of time, the psychologist learns about the patient’s life-events and participants from the Personal Narratives shared during each therapy session. The acquired knowledge is then used to support the patient to reach a healthier mental state by appropriate targeted feedback during each conversation. In this work, we propose an unsupervised approach to automatically extract personal life-events and participants from the patient’s narratives and represent them as a personal graph. This personal graph is then updated at each interaction with the patient. We have evaluated our proposed approach on a dataset of longitudinal Italian Personal Narratives as well as a dataset of English commonsense stories.
Remerciements
The research leading to these results has received funding from the European Union – H2020 Programme under grant agreement 826266: COADAPT.
Texte intégral
1. Introduction
1There is a growing research and clinical interest in developing conversational agents (CA) for mental health support as Personal Healthcare Agents (PHA) (Abd-alrazaq et al. 2019; Fitzpatrick, Darcy, and Vierhile 2017; Inkster, Sarda, and Subramanian 2018). However, the lack of appropriate domain knowledge has resulted in the abundance of rule-based dialogue systems in the mental health domain with shallow interactions and weak user engagement (Abd-Alrazaq et al. 2021). Currently available dialogue knowledge can be adequate for consumer-oriented agents or holding a free-topic social conversation. However, it can not be used to hold a dialogue about personal life-events and emotions. Meanwhile, patients’ conversations in the mental health domain have a unique and complex structure since they encompass personal feelings and situations which vary across patients and interventions.
2In order to carry out a personal conversation regarding the patient’s life-events, it is essential to obtain the required knowledge during each interaction with the patient and from her Personal Narratives. Personal Narratives (PN) are recollections of thoughts and emotions about life-events of the patient. These narratives are used by the psychologist to identify the issues that have activated the patient’s emotional state and provide support accordingly in order to reach a healthier mental status (Tammewar et al. 2019; Vromans and Schweitzer 2011).
3In this work, we present an unsupervised approach, inspired by (Chambers and Jurafsky 2008), to automatically extract the life-events and their participants from the patient’s PNs, and construct a Personal Space Graph. Figure 1 represents the work flow of our model. Through the interaction with the patient, each narrative is parsed and presented in terms of its predicates (the events, the edges of the graph) and their noun dependencies (the participants, the nodes of the graph). Each edge has an index based on its order of appearance in the narrative which makes it possible to reconstruct the order of occurrences among the events (for instance, the event "litigo spesso" is mentioned after "parla male"). Moreover, the events and participants mentioned in a recent narrative are considered to be more relevant for an ongoing interaction. Based on this assumption, older nodes and edges in the graph will become less relevant upon receiving a new narrative (presented by dashed lines in Figure 1). The obtained graph can be integrated with PHAs to automatically identify the life-event that is distressing the patient from his/her PNs to provide support and monitor its recurrence.
4We have evaluated our approach on a dataset of longitudinal Italian PNs collected from patients who were receiving Cognitive Behavioural Therapy to manage their distress1. Besides, the English adaptation of our model was evaluated in the "Story Cloze Test" setting introduced by (Mostafazadeh et al. 2016). The results show that the proposed approach obtains similar results to other unsupervised models on the English dataset, and can be a strong baseline for personal space representation and response selection in Italian.
2. Literature Review
5Unsupervised Event Extraction There have been several interesting works regarding the unsupervised extraction of events and their participants from unstructured text. (Chambers and Jurafsky 2008) introduced the concept of "Narrative Event Chain". In this work, the events with a shared participant are assumed to be parts of a uniform story. They present the events in the narrative by the verbs that have a shared participant, and the participant’s role for each verb. (Chambers and Jurafsky 2009) then extended this model to "Narrative Schema", obtained by merging different event chains extracted from one narrative into an integrated uniform schema in order to model the document by all participants across the verbs. Recently, (Hatzel and Biemann 2021) proposed to further extend the "Narrative Schema" concept to support long documents in German language by 1) performing language adaptation of the model; and 2) dividing the event sequence into multiple strongly-connected schema in order to present different scenes in a long story.
6Evaluation Criteria Regarding the evaluation of the models, the mentioned unsupervised approaches were evaluated in "Narrative Cloze Test" setting. In this setting, an event is removed from a sequence of events in a document and the task is to predict the most probable candidate for the missing event (Chambers and Jurafsky 2008). Later however, (Mostafazadeh et al. 2016) introduced "Story Cloze Test" evaluation criterion. In this setting, the system selects a complete lexicalized sentence as the closure to a story rather than predicting the missing event. For this purpose, the authors crowd-sourced a dataset of commonsense stories, called ROCStories, with right and wrong ending sentences for each story.
3. Personal Space Graph Representation
7In this work, we propose an unsupervised Entity-Relation Extraction (ERE) approach to obtain the personal graph of life-events and participants from the user’s PNs in the mental health domain. Figure 1 shows the workflow of our approach, consisting of five main modules.
8Functional Unit Segmentor Upon receiving a narrative, it is first segmented into its functional units. A functional unit is a contiguous span within a message which has a coherent communicative intention (Oltean et al. 2017). The segmentation into Functional Units was performed by a seq2seq model (Zhao and Kawahara 2019), trained to jointly perform Functional Unit segmentation and Dialogue Act (DA) tagging, based on ISO standard DA tagging in Italian (Roccabruna, Cervone, and Riccardi 2020). The model was trained on the corpus of Italian dialogues in the mental health (Mousavi et al. 2021). The predictions of the model were then post-edited and adjusted by two human annotators with strong inter-annotator agreement (0.87) measured by Cohen’s $\kappa$ coefficient (Fournier and Inkpen 2012).
9Dependency Parser Each functional unit is then passed to the dependency parser to obtain the corresponding dependency tree, for which spaCy natural language processing library2 was used. Using the obtained tree and part-of-speech tags, tokens tagged as nouns and proper nouns are extracted as nodes in the graph (nominal modifier nouns are excluded in this process since they are describing/specifying characteristics of another noun). In cases that pronouns are subjects or objects of a verb, they are extracted as nodes as well.
10Entity Linking In order to make sure repeated nouns or variations of the same noun are mapped to the correct node in the graph, an Entity Linking module is defined. This module queries BabelNet3 and ConceptNet4 semantic networks for the root form of the extracted nouns and matches them consequently to obtain a set of entities and participants in the narrative.
11Null Subject Restorer All the verbs contained in the functional unit are extracted and controlled for possible null subject case. Null subjects are non overtly expressed subject pronouns commonly used in pro-drop languages such as Italian and Spanish (Russo, Loáiciga, and Gulati 2012). In this case, the subject of the verb is restored as a pronoun based on its conjugation using an out-of-the-shelf library MLCONJUG35 to make sure each event participant is detected and extracted correctly.
12Entity-Relation Extraction Lastly, the model navigates through the dependency tree to find the verbs that connect the extracted entities as subjects and objects/oblique nominals. In cases of entity conjunctions, the same verb spans over all the entities in the same conjunction. For a better visualization, the neighbours of the verb in the dependency tree are explored to obtain an entire predicate composed by adverbs, ad-positions and auxiliaries as the edge of the graph.
13The obtained graph is specific to each patient and spans over the life-events shared in the narratives. In each graph, the patient is presented as the node "Io" and all the other participants in the patient’s PNs are connected to it by the corresponding predicate. PNs in the mental health domain are about the events that activated the patient’s emotional state. Therefore, it is important to maintain the consecutive order among events in each PN as well as among subsequent PNs through several interactions with the patient. For this purpose, each edge is indexed based on its sequence of appearance in the narrative in order to reconstruct the ordered chain of events. Moreover, events extracted from prior narratives are considered less relevant to the patient’s mental status, unless they re-appear in recent narratives. Therefore, these events receive lower importance score in time based on the assumption that the issue is resolved and the patient does not feel the need to re-mention it.
4. Evaluations
14We have evaluated our proposed approach in two different settings in the mental health domain for Italian language. Furthermore, we have compared the performance of its English adaptation with other models in the "Story Cloze Test" setting introduced by (Mostafazadeh et al. 2016).
4.1 Personal Narratives Evaluation
15We first collected a dataset PNs from Italian patients who were receiving Cognitive Behavioural Therapy to better manage their distress6. Using the approach introduced priorly by (Mousavi et al. 2021), the patients were asked to write PNs about real-life situations and events that have activated their emotional state for the period of three months. As the result, we collected 241 PNs from 18 patients with average length of 128.2 tokens per PN and average number of 11.9 PNs per patient.
16Using the obtained dataset of PNs, in the first setting we evaluated the model for the task of Closure Selection. That is, the model was tasked to select the correct closure sentence for an incomplete narrative based on the participants and events (verbs) it consists of. Similar to a response-selection setting, we assessed the performance of the model using two pools of 2 and 5 candidates, each consisting of 1 correct closure and n-1 distractors.
17In the second setting, we evaluated whether the obtained graph can correctly represent a personal space of events and participants that varies for each user. To this end, the model was first presented with a set of 2 or 5 consecutive PNs from a specific patient as history. Once the corresponding personal space graph was extracted, the model was tasked to select the next possible PN from that patient from a pool of 2 candidates, consisting of the correct PN and a distractor (a PN written by a different user.)
18The results of these evaluations are presented in Table 1. In the first scenario, while TF-IDF manages to be a strong baseline, our proposed system outperforms the Random baseline and has a much higher success rate than the selection solely based on the recurrence of the nouns. Moreover, by raising the task difficulty and increasing the pool size to 5, our model maintains the same performance trend. Regarding the second evaluation, the results indicate that the recurrence of the nouns is an important factor for the model to select the next possible PN. Nevertheless, our model manages to outperform this baseline by considering the predicates as an additional factor, and get closer to TF-IDF scores.
4.2 Story Cloze Test
19In order to compare the performance of our model with other unsupervised approaches, the English adaption of the model was evaluated in the "Story Cloze Test" setting. In this setting, the model is tasked to select the most probable ending for a four-sentence story from a pool of 2, consisting of the right ending and the wrong one. (Mostafazadeh et al. 2016). The result of this evaluation for the test set of 3744 stories is presented in Table 2, indicating that our model performance is inline with other unsupervised approaches.
5. Conclusion
20In this work, we present an approach to automatically extract life-events and participants from patients’ Personal Narratives in the mental health domain and represent them as a personal graph. This graph can be a source of knowledge for Personal Healthcare Agents (PHA) in this domain, to automatically identify the life-event that is activating the user’s emotional state and causing distress.
21We evaluated our model on a domain-specific dataset of Personal Narratives in Italian as well as an open-domain dataset of commonsense stories in English. The results indicate that our proposed model performs in-line with other unsupervised alternatives and can be a strong baseline for automatic extraction of life-events from Personal Narratives in Italian.
Bibliographie
Des DOI sont automatiquement ajoutés aux références bibliographiques par Bilbo, l’outil d’annotation bibliographique d’OpenEdition. Ces références bibliographiques peuvent être téléchargées dans les formats APA, Chicago et MLA.
Format
- APA
- Chicago
- MLA
Alaa A. Abd-alrazaq, Mohannad Alajlani, Ali Abdallah Alalwan, Bridgette M Bewick, Peter Gardner, and Mowafa Househ. 2019. “An Overview of the Features of Chatbots in Mental Health: A Scoping Review.” International Journal of Medical Informatics 132: 103978.
Alaa A. Abd-alrazaq, Mohannad Alajlani, Nashva Ali, Kerstin Denecke, Bridgette M Bewick, and Mowafa Househ. 2021. “Perceptions and Opinions of Patients About Mental Health Chatbots: Scoping Review.” Journal of Medical Internet Research 23 (1): e17828.
Nathanael Chambers, and Dan Jurafsky. 2008. “Unsupervised Learning of Narrative Event Chains.” In Proceedings of Acl-08: HLT, 789–97.
Nathanael Chambers, and Dan Jurafsky. 2009. “Unsupervised Learning of Narrative Schemas and Their Participants.” In Proceedings of the Joint Conference of the 47th Annual Meeting of the Acl and the 4th International Joint Conference on Natural Language Processing of the Afnlp, 602–10.
10.3115/1690219 :Kathleen Kara Fitzpatrick, Alison Darcy, and Molly Vierhile. 2017. “Delivering Cognitive Behavior Therapy to Young Adults with Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial.” JMIR Mental Health 4 (2): e19.
Chris Fournier, and Diana Inkpen. 2012. “Segmentation Similarity and Agreement.” arXiv Preprint arXiv:1204.2847.
Hans Ole Hatzel, and Chris Biemann. 2021. “Towards Layered Events and Schema Representations in Long Documents.” In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop, 32–39. Online: Association for Computational Linguistics. https://0-doi-org.catalogue.libraries.london.ac.uk/10.18653/v1/2021.naacl-srw.5.
10.18653/v1/2021.naacl-srw.5 :Becky Inkster, Shubhankar Sarda, and Vinod Subramanian. 2018. “An Empathy-Driven, Conversational Artificial Intelligence Agent (Wysa) for Digital Mental Well-Being: Real-World Data Evaluation Mixed-Methods Study.” JMIR mHealth and uHealth 6 (11): e12106.
Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Vanderwende, Pushmeet Kohli, and James Allen. 2016. “A Corpus and Cloze Evaluation for Deeper Understanding of Commonsense Stories.” In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 839–49.
Seyed Mahed Mousavi, Alessandra Cervone, Morena Danieli, and Giuseppe Riccardi. 2021. “Would You Like to Tell Me More? Generating a Corpus of Psychotherapy Dialogues.” In Proceedings of the Second Workshop on Natural Language Processing for Medical Conversations, 1–9. Online: Association for Computational Linguistics. https://0-doi-org.catalogue.libraries.london.ac.uk/10.18653/v1/2021.nlpmc-1.1.
10.18653/v1/2021.nlpmc-1.1 :Horea-Radu Oltean, Philip Hyland, Frédérique Vallières, and Daniel Ovidiu David. 2017. “An Empirical Assessment of Rebt Models of Psychopathology and Psychological Health in the Prediction of Anxiety and Depression Symptoms.” Behavioural and Cognitive Psychotherapy 45 (6): 600–615.
Gabriel Roccabruna, Alessandra Cervone, and Giuseppe Riccardi. 2020. “Multifunctional Iso Standard Dialogue Act Tagging in Italian.” Seventh Italian Conference on Computational Linguistics (CLiC-It).
Lorenza Russo, Sharid Loáiciga, and Asheesh Gulati. 2012. “Improving Machine Translation of Null Subjects in Italian and Spanish.” In Proceedings of the Student Research Workshop at the 13th Conference of the European Chapter of the Association for Computational Linguistics, 81–89.
Aniruddha Tammewar, Alessandra Cervone, Eva-Maria Messner, and Giuseppe Riccardi. 2019. “Modeling User Context for Valence Prediction from Narratives.” In Interspeech, 3252–6.
Lynette P. Vromans, and Robert D Schweitzer. 2011. “Narrative Therapy for Adults with Major Depressive Disorder: Improved Symptom and Interpersonal Outcomes.” Psychotherapy Research 21 (1): 4–15.
Tianyu Zhao, and Tatsuya Kawahara. 2019. “Joint Dialog Act Segmentation and Recognition in Human Conversations Using Attention to Dialog Context.” Computer Speech & Language 57: 108–27.
Notes de bas de page
1 This data collection has been approved by the Ethical Committee of the University of Trento
3 BabelNet babelnet.org
4 ConceptNet conceptnet.io
5 MLCONJUG3 pypi.org/project/mlconjug3
6 This data collection has been approved by the Ethical Committee of the University of Trento
Auteurs
Signals and Interactive Systems Lab, University of Trento, Italy – mahed.mousavi@unitn.it
Signals and Interactive Systems Lab, University of Trento, Italy
Signals and Interactive Systems Lab, University of Trento, Italy – giuseppe.riccardi@unitn.it
Le texte seul est utilisable sous licence Creative Commons - Attribution - Pas d'Utilisation Commerciale - Pas de Modification 4.0 International - CC BY-NC-ND 4.0. Les autres éléments (illustrations, fichiers annexes importés) sont « Tous droits réservés », sauf mention contraire.
Proceedings of the Second Italian Conference on Computational Linguistics CLiC-it 2015
3-4 December 2015, Trento
Cristina Bosco, Sara Tonelli et Fabio Massimo Zanzotto (dir.)
2015
Proceedings of the Third Italian Conference on Computational Linguistics CLiC-it 2016
5-6 December 2016, Napoli
Anna Corazza, Simonetta Montemagni et Giovanni Semeraro (dir.)
2016
EVALITA. Evaluation of NLP and Speech Tools for Italian
Proceedings of the Final Workshop 7 December 2016, Naples
Pierpaolo Basile, Franco Cutugno, Malvina Nissim et al. (dir.)
2016
Proceedings of the Fourth Italian Conference on Computational Linguistics CLiC-it 2017
11-12 December 2017, Rome
Roberto Basili, Malvina Nissim et Giorgio Satta (dir.)
2017
Proceedings of the Fifth Italian Conference on Computational Linguistics CLiC-it 2018
10-12 December 2018, Torino
Elena Cabrio, Alessandro Mazzei et Fabio Tamburini (dir.)
2018
EVALITA Evaluation of NLP and Speech Tools for Italian
Proceedings of the Final Workshop 12-13 December 2018, Naples
Tommaso Caselli, Nicole Novielli, Viviana Patti et al. (dir.)
2018
EVALITA Evaluation of NLP and Speech Tools for Italian - December 17th, 2020
Proceedings of the Seventh Evaluation Campaign of Natural Language Processing and Speech Tools for Italian Final Workshop
Valerio Basile, Danilo Croce, Maria Maro et al. (dir.)
2020
Proceedings of the Seventh Italian Conference on Computational Linguistics CLiC-it 2020
Bologna, Italy, March 1-3, 2021
Felice Dell'Orletta, Johanna Monti et Fabio Tamburini (dir.)
2020
Proceedings of the Eighth Italian Conference on Computational Linguistics CliC-it 2021
Milan, Italy, 26-28 January, 2022
Elisabetta Fersini, Marco Passarotti et Viviana Patti (dir.)
2022