Exploring online social support among infertility treatment patients: a text-mining approach
Vanja Ida Erčulj, Aleš Žiberna and Vislava Globevnik Velikonja.
Introduction. Social support positively influences patients’ health and well-being in both real life and computer-mediated settings. Informational and emotional social support types are most commonly found in online social support group discussions.
Method. Latent Dirichlet Allocation was used to analyse 132,374 infertility posts and identify topics of discussion.
Results. Two types of online social support were found: emotional and informational. Emotional support was established in 17% of all posts. Three subtypes of emotional support were identified: encouragement, offerings of congratulation, and empathy. Several informational social support subthemes were found. The two most common were discussions about the infertility treatment procedure from puncture to pregnancy, and about official procedures.
Conclusions. The results agree with similar studies performed on online social support group discussions dealing with health issues. Participants express a great need for informational support. The areas of concern are identified and some implications for medical practice are provided.
Several studies report the positive effect of social support on patients’ physical and mental health, regardless of the nature of their disease. Researchers generally report two types of social support: emotional and informational. Informational support helps patients manage their disease better and improves their everyday functioning (Wicks et al., 2012), while emotional support lowers their stress level and improves their psychological well-being (Yoo et al., 2014). The beneficial effect of social support was also shown in an online support group environment. This study focuses on online social support among patients coping with infertility. It employs a text-mining approach to automatically detect discussion topics in the largest infertility forum in Slovenia and identify themes of social support types. In other countries, researchers analysed such forums manually with several coders assigning codes to the text. Smaller numbers of posts were analysed, showing that informational support prevailed over emotional support. We expect similar results to also be obtained in Slovenia, but seek to take the analysis further by identifying the topics of discussion within each social support type detected. The analysis includes over 130,000 posts of the biggest Slovenian infertility forum made between 2002 and 2016. The informational support themes will shed light on which information users are looking for. The discussion includes some implications for practice.
Social support encompasses everyday behaviour that directly or indirectly communicate to an individual that they are valued and cared for (Barnes and Duck, 1994). Amongst others, it includes two subdimensions: emotional and informational (Sherbourne and Stewart, 1991; Uchino, Cacioppo, and Kiecolt-Glaser, 1996). Emotional support refers to an expression of sympathy and empathy while informational provides information that is of help to others (Myrick, Holton, Himelboim, and Love, 2016). Cutrona and Suhr (1992) distinguish three other types of social support categories: esteem, social network, and tangible support. These additional social support types were found in studies dealing with offline social support. In a computer-mediated environment, however, esteem, social network, and tangible social support are rarely present (Braithwaite, Waldron, and Finn, 1999; Coulson, 2005; Coursaris and Liu, 2009; Loader, Muncer, Burrows, Pleace, and Nettleton, 2002; Winzelberg, 1997.)
The beneficial role of social support for health outcomes including mortality is shown in several studies (Ali, Merlo, Rosvall, Lithman, and Lindström, 2006; McDade, Hawkley, and Cacioppo, 2006; Tomaka, Thompson, and Palacios, 2006; Zhang, Norris, Gregg, and Beckles, 2007). It seems that not only receiving, but also providing, social support is an important factor for maintaining an individual’s health (Brown, Nesse, Vinokur, and Smith, 2003; Gruenewald, Karlamangla, Greendale, Singer, and Seeman, 2007). Social support also effectively alleviates the recipient’s stress (Sarason, Sarason, and Pierce, 1990) and improves their well-being (Bhanja and Jena, 2012; Klemm and Wheeler, 2005; Lieberman and Goldstein, 2005; Reblin and Uchino, 2008; Uchino, 2006; Uchino et al., 1996). Social support’s beneficial role was also found in a computer-mediated setting (Bhanja and Jena, 2012; Klemm and Wheeler, 2005). Amongst others, information sharing online helps individuals function effectively. For example, patients dealing with epilepsy managed their conditions better due to information sharing on a social network site (Wicks et al., 2012). Other studies showed that, in comparison to those not involved, people involved in online social support groups have a better health-related outcome (Gustafson, Hawkins, Boberg, Pingree, Serlin and Graziano, 1999) and experience less isolation and depression (Bacon, Condon, and Fernsler, 2000; Smyth, Stone, Hurewitz, and Kaell, 1999). In cancer patients, emotional support received from participants of a computer-mediated social support group positively affected their emotional well-being (Erfani, Abedin, and Blount, 2017; Yoo et al., 2014).
Regarding infertility treatment, research reveals the beneficial impact of social support on health complaints, depression, anxiety and complicated grief (Lechner, Bolman, and van Dalen, 2007). Social support reduces infertility-related stress (Gibson and Myers, 2002; Schmidt, Tjørnhøj-Thomsen, Boivin, and Nyboe Andersen, 2005), shown to be associated with worse treatment outcomes (Boivin and Schmidt, 2005). In addition, anxiety and depression in patients who experience failed infertility treatments are lower when they receive greater social support (Verhaak, Smeenk, van Minnen, Kremer, and Kraaimaat, 2005). However, women coping with infertility reported their needs for social support from their social network were unmet (High and Steuber, 2014). Considerable research results suggest that participating in online support groups can successfully close this gap (Himmel, Meyer, Kochen, and Michelmann, 2005; Malik and Coulson, 2010; Van Selm, Tuil, Verhaak, Woldringh, and Kremer, 2008; Wischmann, 2008; Zillien, Haake, Fröhlich, Bense, and Souren, 2011).
This research deals with online social support among people coping with infertility. Several studies reported such groups were a source of social support, which in turn alleviates participants’ distress and helps them cope with infertility (Himmel et al., 2005; Malik and Coulson, 2010; Sherbourne and Stewart, 1991; Van Selm et al., 2008; Wischmann, 2008; Zillien et al., 2011). Informational support is reported to be more common than emotional support (Himmel et al., 2005; Richard, Badillo-Amberg, and Zelkowitz, 2017; Van Selm et al., 2008). Research studies dealing with the type of online support given were based on a relatively small number of manually annotated online support group posts. Specific topics of informational and emotional support were not discussed. One of the objectives of this study is to identify these specific topics. A similar approach was employed in a cancer patients’ study where researchers detected several informational and emotional support subtypes (Meier, Lyons, Frydman, Forlenza, and Rimer, 2007). The most common topics in survivors’ messages were seeking treatment information and advice on how to communicate with health providers. Several other information and advice themes arose during communication, but were less frequent. Further analysis revealed six emotional support themes. The top three dealt with encouragement, empathy and emotional coping strategies. In an online community for children with autism spectrum disorders, nine emotional support subthemes were found: relationship, physical affection, sympathy, empathy, encouragement, prayer, consoling, gratitude and congratulation (Mohd Roffeei, Abdullah, and Basar, 2015). Similar subthemes of emotional support were found in an online community of HIV patients where the last four themes were omitted and replaced by expression of care and confidentiality (Mo and Coulson, 2008). This study has a similar objective of identifying specific discussion topics within common social support types in an extensive number of Slovenian infertility forum posts. In contrast to other studies in this research area where the coding of the text was performed manually, topics of discussion will be identified automatically by employing a text-mining approach. The later was one of the proposed methodologies for biomedical information retrieval in the special issue of the one of the core information science journals (Moskovitch, Wang, Pei, and Friedman, 2017).
Overall, this study seeks to answer the following research questions:
- Which social support types are present within the online support group under study?
- To what extent are social support types present in online discussions?
- Which are the specific discussion topics within each social support type?
A total of 16,086 threads containing 132,374 posts published from 2002 till September 2016 in the Slovenian infertility forum found at the med.over.net website was collected. The algorithm for the automatic collection of posts, usernames and dates was written and employed using the R program (R development Core Team, 2016) . Half the threads included fewer than five posts (interquartile range: 3–9). The minimum number of posts per thread was one and the maximum was 810.
The collected text was pre-processed prior to the analysis. Upper-case words were changed to lower-case words. Stop words (e.g. of, I, we, you, with, is), punctuation, numbers and words with an overall frequency of less than two were removed from the text. Tokenisation (division of text into units such as words, punctuation marks) and lemmatisation (changing different forms of the same word such as 'give' and 'gives' into a single word) were applied using an online text analyser (Erjavec, Fišer, Krek, and Ledinek, 2010; Jožef Stefan Institute, 2013).
The first step of the analysis included the automatic retrieval of topics of discussion. Topic modelling, a text-mining method, was used to uncover abstract topics from the collected posts (Brody and Elhadad, 2010) by using a latent Dirichlet allocation. This iterative algorithm assumes that each document is a collection of topics and each topic has a particular probability of generating a certain word. Prior Dirichlet distribution of topics within each document and words within each topic is assumed (Liu, Tang, Dong, Yao, and Zhou, 2016). An individual word can be present in different topics with a different probability and may co-appear in each document with different words. The posterior distribution is obtained using collapsed Gibbs sampling (Griffiths and Steyvers, 2004). The Dirichlet allocation algorithm has been used successfully in various domains, including analysis of such short textual documents from online discussions or tweets (Alkhodair, Fung, Rahman, and Hung, 2018; Rashad, Mohammed, El-Midany, Kandil, and Ibrahim, 2007). The algorithm is implemented in library lda of the R program (Chang, 2015).
In the current study, each post was considered as a separate document. When using the Dirichlet method, the number of topics must be decided in advance. As a smaller number of broader topics was assumed, only 10 topics were extracted. The most probable topic of each document was calculated. Most probable topic probabilities per document ranged from 0.17 to 1.0, with a median value of 0.69. The Dirichlet method was repeated three times with different random starting points with a comparison of the results showing stable results content-wise. To further validate the results, 100 posts per topic were randomly sampled and manually revised for the content-wise correctness of their classification. The share of correctly classified documents per topic was calculated and a 95% confidence interval was obtained using the bootstrap method (Babu, 2005) with 1,000 replications. The share of correctly classified posts ranged from 70% to 91%. The lowest 95% confidence intervals’ bound was 61% and the highest was 96%. The derived topics were manually assigned to social support categories.
One output of the Dirichlet method procedure is a list of the most probable topic keywords. The same word may be assigned to different topics through the iteration history. Words most frequently assigned to a given topic were defined as topic keywords.
In the second step, the Dirichlet method was reapplied to the largest retrieved topic of emotional support with the aim of identifying emotional support subtypes. The retrieval of a variety of numbers of topics was probed where, of all, the three-topic solution was the most interpretable. Topics were internally homogeneous and externally heterogeneous.
Topic modelling results were graphically presented with a correspondence map. As an input for correspondence analysis, a matrix with the number of times each of the top three most probable topic keywords was assigned to each topic during the Dirichlet method iteration process was used. Relationships of topics and keywords were reduced to a lower dimensional space, revealing the underlying (latent) dimensions in which the topics obtained differed.
Overall, participants give or receive emotional support in 17% of all posts. Three different subtopics can be identified (Figure 1). The first is encouragement whereby participants encourage each other and hope for the best possible outcome. The second encompasses posts offering congratulations and best wishes to those who reported a successful treatment outcome (pregnancy). Empathic and sympathetic posts are included in the third emotional support topic.
All remaining posts (83%) are informational support posts. Stages of the IVF procedure from follicular puncture to embryo transfer and anticipation of the treatment outcome (pregnancy; topic Pregnancy signs) are discussed in 13% of all posts. Patients discuss official procedures, referrals, doctors and appointments in 12% of posts. The exchange of information about medications and their effect is a topic of 10% of posts. Another 10% of posts deal with a situation appraisal. Treatment changes the participants’ view of life. They reflect on changes the treatment has on their everyday life, partnership, friendships and work. They reassess and redefine these changes in order to illustrate them as more positive and easier to accept. Another 10% deals with health problems, medical examination and medical findings that might be the cause of participants’ inability to become pregnant.
In 8% of posts, participants share information on national healthcare system regulations, particularly those dealing with infertility treatment. They discuss associations of common interest in which they can participate or sign governmental petitions for some common cause or their rights regarding infertility treatment and/or child adoption. In a further 8% of posts, the menstrual cycle is discussed in more detail. Participants discuss the signs of ovulation, pregnancy and health problems or medications influencing the menstrual cycle. Of the remaining 14% of posts, half deals with infertility treatment in detail such as puncture, the embryo transfer, and experienced pain. The other half deals with food supplements that could improve fertility, along with alternative treatments like acupuncture, reflexotherapy etc.
|Informational support (83%)|
|PUNCTURE to PREGNANCY (13%)|
KWs: test, puncture, discharge, transfer, sign
KWs: procedure, medication, injection, Gonal, mail
|MENSTRUAL CYCLE (8%)|
KWs: ovulation, Clomiphene, menstruation, cycle, cyst
|Example: I'm in the IVF procedure for the first time. They transferred two blastocysts. Currently I'm waiting for a +. And mail about the frozen eggs. Keeping fingers crossed for all!||Example: I asked if sth was wrong if I took them in the evening and they said no, but I should take them every day then, and it worked. But now I take them in the morning, and we'll see what will be ... regards||Example: Elevated prolactine prevents ovulation, and you can't get pregnant. I had problems years ago, I drank Bromergon, which didn't help. I went to an endocrinologist. For most, the problem is solved by Bromergon.|
|OFFICIAL PROCEDURES (12%)|
KWs: referral, dr. Resh, procedure, medical check-up, medical report
|MEDICAL EXAMINATION (10%)|
KWs: getting pregnant, child, the fallopian tube, medical examination, IVF
|INFERTILITY TREATMENT (7%)|
KWs: cell, embryo, procedure, follicle, transfer
|Example: When is the test for STD done? If a woman doesn't get pregnant naturally, after all examinations or before IVF? Or when do men have a spermiogram?||Example: It's probably polycystic ovaries, which is also the problem I have. You don't have regular ovulation and it's harder to get pregnant. This'll improve so that you'll get pregnant. GOOD LUCK.||Example: I wonder why IUI is done at all if the seed pattern was inappropriate or less appropriate?! I learned this only after the completion of three IUIs and a doctor's check-up.|
|SITUATION APPRAISAL (10%)|
KWs: job, seek leave, session, head, hurt
|INFORMATION, ADVICE (8%)|
KWs: child, man, association, life, infertility
KWs: nutrition, vitamin, tea, drink, royal jelly
|Example: Yes, a familiar problem! Sex for pleasure as sex on call is a killer, I know as I’ve been there, done that! The more you press it, the worse it will be, and you will start arguing! Anyhow, after 5 years, you deserve it!||Example: mmmm, my sixth sense tells me that the government will lower the age limit for IVF! And more procedures will be paid for the younger ones. FAIR? I don’t think so!||Example: This month we were drinking it in the same way as you did, with honey. I also added vinegar to mine. Next month we will try a new regime.|
Topics and their keywords are presented in a perceptual map by using correspondence analysis. Three dimensions explain 51% of inertia, a measure similar to explained variance in factor analysis. The first dimension distinguishes between traditional and alternative treatment of infertility. In this dimension, the topic Supplements is distinguished from all other topics. The second dimension distinguishes emotional and informational support (the horizontal dimension in Figure 2). Patients receive and give emotional support by offering encouragement, compassion and sharing joy when treatment is successful. From emotional support, their discussions shift towards more informational support topics of treatment descriptions, hormonal therapy and the official procedures they must follow in order to receive infertility treatment covered by health insurance. The third dimension (the vertical dimension in Figure 2) summarises the whole treatment process from the first information about infertility until the treatment outcome.
The online social support group for infertility patients in Slovenia has been active since 2002. By September 2016, over 130,000 posts had been published. Within this study, they were automatically retrieved and analysed. The first research question tackled the content of the downloaded forum posts. The text-mining approach for automatic topic retrieval revealed the main function of the online support group community is to provide and receive emotional and informational support (the second underlying dimension of the content discussion; Figure 2). This finding agrees with findings of similar studies in other European countries (Himmel et al., 2005; Malik and Coulson, 2010; Richard et al., 2017; Van Selm et al., 2008; Wischmann, 2008; Zillien, Haake, Fröhlich, Bense, and Souren, 2011). Studies also found that participants offer more informational than emotional support (Himmel et al., 2005; Meier et al., 2007; Richard et al., 2017; Van Selm et al., 2008). This finding is supported by this study. The majority of posts (83%) analysed included an exchange of information between participants and only 17% entailed the giving or receiving of emotional support. This finding answers the second research question that addresses the prevalence of social support types in the online infertility community. Within the emotional support posts, three topics or subtypes of emotional support were identified, namely encouragement, offering congratulations, and expression of empathy and sympathy. Encouragement and empathy were also two of the most common types of emotional support found in cancer survivors messages (Meier et al., 2007), in posts of an online community for children with autism (Mohd Roffeei et al., 2015) and in an online community of HIV patients (Mo and Coulson, 2008).
Regarding informational support, participants of the online infertility support group are most interested in phases of the IVF procedure, from the puncture to the treatment outcome. They want to know which are ‘the right’ feelings accompanying these phases of in-vitro fertilisation and which could be physical signs of pregnancy. Participants exchange information about their feelings throughout these phases and want to know whether certain physical conditions such as pain, vaginal discharge, discomfort are ‘normal’ in the course of IVF treatment. Participants also discuss official (administrative) procedures during treatment including referrals, waiting periods for diagnostic or treatment etc., their ‘standing’ regarding sick leave from work, and sharing information about their difficulties of having a child with their friends or family. Participants also discuss the regime, dosage and application of medications. They discuss medical examinations and share their experience with different examinations or surgical procedures. They exchange information about adoption or other possibilities for infertility treatment abroad and share some general information regarding infertility treatment, women’s menstrual cycle and alternative treatment options. The content of discussion thus ranges from traditional medical treatment towards alternative or complementary treatment of infertility. It follows the infertility treatment timeline: from the first information about infertility treatment till the treatment outcome. It reflects the participants’ vast need for information.
Slovenia has three public medical centres for infertility treatment. The biggest centre is located in the Slovenian capital, within the Ljubljana University Medical Centre. Official informational support is offered when a couple starts infertility treatment by way of an individual consultation with the infertility treatment team. Couples can also obtain informational support by contacting the medical centre electronically or by phone where they obtain information from a medical specialist. Emotional support in terms of psychological counselling is routinely given to couples in the procedure of obtaining a donor sperm or egg cell. Other couples can begin psychological counselling upon a referral from their gynaecologist. Other formal types of social support, such as formal social support group meetings, do not exist within the health centre or broader. The findings of this research can be transferred to medical practice so that infertility specialists provide enough informational support in areas of major concern to infertility treatment patients. Constant social support throughout infertility treatment, within each treatment phase, is needed to prevent unnecessary additional stress associated with a lack of knowledge about possible concomitant treatment signs or symptoms. Patients are eager to understand the effects of the treatment and medications on their physical functioning and overall health. According to a vast share of online informational support posts, it is evident that spending additional time on educating patients would not be in vain. As shown by many studies, social support is directly linked to patients’ health (Ali et al., 2006; Gustafson et al., 1999; McDade et al., 2006; Tomaka et al., 2006) and within infertility treatment could also result in a higher rate of successful treatment outcomes or at least the improved psychological well-being of infertility treatment patients (Klemm and Wheeler, 2005; Reblin and Uchino, 2008).
Regarding future research, it would be interesting to monitor the content of online support group discussions and provide feedback to medical doctors as to where greater emphasis should be placed while providing informational support to their patients.
This study has some limitations. It should be mentioned that the automatic identification of discussion topics is associated with a certain degree of ‘wrong’ post classifications. This mistake was, however, evaluated manually and analysis showed the content of discussions within topics is homogenous. An alternative approach of treating the noise problem could be also introducing a new type of topic to gather noisy words, as was recently proposed by some researchers (Li, Wang, Zhang, Li, Chi and Ouyang, 2018). As the number of topics needs to be decided upon in advance, the decision to extract a larger number of topics would yield a different outcome and some topics would be identified that were now overlooked. However, the inspection of a fifteen-topic solution provided less cohesive results than the retrieval of ten topics.
This study is the first study to automatically retrieve topics of discussion in this area of research in Slovenia and beyond. Specific topics within each type of support were identified that reveal the participants’ needs. Information about this could serve to guide healthcare providers and underpin their attempts to satisfy these needs.
About the authors
Vanja Ida Erčulj is the owner of Rho sigma Research & Statistics, a company dealing with statistical and research methodology consulting in Ljubljana, Slovenia. She is a lecturer at the Faculty of Criminal Justice and Security where she holds statistical and methodological courses. Her research interests are related to information retrieval from textual data, research methodology and biostatistics. She can be contacted at email@example.com
Aleš Žiberna is associate professor at Faculty of Social Sciences, University of Ljubljana, in Slovenia, Ljubljana. He is a member of a chair of Social Informatics and Methodology and a coordinator of Social statistics modules at PhD and masters programs. His research expertise is social network analysis, especially blockmodelling. He is the author and maintainer of the package Blockmodeling in program R. He can be contacted at firstname.lastname@example.org
Vislava Globevnik Velikonja an assistant professor in psychology, is employed at the University Medical Centre Ljubljana’s Division of Obstetrics and Gynaecology as a specialist in clinical psychology. In addition to her clinical work, where she expands the field of healthcare psychology to the field of gynaecology and obstetrics in Slovenia, she is also active in the research and pedagogical field. As a lecturer and mentor she is involved in pedagogical work for students of psychology, medicine, nursing and midwifery, and trainees in clinical psychology, gynaecology and obstetrics, and paediatrics. She can be contacted at email@example.com
- Ali, S. M., Merlo, J., Rosvall, M., Lithman, T., & Lindström, M. (2006). Social capital, the miniaturisation of community, traditionalism and first time acute myocardial infarction: a prospective cohort study in southern Sweden. Social Science and Medicine, 63(8), 2204–2217.
- Alkhodair, S. A., Fung, B. C. M., Rahman, O., & Hung, P. C. K. (2018). Improving interpretations of topic modeling in microblogs. Journal of the Association for Information Science and Technology, 69(4), 528–540.
- Babu, G. J. (2005). Bootstrap techniques for signal processing. Technometrics, 47(3), 374–375.
- Bacon, E. S., Condon, E. H., & Fernsler, J. I. (2000). Young widows’ experience with an Internet self-help group. Journal of Psychosoc Nursing and Mental Health Services, 38(7), 24–33.
- Barnes, M. K., & Duck, S. (1994). Everyday communicative contexts for social support. In B.R. Burleson, T.L. Albrecht and I.G. Sarason (eds.). Communication of social support: messages, interactions, relationships, and community, (pp. 175–193). Thousand Oaks, CA, US: Sage Publications, Inc.
- Bhanja, C., & Jena, S. (2012). Synthetic protocols of a top-selling anti-hypertensive drug “Captopril”: a synthon disconnection approach. Journal of Chemical and Pharmaceutical Research, 4(7), 3610–3616.
- Boivin, J., & Schmidt, L. (2005). Infertility-related stress in men and women predicts treatment outcome 1 year later. Fertility and Sterility, 83(6), 1745–1752.
- Braithwaite, D. O., Waldron, V. R., & Finn, J. (1999). Communication of social support in computer-mediated groups for people with disabilities. Health Communication, 11(2), 123–151.
- Brody, S., & Elhadad, N. (2010). Detecting salient aspects in online reviews of health providers. In AMIA Annual Symposium Proceedings, 2010, (pp. 202–206). Bethesda, MD: American Medical Informatics Association. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3041395/ [Unable to archive]
- Brown, S. L., Nesse, R. M., Vinokur, A. D., & Smith, D. M. (2003). Providing social support may be more beneficial than receiving it: results from a prospective study of mortality. Psychological Science, 14(4), 320–327.
- Chang, J. (2015). Package "lda". Retrieved October 8, 2018. Retrieved from https://cran.r-project.org/web/packages/lda/lda.pdf (Archived by WebCite® at http://www.webcitation.org/75z5khbnK)
- Coulson, N. S. (2005). Receiving social support online: an analysis of a computer-mediated support group for individuals living with irritable bowel syndrome. CyberPsychology & Behavior, 8(6), 580–584.
- Coursaris, C. K., & Liu, M. (2009). An analysis of social support exchanges in online HIV/AIDS self-help groups. Computers in Human Behavior, 25(4), 911–918.
- Cutrona, C. E., & Suhr, J. A. (1992). Controllability of stressful events and satisfaction with spouse support behaviors. Communication Research, 19(2), 154–174.
- Erfani, S. S., Abedin, B., & Blount, Y. (2017). The effect of social network site use on the psychological well-being of cancer patients. Journal of the Association for Information Science and Technology, 68(5), 1308–1322.
- Erjavec, T., Fišer, D., Krek, S., & Ledinek, N. (2010). The JOS linguistically tagged corpus of Slovene. In Calzolari, N., Choukri, K., Maegaard, B., Mariani, J., Odijk, J., Piperidis, S., ... Tapias, D. (Eds.). Proceedings of the Seventh International Conference on Language Resources and Evaluation: LREC 2010, (pp. 1806–1809). Valletta, Malta: European Language Resources Association.
- Gibson, D. M., & Myers Gibson, J. E. (2002). The effect of social coping resources and growth-fostering relationships on infertility stress in women. Journal of Mental Health Counseling, 24(1), 68–80.
- Griffiths, T. L., & Steyvers, M. (2004). Finding scientific topics. Proceedings of the National Academy of Sciences of the United States of America, 101(Supplement 1), 5228–5235.
- Gruenewald, T. L., Karlamangla, A. S., Greendale, G. A., Singer, B. H., & Seeman, T. E. (2007). Feelings of usefulness to others, disability, and mortality in older adults: the MacArthur study of successful aging. Journals of Gerontology - Series B Psychological Sciences and Social Sciences, 62(1), P28-P37.
- Gustafson, D. H., Hawkins, R., Boberg, E., Pingree, S., Serlin, R. E., & Graziano, F. (1999). Impact of a patient-centered, computer-based health information/support system. American Journal of Preventive Medicine, 16(1), 1–9.
- High, A. C., & Steuber, K. R. (2014). An examination of support (in)adequacy: Types, sources, and consequences of social support among infertile women. Communication Monographs, 81(2), 157–178.
- Himmel, W., Meyer, J., Kochen, M. M., & Michelmann, H. W. (2005). Information needs and visitors’ experience of an internet expert forum on infertility. Journal of Medical Internet Research, 7(2), e20. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1550645/ [Unable to archive]
- Jožef Stefan Institute. (2013). ToTaLe analyser for Slovene. Ljubljana: Jožef Stefan Institute. Retrieved from http://nl.ijs.si/analyse/ (Archived by WebCite® at http://www.webcitation.org/75z3rzg8k)
- Klemm, P., & Wheeler, E. (2005). Cancer caregivers online: hope, emotional roller coaster, and physical/emotional/psychological responses. CIN - Computers Informatics Nursing, 23(1), 38–45.
- Lechner, L., Bolman, C., & van Dalen, A. (2007). Definite involuntary childlessness: associations between coping, social support and psychological distress. Human Reproduction, 22(1), 288–294.
- Li, X., Wang, Y., Zhang, A., Li, C., Chi, J., & Ouyang, J. (2018). Filtering out the noise in short text topic modeling. Information Sciences, 456, 83–96.
- Lieberman, M. A., & Goldstein, B. A. (2005). Self-help on-line: an outcome evaluation of breast cancer bulletin boards. Journal of Health Psychology, 10(6), 855–862.
- Liu, L., Tang, L., Dong, W., Yao, S., & Zhou, W. (2016). An overview of topic modeling and its current applications in bioinformatics. SpringerPlus, 5(1), 1608.
- Loader, B. D., Muncer, S., Burrows, R., Pleace, N., & Nettleton, S. (2002). Medicine on the line? Computer-mediated social support and advice for people with diabetes. International Journal of Social Welfare, 11(1), 53–65.
- McDade, T. W., Hawkley, L. C., & Cacioppo, J. T. (2006). Psychosocial and behavioral predictors of inflammation in middle-aged and older adults: the Chicago health, aging, and social relations study. Psychosomatic Medicine, 68(3), 376–381.
- Malik, S. H., & Coulson, N. S. (2010). Coping with infertility online: An examination of self-help mechanisms in an online infertility support group. Patient Education and Counseling, 81(2), 315–318.
- Meier, A., Lyons, E. J., Frydman, G., Forlenza, M., & Rimer, B. K. (2007). How cancer survivors provide support on cancer-related internet mailing lists. Journal of Medical Internet Research, 9(2), e12.
- Mo, P. K. H., & Coulson, N. S. (2008). Exploring the communication of social support within virtual communities: A content analysis of messages posted to an online HIV/AIDS support group. CyberPsychology & Behavior, 11(3), 371–374.
- Mohd Roffeei, S. H., Abdullah, N., & Basar, S. K. R. (2015). Seeking social support on Facebook for children with Autism Spectrum Disorders (ASDs). International Journal of Medical Informatics, 84(5), 375–385.
- Moskovitch, R., Wang, F., Pei, J., & Friedman, C. (2017). JASIST special issue on biomedical information retrieval. Journal of the Association for Information Science and Technology, 68(11), 2525–2528.
- Myrick, J. G., Holton, A. E., Himelboim, I., & Love, B. (2016). #Stupidcancer: exploring a typology of social support and the role of emotional expression in a social media community. Health Communication, 31(5), 596–605.
- R Core Team. (2016). R: A language and environment for statistical computing. Vienna, Austria: the R foundation for Statistical Computing. Retrieved from http://www.r-project.org/ (Archived by WebCite® at http://www.webcitation.org/75z5Fqh5t)
- Rashad, M. M., Mohammed, R. S., El-Midany, A. A., Kandil, A. T., & Ibrahim, I. A. (2007). Application of statistical design to optimize the preparation of barium titanate nanopowders via oxalate precursor method. Eurasian Chemico-Technological Journal, 9(2), 113–119.
- Reblin, M., & Uchino, B. N. (2008). Social and emotional support and its implication for health. Current Opinion in Psychiatry, 21(2), 201–205.
- Richard, J., Badillo-Amberg, I., & Zelkowitz, P. (2017). "So much of this story could be me": men’s use of support in online infertility discussion boards. American Journal of Men’s Health, 11(3), 663–673.
- Sarason, B. R., Sarason, I. R., & Pierce, G. R. (1990). Social support. An interactional view. Oxford: John Wiley & Sons.
- Schmidt, L., Tjørnhøj-Thomsen, T., Boivin, J., & Nyboe Andersen, A. (2005). Evaluation of a communication and stress management training programme for infertile couples. Patient Education and Counseling, 59(3), 252–262.
- Sherbourne, C. D., & Stewart, A. L. (1991). The MOS social support survey. Social Science and Medicine, 32(6), 705–714.
- Smyth, J. M., Stone, A. A., Hurewitz, A., & Kaell, A. (1999). Effects of writing about stressful experiences on symptom reduction in patients with asthma or rheumatoid arthritis: a randomized trial. Journal of the American Medical Association, 281(14), 1304–1309.
- Tomaka, J., Thompson, S., & Palacios, R. (2006). The relation of social isolation, loneliness, and social support to disease outcomes among the elderly. Journal of Aging and Health, 18(3), 359–384.
- Uchino, B. N. (2006). Social support and health: a review of physiological processes potentially underlying links to disease outcomes. Journal of Behavioral Medicine, 29(4), 377–387.
- Uchino, B. N., Cacioppo, J. T., & Kiecolt-Glaser, J. K. (1996). The relationship between social support and physiological processes: A review with emphasis on underlying mechanisms and implications for health. Psychological Bulletin, 119(3), 488–531.
- Van Selm, M., Tuil, W., Verhaak, C., Woldringh, G., & Kremer, J. (2008). Chat about what matters most: an analysis of chat contributions posted to an outpatient fertility website. Cyberpsychology & Behavior, 11(6), 675–677.
- Verhaak, C. M., Smeenk, J. M. J., van Minnen, A., Kremer, J. A. M., & Kraaimaat, F. W. (2005). A longitudinal, prospective study on emotional adjustment before, during and after consecutive fertility treatment cycles. Human Reproduction, 20(8), 2253–2260.
- Wicks, P., Keininger, D. L., Massagli, M. P., la Loge, C. de, Brownstein, C., Isojärvi, J., & Heywood, J. (2012). Perceived benefits of sharing health data between people with epilepsy on an online platform. Epilepsy and Behavior, 23(1), 16–23.
- Winzelberg, A. (1997). The analysis of an electronic support group for individuals with eating disorders. Computers in Human Behavior, 13(3), 393–407.
- Wischmann, T. (2008). Implications of psychosocial support in infertility - a critical appraisal. Journal of Psychosomatic Obstetrics and Gynecology, 29(2), 83–90.
- Yoo, W., Namkoong, K., Choi, M., Shah, D. V., Tsang, S., Hong, Y., … Gustafson, D. H. (2014). Giving and receiving emotional support online: Communication competence as a moderator of psychosocial benefits for women with breast cancer. Computers in Human Behavior, 30, 13–22.
- Zhang, X., Norris, S. L., Gregg, E. W., & Beckles, G. (2007). Social support and mortality among older persons with diabetes. The Diabetes Educator, 33(2), 273–281.
- Zillien, N., Haake, G., Fröhlich, G., Bense, T., & Souren, D. (2011). Internet use of fertility patients: a systematic review of the literature. Journal Für Reproduktionsmedizin Und Endokrinologie, 8(4), 281–287.