I've always been intrigued by the meta aspects of research. Why use a particular research method, or statistical procedure? When a researcher chooses a particular method, or procedure, what guides their decision? What is the current status quo, and are there improvements that can be made? A poor workman blames their tools. A worse workman uses tweezers to dig a hole because they didn't think about grabbing a shovel. I promise my research methods research is mostly better than my analogies. Mostly.
An exploration of the strengths and weaknesses of using text messaging as a tool for self-report data collection in psychological research
This was the topic of my PhD, so much of this content reflects my dissertation. Said PhD was submitted and conferred in 2015, a "thesis by compilation" (read: mish-mash of published and unpublished bits).
As of October 2015, the hard yard was done; fifteen scientific studies, a million things learned (primarily an enumeration of things I've yet to learn), and the official establishment that the floppy PhD hat does indeed make a much better frizbee than an undergrad mortar board.
Most importantly, I'd reached my goal of supporting future ecological momentary assessment work by conducting the first rigorously methodological investigation of SMS as a tool for self-report psychological research.
Yup, reached it all right. Three years work wrapped safe behind scholarly journal paywalls and in physical form, delightfully bound in yellow.
... OK, so my work in actually looking into this is done (inasmuch as it can ever be done. Don't tempt me to poke at another thread or I'll probably fall into another dissertation's worth of investigation). But there's a long way to go in actually supporting ecological momentary assessment research (and broadly any investigation of intra-individual variability, so very important and comparatively neglected).
Short Message Service (SMS) has immense potential for self-report data collection because it makes use of mobile phones that people already own, and allows researchers to communicate with participants regardless of physical location. Super cool. Though interest in the possibilities of SMS as a tool for psychological research is slowly growing, to date, there has been no structured investigation of how this potential may be applied in psychological research. Which is weird, as it seems to be a no-brainer.
I decided to strike out and examine the feasibility of using SMS as a tool for self-report psychological research, focussing on its strengths and weaknesses as a research mode. As much as I wanted to do ALL THE THINGS, my supervisor (Dr. Jay Brinker) rightly pulled my focus back to something a bit more achievable. So, across fifteen studies (OK so there were several more that didn't pan out), this was investigated using a mixture of literature review, meta-analysis, surveys, and interviews. Participant samples varied from the broad (general population, university students) to specific (the elderly, the deaf). Thank you, lovely participants.
So, what did I find? Well, while SMS has great potential as a tool for psychological self-report research, it has a number of weaknesses. Which you can read about below. Also, holy crapsticks, I was able to bother people 20 times in one day with a short questionnaire, and there wasn't a pitchfork-wielding undergraduate mob outside my office. Huzzah!
For more detail you'll find the whole thing on this website, brought to you by some legit copyright hoop-jumping and a heartfelt hope this information (or even approach to enquiry) will help enrich someone's research. See the section links below.
My main scholarly focus has shifted somewhat (because the next step from psychological research methods is epidemiological and neuroimaging ageing research. Obviously.), but anyone who wants to get in touch about any of this work, or the topic of self-report research methods more broadly is more than welcome to say get in touch via firstname.lastname@example.org
Oh, before I forget; for other examples of what other people, scholarships and supervisors can prod into life, also check out the ANU research thesis digital repository; this place is a gold mine of interesting.
A methodological approach for time series ordinal data
Working with E. Christian, R. Wilkinson, and J. Brinker
A variety of psychological concepts, such as IQ, causal judgement, and attachment hierarchies are meaningfully conceptualised on an ordinal level of measurement. Whilst several analytical techniques exist to deal with single measures of such data, few are generalizable to a repeated measures setting. Many psychological processes are related to context (e.g. stress and stressors). Variability in psychological characteristics are stable, informative individual difference characteristics, yet there remains little investigation of the variability in ordinal constructs over time. This may be due to the methodological and statistical challenges of collecting and analysing such data. This methodology paper proposes a methodology encompassing data collection and analysis, to allow researchers to explore variability over time in ordinal data, where ties are allowed. The proposed text messaging based methodology was evaluated with pilot data collected with 33 undergraduates, sampling on 30 occasions over two months, using attachment hierarchies as an example psychological construct. Pilot results indicated a promisingly high response rate, data completeness, and positive participant perceptions of their response experience. Examples of analyses possible within the proposed treatment of the data are discussed.
Click here to read the slides for the talk: Walsh, E., Christian, E., Wilkinson, R., & Brinker, J. (2014) A Methodological Framework For Exploring Attachment Hierarchy Stability. International Association for Relationship Research Conference (Melbourne, Australia, 10th-13th July)
Click here to read the poster: Walsh, E., Christian, E., Wilkinson, R., & Brinker, J. (2014) A Methodological Framework For Exploring Attachment Hierarchy Stability. International Association for Relationship Research Conference (Melbourne, Australia, 10th-13th July)