Initial Research Outline
[NB: This document is taken from an initial outline submitted as part of an assignment. It does not constitute the final research project / questions.]
Virtual World Simulation of Dual Sensory Loss for Workplace Learning in a Social Care Setting: A Mixed Methods Study
Simulation is a common method used with disability awareness training, yet its efficacy or otherwise remains a controversial area, with concern being increasingly voiced over both the superficial nature and potential deficit model of disability it can portray (Herbert 2000; Pfeiffer 2002). In addition, simulation is person-intensive and expensive, requiring experienced trainers as well as sufficient time and equipment. In the light of new understandings of how people learn and the current economic climate, learning and development departments are being forced to ask hard questions about the effectiveness of training and whether that knowledge transfers to the workplace (CIPD 2012b; CIPD 2014; Kirkpatrick & Kirkpatrick 2013). Trends in UK workplace learning show an increasing shift towards elearning primarily for economic reasons, yet innovative forms of elearning such as virtual worlds remain in the minority (CIPD 2011; CIPD 2014). Virtual worlds offer the potential to deliver transferable skills and are already being used for simulations in healthcare (Ghanbarzadeh et al 2014; Hansen 2008; Herrington 2006), but there is much scepticism about the use of these environments to deliver workplace training (CIPD 2012a & 2014; Landers & Callan 2012) and a lack of research into their effectiveness for this purpose (CIPD 2012a).
The aim of this research is to establish to what extent virtual worlds can be an effective environment for developing understanding of dual sensory loss (DSL) in social care workers through simulation.
There are three outcomes of interest for the research; to what extent can:
- virtual worlds effectively simulate DSL;
- simulations of DSL in virtual worlds address the deficit model of disability;
- simulations of DSL in virtual worlds generate knowledge and skills that transfer to the workplace.
Research design and methods
The research is situated within a large (>2,300 employees) not-for-profit social care organisation, which trains all new employees on DSL as part of a two-day taught induction, using a combination of discussion, video and simulation. The participants will be drawn from this cohort of new employees. [Added: this will not be the case for the final research, which will have a different subject group – potentially an international subject group.]
The chosen methodology follows a mixed methods model: integrating and analysing quantitative and qualitative data, and drawing inferences based on the combined strengths of that data (Creswell 2014 p1).
A mixed methods model was chosen because neither a quantitative nor qualitative approach on its own would be sufficient for explaining the area under research: a quantitative approach alone would produce evidence of the extent of effectiveness, but not explain why; and a qualitative approach alone would not produce ‘hard’ evidence which could aid inference of cause and effect. Using a mixed methods approach should address both causal effect and causal mechanisms (Teddlie & Tashakkori 2009 p128).
A convergent design will be used which embeds qualitative data within a quasi-experimental trial. The design will use:
- qualitative and quantitative methods to gather data at the start – to set a benchmark;
- a quasi-experimental trial – the learning activities (simulations); and
- qualitative and quantitative methods to gather data at the end – to evaluate any change.
A pre-test and post-test will be used to measure attitude towards people with DSL. The test will be based on a series of qualitative statements using a Likert Scale to aid analysis, together with open-ended questions.
There are three potential issues with internal validity in relation to the test:
- The administration of the pre-test could influence participant’s response to the simulation by ‘cluing them in’ on the desired outputs (Cohen et al 2014 chp10.4).
- The test must be created as there are no standardised instruments for measuring attitude towards DSL; there are instruments for measuring attitudes towards people with disabilities, but the reliability of these remains contested (Findler et al 2007) and the focus is on disability in general. There is therefore no baseline for validation of the results from the test and validation will rely on integration and comparison with qualitative data from interviews, focus groups, feedback from participants’ workplace line managers, internal audit and external inspection reports.
- The Likert Scale attempts to convert qualitative data into quantitative data, yet respondents may value their scores differently based on personal perspective and experience.
The learning intervention is quasi-experimental in nature. A classical experimental approach is not appropriate because:
- True randomisation is not possible – the participants are those from an existing cohort and are bias in terms of: gender, geographic location, ethnicity, digital literacy, etc., although this will be managed as far as possible through matching (Cohen et al 2014 chp16.3).
- There are difficulties in creating a control group – it would be misleading to presume that the learning activities replicate an actual world simulation due to the factors that influence such learning (Hunsinger & Krotoski 2012), for example: a charismatic tutor; inconsistent training delivery; differences in workplace ethos towards learning and towards DSL; the format of the simulation itself; etc. Instead of a control being based against an actual world group, four virtual world groups with twelve participants per group will be used for the experiment and comparison between these groups will be made.
These factors have implications for validity, in particular for external validity.
In terms of external validity, inference of causal relationships and generalisation will be affected by several internal variables including:
- scale of the research – the sample size is relatively small (48 participants);
- gender bias – the organisation is 80% female, 20% male (removed);
- recruitment strategy – the organisation has introduced new methods to recruit on the basis of ‘attitudes and values’ (removed);
- organisational culture and ethos – which influences attitudes towards DSL;
- comparison of the effectiveness of actual world simulation versus virtual world simulation – which will not be possible as the two are not similar enough for direct comparison and there is little or no data on the effectiveness of actual world simulation of DSL.
These factors mean that more care must be taken when drawing conclusions, and results will need to be qualified based on the context of the research which is ‘highly localized’ (Shadish et al 2014 p18).
Significance of the research
The aim of the research is to determine whether virtual world simulation of DSL is effective.
‘Effectiveness’ is a subjective concept. For this research, within the context of workplace learning, it is taken to mean:
- “does this learning produce the understanding, attitudes and values towards DSL that the organisation desires” and;
- “do these transfer into practice”.
The drive towards elearning together with the affordances of virtual worlds present a vision for innovative organisational elearning described by CIPD as the opportunity to “create an emotionally compelling context for the player and build on nostalgia, curiosity, visual appeal and employees’ interest” (CIPD 2012a p13). However, if virtual world simulation of DSL is not effective, then it would not be an appropriate learning method to introduce. This together with the inherent scepticism and resistance to change within organisational learning and development (CIPD 2011, 2012a, 2012b & 2014; Landers & Callan 2012) provides a strong practical incentive for the research.
Furthermore as part of ‘effectiveness’, there is the issue of the deficit model of disability to consider: “how effective are virtual world simulations at addressing the deficit model of disability?” This links to the understanding, attitudes and values towards DSL desired by the organisation, as well as being an issue of concern within the wider disability community (Herbert 2000; Flower et al 2007; Pfeiffer 2002). Researching and identifying the factors that develop a tendency towards or away from the deficit model is needed. If virtual world simulation reinforces the deficit model of disability, then it is unlikely that it would be a learning method appropriate for organisational learning and development within this context.
Additionally, although there is significant research into the use of virtual worlds for simulations within healthcare settings from the perspective of working with people with disabilities (Ghanbarzadeh et al 2014; Hansen 2008; Herrington 2006), and research into the experiences of people with disabilities in virtual worlds (Krueger 2013; Stendal et al 2011; Stewart et al 2010), there is little or no research into the effectiveness of using virtual worlds to simulate a disability. Thus this research would address a gap in knowledge.
Finally there is the issue of transferrable skills. If the resultant learning from the simulation does not transfer into practice in the workplace, then it is not effective for organisational learning and development (CIPD 2014; Chiaburu & Lindsay 2008; Mavin et al 2010). Hence the final question to be addressed investigates the effectiveness of this transfer. Generalisation here must though be done cautiously as there may be other significant variables, for example the workplace ethos where one workplace may have a line manager who encourages employees to disseminate new learning and build on it, and another may not. In addition, evaluating the impact of learning and development remains a thorny issue with little agreement on the best or most effective way in which to do this (Anderson 2007; CIPD 2012b & 2014; Mavin et al 2010).
Anderson, V. (2007) The Value of Learning: a new model of value and evaluation, CIPD.
Chiaburu, D.S. & Lindsay, D.R. (2008) ‘Can do or will do? The importance of self-efficacy and instrumentality for training transfer’, Human Resource Development International. 11 (2), pp. 199–206.
CIPD (2011) Focus on e-Learning [online]. Available from: http://www.cipd.co.uk/binaries/focus-on-e-learning_2011.pdf [Accessed 12 January 2015].
CIPD (2012a) From e-learning to ‘gameful’ employment. CIPD.
CIPD (2012b) From steady state to ready state: A need for fresh thinking in learning and talent development? [online]. Available from: http://www.cipd.co.uk/binaries/from-steady-state-to-ready-state_2012.pdf [Accessed 12 January 2015].
CIPD (2014) Learning and Development 2014 [online]. Available from: https://www.cipd.co.uk/binaries/learning-and-development_2014.pdf [Accessed 29 December 2014].
Cohen, L., Manion, L. & Morrison, K. (2013) Research Methods in Education. 7th edition [ebook]. London: Taylor and Francis.
Creswell, J. W. (2014). A Concise Introduction to Mixed Methods Research [Kindle Edition]. SAGE Publications.
Ghanbarzadeh, R., Ghapanchi, A.H., Blumenstein, M. & Talaei-Khoei, A. (2014) ‘A Decade of Research on the Use of Three-Dimensional Virtual Worlds in Health Care: A Systematic Literature Review’, Journal of Medical Internet Research. 16 (2).
Findler, L., Vilchinsky, N. and Werner, S. (2007) ‘The Multidimensional Attitudes Scale Toward Persons With Disabilities (MAS) Construction and Validation’, Rehabilitation Counseling Bulletin, 50(3), pp.166–176.
Hansen, M.M. (2008) ‘Versatile, Immersive, Creative and Dynamic Virtual 3-D Healthcare Learning Environments: A Review of the Literature’, Journal of Medical Internet Research. 10 (3).
Herbert, J.T. (2000) ‘Simulation as a Learning Method to Facilitate Disability Awareness’, Journal of Experiential Education. 23 (1) pp. 5–11.
Herrington, J. (2006) ‘Authentic e-learning in higher education: Design principles for authentic learning environments and tasks’. In: World Conference on E – Learning in Corporate, Government, Healthcare, and Higher Education (ELEARN) 2006.
Hunsinger, J. & Krotoski, A. (2012) ‘Learning and Research in Virtual Worlds’. In: Hunsinger, J. & Krotoski, A. (eds.). Learning and Research in Virtual Worlds. Abingdon, Oxon: Routledge. pp. 1–6.
Kirkpatrick, J.D. & Kirkpatrick, W.K. (2013) ‘Creating a Post-Training Evaluation plan’, T + D. 67 (6), pp. 26–28.
Krueger, A. (2013) ‘People with Disabilities in Virtual Worlds: Who, how, Why, and What’s Next?’, Selected Papers of Internet Research (14) [online]. Available from: http://spir.aoir.org/index.php/spir/article/view/862/pdf [Accessed 28 December 2014].
Landers, R.N. & Callan, R.C. (2012) ‘Training Evaluation in Virtual Worlds: Development of a Model’, Journal of Virtual Worlds Research. 5 (3).
Mavin, S., Lee, L. & Robson, F. (2010) The evaluation of learning and development in the workplace: a review of the literature [online]. Available from: http://www.northumbria.ac.uk/static/5007/hrpdf/hefce/hefce_litreview.pdf [Accessed 6 January 2014].
Pfeiffer, D. (2002) ‘The Philosophical Foundations of Disability Studies’, Disability Studies Quarterly. 22 (2) pp. 3–23.
Shadish, W.R., Cook, T.D. & Campbell, D.T. (2002) Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Belmont, CA.: Wadsworth Cengage Learning.
Stendal, K., Balandin, S. & Molka-Danielsen, J. (2011) ‘Virtual worlds: A new opportunity for people with lifelong disability?’, Journal of Intellectual & Developmental Disability. 36 (1), pp. 80–83.
Stewart, S., Hansen, T.S. & Carey, T.A. (2010) ‘Opportunities for People with Disabilities in the Virtual World of Second Life’, Rehabilitation Nursing. 35 (6), pp. 254–259.
Teddlie, C. & Tashakkori, A. (2009) Foundations of Mixed Methods Research: Integrating Quantitative and Qualitative Approaches in the Social and Behavioral Sciences. London: Sage.