ACEN NSW/ACT FORUM SNAPSHOT

Digital images courtesy of Dr Kylie Twyford

Evaluation is something that I have been giving a lot of thought to recently and evaluation of WIL in particular. It seems I am not alone judging by the numbers attending the NSW /ACT ACEN Chapter forum on ‘Realising opportunities through WIL evaluation’ which was held at UTS in Sydney on 14 February.

The purpose of the day was to explore how evaluation can be used for a broad range of purposes in WIL such as informing improvements to practice, influencing policy and strategy, informing  research, and measuring impact (on students, partners and the wider community).

The day commenced with thought provoking presentations from experienced practitioners on each focus area. 

Associate Professor Rachael Hains-Wesson (University of Sydney Business School) presented on evaluating WIL to inform practice. She used the evaluation of an International Study Tours program to demonstrate how a mixed methods approach (e.g. critical friends’ meetings; reflections on literature; online survey for Third Party Providers & student focus group interviews) can inform key recommendations for effective and inclusive collaboration amongst stakeholders to create a high-quality short-term study tour program for Work-Integrated Learning.

JulieanneO’Hara (Manager of UTS Careers) spoke about evaluation of WIL to influence internally and told the story of how UTS Careers developed an Internship Manifesto by evaluation of existing practices and sharing responsibility of WIL through a Careers Community (community of practice) which enabled new ways of influencing internally.

Dr Theresa Winchester-Seeto (Winchester-Seeto Consultancy) provided some useful definitions of evaluation and its formative and summative forms. She spoke about the differences (and overlap) between evaluation and research  -“The purpose of evaluation is to improve, not prove” (Fain et al 2006). She also explained the potential problems with evaluation as research and the points that need to be considered if you want to turn your evaluation into research.

Associate Professor Annette Marlow (University of Tasmania) presented on evaluation to measure the impact of WIL.  She spoke about the Integrated Learning (WIL) Evaluation Tool which she and her colleagues have developed to measure, and report on, student perceptions of their learning while undertaking WIL. She referred to the use of feedback rubrics and the importance of formative feedback (given during the WIL program) to gauge impact. The question was raised as to how to deal with negative feedback about a host organization or indeed a WIL program.

At the end of the presentations participants were given the opportunity to discuss the presentations in groups and then ask the panel questions focusing on the key evaluation topics.

A delicious lunch (thank you UTS!) provided an opportunity for some networking and sharing of ideas before we all reconvened for the afternoon sessions.

The afternoon started with participants working in groups and being presented with a fictitious WIL case study which required evaluation with a focus on either informing practice, influencing internally, measuring impact or informing research. Reflecting on the morning presentations groups were asked to discuss what the desired outcomes of the evaluation would be for their particular case study and purpose. They were also asked to identify what steps might be involved in their evaluation. Lively and engaging discussion within groups enabled them to identify what opportunities, challenges and tensions might arise from such evaluation.  Each table presented their discussions to the whole group.

The afternoon concluded with a presentation by Dr. Anna Rowe and Associate Professor Kate Lloyd on an institutional wide approach to evaluation of Macquarie University’s Professional and Community Engagement (PACE) initiative. This was a good session to end the day on as it brought together many of the themes that had been discussed during the day. The presenters spoke about the opportunities and challenges of implementing a university wide Evaluation Framework to determine the impact of the initiative for the University, its stakeholders and the broader community, with a focus on continuous program improvement and development.

I’m sure that by the end of the day participants felt more confident in approaching evaluation and applying tools and methods to their own programs (I certainly did!). 

Links to all slides from the presentations are available here.

Donna Denyer

Career Development Officer, Careers Centre

University of Sydney