Measuring the Impact of Work Integrated Learning from the Student’s Perspective

Providing Work Integrated Learning (WIL) opportunities to students that are meaningful, relevant and add value to courses are of utmost importance to students, industry partners and education providers. Therefore, devising relevant tools to receive meaningful feedback from students about their WIL learning experiences is not to be underestimated.  

Recently a colleague and I developed a survey instrument to measure and report on students’ perceptions of their learning while undertaking WIL. The survey instrument included questions pertaining to student preparedness for WIL; engagement with active learning while on WIL; and, opportunities to develop a professional sense of self.  Additionally, free text questions asked students to provide examples of what helped their learning; what hindered their learning and to provide other commentary or suggestions on what aspects may improve their learning during WIL.

The results so far demonstrate that supportive WIL environments positively influence student learning.  Factors that students depict as helping their learning are when supervisors/facilitators actively engage in student learning; are available and accessible to respond to questions or concerns; encourage independent learning; and, provide opportunities to contextualise theoretical understandings.  In contrast, students commented their learning was hindered when they were made to feel unwelcome, received little guidance or when expectations were unclear.

While in its infancy, the WIL survey instrument has provided a practical method of analysing and reporting student feedback on the quality of their learning experiences.  Additionally, the tool has provided a mechanism to evaluate how students take responsibility for their own learning. Furthermore, the instrument has enabled cross-disciplinary comparisons related to the quality of student learning experiences. 

For more information about the WIL Survey, please contact annette.marlow@utas.edu.au

Annette Marlow

Director of Professional Experience
 

College of Health and Medicine
 

University of Tasmania
 




Evaluation in WIL: Themes across case studies

In the afternoon session of the NSW/ACT Chapter Forum a number of case studies were analysed by participants in relation to evaluation in work integrated learning (WIL). 

Blair Slater, Senior Careers Consultant (International), UNSW Sydney, has collated the themes across the case studies, and presented a snapshot of those themes below.

It would be great to hear in the comments section below, your views in response to the challenges and tensions identified by the groups. And, more importantly, how have you dealt with these challenges and tensions at your university?

1. Stakeholders:

• Predominate Stakeholders in WIL evaluations:
  Students (current and prospective), academics, designers/Administrators of program or course (careers office, Faculty office), senior executives (decision makers), employers, recipients of work performed (client), 3^rd party providers, university or organisation recruitment team, future employers within the industry since they may potentially benefit from WIL graduates.

  2. Desired Outcomes:

•  Acquire a better understanding of impact to achieve a change.
• Understand the value of a WIL initiative and inform program/course development. “The purpose should be to improve, not to prove.”
• To show positive results in the form of benefits for stakeholders, however, negative results also need to be considered.
• Demonstrate value to increase funding
• To formulate a “toolkit” for success for stakeholders.
• Alignment between WILl and graduate attributes and employability.
• Increase perception of return on investment for students.

3. Steps in the evaluation:

•  Develop research questions and examine if there is any existing theory or literature to help inform questions and area of focus. Ask, will evaluation need to be longitudinal? Is it summative or formative?
• Develop methodology, ethics application if considering research publication.
• Data collection, online surveys, student self-evaluation.  In person interviews are effective for qualitative data.
• Analysis and dissemination to stakeholders.
• All the above need to be assessed in terms of feasibility and cost before starting.

 4. Opportunities, Challenges and Tensions:

 Opportunities

• Evaluation can potentially be applied to other disciplines. For example, faculty to faculty.
• Opportunities for curriculum enhancement/development. Receive greater academic support/financing.
• Evaluation can create a good baseline upon which to base future development and evaluation.
• If evaluation is done well, it could be used for, or extended, into research.

 Challenges

• When evaluating outcomes, be open to the possibility of learning about an unintentional outcome which perhaps can warrant further evaluation or research. Also need to be cautious of censoring negative findings and handling this challenge.
• Not much published on benefits for employers. What evaluations can be done in this area to help bring on more employers?
• Importance of defining/listing the variables in addition to defining and understanding the assumptions being made prior to undertaking evaluation.
• Potential for evaluations to be too broad if variables cannot me limited or identified. For example, students’ background, academics, industry and other factors that may be hard to control or know.
• Not being as successful as originally thought. Do you still report the negative? Does this change the dissemination?
• Tracking alumni in any longitudinal evaluation.

Tensions

• Balancing research/evaluation with program agenda. Often need someone dedicated to the research component. Investment of time and resources in undertaking a good quality evaluation. Cost versus strategic value needs to be assessed.
• Diverse conceptions of “value” depending on stakeholder. Stakeholder who is administrating the evaluation may have different values in mind from those who will be making decisions based on the evaluation.
• Employability skills need to have a clear definition for both the evaluator and subject.
• Employability is also often hard to measure and must be defined. Is it measured through satisfaction (assess against student expectations), learning outcomes, or engagement and activity level of learning longitudinally.
• If students are aware that their reflections will be used for evaluation, will this change what they write or give them self in any self-ratings?
• Need to be aware of comparing same WIL format but with different employers.