Learning Experience:

A Teaching Training Program on The Role of Research: Integrating, Utilizing, and Applying Research-Based Methods into Students’ Assessments

Learning Goals

• Create a teaching training program raising awareness on different assessment methodologies

• Integrate research-based methods into students’ assessments

• Foster a space and culture for research at the university

• Plan a training program based on adult-learning theories to increase intrinsic motivation in the workplace

Rationale 

The traditional methods of teaching focuses on the following assessment outputs: Question and Answer, Multiple Choice, True/False. These are assessments are limited as they solely measure students’ knowledge of content rather than assess critical thinking skills including the following assessment outputs: research papers, infographics, opinion editorial pieces, vlogs, and more. Moreover, the type of assessment determines instruction. This workshop catered to all general education professors in a private university in the Philippines who are teaching first and second-year students. This will take place over the course three-days held at the Multipurpose Conference Room at Lyceum of the Philippines University in Intramuros, Manila.

Context Analysis 

The learners of the workshop are Filipinx professors ages 35-65, teaching general education subjects for liberal arts students at the university-level (Philippine Government and Constitution, The Contemporary World, Rizal History Course). 8 out of the 10 teachers are part of a teaching union and have stated their need and want more financial incentives. The teachers are highly knowledgeable about the subject-matter, as all teachers have a master’s degree in the area they are instructing. However, are still utilizing the traditional methods of assessments – Q+A, True or False, Multiple Choice (data collected from pre-assessment survey). 

Learning Theories 

To motivate the teachers (learners), teaching incentives will be provided. Learners are able to vet their students to get published in an academic journal; they can also co-publish/ peer-review students’ work for publication. Finally, learners had opportunity to upskill and build a unique research/evidence-based skillset, which can be applied professionally. Research skills practiced in the classroom can be transferred into their own work (near and far transfer).

Certifications were given upon the completion of the workshop, and micro-learning was incorporated into shorter learning sessions to ensure optimal participation and capture the attention spans of the participants.

Adult Learning Theories 

I took into account the three adult learning theories and incorporated these three theories of Andragogy, Self-Directed Learning (SDL), and Transformational Learning into the design of the workshop.

Challenges  

Despite these incentives, learners may be resistant to changing their ways if there is no corresponding financial incentive. Furthermore, they may be unfamiliar with new teaching assessment strategies and may feel that they are already confident with their current teaching strategies. Lastly, there is a huge technology gap between the learners given the age gap and generational gap between the learners. Based on the pre-survey assessments, 3 out of 10 professors are part of a teacher’s union. Professors have previously voiced that they are resistant to change and are less receptive to newer and innovative ways of assessing students unless there is a corresponding financial incentive. For this reason, workshops like these are beneficial so that teachers can exchange best practices on how best to assess the students given the post-pandemic learning modalities (flexible, in-person, and fully remote). The main sources of learners’ data will be pre-workshop and post-workshop surveys (Appendix 1), word-of-mouth anecdotal evidence through live and online breakout rooms (provides a candid way of getting professors’ views on similar types of training and workshops), data collected from students’ evaluation of professor both in the midterm assessment and final assessment. 

The 3-day workshop took place at the Multipurpose Hall/Function Room. Chairs and tables for smaller group discussions for 2-3 pax were provided to ensure an open environment promoting collaboration. Seating arrangements that can accommodate peer feedback and evaluation preferably seats facing one another will be utilized. Notable guest speakers - including the Head of Research at Lyceum of the Philippines University, a speaker on Gen-Z’s unique learning styles, and a main keynote speaker on the value of integrating research into students’ assessments were in attendance for the first day. The second day of the workshop focused on getting peer feedback from other learners and where they shared best practices on different types of assessment techniques. Finally for the reflection period, all the learners assessed their learning, as evidenced by the SWOT analysis and reflection sheet, each learner got feedback and applied the comments and iterated their assessment techniques accordingly. 

Workshop Structure (See Appendix 3)

• Ice Breakers

• Guest Speaker Series 

• The Millennial and Gen-Z Learner (PPT, short video presentation) 

• Role of Research into Assessments (Small breakout groups of 2/3 pax of students)

• Brainstorming Session: How to Integrate Research into Assessments 

• Assessment SWOT Analysis (See Appendix 3)

• Peer Review of SWOT Analysis

• Reflection and Sharing (Q+A, sharing best practices) 

Assessment Approach 

a.     Pre-Workshop Assessment Survey 

·  Prior to the workshop, a needs training analysis survey was conducted. A baseline of skills was established from all participants who were required to take a pre-survey. This will be followed-up by a post-survey once the workshop is completed

·  This was done via Google Forms (it can also be conducted via any enterprise software available such as MS Surveys, Survey Monkey) 

· Preliminary results of the survey are to be discussed as a part of the introduction and rationale for workshop

b.    Post-Workshop Assessment Survey 

· Self-assessment surveys given to professors in attendance for self-reflection and to gauge efficacy of workshop and increase buy-in from learners 

c.     Students’ Course Syllabi Outcomes 

· Instructional designers and course developers create random check-ins to see if teachers have incorporated new and innovative measures of assessment into students’ learning outcomes

· Quality control through looking at the syllabi of the students of teachers attending the workshop  

d.    Students’ Evaluation Forms 

· Qualitative and quantitative data collected from students’ evaluations of professors’ assessment methodologies collected during both the midterm and the final period

e.     Random Classroom Check-ins 

· To check if the assessment methods are being applied 

· Evaluation coming from fellow learners (peer-evaluation component)

Key Instructional Design Decisions + Theories

Appendix 3 shows the line-up of workshop activities spread across all three days of the workshop. The icebreaker and the brainstorming activities are based off schema theory (Piaget, 1926). Schemas are mental representations of how the world works. A schema is a cognitive structure which organizes and stores pre-existing knowledge either through assimilation or accommodation (Piaget, 1926). Schemas allow us to categorize and remember information accordingly. After the 3-day workshop, the teachers will strengthen and accommodate new information about the role of research into assessments or assimilate pre-existing knowledge of research-based assessments.  

The SWOT assessment analysis (Appendix 2) is a guided activity, where each learner will get the opportunity to be peer-reviewed. This activity provides scaffolding where the learner is led until one reaches the Zone of Proximal Development (ZPD). Furthermore, the peer-reviewed component of the SWOT analysis is based on Vygotsky’s collaborative dialogue, where peers internalize feedback from their current cohort, and are therefore ably to apply the information to improve individual future performance (Vygotsky, 1978). The component of the workshop that allows learners to create their own assessments for students (preliminary, midterm, and final assessment) is based off the theory of constructionism (Dewey, 1938), which centers on the idea that learning is a process of creating meaning from experience. It is only by engaging in the world via experiences that mental structures can evolve. The process of creating unique assessments for students is also a way for learners to make tangible models of the world around them. 

Tracking User Engagement with Articulate Rise

xAPI: Using xAPI and Javascript, I was able to write xAPI statements and implement the code to track every time a user reached the midpoint of the Articulate Rise course. Using Visual Studio Script, I was able to track when a user would reach half-way through the course, which then prompts a Javascript code to execute.

• Midpoint Data: Shows how engaged a user is to the midpoint of the course.

Analyzing Data

Each time a learner interacts with the Articulate Rise course, the midpoint data gets sent out to the LRS. Using this xAPI tool, this allows me to track when users have interacted with my course and reached the halfway mark, which allows me to measure user engagement and learning.

Impact

By the end of the workshop, teachers (learners) understood and seamlessly applied assessment based-approaches into the classroom assessments.

Research shows that assessment type dictates the quality of instruction. At the end of the workshop, the teachers were able to:

✔️Come up with 3 novel assessment methods which incorporate critical thinking skills (one for the preliminary, one for midterm, and one for students’ final assessment period).

✔️Incorporate the new learning assessments into the current learning modality for the semester (as evidenced by their course site).

✔️ Work with instructional designers and course developers to evaluate the efficacy of the assessments and whether these assessments are meeting students’ learning needs.

References   

Dewey, J. (1938).  Experience and education.  New York: Simon & Schuster.

Piaget, J. (1926). The language and thought of the child. Harcourt, Brace. 

Vygotsky, L.S. (1978). Mind in society: The development of higher psychological process. London: Harvard University Press.