Wednesday, 8 April 2020

How to transform F-2-F #exams into online exams #onlineAssignments #onlineExams #pedagogy

As #COVID19 seems to disrupt our teaching and learning for a longer period of time, I wrote up a document based on requests from colleagues to look into the delivery of online exams and assignments, and what the options are.
The document is entitled "Transforming face-to-face exams into online exams: considering proctoring tool security and creative pedagogical options".

The document is shared under Creative Commons Share Alike license which aligns with the EU directive on transparency and sharing as EIT InnoEnergy for which I work, has a supporting role to the EU and the EU promotes sharing to ensure mutual growth.

This document is an addition to a previous document I wrote with 10 fast tips to move from face-to-face learning to online learning (with a lot of English resources from the University of Cape Town; South Africa; Harvard business, USA; University of North-South Wales, Australia; and EU commission on education, ... as well as tools).

(Disclosure: I don’t get or have any benefits suggesting any of the tools or solutions in these documents).
The following topics are discussed:
  • Comparing online proctoring tools which can be used to ensure safe online exams and assignments (the term ‘safe’ means non-cheating here). Can we use proctoring tools for team exams, how secure are they, what are some of the options and prices?...
  • Limiting online exam costs by looking into the usefulness of using group or team exams and open book exams in a digital learning world (benefits, as all of us can transform some or more of our usual closed book exams (CBE) towards an open book exam (OBE).
  • Using best online exam practices without proctoring tools with audio/video 1on1 only and assignments (useful for those not having the financial resources, or with limited internet access).
The full document (11 pages) can be found here.

Thursday, 19 March 2020

Sharing #oralAssignments and #OnlineExam #bestPractices to limit cheating

Request for expertise sharing on #online #exams #covid19 pro-active planning until the end of this academic year and offering #BestPractices for #OnlineExams below. 

The first rumors are indicating that our international HigherEd students will not be requested to come to their guest universities to plan their exams for the end of the 2019-2020 academic year. 

I am trying to find a solid online learning tool that can be used, but in the meanwhile, I want to share best practices that are already used at our and other institutes. Feel free to add any ideas or measures I might have missed when listing our guidelines.

Best practices for organising online exams and online tests All of us working with international students scattered around the globe as they have rejoined their families in their country of origin, will probably be facing online examination needs. With this in mind, I am listing best practices and in a second stage I will be reviewing #EdTech tools that might come in handy if you have multiple students linking up remotely for their exams (we are preparing for 382 students which is a feasible number, yet demands a streamlined approach). I took my master’s in education (M.Ed.) exams remotely myself (thank you @AthabascaUniversity, so sharing those best practices with some additions below. Best practices using only camera and audio as technology: Preparing the exam Switch any written exam questions you might have to oral exam questions. These can include notes that need to be shared (ask contextualized questions, questions that show they understand the material yet can apply it to new contexts; e.g. ask short oral essay questions). Create original exam questions: i.e. questions are not available in educational textbooks (otherwise tech-savvy students will be able to find them in no time :D Choose an online meeting tool that offers recording options (think legal discussions, you need to be able to show why you gave the examination points you gave) and a tool that allows for lengthy recordings at that (no one wants their exam to suddenly stop). Choose a tool that enables sharing the screen (might come in useful for some short essays, designs, stats…). Prepare an informed consent document and send that to the student, so they know their exam will be recorded and stored at the admin server space for X time. If possible, indicate the amount of time set aside for the exam. Make a designated exam folder structured according to your admin. Additionally: you might want to send out a ‘code of conduct’ to the students, so they know what is expected of them. This is where the penalties might be discussed: what is considered cheating, what is the penalty for each stage of cheating… Once the exam starts Introduce the student to the fact that their online session will be recorded (GDPR) – check that the informed consent was signed and sent back to you. Start recording. Indicate the overall guidelines of the exam: open book, closed book, time available, number of questions (if relevant). The student must be made aware of what they can expect. Ask whether they understood what you have just said. Check identity: ask the examinee to show their passport and take a screenshot, save that screenshot as part of the examination administration. Ask them to show their desk, room, and that they need to be in view mid-torso with hands and keyboard visible. (you know why 😊 In case you choose to go with closed book examination: ask them to share their full screen (look at the tabs that are open!). Of course, there is a workaround if they are tech-savvy, which is why exam questions should preferably be open book, it allows them some freedom, yet they still need to really understand how they come to a solution. Only offer one question at the time. Feedback is important… but: depending on the number of questions you prepared, you might want to choose a different feedback strategy. If you have different questions for each student: give feedback as you see fit. If you want to reuse questions: limited feedback is preferable. As we all know, students quickly inform each other on which type of exam questions they got, what the answers or feedback was to what they gave, and what feedback they got. Feedback is given at the end Stop recording and make sure it is in the right folder. What cannot you address in case you work with audio/video tools only? Disabling the right-click button (copying and pasting options, so that students can quickly save questions). A reason to go to tailored questions per student, based on comprehension and creative thinking. Single function add-on tools Use the Respondus Lockdown browser or similar tool to ensure that students cannot look up answers, but yet again, you need to block students looking up answers https://web.respondus.com/he/lockdownbrowser/ A review of more designated tools such as ProctorU, ProctorExam, … will follow.

Wednesday, 4 March 2020

Free #Horizon2020 report out @educause good inspiration #learning #education

The Horizon 2020 report (58 pages) has been released by @educause on 2 March 2020 and it is an inspiring read for those of us looking at emerging learning designs and techniques. The report covers trends in the social, economical, political, technological and of course higher education realm and new in this report is a nice contextualization of all the different trends and technologies using visual supports. Educause is Northern America based, so most of the examples and projects they refer to are also North-America based.

This report is also more consciously covering multiple scenarios resulting from the interactions between all the different realms of society, which makes it a nuanced reflection of where learning can go in the near future. The report also links to additional reading and complementary material, e.g. articles on micro-credentials and experiential learning, [High on the higher ed agenda: alternative learning and ongoing increase of online education. High on the economic agenda: climate change and the green economy]

Download it now! Why, because it has tons of interesting links with a great synopsis for each subject. See below to get an idea of only a handful of information.

Emphasized learning technologies and practices this year:

  • Adaptive Learning Technologies 
  • AI/Machine Learning Education Applications 
  • Analytics for Student Success 
  • Elevation of Instructional Design, Learning Engineering, and UX Design in Pedagogy 
  • Open Educational Resources 
  • XR (AR/VR/MR/Haptic) Technologies 

Adaptive learning technologies are still hot news, as the search for effective personalized learning is still looking for practical outcomes. One of these listed in the report is the Alchemy tool by UBC (University of British Colombia, Vancouver, Canada). It is described as "Alchemy is a multi-featured online tool that supports teaching and learning in any circumstance that benefits from flexible, scalable, and feedback-rich learning alongside growing learning analytics capabilities." which basically shows where learning/teaching is moving towards in this ever more specialized-topic driven learning world.
Another adaptive example I like is from Deakin University, called the professional literacy suite, where I especially like their focus on digital skills in the first year. Which fits with the demand on data and AI savviness, communication skills.... teach those early on in higher ed curricula.

AI and Machine learning in practice still focus a lot on chatbots (which is basically turn a FAQ into feedback offered by bots). The most interesting option mentioned is the Responsible Operations positioning paper by the worldwide library cooperative (38 pages, great insights) that looks at how AI and ML are embedded in society, and how this changes all parts of society and which research agenda might address these challenges.
And the University of Oklahoma has set up PAIR (a global directory of AI projects in Higher education) ... nice!

The Analytics for Student Success are fairly similar, but the report on Ethics in Learning Analytics (16 pages) by the International Council for open and distance education is a good reference document to keep at close hand.

Elevation of learning design - pedagogy is always of interest to me. In a way, the learning design changes feel as small changes, but with big impact as a growing number of teachers and learning-related professionals are picking up digital learning tools and embedding them into their curriculum to address multiple learning challenges.
Carnegie Melon has an open learning toolbox called OpenSimon (part of the Simon initiative), a great spot to explore tools, techniques, research projects and so on (e.g. the Tetrad project which is an easy visualisation tool for data).

OER: we need more OER, but for those looking for new material, Mason OER metafinder is a great starting point.

XR technologies (extended reality) are increasingly on the rise as just-in-time workplace learning is higher on the agenda of our rapidly changing world. It builds upon prior realizations and needs to simulate emergency actions for students, e.g. augmented reality use for medical students by the University of Leiden, Netherlands.

It is a great read, good to get a fresh perspective on where we are all going.




Thursday, 16 January 2020

Free @eLearningGuild research on #generations in the #workplace @JaneBozarth

The inspiring eLearning Guild keeps disseminating great reports in relation to learning. One of their key authors is Jane Bozarth (= director of research @eLearningGuild), who is always an inspiration. You also know that a report will be of interest if she writes it.

In the eLearning Guild's latest report (19 pages) the focus is on Generations in the Workplace: how they see each other and why this is worth all of our attention. Interestingly, the report starts out with a clear framing on why all of us tend to have possible stereotypes confirmed when thinking about 'the other' in terms of age and what a person in a certain age group can do (see page 2 of the report). And the report concludes that little comparable research in terms of learning outcome is being done, diversity and inclusion of age groups is a concern, training might be affected by trainers encountering different attitudes towards their training, concepts that are being used as indicators (e.g. lifelong learning) to assume a strong will to learn might be based on different interpretations of those concepts by learners. The complete list of ideas and outcomes for L&D practitioners is pasted below. 

Basically, learners of all ages are more similar than different! So, erase any potential stereotypes and move on :) Get the report, it is free (if you register with the eLearning Guild, also free) and it is a really good read.

WHAT DOES THIS MEAN FOR L&D PRACTITIONERS?
It is difficult to find advice for L&D practitioners that is not rooted in often-unsubstantiated data from the popular press. A few takeaways from this review of the empirical literature base:
• It seems worth noting that while much research explores values and attitudes (Do people of different generations comply with rules? Do people of different generations like technology?), little of it compares outcomes. Is one generation more productive? Is one generation more prone to making errors? There is no evidence to suggest one person inherently performs better than another by dint of any grouping by birth year.
• Work has changed. More work is remote, more work is mobile, and people are becoming increasingly accustomed to finding quick answers and help. Workers need content that is available anytime, anywhere, and is accessible via any number of devices. The content may need to be less formal than it has been in the past. Workers may need help understanding how to access that content and how to use the devices.
• A number of researchers (among them Mencl & Lester, 2014; Urick et al. 2017; Woodward, 2015) tie concerns about generations in the workplace to the larger matter of diversity and inclusion. L&D workers involved in efforts in these areas may need to incorporate ideas about generations into that broader context.
• In their 2015 review of the literature, Woodward et al. found that younger generations placed greater emphasis on lifelong learning and personal development at work than older generations. Those in L&D might consider that similarities can take a number of forms. For instance, Mencl & Lester 2014 note that the Baby Boomer interested in “career advancement” and the Millennial interested in “lifelong learning” may be talking about very similar things.
• An interesting take comes from a 2018 Learning Solutions article by Joe Mayer, who reframes the conversation from “managing differences” to “avoiding generational bias”. Among his suggestions are using design thinking to take an empathetic view of the learner, to conduct extensive user-testing of your user base, to choose appropriate vehicles for content rather than try to accommodate some perceived generational preference for a particular medium or instructional approach, and to consider tenure rather than age.
• Finally, we should take care to check our own biases. In a lab experiment in which undergraduates taught a technical task (using Google chat) to learners of varying ages, researchers found that “ostensibly older trainees evoked negative expectancies when training for a technological task, which ultimately manifested in poorer training interactions and trainer evaluations of trainee performance” (McCausland, King, Bartholomew, Feyre, Ahmad & Finkelstein, 2015) p. 693. In short: Trainer stereotypes of learners based on age created a self-fulfilling prophecy. Older trainees received lower-quality training, which could ultimately affect job performance.
(p. 14, eLearning Guild, 2020)