Showing posts with label practical Edtech. Show all posts
Showing posts with label practical Edtech. Show all posts

Thursday, 19 March 2020

Sharing #oralAssignments and #OnlineExam #bestPractices to limit cheating

Request for expertise sharing on #online #exams #covid19 pro-active planning until the end of this academic year and offering #BestPractices for #OnlineExams below. 

The first rumors are indicating that our international HigherEd students will not be requested to come to their guest universities to plan their exams for the end of the 2019-2020 academic year. 

I am trying to find a solid online learning tool that can be used, but in the meanwhile, I want to share best practices that are already used at our and other institutes. Feel free to add any ideas or measures I might have missed when listing our guidelines.

Best practices for organising online exams and online tests All of us working with international students scattered around the globe as they have rejoined their families in their country of origin, will probably be facing online examination needs. With this in mind, I am listing best practices and in a second stage I will be reviewing #EdTech tools that might come in handy if you have multiple students linking up remotely for their exams (we are preparing for 382 students which is a feasible number, yet demands a streamlined approach). I took my master’s in education (M.Ed.) exams remotely myself (thank you @AthabascaUniversity, so sharing those best practices with some additions below. Best practices using only camera and audio as technology: Preparing the exam Switch any written exam questions you might have to oral exam questions. These can include notes that need to be shared (ask contextualized questions, questions that show they understand the material yet can apply it to new contexts; e.g. ask short oral essay questions). Create original exam questions: i.e. questions are not available in educational textbooks (otherwise tech-savvy students will be able to find them in no time :D Choose an online meeting tool that offers recording options (think legal discussions, you need to be able to show why you gave the examination points you gave) and a tool that allows for lengthy recordings at that (no one wants their exam to suddenly stop). Choose a tool that enables sharing the screen (might come in useful for some short essays, designs, stats…). Prepare an informed consent document and send that to the student, so they know their exam will be recorded and stored at the admin server space for X time. If possible, indicate the amount of time set aside for the exam. Make a designated exam folder structured according to your admin. Additionally: you might want to send out a ‘code of conduct’ to the students, so they know what is expected of them. This is where the penalties might be discussed: what is considered cheating, what is the penalty for each stage of cheating… Once the exam starts Introduce the student to the fact that their online session will be recorded (GDPR) – check that the informed consent was signed and sent back to you. Start recording. Indicate the overall guidelines of the exam: open book, closed book, time available, number of questions (if relevant). The student must be made aware of what they can expect. Ask whether they understood what you have just said. Check identity: ask the examinee to show their passport and take a screenshot, save that screenshot as part of the examination administration. Ask them to show their desk, room, and that they need to be in view mid-torso with hands and keyboard visible. (you know why 😊 In case you choose to go with closed book examination: ask them to share their full screen (look at the tabs that are open!). Of course, there is a workaround if they are tech-savvy, which is why exam questions should preferably be open book, it allows them some freedom, yet they still need to really understand how they come to a solution. Only offer one question at the time. Feedback is important… but: depending on the number of questions you prepared, you might want to choose a different feedback strategy. If you have different questions for each student: give feedback as you see fit. If you want to reuse questions: limited feedback is preferable. As we all know, students quickly inform each other on which type of exam questions they got, what the answers or feedback was to what they gave, and what feedback they got. Feedback is given at the end Stop recording and make sure it is in the right folder. What cannot you address in case you work with audio/video tools only? Disabling the right-click button (copying and pasting options, so that students can quickly save questions). A reason to go to tailored questions per student, based on comprehension and creative thinking. Single function add-on tools Use the Respondus Lockdown browser or similar tool to ensure that students cannot look up answers, but yet again, you need to block students looking up answers https://web.respondus.com/he/lockdownbrowser/ A review of more designated tools such as ProctorU, ProctorExam, … will follow.

Monday, 22 May 2017

Rubrics as part of online MOOC peer reviews #mooc #elearning

An addition to the EdTech options that I am currently organising. A rubric is a grading tool used within a course (blended or online courses) which is used to enable students as well as learners to understand what is expected of them in terms of solving an assignment or reviewing assignments from their peers.

Where - within the learning process - can a rubric be used?

Typically, as a teacher you will first introduce a case or project (generic example) that is exemplary for a specific process or project (for instance designing an online course overview). Each concept of interest is highlighted in detail. After explaining that particular example of a case, an alternative is given to deepen understanding. Then the learners are requested to build a similar case, yet adapted befitting their own context, infrastructure or conditions. By asking them to build a contextualised case, you bring the content and the assignment closer to their own previous knowledge. To offer guidance you provide a rubric, including the concepts you described in the detailed example. 

Brief orientation of the rubric: the rubric provided bellow can be used as an example rubric which can be adapted to align the conditions to the course topic. In this case the rubric is used to peer review online course overview. So the assignment include providing a course overview, including content and accompanying assignments consisting of several modules, with one module completely worked out in detail. The overview will provide an idea of the overall structure of the course, the detailed module gives an idea of signposting, descriptions, attained learning objectives. 

What is the benefit of using a rubric
A rubric has multiple purposes and can be used in different settings:
  •  It can be provided to learners prior to having to submit an assignment. That way they understand what the professor will be looking for, what the important criteria of the assignment are, and the rubric will offer a structured overview of how to strengthen a project, proposal or assignment prior to submitting it.
  • A rubric can also be used as a grading or reviewing tool between peers (e.g. learners). A rubric offers a more objective way to review each others work. In addition reviewing each others work will result in a more in-depth understanding of what the project/proposal can be and how your own project can be enhanced by looking at how your peers solve it, or design it.
  • By using a rubric the learners also get an idea of critically looking at other projects, and at the same time knowing the challenges that come along when writing a project based on specific criteria. Which is useful for future project work or collaborations with partners.

Using a rubric triggers deeper reflection in the learner on a specific tasks, as well as trigger additional actions concerning the task by integrating the criteria in a project or task. This leads to higher order thinking.

Example rubric
This example rubric is based on reviewing an online course project, but it can be adapted to any field using criteria that are relevant to that field and the requested project or proposal at hand.

The rubric below is made up out of four grading elements, you can increase or decrease them according to your own preference. In this case I choose to use an even amount of grading elements, as this pushes the learner to make a non-neutral choice, the feedback is either bad or good, not neutral. So you put the learner out of a comfort zone by not providing a neutral option, which would be an option when given three or five grades.

In general, once you have a criteria, you will be able to describe a good quality delivery of that criteria as that is typically what a teacher/professor would hope to get from a learner. From there you work your way back towards what you would consider to be a poor quality delivery of that particular criteria.


Grading criteria
Poor quality
Insufficient  quality
Sufficient quality
Good quality
Overall course structure
There is no coherent course structure.
An attempt is made to provide a course structure, but the course lacks descriptions, has no sign-posting to guide the learner through the course.
The course elements are structured, but not all course units are accompanied by descriptions and/or signposting. Leaving the learner to test those course units for themselves.
The course is well-structured providing clear descriptions and sign-posting throughout the course, enabling self-directed learning.
Online content in alignment with learning objectives
No learning objectives are given.
Learning objectives are given, but they seem to be disconnected from the content that is provided in the course, or they are not covered by the content of the course.
Learning objectives are given, but it is not always clear where the relevant content connected to these learning objectives can be found.
The learning objectives in the course are all clearly reached by the end of the course. The alignment of the learning objectives with the course content can be traced by looking  at the titles of the different course segments.
Course content engagement
The content is boring, lengthy and non-inspiring.
The course content consists of an amalgam of course elements that do not touch any challenges, nor do they inspire to integrate ideas coming from the content into my own context.
Parts of the content are engaging and inspiring. Some course units lack mentioning challenges and solutions, but they do provide informative background material.
The course is captivating, in an engaging way. It provides consize and meaningful content related to the subject matter, highlighting challenges and solutions related to each course unit.
Complexity of the learning path.
The course elements are provided chaotically, without enabling the learner to grow as they go through the different learning units.
An attempt is made to enable the learner to grow throughout the course, but too little stepping stones are provided in between the course units. The learner isn’t provided with enough background to assimilate new knowledge so they can move to the next course unit and understand what is covered there.
The course consists of logical steps, moving the learner towards more understanding by providing new information that supports basic knowledge creation. But some units lack additional, advanced learning material.  
The course evolves from simple concepts to complex combinations of concepts. Within each course unit there is also a consistent increase of content complexity.
Relevance and contextualisation of course assessments
Assignments are lacking.
Too little assessment is available to enable the learner to self-evaluate their own learning.
The assessments provide ample opportunity to see whether the content is understood. However, there are no contextualizable assessments or assignments provided. For example: no challenge or need based assignments.
Course assignments can be contextualised given the learner’s background or field expertise. Course assessments are varied and range from simple to complex. The course offers self-assessment options after each larger content segment covering a learning objective.
Content support through media use
Only one type of media is offered as content throughout the course.
The course integrates two different types of media (video and text), but the visuals add nothing to the story that is told. It could just as well be offered in writing. The video is of very low quality, you can hardly see what is recorded.

The course uses a mix of media, in accordance with the affordances of that particular media (e.g. discussion paper to increase debate, video of an actual engineering plant described in the course module).
Critical viewpoints provided and stimulated.
The content only shows the topic from one particular angle and is not critical.
The content is infrequently critically analysed by the content provider.
The content is enriched with critical arguments, both the challenges and the solutions.
Challenges and solutions related to the content are addressed from multiple angles. The learners are engaged to find additional viewpoints, or add critical content.

Wednesday, 10 May 2017

Online course & MOOC video scripting guidelines #MOOC #edtech

Below are some general guidelines when looking to produce an online course script that can be used to record video content (e.g. MOOC, blended learning,…). The guidelines are divided into two main sections: timing and narrative script. 

A good example of a meaningful, engaging video trailer containing images consistent to what can be expected in the full program can be found in the ‘Through the wormhole’ trailer:
https://www.youtube.com/watch?v=5IawpF062Ds

Timing
The timing of the video depends on the purpose of its use. Although there some general guidelines referring to preferred timing, you as a teacher must consider the  content of the video in order to consider the timing. But no matter how long (e.g. full simulations of a process) or short (e.g. teaser video, content element) the video is meant to be, a good narrative helps to create an engaging and captivating video (or audio, as a good narrative applies to both audio and visual media).
  • Shortest bit of video: the branding (logo). Keep this to an absolute minimum. Branding should happen in a singular instant.
  • Shortest bit of meaningful info: the course teaser: timing around 2 minutes, see further down this document for specific course teaser pointers.
  • Video content describing one coherent part of the course content (bite size: aim at 5 min, max 6 min). For a non-educational video, attention drops at 2 minutes, learner attention for educational videos drops quickly after 6  minutes (Guo, Kim & Rubin, 2013; Fishman, 2016)   
  • Longest meaningful video content: a full process description at a specific authentic location, explaining a full process demands time. But still, make sure to write a consize, brief script and be really weary of having more than 12 min of video length even for long concepts. Focusing only on what needs to be shared to understand that particular bit of content. (Tip: keep in mind your learners are watching a movie, if a scene in a movie takes too long or does not seem to lead anywhere, their mind starts wondering. If too many scenes are not goal-driven, they turn to another movie).

Overall, the length of a video should be kept to the minimum that is needed to describe or illustrate one coherent learning objective. And you must always consider, what does this video add to the content that is needed to understand that particular bit of content which is one learning objective of the full course.

Narrative script
Before building a script, think back and try to recall any professor or teacher you have had who gave inspiring lessons… and try to figure out why they were inspiring? Being inspiring enhances motivation.

General idea: know what you want to talk about, cut it into consize and meaningful pieces, see how you can build a story with the pieces you have (find a narrative to connect all the pieces together that captures attention).

Main action to keep in mind:
  • is this content relevant to the topic of the course?
  • does this multimedia segment align with the learning objectives/results you want your learners to achieve with this learning segment?
  • is the story of interest to is it explained as briefly yet meaningful as possible (length can vary, but being consize and to the point is necessary to keep the learners’ attention)?
  • if you want to reuse the material, it is important that the content is self-contained, and not referring to other parts (eg. avoid: “as we have seen in a previous video”, this saves time in the editing room).
Why create a narrative script? Because it gives you an overview of what you want to say (or what you want others to say), you can check whether all the elements of a particular subject area are actually covered, and it saves time when recording parts of a course when you are sure all the content is covered (revisions of recorded material can be costly and time consuming).
Remark: if you are writing a script for someone else’s course, make sure the review your script before going into production.

What makes a great lesson?
In many cases an inspiring teacher is both a real passionate expert in her or his area of expertise, and a great story teller. This means the content they provide is relevant, timed in accordance to the content that needs to be delivered in a specific time frame, and it is coherent. The content that is provided needs to be (and look like) high quality material (evidence-based, well structured, and be open to some sort of assessment).

Some pointers writing an engaging script
Get an idea of all the ‘actors’ in your course. Which elements are at the core of your course, how can you make them more interesting? How can you use these actors to build a course? (eg. Energy is the ‘good’ guy, while an earthquake can disrupt the energy equilibrium, in that case the earthquake can be the ‘bad’ guy, and as such become part of a narrative to make a specific part of the course more intriguing.
Start with an immediate hook. Capture the audience attention from the start. Start with a general idea that you know many of the students/professors wonder about (a good example can be seen here, through the wormhole trailer - https://www.youtube.com/watch?v=5IawpF062Ds  ). Integrate moments of reflection: provide very brief cliff hangers (what would you do in case X or Y happened? Offer options to trigger reflection, or offer challenges that learners can relate to in order to get their attention)
Create a learning journey and make it compelling. Provide narrative throughout the course: if the course has elements which flow one to the other, try and find a narrative that feels natural to these steps. Try to find an authentic learning context, related to actual cases in the field, and distill an overarching story from that: “All was well on the island of Eden, the energy that was used was stable and people used that source of energy for transportation, food production and general comfort. Then the unthinkable happened and due to a power breach, the island was cut off from its main energy supply. Unfortunately it was in the middle of winter so priorities needed to be set in order to safeguard the islands citizens…. And from there different case studies with solutions, or different approaches can be shared as content modules. Remark: sometimes this umbrella story that can embed content from all the modules given in the course, only starts to emerge once you see the modules of the course itself.
Keep it poignant. Although a narrative is built, all the content should be brief, consize and relevant. In a classroom you can elaborate, learners do not dare to leave an auditorium, but if a video is elaborating too much or not getting to the point, learners simply move to the next bit of information. Therefore, it is good to constantly ask yourself: which message do I want to put across, what is at the core of this content, and what do I want the learner to get out of this section. If you feel that more information needs to be provided, consider adding it as additional reading.
Use clear structure. A story needs to have a beginning (situating what is to be expected), a middle (where you gradually go deeper into the content, takes more time. Needs sign-posting in between modules for ease of learning), and an end (wrap up with conclusions, recapturing what is said and consequently should be learned).
Provide authentic scenes and situations. Are there elements in the course that can be used as visual extra’s? For instance, do not talk about lab work, but show a lab while explaining something. Are you particularly happy with a new invention, show the invention. Did you create a new model or design, show the design. .

Special video purposes
Teaser script
In case you want to create a course teaser script (short video that can be used for promotional purposes for your course), it is good to keep the following elements in mind:
The teaser should be truthful to the content that can be expected (mention the big lines, or big topics/modules). This means that the topics listed, should be part of the actual course once it starts. It also means that the teaser should be a representation of the other audiovisual media in the course (do not use high-impact simulations in your teaser video, if you are not planning to use them in your actual course).
Trigger interest. Remember that an idea that is not concluded, is an idea you keep in your mind for a longer period of time. So provide some open questions, or open ideas in the teaser… challenges or enigma’s that might trigger curiosity in the potential learners, yet will only be answered in the course.

Plant/laboratory/ad hoc meaningful location
Some locations provide additional meaning to a video. Adding a meaningful location can add to the authenticity of learning by using a real life location to describe particular content. If you talk about a particular part  in the process of a windmill energy park, it becomes more meaningful if you show that part in action while describing it.

Additional ad hoc tools enabling advanced learning options:
Consider using a 360 camera: a 360 camera enables the learner to look at something from all angles (360 degrees). For example: if you are recording a video on the topic of emergency health care in an ambulance, it is of use to provide a full angle perspective of who does what in the ambulance, and which instruments are available and used during the drive to the hospital. This means that a full view of the inside of an ambulance will provide more meaning and will allow the learner to zoom in and out to a point of interest to them, or that you can use a virtual 360 recording to train ambulance drivers (see short example here: https://www.youtube.com/watch?v=XhUi-jrrzgI).  Free MOOC on recording a 360 degrees virtual reality movie: https://www.udemy.com/cinematic-vr-crash-course-produce-virtual-reality-films/
These recordings integrate an additional tool is to add a virtual reality simulation that can be seen using virtual goggles of any kind to really get a feeling of what that particular plant looks like.

Consider making an augmented simulation based on a meaningful location, which can be manipulated by the learner (e.g. using an augmented simulation of that particular place that is triggered by a QR-code provided inside of a course which the learner can scan with their smartphone). An augmented reality simulation provides a virtual design or model of a specific object. Because it is virtual, you can look at it from different angles offering a fuller (3D) understanding of the object. A straightforward implementation of Augmented Reality for learning can be seen in this integration inside the Roman Forum: https://www.youtube.com/watch?v=gdzwDVR93-c or a well known example from Ikea: https://www.youtube.com/watch?v=xC6t2eEPkPc  . Another example is now looking at collaboration through augmented reality: https://medium.cinematicvr.org/enhancing-collaboration-with-virtual-reality-5e168f1548d2

References
Guo, P. J., Kim, J., & Rubin, R. (2014). How video production affects student engagement: An empirical study of mooc videos. In Proceedings of the first ACM conference on Learning@ scale conference (pp. 41-50). ACM.
Fishman, E. (2016). How long should your next video be? Wistia. Retrieved from https://wistia.com/blog/optimal-video-length