Assessment Recipe: Multiple Choice

See an overview of Getting Started with Short Duration Online Assessment.

In person arrangements

When this type of exam is sat in person students sit spaced apart in a large hall. Students have a printed question paper with rubric on the front. The question book contains a multitude of multiple response questions sometimes known as “bubble sheets”.  Questions are set out for each of which there may be one or more correct answers and additional distracters.  Students select out of the possible answer which answer/s they believe to be correct.

A further example is the computer based exams that have taken place in workstation rooms on the University campus.  These were run by the Exams Office and iSolutions.  Students used a dedicated exam launcher that was custom built for our workstation rooms, from where they could launch into a secure environment where they had only access to the test taking software.  Frequent backups were taken and invigilators patrolled the workstation room communicating with the Exams office and iSolutions using two way radios.  If a computer failed the invigilator would move the student to another computer where they could continue their exam.  This practice was the result of more than ten years of development and lesson learning between the Exams Office and iSolutions, you can read more about this topic here.

ExamStart is a tool used to secure computer-based, invigilated exams. It only works on University workstations within the University network. It can only be used by students who are physically attending exams on campus.

How to transfer this online

A new Final Assessment menu item has been added to most Blackboard courses. Please use this content area to add all content relating to your final assessment, or move your final assessments here if they are elsewhere. If you do not see a Final Assessment menu item, you can create one by following this guide:  How to add a Blackboard menu item.

Within this content area, add a content item explaining that this is where the assessment will be found on the day. This should include the rubric for the assessment (usually found on the front of the physical paper) so that students have a clear understanding of what the format will be. Include information such as who in the school students should contact with questions about the assessment or the paper, including technical questions*. Also, include a link to this Student Guide: Preparing for short duration online assessment.

*this is based on on this section of Section IV: General Information and Regulations of the University Calendar that states that “The nominee(s) of the Associate Dean (Education) must be available during the whole of the examination period in case of queries”, to keep the Short Duration Online Assessment process aligned with this exams process that it replaces, the nominee (usually a course instructor) should be available for student queries during their working hours.

Create a Blackboard test, set the creation options, then start building the test.  Once ready deploy the test to a content area using our recommended options.Exams will be scheduled by the Exams Office so you should match start and end dates/times for the assessment to match the exam schedule.

Recommended: Blackboard Test

These steps assume that the test has already been created but not yet deployed to a content area for students to access.

Create the Test

Create a Blackboard test, set the creation options, then start building the test.

When creating test questions and answers, use the Accessibility checker for each and follow its guidance to resolve any accessibility issues.

Deploy the Test

Once ready deploy the test to a content area using the below settings that are particularly relevant for a Short Duration Online Assessment.

Open test in new window: No – this reduces the possibility of students accidentally logging out of Blackboard in a different window and being locked out of the test

Make available to students: Yes – students will not be able to access the test unless this option is set.

Multiple Attempts: tick and set a two attempts.  If a student is locked out of their test they can start at least one more attempt before requiring technical support or raising an appeal.

Score attempts using:  set this to set to First Marked Attempt if more than one attempt is set as an option (above). Students should be told that the second attempt is only to be used if they had technical issues with the first attempt (not an opportunity to improve their score). They should contact the school office, or the course instructor, if they want the second attempt counted.

Force Completion: Do not tick.  If you tick this box, then once participants have begun the Test, they may not exit and continue working on it at a later date.  This means if students lose their internet connection or have a power outage while taking the test they will not be able to complete it.

Set timer: Do not set a timer.  Students who lose their connection will find that the timer continues even though they have lost the ability to continue their attempt.

Display after/until: tick the boxes and set the dates and times to correspond with dates and times that the assessment should be accessible to students.

Password: to avoid increased support requests, do not set a password.

Due Date: do not set a due date, use the display until settings to set when the test is no longer accessible.  We recommend this because if you allow exceptions for AER students they will not affect the due date, and the due date is not a setting which can be added to exceptions.

Include this Test in Grade Centre Score Calculations: Do not tick – this will prevent the score of the test affecting the running total in the Grade Centre, potentially giving students an indication of their score even if results are hidden.

Hide results for this test completely from the instructor and the Grade Centre:  Do not tick: ticking this box will mean the results will never be recorded, once students start taking this assessment this option can never be undone.

Show test results and feedback to students: untick everything.  In most circumstances for summative assessment, the marks should not be shown to students after they have completed the assessment.  This can be adjusted later to reveal feedback at a set date if desired.

Test presentation: One at a Time – this keeps the screen space small, and keeps a connection with Blackboard when students move to the next question.

Prohibit Backtracking: Do not tick.  If this is ticked students will not be able return to an earlier question.

Randomise Questions: Tick if you wish for the order of questions in the test to be presented randomly.  This affects the question order.  Note that other areas of randomisation are:

• When building a test question you may specify if possible answers (where applicable) should be presented in a random order.
• When building a test you can state that there should X number of questions chosen at random from a question pool. Multiple pools can be used.

After submitting these options, make sure to follow the steps in this guide to ensure that the results will not be visible to students.

Hide Blackboard Grade Centre column

It’s really important to ensure that students do not see scores before you want them to.  Follow the steps in this guide to ensure that the results will not be visible to students.

Online considerations

This assessment will not be proctored (invigilated) which means that it is effectively open book. Students have access to all of Google and the wider web. While you can set a timer and enforce the attempts are made only within a limited time this invites an array of issues if the student experiences a fault and is not able to continue within the time limit, this is why we do not recommend setting a time limit in online tests. This leaves only the time between the start date and the display until date.   This potentially means students spend the whole of the availability period working on the exam. Your questions should be structured to take this into consideration. Questions which hinge on them to remember facts are unlikely to be useful identifying strong students.

Risks and possible mitigation

In addition to the general risks outlined on the main Online Short Duration Assessments web page, the table below covers risks specific to this scenario along with suggested mitigations.

What could go wrong?	Possible Mitigation
Error found in question/answers after assessment made available.	Ensure that the questions, rubric and all settings have been thoroughly proof read and reviewed for errors by yourself and relevant colleagues in good time before the assessment is due to take place.Where a mistake is found after students have started the test please follow the appropriate AQSC Policy.
Student is disconnected and cannot continue assessment.	Note: This advice may change subject to AQSC policyAllow at least two attempts on the assessment when deploying the assessment.Do not select Force Completion. For details of these and other settings view this page.
An assessment has been scheduled to be available for the wrong time, or for too little time.	Scheduling can be adjusted when the test is deployed. If this needs to be done after the test you must refer to the AQSC Policy which will explain how respond to this should it arise.
Blackboard becomes unavailable during an assessment.	Allow at least two attempts on the assessment when deploying the assessment.Do not select Force Completion. For details of these and other settings view this page. If Blackboard does not become available again within the time window available for the student to complete the assessment it may be a question of an appeal to go via the School’s standard process.
A student sees a score or correct answer feedback when they should not.	Leave the “show test results and feedback to students” options empty in the Test Options. Set the ‘hide from students’ setting for the Grade Centre column and make sure the score is not included in the running total.
A student was still doing a test when the deadline was reached and did not submit on time, leaving their attempt marked “in progress”	You can submit a test on behalf of a student if required.

Marking and giving feedback

To replicate the exams experience it is expected that no feedback will be given to students for the submissions other than a final score which would be delivered via Banner Self Service and the “Marks Release” process.  More information will be placed here shortly.  In the meantime it is important to ensure that marks and feedback are hidden from students and ensure when deploying the test that no feedback options are ticked.