Scoring Alternate Versions of a Test | Scoring Alternate Versions of a Test | Scoring Multiple Choice Tests | TSQS | Academic Technologies

Scoring Alternate Versions of a Test

A popular practice in a number of departments is to create multiple forms of a test for purposes of administration in large (crowded) classes. This is usually done by reordering the items in the test. In order to analyse all forms of a test as a single set, mapping instructions must be provided so that the items can be aligned to correspond from one form to another before proceeding with the item analysis.

When the different forms of the test are submitted for processing in separate packages, each with their respective key sheet, use of the Reorder program may be requested which will combine the separate files into a single file after rearranging the items so that they are all in the same order from one form of the test to another. This approach is necessary in order to use the Itemanal program to perform the item analysis. The following steps are required in order to accomplish this process:

  • The answer sheets must be packaged in separate envelopes--one for each form.
  • A key must be provided for each form. It should be placed on top of the answer sheets in the respective envelope.
  • Each envelope should be clearly marked regarding the form # of its contents.
  • For forms 2 through n, we require a list of numbers which indicate the correspondence of the items in the alternate form to those in the first form, e.g.,

Form 1 Form 2 Form 3
1 1 10
2 3 9
3 5 8
4 7 7
5 9 6
6 2 5
7 4 4
8 6 3
9 8 2
10 10 1

 

This example indicates that item 1 from Form 1 appears as item 1 in Form 2 and as item 10 in Form 3 while item 5 on Form 1 appears as item 9 in Form 2 and item 6 in Form 3. After composing this table, please ensure that each item appears only once in each column. Note that Form # 1 must have the items numbered sequentially.

The item-mapping table illustrated above can also be used with the Examtab test analysis program. For this program, it is not necessary to segregate the different forms into different packages. Instead, the different forms may be intermingled provided that they each have their form number coded in the Special Codes section of the answer sheet. This same form number must also be coded in the same columns on the corresponding answer key. Using a form number which is more than a single digit with a different pattern of digits, e.g., 123, 444, 987, is recommended since this reduces the possibility of a miscoded form number being a valid number from a different form.  It is helpful to provide instructions to the students about which columns of the Special Codes to use: for a 3-digit form number, they could be instructed to use columns J, K and L on the U of A General Purpose Answer sheet.

When submitting a job for scoring with the intent of using Examtab to perform the test analysis, the number of keys specified on the Request for Service sheet should be equal to the number of versions of the test that are to be scored. This program produces its own copy of any Class Lists that have been requested. By default, it also puts one copy of the class list data into a file that is formatted as fixed fields. You may ask for a file that is comma-delimited which will give you a file that is suitable for input into programs such as Excel or eClass.

When Examtab is used, the only score that appears in the Class Lists is the raw (number correct) score computed from the appropriate key. The Reorder program, on the other hand, will allow subsequent generation of output with all the options available in the more general Class Lists program.

Even when the Examtab option is used, it is still possible to ask for Itemanal processing of the file resulting from reordering the items in the different versions of the test that is described above.  Note that an advantage of using the Examtab option is that alert messages will be generated when it appears that the wrong form number has been coded on the answer sheets.

Academic Technologies Sitemap

Classrooms

Book a Classroom
Classroom Listing

Computer Labs

Book a Lab
Lab Listing

Software

N/A

Test Scoring & Questionnaire Services
General Data Collection
Instructor Designed Questionnaire (IDQ) System
Scoring Multiple Choice Tests
Universal Student Ratings of Instruction (USRI)
Web Surveys
High Performance Computing

Shared Computing
Visualization