CAT Testing Programs
Operational CAT Tests and Testing Programs
CAT Programs and Tests Under Development
A number of testing programs have implemented CAT in the last several years. Following is a first list of these CAT testing programs. I will be adding further information about these programs as it becomes available, as well as links to further information. I will also add other operational CAT programs as I become aware of them. If you are involved in a testing program that has implemented CAT, please let me know about it so we can include information on it.
The Adaptive Matrices Test (AMT). This test, which is part of the Vienna Test System, is a non-verbal assessment of general intelligence based on deductive thinking, and is thus mainly independent of cultural and social backgrounds. It can be applied to persons aged 15 years and older. For further information see www.schuhfried.at/eng/wts/amt.htm
The AICPA -- the American Institute of Certified Public Accountants -- put into operation a computer-based testing program for the Uniform CPA Examination, which is used in licensing CPAs in the U.S. The CPA Exam uses a multi-stage model, which includes adaptive, content-balanced testlets. In this model, branching decisions are made after completion of each 20- to 25-item testlet. Three testlets are administered, followed by two non-adaptive accounting simulations. The model, though not providing optimal measurement, allows candidates to review their answers within testlets and simulations. Also, the model allows for substantial quality controls and ensures content balance before the exam is released into the field. For further information see www.cpa-exam.org
The ASCP -- American Society of Clinical Pathologists -- Board of Registry Certification Examinations. These examinations are designed to measure in specific skills for Medical Technology occupations, Medical Technicians, and a number of medical laboratory specializations.
For further information see www.ascp.org/bor/certification/procedures/scoring.asp
The ASVAB -- the Armed Services Vocational Aptitude Test Battery -- developed and used by the U. S. Department of Defense to screen and place potential recruits for the military services. The ASVAB is a multiple-ability test battery that measures the following vocationally relevant abilities: General Science, Word Knowledge, Arithmetic Reasoning, Paragraph Comprehension, Numerical Operations, Coding Speed, Auto and Shop Information, Mathematics Knowledge, Electronics Information, and Mechanical Comprehension.
For further information see www.usmilitary.com/placementtests.html and the book Computerized Adaptive Testing: From Inquiry to Operation, edited by William A. Sands, Brian K. Waters, and James R. McBride.
CAT of Written English for Spanish Speakers. The CAT research group at the Autonoma University of Madrid (Spain) has developed a CAT on written English for Spanish speakers, to be administered on demand by Internet. For further information (in Spanish): http://www.iic.uam.es/pdfs/eCatPDF.pdf
The BULATS (Business Language Testing Service) Computer Test is a collaborative venture between University of Cambridge ESOL Examinations, the Alliance Française, the Goethe-Institut, and the Universidad de Salamanca. It is a sophisticated CAT test of language ability in the world of work, allowing on-demand assessment in English, French, German, or Spanish. It uses an extensive range of different task types including multiple-choice questions, listening and reading exercises and cloze tests. Results are reported in CEF (Common European Framework) levels. On-screen instructions are available in all test languages, as well as in Italian and Portuguese. For an online demo see: http://www.bulats.org/about/demo.html. BULATS is used by a growing number of organzsations that rely on the language skills of their employees in order to operate effectively.
For more information about the test and its availability in different regions visit BULATS Computer test at: http://www.bulats.org/about/about_computer_test.html
CATE -- the Computerized Adaptive Test of English -- developed by English Language and Learning Support of the Information and Learning Resource Services at Middlesex University in the United Kingdom. This test is administered to all incoming students who are not native English speakers. For further information see www.ilrs.mdx.ac.uk/lang/ells/cate.htm.
Cito, the National Institute for Educational Measurement in The Netherlands has developed a number of CATs for use in educational settings. Currently operational CATs include tests for mathematics and arithmetic skills in adult education and secondary vocational education; tests for reading Dutch as a first language, and reading and listening to Dutch as a second language; and basic skills and knowledge in geography. Several CATs designed for ages 4 - 6 are scheduled for release in December 2003, including tests of basic arithmetical skills, such as ordering; basic verbal skills; and orientation in space and time. Several additional CATS are under development for release in 2004 or 2005: spelling ability and vocabulary for ages 6 - 12; and reading comprehension, arithmetic ability, and world orientation for ages 7 -12. For further information see CAT tests summary document (PDF file, 150 KB).
The COMPASS series of tests from ACT. COMPASS provides placement and diagnostic testing in mathematics, reading, and writing. The new COMPASS/ESL provides placement testing for English as a Second Language (ESL) students in Grammar/Usage, Reading, and Listening. For further information see
The GMAT -- The Graduate Management Admission Test -- of the Graduate Management Admission Council, administered by Educational Testing Service. For further information see
The GRE -- the Graduate Record Examination -- developed and implemented by Educational Testing Service. Both the Verbal and Quantitative sections of the GRE General Test use CAT.
For further information see www.gre.org/cbttest.html#howwork
The LPCAT -- Learning Potential CAT -- an adaptive testing system that measures learning potential by using a test-train-test paradigm with culture-free items. LPCAT was developed by Marie DeBeer at the University of South Africa. It is designed to assess learning potential independently of the examinee's cultural environment.
For further information see http://www.lpcat.co.za
MAP -- Measures of Academic Progress -- developed by the Northwest Evaluation Association. This integrated system of CATs measures student performance in math, reading, language development, and science for grades K through 12. The NWEA MAP system is currently used by approximately 900 school districts across the United States. For further information see www.nwea.org/Products/MAP.HTM
The Microsoft Certified Professional exams. Microsoft Corporation uses adaptive testing procedures in a number of the examinations it has developed to certify information technology professionals on their knowledge of Microsoft products and applications. For further information see www.microsoft.com/traincert/mcpexams/faq/procedures.asp
The NAPLEX -- the North American Pharmacist Licensure Examination -- developed by the National Association of Boards of Pharmacy (NABP). These examinations are used by the boards of pharmacy as part of their assessment of competence to practice pharmacy.
For further information see www.nabp.net
The NCLEX -- the National Council Licensure Examinations -- are developed by the National Council of State Boards of Nursing (NCSBN). There are two tests: one for Registered Nurses (RN) and one for Practical and Vocational Nurses (PN). For further information see
http://www.ncsbn.org/testing/psychometrics.aspThis Web address includes three PowerPoint presentations: How CAT works in the NCLEX exam, an example of a CAT in which the examinee passes the license test standard, and an example of a CAT in which the examinee fails.
PreVisor - Offers hundreds of different CATs for use in employment selection. Currently, the majority of the CATs focus on IT, software, and related skills. By the end of 2007, many additional "hard skills" CATs for other areas will be available. For more information, contact Michael Fetzer.
QualityMetric Inc. has developed several CATs based on application of the polytomous IRT generalized partial credit model. These scales, designed for use in health care applications, include the Headache Impact Test, the Osteoarthritis Impact Survey, and the Dynamic SF-36 Health Survey. These scales are available for delivery through the Internet at www.amIhealthy.com. Further information on the development of these scales is available from jbjorner@qualitymetric.com.
STAR Math, Reading, and Early Literacy tests from Renaissance Learning Corporation use 1-parameter logistic (Rasch) adaptive testing procedures for efficient testing in an elementary through high school environment. For further information see www.renlearn.com/lis/
CAT Programs and Tests Under DevelopmentA number of testing programs and tests are working toward the implementation of CAT. Following is a list of those programs of which I am currently aware that are doing the necessary research to implement CAT in the future. If you know of any other testing programs that are planning a conversion to CAT, please send me some details so that they can be listed here.
The NBME -- the National Board of Medical Examiners -- has for a number of years supported research and development concerning the conversion of their United States Medical Licensing Examination™ (USMLE™) to computer-based adaptive administration. Current research focuses on the application of branched adaptive testing approaches using the CAST (computer adaptive sequential testing) method to delivering this series of examinations. For further information see www.nbme.org
Government of Alberta, Canada. Castle Rock Research Corporation has been contracted by the Government of Alberta to provide on-line, computerized adaptive assessments (CAA) in grades 3 through 12 in English language arts, mathematics, science, and social studies. Assessments will consist of high-quality items that relate directly to Alberta curricula. Assessments will be available at both curriculum-specific units (e.g., patterns and relations, shape and space) and across the whole course curriculum. Teachers will be able to use the Alberta CAA system to assess student progress throughout the year (formative) as well as student performance at the end of a course (summative). Detailed reports, which will provide information down to the specific outcome level, can be used to inform instruction. Teachers will decide if and how to use the Alberta CAA system to complement other classroom assessment tools. It is anticipated that the first computerized adaptive assessments will be available in the fall of 2007. The system will be fully operational in the fall of 2008. For further information, see www.castlerockresearch.com/caa or contact "Kate Mckenzie" <kmckenzie@castlerockresearch.com>.