Carol Price from CTB-McGraw Hill
Presentation at NCDAE summit follow-up meeting held January 19, 2005 at the ATIA conference

Table of Contents

• Introduction
• Barriers
• Solutions
• Compatibility
• Research
• Practices
• Conclusion

Introduction

Good morning,

Though leaders for almost eighty years, CTB has led the industry and has made a difference in the development of standardized achievement tests for children and adults. With the demand for quick turn around of student scores, with the movement toward online assessment, and with the legal and ethical mandates to assess all students, CTB has embraced these challenges with the creation of national and state-specific online solutions.

This discussion will focus on five critical issues surrounding distance learning in terms of assessment of individuals with disabilities:

1. Barriers faced
2. Solutions to consider
3. Compatibility
4. Research, and
5. Practices that have promise for the future

Barriers

The greatest barriers we, in the assessment industry, face today are:

1. Creating assessments that incorporate the principals of universal design to afford accessibility to all;
2. Ensuring that technology infrastructure can provide access in multiple formats, various media, and is Section 508 compliant;
3. Developing item pools with quality items in sufficient quantity for online delivery; and
4. The lack of a common set of standards and a universal vocabulary that is ubiquitous across all states.

These barriers make it difficult for test publishers to respond to RFPs.

Solutions

We can make a difference through practical solutions such as education and training which are the very foundation upon which solutions are created. Education eliminates fear. Knowledge generates buy-in. When people are empowered with understanding, they get onboard and embrace change. Training has to occur with the internal staff across departments in test publishing companies, with state department staff at state education agencies, with administrators in districts, with teachers in schools, with students in classes, and with parents in homes.

Let’s look at possible solutions for online assessment:

1. In the development of paper-and-pencil assessments, CTB content specialists review items to ensure that assessments are bias free and incorporate the principals of universal design; however, online assessment causes us to rethink how we develop tests. Tests need to be developed with items that are created to truly measure the construct of the skill we are assessing and are not dependent upon the student having advanced computer skills. So it is important for us to be confident that we are measuring what students know and what they can do.
   
2. Technology infrastructure must be put in place for 508 compliance and this infrastructure must be designed to address IDEA and 504 compliance via a myriad of accommodations for students with visual, auditory, physical, cognitive, and multiple impairments that range in severity from mild/moderate to severe/profound.
   
3. Computerized assessments require large numbers of items in a pool or item bank. Issues related to item banks involve security breaches, copyright permissions, and adaptability of the item or passage to online presentation. New ways of addressing these concerns must be investigated.
   
   • For example: a student who has the accommodation “test read aloud” the use of a text reader may create the potential for a security breach if the text reader requires that the text is imported/exported or copied and pasted. Only a text reader that sits on top of the application and reads what is on the screen will provide 100% security.
     
   • Permissions for copyrights present two major challenges:
     • Some authors will grant permission to publish a passage, poem, or item on a paper-and-pencil test, but refuse permission if the piece is used in a computer-based assessment.
     • The cost of a passage, poem, or item can increase dramatically when the request is for online use.
     • One way test publishers address this issue is to have item writers create original work; however, there needs to be further study regarding the use of authentic text and permissionable passages.
       
   • Often states requesting online assessment merely want their paper-and-pencil test replicated online and the full potential of the computer is not being utilized. States need to think past the paper-and-pencil assessment and investigate audio taped speeches, video clips, computerized scenarios, and three dimensional graphics.
     
   • Online assessment creates a particular problem for students with low vision or who are blind. Test publishes use the services of Braille vendors to create Braille versions of their printed assessments. Online testing presents new challenges that we need to explore.
     
   • When items or passages are delivered in online assessments, the students may need to scroll to read the entire piece. This presents several issues:
     • Are we truly assessing the student’s ability to read and to comprehend the passage or are we assessing their ability to use the computer?
     • Formatting issues cause psychometric concerns when states want a paper-and-pencil assessment equated to an online assessment and when states want comparability studies of these assessments.

Compatibility

In terms of compatibility, whether we’re delivering computer-based tests, professional development, or instructional links, customer satisfaction is largely contingent upon the user’s equipment. Downloading information via a dial-up network is time consuming and frustrating. A broadband connection is better, but often these services are not available in some rural school districts.

Some schools have antiquated computers that do not have the capacity to handle the sophisticated programs that may be required for an online assessment. CTB is aware of this need and we are moving forward to provide readiness services to states, districts, and schools to enable them to deploy online assessments.

We make a difference by helping our clients assess their readiness and we assist them with the development of solutions that encompass PC readiness, network readiness, and security readiness.

Research

Research has to be addressed on two fronts:

1. As related to assessment, research to improve accessibility has to consider:
   • Comparability studies between online and paper-and-pencil assessments — these studies should be completed across states and longitudinal information should be collected.
     
   • Computer-based and computer-adaptive assessments should be investigated to determine the best and most appropriate methods of delivery.
     
2. We can make a difference by fostering the idea that compliance with Section 508 standards demands more research. 508 standards should be considered the roadmap – the big picture. However, the standards leave much open to individual interpretation. If we are going to make a difference, we need to complete research with the goal of creating specificity for each standard so test publishers and state departments agree on what components need to be released and in what order the roll outs should be accomplished. One suggestion is to divide the standards into categories and rank those categories in order of importance. Group the categories into Phases I, II, and III. Then focus on the delivery with states, policy makers, and test publishers to decide what will be included in each phase and when will it be released.

Practices

We an make a difference when we explore best practice and when we create an understanding that there are low-tech and high-tech solutions available to persons with disabilities to provide accommodations and accessibility. I’ve provided examples in this handout (Computer-Based Assistive Technology and Web-Based Accessibility); please turn to page 5. The first column lists the area of difficulty, the second column provides a low-tech solution and the last column provides a list of high-tech options. There are many products available that provide accessibility accommodations, but often they address only one type of disability or area of difficulty.

In this same handout and beginning on page 6, I’ve listed the disability, possible accessibility accommodations, and websites that provide one or more of these accommodations. Integrating the features of various products to maximize accommodations should be the ultimate goal.

Deciding what practices, instructional techniques, and educational products are designed with the elements of Universal Designed is a time-consuming task. I’ve located two websites that incorporate a juried review process to ensure that it is reflective of one or more of the nine principles of Universal Design for Instruction.

Principle Definitions

1. Equitable use - Instruction is designed to be useful to and accessible by people with diverse abilities. Provide the same means of use for all students; identical whenever possible, equivalent when not.
   
2. Flexibility in use - Instruction is designed to accommodate a wide range of individual abilities. Provide choice in methods of use.
   
3. Simple and intuitive - Instruction is designed in a straightforward and predictable manner, regardless of the student's experience, knowledge, language skills, or current concentration level. Eliminate unnecessary complexity.
   
4. Perceptible information - Instruction is designed so that necessary information is communicated effectively to the student, regardless of ambient conditions or the student's sensory abilities.
   
5. Tolerance for error - Instruction anticipates variation in individual student learning pace and prerequisite skills.
   
6. Low physical effort - Instruction is designed to minimize nonessential physical effort in order to allow maximum attention to learning.
   • Note: This principle does not apply when physical effort is integral to essential requirements of a course.
     
7. Size and space for approach and use - Instruction is designed with consideration for appropriate size and space for approach, reach, manipulations, and use regardless of a student's body size, posture, mobility, and communication needs.
   
8. A community of learners - The instructional environment promotes interaction and communication among students and between students and faculty.
   Instructional climate Instruction is designed to be welcoming and inclusive. High expectations are espoused for all students.
   

Product reviews and these principles are available these websites.

The University of Connecticut Center on Postsecondary Education and Disability website

http://www.facultyware.uconn.edu/home.htm

Another product review website is the Multimedia Educational Resource for Learning and Online Teaching.

http://www.merlot.org

Conclusion

Last week, eSchool News online offered seven recommendations for policy makers and school leaders:

• Strengthen ed-tech leadership at the state and local levels;
• Consider innovative budgeting;
• Improve teacher training;
• Support eLearning and virtual schooling initiatives;
• Encourage broadband access;
• Move toward digital content; and
• Integrate data systems.

We can make a difference by accepting that barriers exist and there are a myriad of possible solutions on the horizon. Test publishers and states need to stand united and work cooperatively to create accessible online solutions to meet the demands of NCLB requirements for accurate and speedy reporting of student progress.

Together we can make a difference.

Thank you.

Carol Price
CTB-McGraw Hill