Lately I’ve been struck with this phrase; “We value what we measure rather than measure what we value.” In a postsecondary culture of measurement and transparency, it is good for us to think about the ways in which our own institutional data help us measure those things that we value. For us at the National Center on Disability and Access to Education (NCDAE), something we value dearly is the education outcome for any student with a disability. However, it is unclear that these data are available at most postsecondary institutions.
Many readers will know that NCDAE is engaged in a US Department of Education (FIPSE-funded) project to work with institutions on system-wide web accessibility (i.e., Project GOALS). While the focus is on resources and tools to support institutional efforts, ultimately we would love to see that increases in accessibility have a positive impact on students’ graduation, or persistence in postsecondary settings.
I am learning, that the US Department of Education, and most states, requires each institution to report data on graduation rates, and rates of persistence in school for students across gender and racial lines, but it does not ask institutions to provide similar data on students with disabilities. What is collected and reported, through the Integrated Postsecondary Education Data System (IPEDS) is the percentage of all enrolled undergraduates each fall who are formally registered with the disability services office. That’s it. Data are collected on numbers of students with disabilities, but not on their outcomes. Now, to be fair, there may be some institutions that gather these data on their own, and track the outcomes of those with disabilities, but I am not aware of any who do. If you are one of them please let me know. Since outcome data is not required by the Feds, it is also not required or tracked by the regional accreditation commissions. In a different postsecondary data set, the National Survey of Student Engagement (NSSE) asks about disability services, but this survey is not designed to track postsecondary completion. Moreover it is a measure that is self-reported by individuals who volunteer to respond to the survey rather than one that is derived by the institution from all students.
The Governmental Accounting Office released a report in 2009 “Higher Education and Disability Education Needs a Coordinated Approach to Improve Its Assistance to Schools in Supporting Students.” In it, the authors reported that while the Department of Education provides assistance to postsecondary institutions on supporting students with disabilities through 3 separate offices (i.e., Office of Civil Rights; Office of Special Education and Rehabilitative Services; Office of Postsecondary Education), there is no mechanism for these offices to share information or coordinate their efforts. It is made more difficult by the fact that each “have different missions and priorities, focus on different clients, and provide different types of assistance to schools.” Certainly asking institutions to take their existing data on student persistence and completion and disaggregate it by disability status–in the same way they do with gender and race–should not be an onerous task. It represents one data need shared by many.
If we are to see the disability-related efforts of an institution improve outcomes for students with disabilities, we must have some data from which to start. For our Project GOALS, it will be impossible for us to test hypotheses at an institutional level; that increases in the real accessibility of web content have any effect on the important outcome – those felt by students. This is unfortunate.
I want to be clear that I am not wagging a digital finger to infer that since postsecondary institutions do not measure educational outcomes for students with disabilities they do not value those students. The point in this post is to share what I uncovered, and hopefully create a conversation about whether, in this instance, we are truly measuring that which we value. I welcome comments and discussions, please contact Cyndi Rowland.