Evaluating your Institution's Web Accessibility Efforts
Part 1: Evaluating the Process

Cyndi Rowland

Anyone who has ever been involved in system-wide change at a postsecondary institution is keenly aware that change takes time. In the case of web accessibility, we have learned that the period of this change can be 4 or more years (e.g., the Cal State System initially created a 4 year timeline to drive their transformation and then extended it). Because of lengthy timelines, it is vital that institutions include periodic assessments of their process. By process, we mean the efforts you make to achieve the goals and objectives, or milestones and activities, you have targeted as a way to improve accessibility. The assessment of your process should be done alongside assessments of the outcome, or product, of accessible web content (i.e., how many of our pages or courses are now accessible).

A focus on process-based assessment is necessary because of the interconnected nature of the work to be done. A delay or change in one focus area or part of your implementation plan can have a domino effect across the effort. Unfortunately it can result in delays achieving accessibility or less than optimal accessibility outcomes. For many, the notion of continuous improvement is not new yet they may not know how to apply it to broad accessibility efforts at their institution.

This resource will walk you through a strategy intended to be useful in a process-based assessment of your institutional efforts for web accessibility. The review itself is typically conducted by the institutions web accessibility committee (See our resource on the composition of this committee). For those of you who are using the GOALS Benchmarking and Planning Tool, this periodic evaluation of your process and outcome is reflected in Indicator 4, "Assessment".

Process assessment checklist:

The checklist below presents a 6-step plan for the process assessment. Following the checklist are detailed descriptions of each step.

Steps for a successful assessment:

1. Make sure you have a written plan for your accessibility efforts. If you are using the GOALS Benchmarking and Planning Tool, you will have your Action Plan you can use. Any written plan should include:
1. Goals and objectives (or benchmarks and milestones) to be obtained
2. Activities that lead directly to those goals and objectives
3. Persons who are responsible for the work
4. Timelines for completion
2. Select a timeframe to review accomplishments and challenges in your intended process (e.g., annually, every 6 months, or other).
3. Create a review document that captures the work you intended to complete by the end of the timeframe
4. Determine if you require any independent sources of data to be gathered to verify your progress. (Note- this will not be necessary for every goal or activity that is reviewed)
5. Engage in the review. This becomes your formal process-assessment.
6. Document the assessment. This documentation can take the form of an email or informal meeting minutes to the committee or it may take the form of a formal report to an administrator.

Steps in more detail

Written Plan

At a global level, GOALS staff and partners do believe there is a mechanism for process-based assessment that can be derived from the use of the Benchmarking and Planning Tool. With that said, many institutions may engage in benchmarking their efforts and adjusting their written plans in large cycles (e.g., every 2 years). So how does the committee attend to progress in shorter cycles without redoing the entire tool or engaging in a lengthy process?

Hopefully you have a written implementation plan that details the amount of work you anticipate you'll complete across your cycle (i.e., if you won't re-engage in benchmarking and planning for another two years, your action plan should incorporate activities that span the 2 year timeframe). Having this written plan allows you to revisit those goals that are to be completed in smaller increments. In some instances the written document is a comprehensive implementation plan and in other instances it will be a more specific action plan.

Timeframe for your assessments

Next, select your time span for these process-based reviews. For example, your committee may establish an informal review meeting in 6-month increments to specifically evaluate where you are in your process (i.e., what has been going on, what are the accomplishments, what are new challenges) and determine if any shifts in the plans are needed. Or they might determine that an annual review meeting is sufficient. For most of us, understanding that we'll sit with our peers and review our work during a specific period helps keep us on track with our intended activities.

Create Your Review Document

Given your written plan and selected timeframe for the review, you'll want to create a document that is customized to contain the selected elements that will be addressed during the review. This way you are not having to look at a 20 page action, or implementation, plan when the specific items that you want to review for your timeframe cover only 3 of those pages. Also, this is helpful as you make notes and changes during the review process. Creating a stand-alone document sets this review apart from other periodic reviews and may help you focus on completing the activities at hand.

Gather Data Helpful in the Review

Before the review meeting you will want to understand if there are any data sources you need to gather, either by yourself or from others. You will only need to gather data if it is important to document, or independently verify, completed work during the review meeting. Identifying these data sources early will help you secure the necessary information in time for the review meeting. Examples might include new policy documents, email communications verifying a new procurement process, new data on the accessibility of a sample of public-facing web pages or courses, the results of a survey of students and staff with disabilities, or data on accessibility of a sample of institutional pages. The committee chair may want to archive what you bring in so the institution's efforts can be captured and reflect effort over time. Don't forget that you'll want to identify those persons who will be responsible for bringing this information to the review meeting early in the process.

Review

Now it is time to engage in the review itself. For most institutions the review will take place as a typical meeting (e.g., synchronous either in person or through technology), however it could be constructed as an asynchronous event through a wiki or discussion list. As each item is discussed, pay attention to both what is, and is not, working and why. Discussing challenges in a group typically spurs creativity and new approaches. For any item where the strategy, timeline, or persons responsible has changed, make sure you document these changes. It goes without saying that this is also true for new items that are added to your list of milestones and activities. Expect that this will be a rich discussion that will benefit your accessibility work greatly. This would also be the perfect time to amend or update your overall implementation, or action, plan if you have changes as a result of the evaluation of your process.

Document your Review

The final step in the assessment of your institution's process is a written record. For some, minutes of this meeting in conjunction with documented changes in activities or timelines will suffice. For others, they may want to take this opportunity to write a summary for their Chancellor or Provost, or whoever is in an oversight position of this institution-wide effort. While it is tempting to not create a written record, it is vital. Please remember that as people change titles and roles, or leave the institution, the memory of the work may fade. As new individuals join the committee, this documentation serves as a summarized history of what has gone before. Collecting and documenting your progress over time has many benefits.

If there are other practices that you use in your own process-based assessments, please let us know. We consider our resources to be living documents and would be happy to share your success with others. If you have any feedback, questions, or suggestions on this resource sheet, please contact Cyndi Rowland.