Bobby Program From The Center For Applied Special Technology Resources

The original Bobby was a free online tool, written by Josh Krieger and provided by the Centre for Applied Special Technology (CAST) used to validate websites for WAI and Section 508 compliance. Launched in 1995,[1] it became well known for the usage of the Bobby Approved icon that website authors could use to indicate they have successfully passed the Bobby online test. Development was coordinated by Chuck Hitchcock, CAST's Chief Officer of Policy and Technology, and further developed by David Clark and Michael Cooper.

  1. Bobby Program From The Center For Applied Special Technology Resources Job
  2. Bobby Program From The Center For Applied Special Technology Resources Inc

The CAST tool was officially closed on May 1, 2005. However, the Bobby name lived on in Watchfire Corporation's Watchfire Bobby program. Watchfire provided the same free service that CAST did with Bobby in their Watchfire WebXACT tool. Watchfire's offering, now part of an IBM suite described below, tests pages of web content for quality, accessibility and privacy issues.

See also visual impairment legal, 347, 349 resources and organizations for, 605 blogs, 541 Board of Education of Topeka, Brown v., 315 Bobby program for Web. Defined, 151 CDC (Centers for Disease Control and Prevention), 609 Center for Applied Special Technology Bobby program for Web page accessibility, 547.

  • Jun 04, 2017  Back to the Center for. Disability Resource Links. Bobby is a free service provided by the Center for Applied Special Technology to. NLS administers a free library program of. Additional Resources for Creating and Assessing. Bobby was created by the Center for Applied Special Technology. Disability Support Links. Bobby is a free service provided by the Center for Applied Special Technology.
  • CAST’s achievements include: Creation or co-development of software such as Thinking Reader, WiggleWorks, and Bobby; leadership of federally-funded initiatives to develop and promote a National Instructional Materials Accessibility Standard (NIMAS), which will guide the production and electronic distribution of curricular materials in accessible, student-ready versions, including Braille and Digital Talking Books; and provision of support for school teachers and administrators through.

The free tool was officially closed by the owners, IBM, on February 1, 2008.[2] The software is now available as part of IBM's Rational Policy Tester Accessibility Edition.[1][3]

Currently, the Web Accessibility Evaluation Tool (WAVE) provides this free service at wave.webaim.org.

See also[edit]

References[edit]

  1. ^ abCentre for Applied Special Technology (CAST), BobbyArchived May 1, 2008, at the Wayback Machine, cited 4 May 2008.
  2. ^Watchfire, Bobby and WebXact, cited 4 May 2008.
  3. ^IBM: Rational Policy Tester Accessibility EditionArchived October 20, 2009, at the Wayback Machine'. Accessed 2009-09-01.

External links[edit]


Retrieved from 'https://en.wikipedia.org/w/index.php?title=Bobby_(software)&oldid=853689775'
Published online 2004 Jun 21. doi: 10.2196/jmir.6.2.e19
PMID: 15249268
This article has been cited by other articles in PMC.

Abstract

Background

The World Wide Web (WWW) has become an increasingly essential resource for health information consumers. The ability to obtain accurate medical information online quickly, conveniently and privately provides health consumers with the opportunity to make informed decisions and participate actively in their personal care. Little is known, however, about whether the content of this online health information is equally accessible to people with disabilities who must rely on special devices or technologies to process online information due to their visual, hearing, mobility, or cognitive limitations.

Objective

To construct a framework for an automated Web accessibility evaluation; to evaluate the state of accessibility of consumer health information Web sites; and to investigate the possible relationships between accessibility and other features of the Web sites, including function, popularity and importance.

Methods

We carried out a cross-sectional study of the state of accessibility of health information Web sites to people with disabilities. We selected 108 consumer health information Web sites from the directory service of a Web search engine. A measurement framework was constructed to automatically measure the level of Web Accessibility Barriers (WAB) of Web sites following Web accessibility specifications. We investigated whether there was a difference between WAB scores across various functional categories of the Web sites, and also evaluated the correlation between the WAB and Alexa traffic rank and Google Page Rank of the Web sites.

Results

We found that none of the Web sites we looked at are completely accessible to people with disabilities, i.e., there were no sites that had no violation of Web accessibility rules. However, governmental and educational health information Web sites do exhibit better Web accessibility than the other categories of Web sites (P < 0.001). We also found that the correlation between the WAB score and the popularity of a Web site is statistically significant (r = 0.28, P < 0.05), although there is no correlation between the WAB score and the importance of the Web sites (r = 0.15, P = 0.111).

Conclusions

Evaluation of health information Web sites shows that no Web site scrupulously abides by Web accessibility specifications, even for entities mandated under relevant laws and regulations. Government and education Web sites show better performance than Web sites among other categories. Accessibility of a Web site may have a positive impact on its popularity in general. However, the Web accessibility of a Web site may not have a significant relationship with its importance on the Web.

Keywords: People With Disabilities, World Wide Web, Internet, Health Services Accessibility

Introduction

The World Wide Web (WWW) has become an increasingly essential resource for health information consumers. One recent study estimated that 73 million US residents searched for health information online during the year 2002 [1]. The investigators estimated that seventy-odd percent of the online population search for health-related information for their decision-making [1]. Eysenbach and Kohler [] estimated that approximately 4.5% of all search queries submitted to Web search engines are health related, which is equivalent to a global minimum of 6.75 million health-related searches on the Web every day. With the advances of computer and Internet technology, the distribution of the online population is becoming representative of the general population in terms of demographic and socioeconomic status [3].

The ability to obtain accurate medical information online quickly, conveniently, and privately provides health consumers with the opportunity to make informed decisions and participate actively in their personal care []. Little is known, however, about whether this online information is equally accessible to people with disabilities who must rely on special devices or technologies to process online information due to their visual, hearing, mobility, or cognitive limitations.

The latest report on Internet use from the National Telecommunication and Information Administration (NTIA) demonstrated that people of all ages, races, and ethnicities, including people with disabilities, are moving more and more of their activities online [3]. A recent investigation on Internet use by people with disabilities reported that people without disabilities are four times more likely (38.1%) to use the Internet than are people with disabilities (9.9%) [5]. Similar patterns remain even when factors, such as income, gender and educational attainment, are taken into account [5]. The large disparity in Internet usage may be attributable to problems with the accessibility of Web content [5]. Nielsen (2001) reported that the usability of the Web is about three times better for users without disabilities than it is for users with disabilities [6].

Bobby Program From The Center For Applied Special Technology Resources Job

For people with disabilities, the Web is very often the only source of information that they may access without having to depend unduly on others. Equivalent Internet access to health information will open a door to people with disabilities by offering them exciting possibilities for independent living and community participation [7]. People with disabilities can find a wealth of information on the Internet that addresses many issues of special concern to them, including chronic disease information and rehabilitation and assistive technology services [8]. According to a recent report, people with disabilities tend to seek health related information online more frequently than the able-bodied population [9]. Nevertheless, for health information Web sites to be of real use to people with disabilities, they must first be accessible to them. Health information Web sites are a classic example of the 'inverse information law': access to appropriate information is particularly difficult for those who need it most [].

Background and Prior Work

Web content accessibility helps people with disabilities access Web pages directly or use assistive technologies. Many people with disabilities have to rely on specialized software or hardware to access the Web. For example, people who are visually impaired have to install a software package called a screen reader to read all the content on the Web page aloud to them. Some people who are blind also use a talking browser like IBM Home Page Reader to access the Web page aurally. Some people who are blind prefer a hardware-level solution like the computer-controlled Braille embosser to help them perceive content of the Web page haptically. Regardless of the solution favored by the users with disabilities, if the content of the Web page is not available to their remaining sensory channel, then the page is not accessible to them.

The Web inadvertently has become increasingly inaccessible to people with disabilities as it adopts numerous emerging multimedia technologies. The Web at its beginning was designed for sharing and accessing documents across different computer systems and platforms. These documents are primarily text-based and mostly accessible to assistive technology, such as screen readers. With the introduction of appealing multimedia content, however, the Web is becoming an information medium that is not accessible to or not easily interpreted by assistive technology. Graphics, animations, and even video/audio clips, now commonly appear on the Web. The absence of alternative information about multimedia content makes them less accessible to people with disabilities than those with multimodal access to the multimedia content. The rapid expansion of e-commerce also makes the Web even more complicated and less accessible for people with disabilities. As Herbert A. Simon [10] once stated, 'What information consumes is rather obvious: it consumes the attention of its recipients. Hence a wealth of information creates a poverty of attention, and a need to allocate that attention efficiently among the overabundance of information sources that might consume it.' Web page developers believe that multimedia content could lure more visitors to the Web site and make them stay longer. However, they may overlook or ignore the accessibility for people with disabilities to that multimedia content because its primary purpose is to draw attention from potential consumers, the majority of whom are not people with disabilities.

Realizing this dilemma, the World Wide Web Consortium (W3C), the international organization that oversees the standardization and operation of the Web, announced the establishment of the Web Accessibility Initiative (WAI) on April 7, 1997 [11]. Supported by all W3C members, including such heavyweight stakeholders as Microsoft and IBM, the WAI plays a central role in promoting and correcting the functionality of the Web for people with disabilities. The first major responsibility of the WAI was to formalize guidelines for Web content developers and designers. WAI introduced Web Content Accessibility Guidelines (WCAG) to the public as a draft in 1998, and developed it into a full recommendation in 1999 [12]. WAI expanded the guidelines to be applicable in the design of user agents (e.g., Web browsers or assistive technology agents like the screen reader JAWS for Windows), authoring tools (e.g., Microsoft FrontPage or Macromedia Dreamweaver) and related techniques, and a practical checklist [13,14].

There are two basic themes reflected in the WCAG: ensuring graceful transformation of Web pages, and making content understandable and navigable. By providing Web pages that transform gracefully, people with disabilities or users with device limitations will be able to access them without constraints. Keys to graceful transformation include separating structure from presentation, providing text equivalents to non-textual elements, creating documents that work even if the user cannot see and/or hear, and creating device-neutral documents. When the content is understandable and navigable, end users can utilize the page in a more effective, efficient and satisfactory manner. Keys for making content understandable and navigable include providing a navigating context and orienting information, providing a clear navigation mechanism, and ensuring succinct content descriptions.

Another initiative in the development of accessibility standards is Section 508, conducted by the US Access Board [15]. The Access Board issued standards for accessible information technology under the Reauthorized Rehabilitation Act. These amendments strengthen Section 508 of the Rehabilitation Act of 1973. It mandates that when federal agencies develop, procure, maintain, or use electronic and information technology, they shall ensure that the electronic and information technology will allow federal employees with disabilities access to and use of the same information and data as that accessed and used by federal employees who are not individuals with disabilities, unless an undue burden would be imposed on the agency. Section 508 also mandates that agencies ensure equal access to individuals with disabilities who are members of the public seeking information on data that are comparable to that provided to those who are not individuals with disabilities, unless undue burden would be imposed on the agency. Section 508 clearly defines the accessibility for people with disabilities for federal government Web sites. Section 508 took effect on February 20, 2001.

Many software packages have been developed and commercialized to help Web developers evaluate the accessibility of their Web sites to people with disabilities [16]. These packages can scan Web pages, list computer detectable violations of Web accessibility standards, and give warnings for suspicious HTML snippets. Some tools integrate themselves into Web site developing or quality control programs to assist Web developers in quickly eliminating the inaccessible parts. Bobby, one of the earliest and most well known packages for checking Web accessibility, was used in our study.

Researchers from different disciplines have evaluated Web accessibility and usability of Web sites in various domains. The Journal Library Hi Tech published two special issues dedicated to Web content accessibility of Web-based information resources for people with disabilities [17,18]. Axel Schmetzke [19] maintains a Web accessibility survey site that aspires to be a clearinghouse for studies involving the collection of accessibility data pertaining to Web sites and online resources in education. The site listed many Web accessibility evaluation studies on libraries and higher education Web sites. Another related effort is the Web Usability Index (WUI), a free Web usability statistics database provided by UsableNet [20]. It employs an automatic Web usability evaluation tool for testing Web accessibility and obtains daily statistics of Web usability of sample Web sites from the Internet. According to WUI, only about 43% of current Web sites provide excellent or good Web usability design.

Although the Web is considered a powerful force for reshaping the healthcare infrastructure, the accessibility of Web content to people with disabilities is not a primary consideration for most designers of Web sites providing health related information [21]. Very few research studies have been conducted on the accessibility of health information Web sites for people with disabilities. Research studies on the accessibility of health information Web sites are, for the most part, about the find-ability and search-ability of Internet Web sites by online search engines or about the availability of information technology for the people who need it -]. Previous guidelines related to the quality of health information Web sites failed to emphasize the accessibility of Web sites by people with disabilities [] until the National Cancer Institute (NCI) published research-based guidelines addressing Web usability [28]. Chapter 3 of the NCI report is specifically dedicated to the issue of Web accessibility for persons with disabilities although the rest of the guidelines can also benefit general Web users.

The only study known to us that covers health information Web sites was the study conducted by Joel Davis in 2002 []. Davis explored the extent to which Internet-based health information is accessible to visually impaired individuals who rely on automated screen readers. Davis selected 500 individual Web sites representing 50 common illnesses and conditions for evaluation. The study found that accessibility is currently very low-only 19% of the examined sites' home pages were accessible. It also found that the reason for the inaccessibility of the Web pages was noncompliance with the recommended design and coding changes.

Our study will be different from other studies in several ways: first, the study will check the degree of accessibility not only of home pages (main pages) of health information Web sites, but also of other Web pages within certain levels below the home pages. Second, the majority of other studies report the state of accessibility in terms of the absolute number of violations of accessibility checkpoints. Although absolute numbers of violations of Web content accessibility provide useful information about the state of accessibility, it is not straightforward for direct comparison of general accessibility between Web sites, and it does not include the complexity of the webpage into the evaluation. Third, we will investigate the relationship between Web accessibility and other features of a Web site including function, popularity and importance.

Research Questions

The overall objective of the study was to evaluate the accessibility of consumer health information Web sites for people with disabilities. We were interested in the following specific research questions:

  1. What is the current level of accessibility for consumer health information Web sites?

    We were interested in using automated computer programs to evaluate the current state of content accessibility of Web sites providing health information to consumers. The checkpoints used in the program were derived from Web accessibility specifications -- WCAG 1.0 and Section 508.

  2. What is the relationship between Web accessibility and the functional category of the Web site?

    We were interested in determining the distribution of the level of accessibility among these Web sites after we categorized them into functional groups. We expected government and education Web sites to provide information that is more accessible to consumers than other types of Web sites because of the existing specifications and initiatives.

  3. What is the relationship between Web accessibility and the popularity of the Web site?

    The hypothesis for this research question is that there is a positive correlation between the degree of Web accessibility and the popularity of the Web sites. The variable representing popularity of a Web site was determined by its visiting traffic.

  4. What is the relationship between Web accessibility and the importance of the Web site?

We wanted to investigate whether there is any correlation between the level of Web accessibility and the importance of the Web site. We expected to find that more important Web sites would be more accessible to people with disabilities. The variable representing the importance of a Web site was determined by the page importance ranking data provided by a Web search engine.

Materials and Methods

Design

The study is a cross-sectional study concentrating on the degree of accessibility of Web sites providing consumer health information. We used established Web accessibility specifications as the sources for constructing the measurement framework. Additionally, we investigated the relationship between Web accessibility and other features including function, popularity, and importance.

Materials

An individual Web site providing consumer health information is the unit of analysis in the study. Because the exact number and distribution of Web sites are not pre-determinable due to the tremendous size and rapid growth of the Web, probability based sampling methods, such as random or stratified sampling, are not applicable in the study. An alternative sampling approach widely adopted by researchers conducting studies on Web sites is to use search engines or online Web site directories.

We acquired a list of consumer health information Web sites from the directory service of the Google search engine (See Appendix A). Google's directory service obtained data from the Open Directory Project, the largest, most comprehensive human-edited directory of the Web [30]. We included all Web sites under the subdirectory 'Health/Resources/Consumer' as our candidate Web sites for evaluation. These are health information Web sites for the public, and their content are not necessarily specific to issues related to disability. We excluded ones that had their content changed to non-health related areas or were continuously unavailable during our study period after we reviewed the home page of each Web site.

After selecting the sample Web sites, we needed to establish a limit to the scope of the Web pages to be included within each site. Because WCAG only applies to Web pages, other content formats such as PDF (Portable Digital Format) files were not considered. However, server side scripting such as Active Server Page (ASP), or JavaServer Page (JSP) is able to dynamically produce HTML-based code at the client side, therefore we took these types of pages into consideration. Second, we needed to determine the number of Web pages from each Web site to be included in the analysis. Due to the large number of Web pages in some Web sites, it was not feasible to include all the pages into the study. We selected only the first two layers from the home page within a domain of a Web site in our sample. We hypothesized that the first two layers would be the most visited and would reflect the overall accessibility of the Web site for the study. The other reason for choosing only the first two layers was that Bobby version 4.01 has the ability to only process a limited number of pages on a given Web site because it consumes a large amount of computer memory during the analysis. When we selected three layers from the home page, Bobby encountered an 'out of memory' error when analyzing large Web sites using a Pentium 2.4Ghz desktop computer with 1Gb memory.

Measurements

Web Content Accessibility

One of the objectives of the study is to construct a measurement framework to assess the accessibility of consumer health information Web sites. As we discussed in the background section, two major specifications served as the normative guidelines for Web content accessibility design. The first-the W3C Web Content Accessibility Guideline 1.0 (WCAG)-is a stable international specification developed through a voluntary industry consensus. The US Access Board published the second specification-Electronic and Information Technology Accessibility Standards-in December 2000, pursuant to the US rulemaking process as required by Section 508 of the Rehabilitation Act Amendments of 1998 [31]. Both specifications offer checklists or rules that Web developers should follow with regard to content accessibility for people with disabilities. These two specifications largely overlap; only three of the checkpoints defined in Section 508 are not mentioned in the WCAG guideline 1.0. WCAG is more comprehensive than Section 508 on checkpoints of Web content accessibility, and it provides a priority level to each checkpoint to reflect severity of violations. Therefore, WCAG was used as the foundation for the accessibility metrics we developed.

The number of violations of each checkpoint is a component of our scoring method called the Web Accessibility Barrier (WAB) score. For example, a Web page with fewer accessibility checkpoint violations, e.g., providing an alternative description for an image object, would be considered to present fewer barriers for people with disabilities and will have a lower WAB score.

Because we are interested in automated evaluation of the degree of accessibility of a Web site, the subset of Web accessibility checkpoints demanding manual checking are not included in the calculation of the WAB score. For example, compliance to the rule 'If you use color to convey information, make sure the information is also represented another way,' cannot be verified until a manual check is done. For a list of Web accessibility rules that need to be manually checked, please see the WAI references [32].

WCAG attaches a three-point priority level to each checkpoint from its impact on Web accessibility. Priority 1 checkpoints mandate the largest level of compliance while Priority 3 checkpoints are optional for Web content developers. In weighting the calculation of the WAB score, we used the priority level in reverse order. The weighting factor for Priority 1 violations is 3, for Priority 2 violations is 2, and for Priority 3 violations is 1.

Using only the number of violations of Web accessibility checkpoints, however, may bias the results of the measurement. For example, a Web page with five 'image without alternative text' violations may have 500 image objects embedded in the page and the Web page with one 'image without alternative text' violation may have only one image object in the page. The developer of the first page may have already paid a great deal of attention to and put great effort into complying with the Web accessibility specifications while the developer of the second page may be completely unaware of accessibility. Therefore, the number of true violations of a checkpoint must be normalized against the number of potential violations of the checkpoint. In the last example, true violations are the image objects without alternative text, and the potential violations include all image objects on the page. Whenever a Web developer puts an image element into a Web page, he increases the potential that there could be a violation of the 'alternative text' checkpoint. Table 1 explains the selection of potential violations from HTML code. The average WAB score of all Web pages within a Web site will be the WAB score of the Web site.

Table 1

Checkpoints and the Determinant of the Number of Potential Violations

WAI PriorityCheckpointDetermining the number of potential violations
1Provide alternative text for all images.All <img> elements
1Provide alternative text for each APPLET.All <applet> elements
1Provide alternative content for each OBJECT.All <object> elements
1Provide alternative text for all image-type buttons in forms.All <input type='image' …> elements
1Provide alternative text for all image map hot-spots (AREAs).All <area> elements
1Each FRAME must reference an HTML file.All <frame> elements
1Give each frame a title.All <frame> element
2Use a public text identifier in a DOCTYPE statement.1*
2Use relative sizing and positioning (% values) rather than absolute (pixels).All <table>, <th>, <td>, and <frame> elements
2Nest headings properly.All heading elements
2Provide a NOFRAMES section when using FRAMEs.All <frameset> element
2Avoid blinking text created with the BLINK element.Same as the number of true violations#
2Avoid scrolling text created with the MARQUEE element.Same as the number of true violations#
2Do not cause a page to refresh automatically.1*
2Do not cause a page to redirect to a new URL.1*
2Make sure event handlers do not require use of a mouse.Number of event handler for both keyboard and mouse
2Explicitly associate form controls and their labels with the LABEL element.Number of form elements such as <input>, <select>, and <textarea>
2Create link phrases that make sense when read out of context.Number of <a> elements
2Do not use the same link phrase more than once when the links point to different URLs.Number of <a> elements
2Include a document TITLE.1*
3Client-side image map contains a link not presented elsewhere on the page.Number of <area> elements
3Identify the language of the text.1*
3Provide a summary for tables.Number of <table> elements
3Include default, place-holding characters in edit boxes and text areas.Number of <input type = 'text'>, <text area>, and <select> elements
3Separate adjacent links with more than white space.Number of links

* This feature is determined at the entire page level. Therefore, we assign 1 to the number of potential violations.

# The number of potential violations of this feature was not able to be determined. Therefore, we used the same number of the true violations as the number of potential violations. The frequency of the violations is simply 0 or 1 according to the formula of Web Accessibility Barrier (WAB) score.

Figure 1 summarizes the calculation of the WAB score of a Web site as a formula. A higher score means there are more accessibility barriers on the site, while a lower score indicates fewer barriers. A score of zero denotes that the Web site does not violate any Web accessibility guidelines and should have no automatically detectable accessibility barriers to people with disabilities.

Formula for Calculating the Web Accessibility Barrier (WAB) Score

We employed several program tools to examine the true and potential violations of the Web pages. Bobby is a checking program that can examine a Web page and report violations of Web accessibility checkpoints [33]. It is the most widely used accessibility checking software package and has been around longest. Bobby was originally developed by the Center for Applied Special Technology [34], and is now maintained and distributed by Watchfire Corporation [35].

Bobby desktop version 4.0.1 was used in this study. The desktop version can check compliance with WCAG of an entire Web site or only certain layers from the main page of the Web site. The version 4.0.1 can check non-compliance issues with both WAI and Section 508 checkpoints. After checking a Web site, Bobby generates a report in eXtensible Markup Language (XML) format that can be further processed to extract data about true violations.

Bobby implements 91 distinct testing rules, each of which maps onto a specific WCAG checkpoint. The Bobby tests are classified into a number of different 'checking' categories, as follows: (1) Full: Bobby automatically checks this rule and decides whether there is an error. (2) Partial: Bobby automatically performs some checking of the rule, but cannot decide the existence of violations. Instead, the line number is used as a warning to the testers. (3) Partial Once: Similar to the Partial category, but the warning is not specific to an individual line. (4) Ask Once: Bobby does not have a mechanism to check the rule, so the rule is presented as a reminder to the testers.

For all categories other than Full, a human tester must manually evaluate the Web site further to determine the WCAG compliance, which is not viable for a large scale Web site study like this one. We used only the 25 rules that Bobby implements with Full checking capacity for our evaluation. Even for the rules with 'Full' checking capacity, we still could not determine the quality of the compliance with WCAG. For example, the Web page developer could simply put the file name of the image into the 'alt' attribute of the <IMG> element to avoid a flag from Bobby. The quality of such compliance is much less acceptable than providing detailed description in the 'ALT' attribute.

The data of corresponding potential violations for each checkpoint can be extracted using a Web crawler program, which is an automated program that follows hyperlinks to visit Web pages. We developed a lightweight Java-based Web crawler program to access Web pages at remote Web sites and determine the number of potential violations of Web accessibility checkpoints. We did not use the built-in Web crawler in Bobby because it cannot be customized to check potential violations of checkpoints in a Web page. We also made use of the 'homemade' crawler as the basis for future development of tools for Web accessibility evaluation. For a list of rules for extracting data of potential violations, please see Table 1. Since the crawler embedded in Bobby and the 'homemade' Web crawler may retrieve an unmatched number of pages for the different capacities of both crawlers, we only used the Web pages retrieved by both programs in the study.

Function of the Web Sites

We measured three variables-function, popularity and importance-as other features of the Web sites. We classified the candidate Web sites based on their functions. We used a taxonomy that classifies the Web sites into six functional categories: e-commerce, corporate, portal, community, government, and education. We derived the taxonomy from a similar one from the Web Usability Index database [20]. An e-commerce Web site conducts online transactions of health related products or services. A Corporate Web site represents a health care service corporation online. A Portal Web site provides entrance to various health related information resources. A Community Web site hosts online activities for patients or health information seekers. Government and education Web sites have the postfix '.gov' and '.edu', respectively in their domain names. Table 2 lists example Web sites from each category.

Table 2

CategoryDefinitionExamples
PortalWeb site provides entrance to various health related information resourcesWeb MD (http://www.webmd.com)
GovernmentWeb site has the postfix '.gov' in the domain nameHealth Finder from U.S. Department of Health and Human Services (http://www.healthfinder.gov)
CorporateWeb site represents a health care service corporation onlineMayo Clinic (http://www.mayoclinic.com)
E-commerceWeb site conducts online transaction of health related products or services.Health Windows (http://www.healthwindows.com)
CommunityWeb site hosts online activities for patients or health information seekers.Health Forum (http://www.healthforum.com)
EducationWeb site that has the postfix '.edu' in the domain nameHealthLink from medical college of Wisconsin (http://healthlink.mcw.edu)

Two evaluators individually assigned each Web site to one of the aforementioned categories. In case of a disagreement about the assignment, both evaluators discussed it until reaching a consensus. Each Web site fell into only one of the categories. Government (.gov) and education (.edu) Web sites had precedence over other function categories. For example, HealthFinder.gov is a government Web site, but its function is also to provide health information as a portal. We assigned it to the government instead of portal category. The reason for the precedence is that we were especially interested in the degree of Web accessibility of these two functional categories because of the existing specifications and initiatives.

Popularity of the Web Sites

We used daily traffic-ranking data of each Web site that was provided by the search engine Alexa as the measurement variable for the popularity of the Web sites [36]. Alexa calculates statistics about the traffic patterns of a Web site after aggregating visit data from all users who install Alexa's toolbar in their Web browsers during a three-month period. Because the Alexa toolbar is currently only available for Microsoft Windows and Internet Explorer, the accuracy of the traffic ranking of the Web site is limited. However, it may reflect the popularity of the Web site on the Web to a certain extent. We retrieved the ranking data of the entire candidate Web sites from Alexa on February 25, 2003.

Importance of the Web Sites

We measured the degree of importance using the PageRank score of each Web site available from the Google search engine. The PageRank score relies on the uniquely hypertext nature of the Web by using its vast link structure as an indicator of an individual page's value. In essence, Google interprets a link from page A to page B as a vote by page A for page B. Therefore, the PageRank score of a page can be viewed as an indicator of the importance of the page. But Google looks at more than the absolute volume of votes, or links that a page receives; it also analyzes the page that makes the vote. Votes cast by pages that are themselves 'important' weigh more heavily and help to make other pages 'important.' [37] Because Google does not provide PageRank in a numerical value from its searching interface, we had to rank the sites according to an implicit PageRank score and use the ranking number as the value of the variable of importance. We retrieved the ranking of importance of all candidate Web sites from Google on February 26, 2003.

Data Analysis

All statistical analyses were performed with alpha value at 0.05 and power at 0.80. Descriptive statistics (means and standard deviation) were calculated for each variable considered in the study. Univariate statistics of the WAB scores were calculated at the level of each category. Then a one-way ANalysis Of VAriance (ANOVA) test was applied to the WAB scores at the level of the Web site's functional category. If the ANOVA test indicated a large difference in the WAB scores among different categories, the post hoc Bonferroni test of the WAB scores between different categories was conducted. The alpha level was adjusted for multiple comparisons in the Bonferroni test.

Google ranked Web sites with a sub-category from highest to lowest PageRank value. Therefore, we used the ranking sequence as the value of Web page importance for nonparametric Spearman correlation. Nonparametric Spearman correlation statistics were also conducted to measure the level of correlation between the WAB scores and the popularity of the Web sites. All statistical analyses were conducted using the SPSS 11.0 software package.

Results

Descriptive Statistics

The Google subdirectory 'Health/Consumer/Resources' lists 122 Web sites, 14 of which were excluded because their content are no longer healthcare related or they were not active during the study period. The assessing program retrieved 7,109 Web pages from the remaining 108 sites. Means and standard deviations of WAB scores for the remaining 108 Web sites were calculated. The average WAB score was 9.31 with standard deviation of 6.29. None of the 108 Web sites was absolutely accessible (WAB score = 0). The National Institutes of Health (NIH) Combined Health Information Database (CHID) Web site (http://chid.nih.gov/) achieved the lowest WAB score, i.e., it had the fewest accessibility barriers, of the sites tested (0.97), while a community Web site (http://www.discussyourhealth.com/) received the highest WAB score (24.99). The five most frequently violated WCAG checkpoints of all webpages were: 'identify language of the text' (77.0%), 'use a public text identifier in a DOCTYPE statement' (65.6%), 'provide a summary for tables' (61.6%), 'use relative sizing and positioning (% values) rather than absolute (pixels)' (60.0%), and 'provide alternative text for all images' (52.2%).

WAB and Categories

Among the six functional categories of Web sites, government Web sites were most accessible and had the lowest WAB scores, and portal Web sites were least accessible to people with disabilities, indicated by higher WAB scores (Table 3).

Table 3

Means and Standard Deviations of the Web Accessibility Barrier (WAB) Scores Across Functional Categories

CategoryMeanNumber of Web sites (n)Standard Deviation
Portal13.17306.16
Government1.4260.39
Corporate9.03253.94
E-commerce8.5383.39
Community9.92296.8
Education2.06101.16
Total9.311086.29

The average scores of Web accessibility were calculated for each of the Web categories and the results indicate possible clustering among the six categories, as shown in Figure 2.

Means of the Web Accessibility Barrier (WAB) Score of Each Category. Height of Each Bar Represents Mean WAB Score. The Horizontal Tick Above Each Bar Represents Standard Deviation of WAB Score

Statistically significant differences among the category groups were found using the ANOVA test on the WAB scores (F = 9.705, P < 0.001). In addition, the post hoc Bonferroni test found that the mean WAB scores of governmental and educational Web sites were significantly different from the rest of the categories (P < 0.001). There is no statistically significant difference between any two categories within each of the two clusters.

WAB Score vs. Popularity and Importance

Furthermore, the Spearman correlation test indicates a statistically significant, though modest, correlation between the WAB score and the Alexa traffic ranking (r= 0.28, P < 0.01). No statistically significant correlation between the WAB score and the PageRank of Web sites was found (r= 0.15, P = 0.111) using the Spearman correlation test (Table 4). The correlation between the Alexa's traffic ranking and Google's PageRank was statistically significant (r= 0.32, P < 0.01) using the Spearman correlation test.

Table 4

Spearman Correlation Coefficients Between the Web Accessibility Barrier (WAB) Score, Alexa Ranking and Google's Pagerank

WAB scoreAlexa rankingPageRank
WAB score1.000.28*0.15
Alexa ranking0.28*1.000.32*
PageRank0.150.32*1.00

* Correlation is significant at the 0.01 level (2-tailed). The complete results data set is included as a data supplement with this article.

Discussion

Awareness of accessibility issues is increasing among developers of Web sites due to law enforcement, public initiative, and prospective commercial incentives [21]. Even though many evaluation tools are now available to developers intending to improve the accessibility of their Web sites, the status of Web accessibility, especially among health information Web sites, is largely unknown. Compliance with the specifications of Web content accessibility is necessary to narrow the digital divide between the information affluent and digitally underserved people, in this case, those with disabilities. Ours is the first study to address the issue. It provides a relatively comprehensive evaluation of the Web accessibility of consumer health information Web sites, and proposes a metric evaluation for measuring the accessibility of a Web site, taking into account both accessibility violations and the complexity of the Web site presented as potential violations of accessibility checkpoints. This approach provides a more accurate and impartial measurement about the level of accessibility barriers than using only the absolute number of violations as has been employed by most other evaluations. Additionally, the study investigates the relationship between the level of accessibility and the function, importance and popularity of a Web site.

Current Level of Web Accessibility Across Consumer Health Information Web Sites

No consumer health information Web sites satisfied all of the Web accessibility requirements, which may be attributed to Web site developers knowing little about accessibility standards, the lack of effective and efficient evaluation and repair tools, and the pressure to update information on the Web site quickly. Web accessibility, if ever considered, is often an afterthought once Web content design is finished. This implies that program tools that produce efficient, effective post-hoc repairs of Web content accessibility violations, or an accessible proxy server that transforms and filters inaccessible online content for people with disabilities may be more accepted by both the developers and Web site visitors.

Web Accessibility and Functions of the Web Sites

Of the sites providing health information, government sites followed by education sites are the most accessible. This compliance may be attributed to Section 508, since it is mandatory for all federal agencies [38]. High compliance among sites that fall under this mandate also indicates that legal activities would facilitate the removal of accessibility barriers for people with disabilities.

None of the tested Web sites, including the most accessible government sites, passed the WCAG guideline priority 1 checkpoints, even though the five most frequently violated checkpoints have technically uncomplicated solutions if designers pay attention to them. This may imply that the Web site editor simply overlooked the errors and, for such editors, an automatic Web site monitoring program could be very helpful in identifying and correcting these errors. Other possible reasons for such imperfection are the lack of integrated accessibility tools or functions within Web site editing software. Most Web site editing tools make it optional to strictly follow accessibility rules.

The education Web sites are the second most accessible category. Section 508 is not strictly mandatory for the information technology available on educational Web sites, but high awareness of WCAG rules and legal requirements on most campuses may contribute to better accessibility among the education Web sites. Furthermore, although Section 508 does not mandate all education Web sites, it does apply to educational programs and projects that receive federal funding, as many do, which may explain the high compliance to WCAG rules among education sites.

Web Accessibility and Popularity of the Web Sites

The accessibility of a Web site also correlates with its popularity, possibly implying that people with disabilities are more likely to visit sites that contain fewer or no barriers to them. A more accessible Web site may be more usable for the general population because it can also improve the efficiency, effectiveness, and ease of using the Web site [39]. Meanwhile, accessible Web pages will have better opportunities for indexing by Web search engines, which use programs called crawlers to access Web pages on the Internet and store Web page indexes in a database for fast Web information retrieval. Web crawlers work similarly to Web users who are blind and using screen reader programs. Therefore accessible Web pages will have more chances to be indexed by a Web crawler [40]. Subsequently the overall popularity of the Web sites increase since they attract a group of visitors who have difficulties accessing other sites containing more Web accessibility barriers. Other reasons for the correlation between accessibility and popularity include the possibility that people may take notice that a Web site is accessible and tend to visit it often, or Web developers of accessible Web sites spend more time ensuring their Web sites are appropriate in following other usability rules that make visiting easier for the public.

Web Accessibility and Importance of the Web Sites

The correlation between Web accessibility and a Web site's importance was not statistically significant in our study, although the correlation between its importance and popularity was statistically significant. The measurement of the importance of a Web site was derived from comprehensive link analysis on the Web. It revealed the value of the Web site by measuring how many and what kind of other Web sites link into it. It does not necessarily reflect the value of other HTML elements, especially those Web accessibility related elements. A Web site can be very important in terms of PageRank because many other Web sites have links to it, even though it is not accessible to persons with disabilities when they directly visit it.

Limitations

Please note that there are several limitations to this study. First, although this study attempts to comprehensively assess the accessibility of a Web site, it is not practical for some Web sites, especially those with large numbers of archived documents. The Bobby program often freezes when checking all layers of a Web site, and this resulted in the decision to check only a manageable two layers of Web pages in this study. A more robust tool needs to be adopted or developed for future studies.

Second, only the checkpoints of Web accessibility that can be examined automatically by a computer program were studied. Many other checkpoints require a manual check of pages to ensure the compliance of the content with the guidelines of Web accessibility. WAI proposed a comprehensive framework for evaluating Web content accessibility which requires multiple steps involving several evaluation tools to ensure the accuracy of the evaluation results. Although this type of evaluation is important for quality assurance of individual Web sites, the cost of such a large operation makes it impractical for an evaluation study involving many Web sites. This study assumes that the checkpoints that can be automatically evaluated will strongly correlate to the manual checkpoints and can be used as a surrogate assessment for accessibility of a Web site. Future studies might explore the agreement between these two groups of checkpoints.

Furthermore, the traffic ranking information provided from Alexa may skew towards users of Internet Explorer on a Windows operating system, underestimating the traffic to sites that are disproportionately accessed by people using other browsers or operating systems. The site most likely to suffer from this bias is AOL (America Online), since their members commonly use AOL browsers to access the site.

The WAB score in the study can be used to measure the degree of accessibility of a site. However, it should not be used as the only indicator for Web accessibility, which includes other checkpoints that can not be automatically assessed by computer programs. An experienced Web developer can fine-tune a Web site to produce a perfect WAB score. However, this does not necessarily mean that the Web site is entirely accessible to people with disabilities when they visit it.

Conclusions

This study evaluates the current state-of-accessibility of consumer health information Web sites for people with disabilities. Accessibility barriers are present in all site categories, especially commercial Web sites. Government and education Web sites show better performance than those in other categories. Accessibility may have an impact on its popularity because people with disabilities will feel more comfortable visiting those sites with fewer accessibility barriers. This study attempts to increase the awareness of Web accessibility among the designers of consumer health information Web sites.

Acknowledgments

This work was supported, in part, by grants number 42-60-I02013 from the National Telecommunications and Information Administration (NTIA) and H133A021916 from the National Institute on Disability and Rehabilitation Research (NIDRR). The authors also thank Ms. Yueh-Mei Yang for acquiring and formatting the data.

Abbreviations

WWWWorld Wide Web
WABWeb Accessibility Barriers
NTIANational Telecommunications and Information Administration
W3CWWW Consortium
WAIWeb Accessibility Initiative
WCAGWeb Content Accessibility Guidelines
WUIWeb Usability Index
NCINational Cancer Institute
PDFPortable Digital Format
ASPActive Server Page
JSPJava Server Page
XMLeXtensible Markup Language
ANOVAANalysis Of VAriance
NIHNational Institutes of Health
CHIDCombined Health Information Database

Appendix A

Consumer Health Web Sites Selected from Google for Accessibility Evaluation

Table A1

Consumer Health Web Sites Selected from Google for Accessibility Evaluation

NameAddressDescriptionCategoryAlexa RankWeb Accessibility ScorePage Rank
Consumer Reports Onlinehttp://www.consumerreports.orgInformation and advice on health products, services, and decisions.community2,0153.444
Health A to Zhttp://www.healthatoz.com/Includes a directory of more than 50,000 professionally-reviewed Internet resources, supportive online communities, and a calendar.community22,4362.338
Dr. Gabe Mirkinhttp://www.drmirkin.comReports on health, fitness, and nutrition news from talk show host Gabe Mirkin, M.D., in text and audio form.community52,8513.5628
Body1.comhttp://www.Body1.com/Health news and medical information community for consumers.community211,7056.9930
ProWhohttp://www.prowho.com/Locate health professionals anywhere in the world.community1,098,1886.9134
MDAdvice.comhttp://www.mdadvice.com/Provides health and medical information, health tips, resources, experts, news, chats, and community support.community88,68911.0945
Askapatient.comhttp://www.askapatient.com/Provides a database of patient opinions and ratings of medicine effectiveness. Also includes weekly consumer opinion polls on healthcare topics, and a health care research assistance section.community424,9063.2246
Health & Family Resource Guidehttp://www.noeasytask.comPersonal and professional sites containing valuable information and links.community871,197248
1UpHealth: Your Health Resource on the Nethttp://www.1uphealth.com/Offers information concerning condition and diseases. Listed by alphabet, systems/types, and by demography.community57,3955.6751
CountryNurse.comhttp://www.countrynurse.comIncludes information on clinics, family wellness, disease prevention, diet, exercise and pharmacies.community2,742,1823.4558
Selfhealthhttp://www.selfhealth.com.au/Consumer health information including a medical Q&A database and an Australian drug database. Covers infertility, emotional health, sexual health & integrative & complementary health issues.community1,055,4934.8759
Oneday MD Programhttp://www.onedayMD.comA medical e-course, written for the everyday layman. Easy to understand and digest.communityND7.9860
Wonderful World of Diseaseshttp://www.diseaseworld.com/Catalog of links and information on diseases and human conditions. Includes an online bookstore.community375,3094.9963
Medidoctorhttp://www.medidoctor.com/A home health guide to diagnosis and treatment, and when to see your doctor or go to hospital.community603,47511.9865
Wellness.comhttp://www.wellness.comIncludes health resources, discussion and news.community403,79113.9972
Digital City - Healthhttp://www.digitalcity.com/health/Health resources and providers across the United States.community1,03024.8877
Discuss Your Healthhttp://www.discussyourhealth.com/Discussion forums and health information.communityND24.9981
Mindy Machanic's Change Pages: Wellness and Health Infohttp://www.mindymac.com/Health.htmlArticles on healthy foods, cancer and breast cancer. Includes comprehensive links to additional resources for health and wellness.community3,156,1545.9983
Health-Center.comhttp://www.health-center.com/default.htmResources on numerous health topics. Includes a bulletin board and discussion forum.community49,6075.4486
C.S.S. Doctor's Credentials Searchhttp://www.tese.com/css/index.htmlSearch for a Doctor's Medical School, Board Certification, residence training, licensing, disciplinary action (if any), and other important information.community668,45910.3488
Health In Depthhttp://www.healthindepth.com/Health information links to newspapers, magazines and internet resources.communityND14.9889
American Carehttp://www.americancare.net/Provides access to a network of medical professionals and medical facilities.communityND20.9993
DoctorInfohttp://www.maxpages.com/doctorinfo/Provides searches for background information on medical doctors or doctors of osteopathic medicine.community3,23221.6698
Internet Health Libraryhttp://www.health-library.comSearchable index to healthcare sites.community247,22915.89103
Urgent Medical Helphttp://www.urgentmedicalhelp.com/Offers a list of medical topics, and expert advice.communityND16.88108
Health+Plus / Dr. David Clark [1]http://communities.msn.com/DrDavidClarkHealthPlusInformation and alternatives on various health challenges.community4.66109
Well-awarehttp://www.well-aware.co.ukProvides information on conditions, complementary treatments and expert views, all written by doctors in the United Kingdom.community279,8766.77110
AOL Anywhere Health Web Channel: Tests and Toolshttp://search.aol.com/dirsearch.adp?query=health%20toolsInformational question and answer tool for assessing your personal health and fitness. Addresses a variety of common conditions, diseases, and disorders.community2310.44112
SciTalk.comhttp://www.scitalk.com/Science related resources for the public on health and disease. Discussion boards, chat, news, patents, clinical trials and books.community831,82811.21113
community
Health Communication Networkhttp://www.hcn.net.au/Provides the up-to-date health information on a variety of subjects.corporate62,5225.92836508521
BluePrint for Healthhttp://blueprint.bluecrossmn.com/A health and wellness portal which provides health information, personalized newsletters and interactive health tools.corporate55,3096.11727167722
Apples For Healthhttp://www.applesforhealth.com/Consumer news on healthcare topics.corporate109,0782.77020690226
The Health Resource, Inc.http://www.thehealthresource.com/Specialized medical research reports on mainstream, experimental, and alternative treatments, specialists, and support organizations.corporate751,7416.47385187235
HealthLink Plushttp://www.healthlinkplus.org/Consumer health information on general health, health care providers, medical research, insurance, wellness, mental health, and alternative medicine.corporate1,012,0199.22370668849
HealthFrontier.comhttp://www.HealthFrontier.com/Offers information including diseases and conditions, nutrition, exercise, mental health, live discussions and a message board.corporate1,118,4834.70684918252
Accenthealthhttp://www.accenthealth.com/Database of news, articles, and information about conditions, medications, and tips for living a healthy lifestyle.corporate215,0953.09458941553
HealthStatushttp://www.healthstatus.comFree reports on body fat percentage, body mass index, calorie burning activities, target heart rate and smoking costs. Online health risk assessment which provides resources based on your health risks.corporate100,17614.4885199255
Health-MDhttp://www.health-md.netProvides informational links covering all aspects of health.corporate1,894,24414.2878745557
eCureMe.comhttp://www.ecureme.com/Identify symptoms to make a self-diagnosis; set up online consultations with physicians and therapists; view online medical dictionary of diseases, treatments, drug information.corporate28,6268.17455953961
Patient Protecthttp://www.patientprotect.com/en/Medical consultation devoted to protecting and defend patients. Contributes to reducing health costs, by preventing abuses, negligences, medical errors and incompetence in the health field.corporateND14.2629062262
A Second Opinion Medical Information Serviceshttp://www.physicians-background.comMedical treatment options, physician background check service, best hospitals and doctors. (Ft. Walton Beach, FL)[Fee based service - ed]corporate1,164,3458.51367019764
50+Healthhttp://www.50plushealth.co.ukHealth topics, lifestyle magazine, discussion forum, news and research.corporate986,9028.03537374566
Medical Elitehttp://www.medical-elite.com/International medical consulting and information company that specializes in locating medical specialists. Translated into English, Arabic, Chinese, Portuguese, Russian, Spanish, and other languages.corporate1,574,9685.08342491771
GetWell.orghttp://GetWell.org/Offers resources for consumers on medical conditions, treatment and research.corporate3,373,4733.70408176374
Global Health News and Resourceshttp://www.globalhealthnews.org/Offers news and resources in the health industry.corporateND9.24444089275
The Lifestyle Doctorhttp://www.lifestyledoctor.uk.com/Information on lifestyle issues and simple ways to help oneself.corporate3,097,64912.3946205978
UHealthy Networkhttp://www.uhealthy.com/Global health information network and community that integrate every aspect of Health and Fitness in one place.corporate94,3644.34144667280
Vital Star Health, Science and Technology Resource Centerhttp://www.vitalstar.com/health.htmlFree online Medical Check up How healthly your are? Test your eye, BMI, carbs, protein, cholestrol, heart, height, calories, depression. Plus articles, news and updates related to health and fitness.corporate1,154,16611.1072341484
WoundHeal.comhttp://www.woundheal.com/info/infoIndex.htmEducational information and resources for the non-surgical healing of pressure ulcers, at home.corporate2,266,56614.1259730285
Wellness Hour Medical Informational Talk Showhttp://www.wellnesshour.comA medical talk show aired in over 100 cities throughout the United States.corporate1,523,3367.01261501297
Health and Wellnesshttp://www.health-and-wellness.org/Explains the importance of 'wellness', information on how to rate your own wellness, and how onsite programs can boost employee productivity.corporateND13.80270648104
Medical Information Dictionaryhttp://www.medical-information-dictionary-and-videos.com/Dictionary with extensive listings on treatments. Current information on new medical procedures and definitions.corporate777,9949.957871849105
Discovery Healthhttp://www.discoveryhealth.com/Offers news and a variety of health information resources.corporate4957.237195863121
corporate9.433343772
HealthAnswershttp://www.healthanswers.com/Contains health news and information, including a health encyclopedia.corporate,portal80,99612.46205114119
HealthWindowshttp://www.healthwindows.comA membership healthcare network that helps individuals to become more knowledgeable and active participants in managing their personal health.e commerce723,9992.1391391723
Quackbustershttp://www.quackbusters.com.au/Offers nutrition and general health information.e commerce507,1455.36235749433
Clinnix: Health Care Informationhttp://www.clinnix.netIncludes daily news, travel information and disease management.e commerce802,71313.1282227940
Prozac Prescription Online Pharmacyhttp://www.prozac-prescription-online-pharmacy.comConsult this Prozac guide to get prices, medical facts and tips on where to buy. Includes interaction data and uses.e commerce2,776,05015.7872977568
HealthCheck Risk Assessmenthttp://www.bodybalance.com/hra/Useful health risk assessment.e commerce606,67415.2680982369
Health Depothttp://blakkat.com/health.htmDirectory to health and medical sites about diet, fitness, disabilities, diseases, health resources, products and sales.e commerce148,97315.0475309676
Alternative Healing and Lifestyleshttp://alternatehealing.comOffering resources ranging from weightlifting to mind, body, and Nutrition. Many links to health sites.e commerceND15.50132577100
DoctorOnNet.com [2]http://www.doctoronnet.com/Resources for information regarding medicine, doctors, health, fitness and related topics.e commerce445,2558.931488165102
e commerce
CNN Healthhttp://cnn.com/HEALTH/Health news, chats and advice from CNN.education2621
MCW HealthLinkhttp://healthlink.mcw.edu/Features health news and information, produced by the Medical College of Wisconsin.education15,1092.015
McGill Molson Medical Informatics: Student Projectshttp://sprojects.mmi.mcgill.ca/A growing collection of multimedia projects in medical teaching. Developed by McGill medical students under the supervision of the McGill Medical Faculty. Includes a student/faculty forum.education5,5361.6627
Evaluation of English and Spanish Health Information on the Internethttp://www.rand.org/publications/documents/interneteval/The findings of a large study that describes and evaluates English and Spanish health information on the Internet. Assesses search engine performance and the quality and readability of health information on the Internet, and provides conclusions and recommendations.education25,9664.5631
Duke University Healthy Devil On-Linehttp://gilligan.mc.duke.edu/h-devil/Online medical resources and information.education4,8083.4432
Health Leaderhttp://www.uthouston.edu/hLeader/index.htmlA webzine produced by The University of Texas Health Science Center, which provides information to help you make better decisions about your health.education563,6312.4436
BBC Onlinehttp://www.bbc.co.uk/health/conditions/Information about a wide range of health conditions, summaries of illnesses and treatments, and details of organizations that can provide medical and emotional help and support.education391.03120
Your Health IS Your Businesshttp://weber.edu/hp/Faculty/molpin/bushea/index.htmlSite includes information on health and wellness including primarily links to sites on the internet on health and wellness.education28,2911.46122
BBC Health Your Rightshttp://www.bbc.co.uk/health/consumer/index.shtmlAdvice on everything from finding the shortest waiting lists to what to do if you think you are a victim of medical negligence.education391123
BBC News - Medical noteshttp://news.bbc.co.uk/hi/english/health/medical_notes/Information briefs on health topics related to the news, including several on environmental health topics. Listed alphabetically.education391125
education
National Institutes of Health -- Health Information Indexhttp://health.nih.govMain consumer health information page for the National Institutes of Health (NIH)government2861.452
Healthfinder (tm)http://www.healthfinder.govResource for consumer health and human services.government22,6601.023
Combined Health Information Databasehttp://chid.nih.gov/A database produced by health-related agencies of the Federal Government. Provides titles, abstracts, and availability information for health information and health education resources.government2860.9710
Agency for Health Care Policy and Researchhttp://www.ahrq.gov/consumer/Consumer health and patient information on health plans and insurance, prescriptions, conditions and diseases, surgery, quality of care, quitting smoking, and prevention and wellness.government68,2551.6312
Michigan Electronic Library - Health Information Resourceshttp://mel.lib.mi.us/health/health-index.htmlExtensive resources and links of interest to the health consumer and to professionals.government21,4042.0129
Buying Medical Products Onlinehttp://www.fda.gov/oc/buyonline/Information for consumers from the US Food and Drug Administration. How to determine if a site is legitimate; how to spot health fraud; and how to report fraudulent sites.government3,0671.44114
government
MayoClinic.comhttp://www.mayoclinic.com/Clinical experts provide current medical information and news on health topics.portal4,15616.616
Dr. Koop's Communityhttp://www.drkoop.com/Former Surgeon General Koop's resources for health information. A wide variety of topics, an encyclopedia, pharmacopeia, and resources guide.portal14,24415.067
WebMD - Consumerhttp://my.webmd.com/Frequently updated portal for healthcare, chat forums, health quizzes, news and consumer product updates.portal51414.109
Diseases, Disorders and Related Topicshttp://www.mic.ki.se/Diseases/index.htmlKarolinska Institutet, Stockholm, Sweden. Comprehensive listings of links to medical information, most reliable, some not.portal9,8444.9411
Achoo Healthcare Onlinehttp://www.achoo.com/Portal and directory for medical news and information.portal101,65812.5913
Medinexhttp://www.medinex.comProvides a safe health search for medical information available on the internet.portal504,72614.9614
Healthgrades.comhttp://www.healthgrades.com/Grades the performance of hospitals, physicians, health plans, nursing homes and other health care providers in the United States.portal71,77522.9015
Halls MDhttp://www.halls.md/Clinical calculators of body surface area, breast cancer risk and body mass.portal94,8804.7916
Health In Focushttp://www.healthinfocus.co.uk/Independent UK health information and medical information resource.portal293,59816.4817
Laurus Health Informationhttp://www.LaurusHealth.comInformation on health conditions, pharmaceuticals, medical news, plus profiles of physicians and hospitals. Free registration.portal47,1084.2018
Cochrane Consumer Networkhttp://www.cochraneconsumer.com/This international group dedicated to the study of evidence-based medicine, explains how to decipher clinical studies and how to use them when making decisions about medical care.portal688,47421.3419
Internet Pharmacy and Online Pharmacies Verificationhttp://www.nabp.net/vipps/intro.aspNational Association of Boards of Pharmacy provides searchable listings of approved online pharmacies.portal97,56317.5420
Medicine OnLinehttp://www.meds.comIn-depth information on cancer for health care professionals and patients.portal87,46912.9525
Find a Doctor in Your Areahttp://www.vab.comGeographic directory of doctors with links to their web sites.portal352,92020.3137
Best Doctorshttp://www.bestdoctors.com/Comprehensive knowledge-based medical referral service.portal102,2329.4739
Health Consumer Alliancehttp://www.healthconsumer.orgProvides information to consumers and advocates about access to health care for low-income consumers, including consumer education materials in 13 languages.portal2,026,5867.4942
Doctor Healthynethttp://www.doctorhealthynet.com/Offers diagnosis and treatment of conditions and diseases, medical procedures, preventive health guidelines, and sources of free medicines.portal630,05218.5944
AnswerMed.comhttp://www.answermed.com/Provides basic information on medical conditions and procedures including symptoms, diagnosis, treatment, predicted outcome and alternative diagnoses.portal1,178,6903.1647
Health, Nutrition and Fitnesshttp://www.health-nutrition-and-fitness.comSearch this extensive directory of sites, focusing on exercise and fitness, nutrition, mental health, depression and therapy, and diseases such as osteoporosis.portalND7.3754
Health Plughttp://www.healthplug.com/Provides information on prescription drugs and other medications, with a message board and news links.portal2,136,1218.0167
MDinteractivehttp://mdinteractive.com/Providing consumers with healthcare information and resources in every medical specialty. Providing physicians and patients with an efficient way to create and store medical records interactively.portal2,494,7959.1779
Access Place Healthhttp://www.accessplace.com/health.htmLink collection about medical news, health and fitness and some medical specialties.portal109,3383.3590
SymptomTrackerhttp://www.symptomtracker.comAn interactive medical diagnosis and treatment reference that uses brief yes/no questions about a users symptoms to arrive at possible conditions and treatments. [Please note the 'Warning' before proceeding - ed]portal467,9224.3392
Surgery Door Home Healthcare Guidehttp://www.surgerydoor.co.uk/HomeHealthcareGuide/Symptoms of common illnesses and ailments. From the UK's on-line health service.portal18.7395
iMedNetworkshttp://www.imednetworks.com/An internet-based healthcare network that connects physicians and patients to each other and to a virtual world of medical information, tools, and services.portal2,217,7206.54111
Germ Detectives [3]http://www.germdetectives.com/Portal site to health information, hoax busters, and ways of avoiding Bad Science.portal15.78115
KnockDoctor.Comhttp://www.knockdoctor.comHealth portal covering subjects such as family health, beauty, yoga, ayurveda, health and fitness.portal3,067,98322.95116
MedicalClub [4]http://www.medicalclub.comProvides interactive free health information on Womens, Childrens and Family health concerns. The site also includes extensive information on herbal medicines, supplements and First Aid. Bilingual, English/Spanish.portal17.81117
Planetamberhttp://www.planetamber.com/Global International health, medical and disability resources database. Categorized medical condition search for people with disabilities or health impairments, their families and those providing services and support.portal2,783,43610.16124
Search-It-Allhttp://www.search-it-all.com/biomedical.aspDoctor and hospital search, nutrition facts, drug and disease lookup and health information.portal4,129,6425.40126
portal
Mylifepathhttp://www.mylifepath.comProvides information on health and wellness, daily health news and message boards.Site not available410,70924
Consumer Laboratory Testing Informationhttp://www.ascls.org/labtesting/index.htmA thorough guide to medical laboratory tests, why they are performed, and what they might mean.Site not available933,91241
Carepanion [5]http://www.carepanion.com/Provides life care products, services and tools. Contains links, news, articles and suggested further resources on medical issues.Site not available1,097,49843
The Medical Information Warehousehttp://www.medfindnow.com/Offers medical and disease information including poison control and child abuse areas.Site not available1,498,88150
SearchPointe [6]http://www.searchpointe.com/Free verification of name, location, and education of doctors and chiropractors.Site not available3,423,19656
HealthExpos.comhttp://www.healthexpos.com/Information on upcoming events and expos in Minnesota.Site not availableND70
Health Forumshttp://www.healthforums.com/Customized libraries of health and well-being information. Log in to access an extensive library of resources.Site not availableND73
LivingandHealth.comhttp://www.livingandhealth.com/Offers information on topics such as diabetes, irritable bowel syndrom (IBS), hypertension, and epilepsy.Site not available16,12882
No Frills Healthhttp://www.nofrillsguide.com/health.htmAn easy to use and useful guide to health sites on the net.Site not availableND91
Citypractice.com [7]http://www.citypractice.com/Provides information on preventitive approaches in physical, behavioural and emotional healthcare.Site not availableND94
Health4mhttp://www.health4m.comHealth forum for support, information, or exchanging ideas. Topics of discussion include general health, fitness, nutrition, diets, women's/teen's/men's issues, depression, A.A/N.A recovery, acne, mental illness.Site not available1,454,52796
Worldnethealth.comhttp://worldnethealth.com/Offers an online medical encyclopedia with a large medical slide library and videos.Site not availableND99
Health and Wellness Topic Center [8]http://health.medscape.com/wellnesscenterProvides information and interactive health management tools across a variety of disciplines.Site not available514101
Journey To Wellness [9]http://www.ihealthradio.comAfrican American health magazine and radio show. Listen to archived radio programs of the nationally syndicated radio programs, as well as read program related articles and link to credible related resources..Site not available106
Medicalresourcesusa.comhttp://www.medicalresourcesusa.com/Offers guides to American hospitals, health clinics, medical practices and specialties.ND38
Healing Action [10]http://www.healingaction.com/Health related articles on food, aging, ecology as it relates to health.3,802,73787
A Patient's Guide to the Internethttp://www.patientsguide.comA step-by-step guide for patients seeking medical information on the Internet.203107
Building Better Healthhttp://www.pcsrx-online.comProvides information on health and wellness, along with daily health news, full-text journal and magazine articles.4,103,022118

Footnotes

References

1. Horrigan JB, Rainie L. Counting on the Internet. Pew Internet & American Life Project. 2002. Dec 29, [9 June 2004]. http://www.pewinternet.org/pdfs/PIP_Expectations.pdf.
2. Eysenbach G, Kohler Ch. What is the prevalence of health-related searches on the World Wide Web? Qualitative and quantitative analysis of search engine queries on the internet. AMIA Annu Symp Proc. 2003:225–9.D030003690 [PMC free article] [PubMed] [Google Scholar]
3. National Telecommunications and Information Administration, authors. A Nation Online. Government Report. 2002. [9 June 2004]. http://www.ntia.doc.gov/ntiahome/dn/index.html.
4. Eysenbach G. Consumer health informatics. BMJ. 2000 Jun 24;320(7251):1713–6. doi: 10.1136/bmj.320.7251.1713.http://bmj.com/cgi/content/full/320/7251/1713?view=full&pmid=10864552. [PMC free article] [PubMed] [CrossRef] [Google Scholar]
5. Kaye H Stephen. Computer and Internet use among people with disabilities (SuDoc ED 1.310/2:439579) Washington, DC: U.S. Dept. of Education, National Institute on Disability and Rehabilitation Research Office of Educational Research and Improvement, Educational Resources Information Center; 2000. http://dsc.ucsf.edu/view_pdf.php?pdf_id=23. [Google Scholar]
6. Nielsen Ratings, authors. A Framework for the Evaluation of Internet-based Diabetes Management. [30 Nov 2003]. http://www.nielsen-netratings.com/hot_off_the_net.htm.
7. Paciello Michael G. Web Accessibility for People With Disabilities (R & D Developer Series) Lawrence, KS: CMP Books; 2000. http://dsc.ucsf.edu/view_pdf.php?pdf_id=23. [Google Scholar]
8. Hayes MG. Individuals with disabilities using the internet: a tool for information and communication. Technology and Disability. 1998;8(3):153–158.[Google Scholar]
9. Fox S, Fallows D. Internet Health Resources: Health searches and email have become more commonplace, but there is room for improvement in searches and overall Internet access. Pew Internet & American Life Project. 2003. Jul 16, [9 June 2004]. http://www.pewinternet.org/pdfs/PIP_Health_Report_July_2003.pdf.
10. Varian HR. The Information Economy. How much will two bits be worth in the digital marketplace? Scientific American. 1995 Sep;273:161–162.[Google Scholar]
11. W3C, authors. Web Content Accessibility 1.0 Conformance Logos. [7 Dec 2003]. http://www.w3.org/WAI/WCAG1-Conformance.
12. Balas JL. Library consortia in the brave new online world. Computers in Libraries. 1998 Apr;18(4):42–44.http://www.infotoday.com/cilmag/apr98/cilab5.htm. [Google Scholar]
13. WAI, authors. Authoring Tool Accessibility Guidelines. [1 Aug 2003]. http://www.w3.org/TR/ATAG10/
14. WAI, authors. User Agent Accessibility Guidelines 1.0. [1 Aug 2003]. http://www.w3.org/TR/UAAG10/
15. US Government, authors. Section 508 of Rehabilitation Act. [11 Mar 2003]. http://www.section508.gov.
16. Abou-zahra S. Evaluation, Repair, and Transformation Tools for Web Content Accessibility. [3 Apr 2004]. http://www.w3.org/WAI/ER/existingtools.html.
17. Schmetzke A. Editor. Accessibility of Web-based Information Resources for People with Disabilities (Part One) Library Hi Tech. 2002;20(2):135–136.http://puck.emeraldinsight.com/vl=3205743/cl=31/nw=1/rpsv/cgi-bin/linker?ini=emerald&reqidx=/cw/mcb/07378831/v20n2/s1/p135. [Google Scholar]
18. Schmetzke A. Editor. Accessibility of Web-based Information Resources for People with Disabilities (Part Two) Library Hi Tech. 2002;20(4):397–398. doi: 10.1108/07378830210452596.http://lysander.emeraldinsight.com/vl=2502870/cl=23/nw=1/rpsv/cw/www/mcb/07378831/v20n4/contp1-1.htm. [CrossRef] [Google Scholar]
19. Schmetzke A. Web accessibility at university libraries and library schools. Library Hi Tech. 2001;19(1):35–49.http://oberon.emeraldinsight.com/vl=2833206/cl=12/nw=1/rpsv/~1137/v19n1/s8/p35. [Google Scholar]
20. UsableNet, authors. Web Usability Index. [8 Mar 2003]. http://www.usablenet.com/wui/wui_index.html.
21. Lazar J, Dudley-sponaugle A, Greenidge KD. Improving web accessibility: a study of webmaster perceptions. Comp Hum Behav. 2004;20(2):269–288. doi: 10.1016/j.chb.2003.10.018.http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6VDC-4BRKMR8-B&_user=10&_handle=B-WA-A-A-AB-MsSAYWW-UUW-AUYEBZUBBB-AUYZEVAABB-VCDZWVUBZ-AB-U&_fmt=summary&_coverDate=03%2F31%2F2004&_rdoc=9&_orig=browse&_srch=%23toc%235979%232004%23999799997%23482001!&_cdi=5979&view=c&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=663e23ea1165d97ce29b6e445d1f68be. [CrossRef] [Google Scholar]
22. Lawrence S, Giles C L. Accessibility of information on the web. Nature. 1999 Jul 8;400(6740):107–9. doi: 10.1038/21987. [PubMed] [CrossRef] [Google Scholar]
23. Mandl K D, Katz S B, Kohane I S. Social equity and access to the World Wide Web and E-mail: implications for design and implementation of medical applications. Proc AMIA Symp. 1998:215–9.[PMC free article] [PubMed] [Google Scholar]
24. Berland G K, Elliott M N, Morales L S, Algazy J I, Kravitz R L, Broder M S, Kanouse D E, Muñoz J A, Puyol J A, Lara M, Watkins K E, Yang H, Mcglynn E A. Health information on the Internet: accessibility, quality, and readability in English and Spanish. JAMA. 2001 May 23;285(20):2612–21. doi: 10.1001/jama.285.20.2612.joc02274 [PMC free article] [PubMed] [CrossRef] [Google Scholar]
25. Griffiths Kathleen M, Christensen Helen. The quality and accessibility of Australian depression sites on the World Wide Web. Med J Aust. 2002 May 20;176 Suppl(Suppl):S97–S104.http://www.mja.com.au/public/issues/176_10_200502/gri10081_fm.html.gri10081_fm [PubMed] [Google Scholar]
26. Bykowski J L, Alora M B, Dover J S, Arndt K A. Accessibility and reliability of cutaneous laser surgery information on the World Wide Web. J Am Acad Dermatol. 2000 May;42(5 Pt 1):784–6. doi: 10.1067/mjd.2000.103044.S0190962200664538 [PubMed] [CrossRef] [Google Scholar]
27. Eysenbach Gunther, Powell John, Kuss Oliver, Sa Eun-Ryoung. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA. 2002 May;287(20):2691–700. doi: 10.1001/jama.287.20.2691.jrv10005 [PubMed] [CrossRef] [Google Scholar]
28. Koyani Sanjay J, Bailey Robert W, Nall Janice R. Research-Based Web Design & Usability Guidelines. Washington, DC: Computer Psychology; 2004. Sep, http://usability.gov/guidelines/[Google Scholar]
29. Davis Joel J. Disenfranchising the disabled: the inaccessibility of Internet-based health information. J Health Commun. 2002;7(4):355–67. doi: 10.1080/10810730290001701. [PubMed] [CrossRef] [Google Scholar]
30. Google, authors. About the Open Directory Project. [13 Mar 2003]. http://dmoz.org/about.html.
31. Astbrink G. Web page design: something for everyone. Link-Up. 1996 Dec;:7–10.http://www.nla.gov.au/linkup/961204.html. [Google Scholar]
32. WAI, authors. Web Content Accessibility Guidelines 1.0. [1 Aug 2003]. http://www.w3.org/TR/WCAG10/
33. Axtell R, Dixon JM. Voyager 2000: a review of accessibility for persons with visual disabilities. Library Hi Tech. 2002;20(2):141–147. doi: 10.1108/07378830210432507.http://puck.emeraldinsight.com/vl=3442644/cl=36/nw=1/rpsv/cgi-bin/linker?ini=emerald&reqidx=/cw/mcb/07378831/v20n2/s3/p141. [CrossRef] [Google Scholar]
34. Center for Applied Special Technology, authors. Home page. [4 Aug 2003]. http://www.cast.org/
35. Watchfire, authors. Home page. [3 Aug 2003]. http://www.watchfire.com/
36. Alexa, authors. Home page. [4 Aug 2003]. http://www.alexa.com/

Bobby Program From The Center For Applied Special Technology Resources Inc

Bobby program from the center for applied special technology resources management
37. Google Technology, authors. Home page. [2 Mar 2004]. http://www.google.com/technology/
38. Jaeger PT. Section 508 goes to the library: Complying with federal legal standards to produce accessible electronic and information technology in libraries. Information Technology and Disabilities. 2002;8(2)http://www.rit.edu/~easi/itd/itdv08n2/jaeger.html. [Google Scholar]
39. Kirkpatrick CH. Getting two for the price of one: accessibility and usability. Computers in Libraries. 2003;23(1):26–29.http://www.infotoday.com/cilmag/jan03/index.shtml. [Google Scholar]
40. Pemberton S. The Kiss of the Spiderbot. Interactions. Interactions. 2003 Jan-Feb;10(1):44. doi: 10.1145/604575.604594.http://portal.acm.org/citation.cfm?id=604575.604594&dl=GUIDE&dl=ACM&idx=604575&part=periodical&WantType=periodical&title=interactions&CFID=22307618&CFTOKEN=50909674. [CrossRef] [Google Scholar]
Articles from Journal of Medical Internet Research are provided here courtesy of Gunther Eysenbach