Does data warehouse end-user metadata add value?

Many data warehouses are currently underutilized by managers and knowledge workers. Can high-quality end-user metadata help to increase levels of adoption and use?

DOES DATA WAREHOUSE END-USER METADATA ADD VALUE?

W

By Neil Foshay, Avinandan Mukherjee, and Andrew Taylor

hile organizations routinely realize data is a key asset that must be exploited in order to achieve success in today’s competitive business environment, unlocking the potential business benefits of this data remains an elusive goal for many firms. In order to exploit the value of their data, many organizations have implemented data warehouses and business intelligence applications over the last decade, often at significant cost expenditure. These data warehouse environments serve to collect and integrate data, and to turn it into information that is accessible for query and analysis to produce insights that can inform and influence business decisions. illustration by jason schneider

70

November 2007/Vol. 50, No. 11 COMMUNICATIONS OF THE ACM

Table 1. End-user metadata taxonomy.

T

Category

Explanation

Definitional

Definitional metadata is any information that conveys the meaning of data in the warehouse (or, for example, on reports) to end users. Included in this category are business definitions, calculations, business rules, and allowable values. Definitional metadata answers the question: What does this data mean, from a business perspective?

he primary consumers of the contents of data Data Quality Data quality metadata advises users about the currency (freshness), accuracy, validity, or completeness of the data in the warehouse (or on reports, queries, or OLAP cubes). Data warehouse and business quality metadata answers the business user's question: Does this data possess sufficient quality for me to use it for a specific purpose? intelligence applications are decision mak- Navigational Navigational metadata provides users with a means to search for data (or other resources, such as a report). In other words, navigational metadata lets users query the data warehouse ers and knowledge to search for what they need and to get an understanding of relationships between data objects of various types. Navigational metadata answers the question: Where can I find the workers within the organization. data I need? Often, the performance of these Lineage information tells users about the original source of data in the warehouse (or, for end users is measured by how Lineage example, on a report) and describes what has been done to the data (for example, cleansing, transformation, or aggregation) prior to being loaded into the warehouse. Lineage information effectively they are able to use answers the user's questions: Where did this data originate, and what’s been done to it? available data to make good deciTable 1. End-user this situation is common in organizations. However, sions. Many of the end users are metadata taxonomy. not technically oriented and need there may be a perceptual gap between what end users a significant amount of support to want from the data warehouse and what data wareFoshay table 1 (11/07) use a data warehouse effectively. house practitioners think end users want. A good deal What would it mean if many of these managers has been written regarding one method for supportand knowledge workers don’t fully understand and ing end users: metadata. trust the data they are being provided? What if this Metadata has been described, generically, as “data lack of understanding and trust of data means end about data.” This definition is not particularly helpusers don’t perceive their data warehouse to be easy to ful in understanding the value of metadata in an use or useful, and consequently don’t use it? information systems context. Dempsey and Heery There is a good deal of evidence, based on the offer this more extensive definition: “Metadata is data experience of data warehouse practitioners, that the associated with objects which relieves their potential situation described here is the current reality in many users of having full advance knowledge of their exisorganizations. The primary evidence: the perception tence or characteristics. It supports a variety of operathat many data warehouses are underutilized. For tions” [3]. example, a study by Raden [9] found data warehouses Additional understanding of metadata is offered by and business intelligence applications have low adop- Tannenbaum [10] who states that metadata serves to tion rates within organizations (compared to spread- answer five important questions regarding informasheets and standalone databases). While there are tion in an organization: likely many factors involved causing this situation (poor data quality, dissatisfaction with the business • What do I have? intelligence tool used, and lack of data training, to • What does it mean? name a few), it may be the case that data warehouse • Where is it? end users often don’t fully understand or trust their • How did it get there? data. Consequently, these end users are not willing to • How do I get it? risk making key decisions with data they have little Metadata helps data warehouse end users to underinsight into. Data warehouse practitioners seem to understand stand the various types of information resources avail-

METADATA helps data warehouse end users to understand the various types of information resources available from a data warehouse/business intelligence environment.

72

November 2007/Vol. 50, No. 11 COMMUNICATIONS OF THE ACM

able from a data warehouse/business intelligence envi- warehouse). Thus there is metadata for IT users and ronment. These resources can take many forms metadata for business users. This research focuses including data elements, queries, reports, and pub- exclusively on the latter. Laney [8] suggests there are lished documents. Gardner [6] states that metadata is three types of metadata: definitional, navigational, critical for all aspects of a data warehouse and provides and administrative. Laney’s framework was intended a roadmap or blueprint for many warehouse func- to be a generalized categorization scheme for metations. Haley and Watson describe the value of end- data. Based on Laney’s definition, and refinements user metadata as follows: “We have found that users made through interviews with data warehousing without this [metadata] refrain from using the data experts, the authors developed the end-user metawarehouse, spend inordinate amounts of time devel- data taxonomy presented in Table 1. The proposed oping and testing queries, or ask someone more taxonomy expands on Laney’s definition and is speskilled to write their queries” [7]. cific to the metadata requirements of data warehouse That metadata provides benefits to end users seems end users. to be well understood. However, there1.is End-user a perception Figure metadata conceptual model. in the data warehouse industry that many data ware- RESEARCH MODEL houses today do not provide “good” metadata to end To operationalize the final objective of the study, we developed and tested a users. Further, there are composite research model, many types of metadata DW Metadata Perceived presented in Figure 1, and research into percepQuality Usefulness User Attitudes which explores the relations of end-user metadata DW Use H1: H2: H3: Toward Data DW Metadata tionship between end-user is lacking. In light of the Perceived Use Ease of Use perceptions of metadata apparent paucity of quality, their attitudes research on use of metaUser Perceptions of: Other toward the data available data, the study that is the * Data Quality Factors * Business Intelligence from the data warehouse, basis for this article was * Tool Effectiveness * Data Training Quality their overall satisfaction conceived with the followwith the data warehouse, ing objectives: Figure 1. End-user metadata and their level of use of the warehouse. The model was conceptual model. • To propose a taxonomy developed through consultation with a panel of data for classifying end-user Foshay fig 1warehouse (11/07) and metadata experts and by leveraging metadata; the Technology Acceptance Model (TAM) devel• To gain an understanding of what types of metaoped by Davis et al. [2] as well as the work on indidata are provided today to data warehouse end vidual attitudes and beliefs pioneered by Fishbein users; and Ajzen [4]. • To understand how data warehouse practitioners and the end users they support perceive different types of metadata; and ur model is composed of four levels. • To develop and test an empirical model, for end Starting from the left side of Figure 1, users, that links perceptions of metadata quality the first level consists of the variables and use with user attitudes toward data, perceived “metadata quality” and “metadata use.” usefulness and ease of use of the data warehouse Metadata quality measures how “good” and, ultimately, data warehouse use. end users perceive available metadata to be, incorporating end-user perceptions of the clarity, END-USER METADATA TAXONOMY accuracy, and completeness of metadata. Metadata use To address the first three objectives of the study, we measures the extent to which available metadata is propose a standard taxonomy for classifying end- actually used. The next level in the model is “user attiuser metadata. Data warehouse practitioners recog- tudes toward data.” This variable measures the degree nize there are numerous types of metadata that serve to which users understand, trust, and are willing to the needs of different types of users. Metadata serves use data in the warehouse to help them do their jobs. the needs of technical stakeholders (data warehouse The third level of the model is composed of the “perpractitioners responsible for developing and main- ceived usefulness” and “perceived ease of use” of the taining the data warehouse) as well as business stake- data warehouse/business intelligence environment holders (decision makers and knowledge workers they use. Together, these variables are a measure of who consume the information generated by a data users’ overall satisfaction with the data warehouse.

O

COMMUNICATIONS OF THE ACM November 2007/Vol. 50, No. 11

73

Table 2. Usefulness of and satisfaction with end-user metadata: Technical and end-user perceptions.

other at data warehouse end users. Overall, responses were received from 268 data warehouse practiDefinitional Quality Navigational Lineage Definitional Quality Navigational Lineage tioners from 266 organizations 8.0 7.5 6.9 6.6 5.9 5.5 5.1 5.1 Technical and 621 end-user respondents 6.7 7.5 7.2 6.7 5.4 6.2 5.5 4.8 End users from 104 organizations representThe final level in the model is “data warehouse use.” Table 2. Usefulness of ing a subset of the 266 organizasatisfaction with This variable seeks to measure the extent to which and tions. Thus, we had 104 end-user metadata: users currently access the data warehouse and their Technical and end-user organizations from which we perceptions. plans for future use. Foshay table 2 (11/07) obtained matched-pair or dyadic The study also evaluated the influence of other facresponses, that is, responses from a tors on user attitudes and data warehouse use. These data warehouse practitioner and at least one end user Figure 2. Response profile: Industry and data Warehouse Size. include end-user perceptions of: working in the same organization. Generally, technical respondents played a techno• The level of quality of the data in the warehouse; managerial role in supporting one or more data ware• The usefulness of the business intelligence tool used to access Responses by Industry the data warehouse; and Banking / Finance 13.9% • The quality of training received Manufacturing 13.9% Insurance 10.8% regarding the content of the Data Warehouse Size Healthcare / Medical 8.8% data warehouse (data training). Retailer / Wholesaler / Distributor 7.2% Satisfaction 1=Very unsatisfied ...5=Neutral …9=Very satisfied

Usefulness 1=Not useful at all …5=Neutral …9=Very useful

T

hese factors have been recognized within the data warehouse industry as being important to data warehouse success. As such, they have been included in our research model. The core hypotheses for the study are:

Other

6.8%

Education

5.2%

Government Professional Services / Consulting

Telco Transportation / Shipping / Travel Services Computer Hardware / Software / Data Processing Utilities

H1: End-user metadata quality and use influence end-user attitudes toward the data in their data warehouse. H2: User attitudes toward data influence user perceptions of both the usefulness and ease of use of the data warehouse. H3: User perceptions of ease of use and usefulness of the data warehouse influence the level of use of the warehouse. The hypotheses were tested using multivariate statistical methods. FINDINGS Respondent Profile. The study, which was conducted from April 2005 to October 2005, had the primary focus on the end user. However, the study gathered valuable insights from data warehouse practitioners as well. Data was collected through two online surveys— one aimed at data warehouse practitioners and the 74

November 2007/Vol. 50, No. 11 COMMUNICATIONS OF THE ACM

1TB–5TB

4.0% 3.6% 2.8%

5TB–10TB

6.2%

6.2%

6.8%

500GB–1TB

250GB–500GB 100GB–250GB

Less than 100GB

8.6%

11.7%

19.8%

22.8%

17.9%

2.0%

Professional Association or Non-Profit

Figure 2. Response profile: industry and data warehouse size.

4.8% 4.4%

Mining / Oil /Gas

Agriculture / Forestry

Larger than 10TB

4.4%

Publishing / Media /Advertising

Don't know

6.4%

0.8% 0.4%

houses within their organizations. The most common job titles for technical respondents involved: data warehouse manager, data warehouse architect, and Foshay fig 2 (11/07) data warehouse project manager. Technical respondents completed a survey aimed at gathering information about the data warehouse environment, understanding the types of metadata provided to end users, and assessing their views regarding the perceptions of their end-user community toward currently available metadata. The data warehouse practitioners were requested to identify representative end users in their organizations and invite them to participate in the study by utilizing a snowballing technique. End users participating in the study had, on average, 16.5 years of professional experience, been in their current position for just over four years, and were using a data warehouse for approximately three years. End-user respondents played a wide variety of organizational roles. Responses were received primarily from North American organizations, representing a variety of industries and data warehouse sizes, as presented in Figure 2.

Figure 3a. End-user metadata model findings.

Types of End-User While metadata is Metadata Provided. The deemed useful, end-user DW Metadata Perceived R =0.45 second objective of the responses indicate they are Quality Usefulness R =0.38 R =0.36 User Attitude DW Use R =0.65 study is to explore the only slightly satisfied with Toward Data DW Metadata H3: H1: H2: Perceived types of metadata provided the metadata available to Use Ease of Use to end users. Approxithem (overall average satisR =0.14 R =0.40 mately 88% of the data faction score—5.5 on a 9User Perceptions of: Other * Data Quality tested relationships Factors warehouses studied pro- All were point scale). Users are Business Intelligence * statistically Tool Effectiveness * Data Training Quality vide one or more types of significant at p