CHAPTER 4

Measuring Information Technology’s Success

Key performance indicator

– measures that are tied to business drivers

- Metrics are detailed measures that feed KPIs

- Performance metrics fall into the nebulous area of business intelligence that is neither technology, nor business centered, but requires input from both IT and business professionals

EFFICIENCY AND EFFECTIVENESS

§Efficiency IT metric

 – measures the performance of the IT system itself including throughput, speed, and availability

§Effectiveness IT metric

– measures the impact IT has on business processes and activities including customer satisfaction, conversion rates, and sell-through increases

BENCHMARKING – BASELINING METRICS

§Regardless of what is measured, how it is measured, and whether it is for the sake of efficiency or effectiveness, there must be benchmarks – baseline values the system seeks to attain

§Benchmarking

- a process of continuously measuring system results, comparing those results to optimal system performance (benchmark values), and identifying steps and procedures to improve system performance

THE INTERRELATIONSHIPS OF EFFICIENCY AND EFFECTIVENESS IT METRICS

Efficiency IT metrics focus on technology and include:

• Throughput

• Transaction speed

• System availability

• Information accuracy

• Web traffic

• Response time

Throughput - the amount of information that can travel through a system at any point

Transaction speed - the amount of time a system takes to perform a transaction

System availability - the number of hours a system is available for users

Information accuracy - the extent to which a system generates the correct results when executing the same transaction numerous times

Web traffic - includes a host of benchmarks such as the number of page views, the number of unique visitors, and the average time spent viewing a Web page

Response time - the time it takes to respond to user interactions such as a mouse click

Effectiveness IT metrics focus on an organization’s goals, strategies, and objectives and include:

v Usability

v Customer satisfaction

v Conversion rates

v Financial

Usability - The ease with which people perform transactions and/or find information. A popular usability metric on the Internet is degrees of freedom, which measures the number of clicks required to find desired information.

Customer satisfaction - Measured by such benchmarks as satisfaction surveys, percentage of existing customers retained, and increases in revenue dollars per customer.

Conversion rates - The number of customers an organization “touches” for the first time and persuades to purchase its products or services. This is a popular metric for evaluating the effectiveness of banner, pop-up, and pop-under ads on the Internet.

Financial - Such as return on investment (the earning power of an organization’s assets), cost-benefit analysis (the comparison of projected revenues and costs including development, maintenance, fixed, and variable), and break-even analysis (the point at which constant revenues equal ongoing costs).

• Security is an issue for any organization offering products or services over the Internet

• It is inefficient for an organization to implement Internet security, since it slows down    processing :

ü However, to be effective it must implement Internet security

ü Secure Internet connections must offer encryption and Secure Sockets Layers (SSL denoted by the lock symbol in the lower right corner of a browser)

THE INTERRELATIONSHIPS OF EFFICIENCY AND EFFECTIVENESS IT METRICS

Interrelationships between efficiency and effectiveness

METRICS FOR STRATEGIC INITIATIVES

§Metrics for measuring and managing strategic initiatives include:

Ø Web site metrics

Ø Supply chain management (SCM) metrics

Ø Customer relationship management (CRM) metrics

Ø Business process reengineering (BPR) metrics

Ø Enterprise resource planning (ERP) metrics

WEB SITE METRICS

Web site metrics include:

Ø Abandoned registrations

Ø Abandoned shopping carts

Ø Click-through

Ø Conversion rate

Ø Cost-per-thousand

Ø Page exposures

Ø Total hits

Ø Unique visitors

Abandoned registrations: Number of visitors who start the process of completing a registration page and then abandon the activity.

Abandoned shopping carts: Number of visitors who create a shopping cart and start shopping and then abandon the activity before paying for the merchandise.

Click-through: Count of the number of people who visit a site, click on an ad, and are taken to the site of the advertiser.

Conversion rate: Percentage of potential 

customers who visit a site and actually buy 

something.

Cost-per-thousand (CPM): Sales dollars generated per dollar of advertising. This is commonly used to make the case for spending money to appear on a search engine.

Page exposures: Average number of page 

exposures to an individual visitor.

Total hits: Number of visits to a Web site, many 

of which may be by the same visitor.

Unique visitors: Number of unique visitors to a site in a given time. This is commonly used by Nielsen/Net ratings to rank the most popular Web sites.

SUPPLY CHAIN MANAGEMENT (SCM)METRICS

Ø Back order

Ø Customer order promised cycle time

Ø Customer order actual cycle time

Ø Inventory replenishment cycle time

Ø Inventory turns (inventory turnover)

Back order: An unfilled customer order. A back order is demand (immediate or past due) against an item whose current stock level is insufficient to satisfy demand.

Customer order promised cycle time: The anticipated or agreed upon cycle time of a purchase order. It is a gap between the purchase order creation date and the requested delivery date.

Customer order actual cycle time: The average time it takes to actually fill a customer’s purchase order. This measure can be viewed on an order or an order line level.

Inventory replenishment cycle time: Measure of the manufacturing cycle time plus the time included to deploy the product to the appropriate distribution center.

Inventory turns (inventory turnover): The number of times that a company’s inventory cycles or turns over per year. It is one of the most commonly used supply chain metrics.

CUSTOMER RELATIONSHIP MANAGEMENT (CRM)METRICS

Customer relationship management metrics measure user satisfaction and interaction and include

Ø Sales metrics

Ø Service metrics

Ø Marketing metrics

Sales Metrics

- Number of prospective

 customers

- Number of new

 customers

- Number of retained 

customers

- Number of open leads

Service Metrics

- Cases closed same

 day

- Number of cases

handled by agent

- Number of service

 calls

- Average number of

 service requests by 

type

Marketing Metrics

- Number of 

marketing campaigns

- New customer

 retention rates

- Number of 

responses by 

marketing campaign

- Number of 

purchases by 

marketing campaign

BPR AND ERP METRICS

The balanced scorecard enables organizations to measure and manage strategic initiatives



The balanced scorecard is a management system, (in addition to a measurement system), that enables:

•organizations to clarify their vision and strategy and translate them into action.

•It provides feedback around both the internal business processes and external outcomes in order to continuously improve strategic performance and results.

•When fully deployed, the balanced scorecard transforms strategic planning from an academic exercise into the nerve center of an enterprise

The balanced scorecard views the organization from four perspectives, and users should develop metrics, collect data, and analyze their business relative to each of these perspectives:

vThe learning and growth perspective.

vThe internal business process perspective.

vThe customer perspective.

vThe financial perspective