250 Part V: Real Life with Service Management
Partners HealthCare
Partners HealthCare, a not-for-profit organization, is the largest provider of medical services in Massachusetts. It was founded in 1994 by two of the top hospitals in the country, both in Boston: Brigham and Women’s Hospital, and Massachusetts General Hospital. It includes large hospitals, community hospitals, primary care and specialty physicians, the two founding academic medical centers, specialty facilities, community health centers, and other health-related entities. Its goal is to provide “a continuum of coordinated high-quality care.”
In 2000, Partners began to implement a service oriented architecture (SOA) strategy to share information in-network and among partners. In this way, patients can get the best possible care regardless of which facility they go to.
As Partners developed its portfolio of services, it realized that it needed a way to manage these services more effectively. Partners started a rudimentary process for vetting service requests, part of which involved arranging the services in a catalog that people could access. At first, the catalog was nothing more than a spreadsheet with information about the services; over time, it became a database that allowed users to request services.
As the services strategy evolved, IT had to provision these services and grant license to use them. This requirement set in motion a whole set of concerns related to use monitoring, performance monitoring, and security assur-
ance. Partners needed to refine its approach to service management. This refinement included developing a monitoring and performance management strategy to ensure that services were meeting SLAs. In addition to monitoring service performance, the team made sure that it had a sound capacity management process in place so that unexpected spikes in service consumption didn’t bring down the house.
Monitoring services
You can monitor a service at many levels. One of the first things that Partners needed to monitor was whether the consuming application was authorized to use a particular service. The IT team, under the direction of Steve Flammini, chief information officer of the information services group at Partners, implemented a security gateway to examine the credentials of the service call. That way, IT confirmed an authorized user and could attribute a level of use with that consumer.
The team also monitors the performance of the service at any given time and determines whether that performance meets its SLA requirements (if a SLA has been negotiated for the service). Flammini notes that in many cases, his team negotiates a SLA that is purely between the provider and the consumer. No blanket SLA applies to all consumers. IT works with different consumers
Chapter 20: Health Care 251
to analyze their needs. If a specific performance level is required, a cost can be established, the consuming organization agrees, and the performance level can technically be met, the SLA is implemented.
A given application may need to do a medication lookup on a medication formula, for example; that service is a standard service. According to Flammini, you don’t want several applications designing their own medication lookups. This service is a core piece of functionality that should be offered in the enterprise as a service. The SLA may be that when the consuming application (such as an electronic record in which prescriptions are written) issues a call to do a drug search, that result needs to be returned within 200 milliseconds. Before the implementation, the team does some load and performance testing, and performs simulations to ensure that the service can really turn this functionality around within that amount of time.
A lot of services used by a lot of applications can mean a lot of negotiation. The team delivers high-performance services that can scale, and in most cases those services are well within the performance window of any application’s potential needs. The team can deliver this result because it has a sound underlying architecture and has done ample load testing.
Service performance may be affected by how often a service is consumed, so the team also tracks consumption patterns within the infrastructure. And, when a service doesn’t meet its SLA, the team has to do some diagnostic investigation to see whether the service is being overly consumed. If someone applied to consume a service 10,000 times a day but actually calls it
100,000 times a day, that’s overconsumption. The team looks at the consumption pattern and the infrastructure to find hot spots.
Planning capacity needs
In some cases, the team sees a much larger jump in usage on a particular service than it thought it would see. According to Flammini, the cause could be a bad design decision, an invocation bug, or explosive uptake in these services for various applications.
One example of a service that’s experienced a large uptick in demand is the Enterprise Master Patient Index (EMPI), which was rolled out in 2005. In this identity-driven system, a patient’s medical-record number is attached to all of his charts and paperwork. If a patient who’s normally treated at one hospital in the network goes to another hospital or provider, that provider can request patient information by using these services. In effect, the EMPI is a unique customer identifier. Virtually all applications across the enterprise — whether they were developed in-house or purchased, and whether they are clinical, administrative, or data warehousing applications — depend on the EMPI as their source of patient information and identity. This service may be called upon tens of millions of times per day.
252 Part V: Real Life with Service Management
Partners on the back end must to be prepared to handle the growth. This requires capacity planning. According to Flammini, it sounds like “motherhood and apple pie” to say that you have to sit down with IT and clinical systems leadership to understand how aggressively that organization wants to push out systems and understand their growth projections. However, the IT team needs to understand these plans so they don’t get caught in a growth situation they may be unable to handle.
For example, in the case of the EMPI, IT sits down with the consumer to try to understand their consumption patterns. Are they going to have to call the EMPI service a dozen times each time they open a patient record — or just once? Or are they going to continually call upon a service for data, or will they store that data locally to avoid calling repeatedly?
Identifying team roles
What happens when a problem occurs? Where does the organization turn? Partners has multiple service-provider teams that are experts in certain clinical domains such as medications or the EMPI. Each team has designers, developers, and support people who really understand the inner workings of these services and can work with consumers to resolve issues.
In addition to the specialists, Partners has a group of generalists who support the infrastructure, look at the underlying technology platform, and monitor whether CPU use is out of bounds. This group also looks at various levels of the infrastructure.
Partners has had great success with its SOA strategy, and part of that success has come from learning to manage these services. Although the need for a service management strategy was an outgrowth of a technical requirement, the result is that the company is now using its services in applications that it hadn’t even imagined, and the service management strategy is enabling IT to deliver real value to the organization.
Chapter 21
Retail
In This Chapter
Virgin Entertainment Group
The retail market is highly competitive. If consumers don’t like what they see in your stores, they might simply take their business else-
where. Retailers understand that today, more than ever, they must deliver a memorable and positive customer experience. For companies that focus on entertainment-related products, such as music and movies, the competition is even more intense.
Although mom-and-pop stores still exist, the trend has been toward larger chains and giant multinational corporations. To win the hearts and minds of customers, many large retail stores need to provide top-notch service as well as a memorable experience. They may provide some form of entertainment in the store itself (such as a light-and-sound show or memorabilia), or they may furnish kiosks where customers can listen to music or sample movies, which helps close the deal faster.
The key to customer satisfaction is the same whether you’re in a small or large store: Exceed customer expectations.
In this chapter, we profile Virgin Entertainment Group and, in particular, its U.S. Virgin Megastores. The IT group servicing these stores realized that it needed to provide its customers an experience that was always on. This goal meant that the hardware and software used in the stores — especially the software that completed sales — had to be maintained properly, which led the company to deploy an aggressive service monitoring strategy.
Virgin Entertainment Group
Virgin Entertainment Group is a subsidiary of Richard Branson’s Virgin Group conglomerate. The company has annual revenue of about $200 million. It sells everything from music, movies, books, and videogames to electronics and lifestyle fashion. It also has an e-commerce and social networking site. Although its stores in the United States are closing, the company owns and
254 Part V: Real Life with Service Management
operates about 150 Virgin Megastores in France, Japan, Australia, and the Middle East, all owned by local companies with licensing agreements that lead back to Branson.
According to Robert Fort, chief information officer of Virgin Entertainment, the company had a critical need for real-time information about everything from sales to overall operational data. When Fort was walking through a store early in his tenure, he realized how important the customer experience was to the success of the brand. The company prides itself on providing a dynamic in-store environment, knowing that it has only a short time to make a positive impression on a customer. “Once we had established that customer experience, we realized it could be easily undermined by an inoperative kiosk or a register that was not working properly,” he says. These kinds of problems could damage the company’s bottom line.
(Store) room for improvement
Fort quickly determined that the company needed to monitor — proactively and persistently — all the services and features that it offered customers. By doing so, it could get right on top of any outages and, more important, “get out in front of those so that they don’t even occur,” he says.
Fort assessed his own organization and realized that he didn’t have the resources to check every kiosk touchscreen. He wanted to monitor the uptime of all critical components, which included in-store technology and all the backbone components from IT, such as key servers, point-of-sale (POS) servers, and the registers right on down to the kiosks themselves.
A critical element of the monitoring strategy is its business focus. When a problem occurs, that problem is immediately tied to a business process. If a key network component fails, the IT department immediately becomes
aware of which business processes are affected, such as credit processing or kiosks. Then it notifies the stores involved about the issue and takes corrective actions to prevent any effect on customers. A kiosk’s failure at the same time as a credit failure is deemed to be less critical than the credit failure, so IT prioritizes its responses and resources appropriately.
The circle game
As the IT team embarked on its service monitoring effort, Fort realized that he needed to determine the most critical components of the Megastore infrastructure. He drew a series of concentric circles to help him organize his thoughts around these components. The innermost circle included core critical servers, network, and voice over IP (VoIP) components. The next circle contained the POS servers, which would bring all sales at a store to a
Chapter 21: Retail 255
screeching halt if they went down. This circle also contained credit services, which depend on the network to get customers’ credit cards validated. (A store could continue to accept credit cards if the network was down, of course, but it would assume greater risk.) The next circle out included the registers, the circle beyond that one included the kiosks . . . you get the picture.
The failure of a single register or a single kiosk is less important than the failure of the whole system, but it still leaves a negative impression with customers. Typically, one register would fail and wouldn’t be reported to the help desk. When the help desk finally got a call, the report was that multiple registers were down and associated monitors weren’t functioning.
Overarching everything in the retail space is the Payment Card Industry (PCI) initiative, which includes mandates from card companies such as Visa MasterCard on how to protect customer data. Fort knew that he had to monitor the credit process very tightly, making sure that IT was aware of any outages or anomalies so that the company was PCI-compliant.
Monitor the infrastructure; think about the business
The team implemented a commercial monitoring application that could answer questions like these:
Is the machine running?
Does it have the appropriate amount of disk space?
Does it have memory problems?
Is it connected to the network?
The software also monitors application, database, and operating systems. In all, the team monitors more than 2,000 components. Any number of components make up a business process, and a failure in any physical component could affect one or more business processes. The team also can monitor individual business transactions such as credit card purchases, which helps Virgin remain compliant with PCI.
Fort is quick to point out that although the company is monitoring these components at a technical level, it receives information about the impact of events at a business level. At the technical level, you can imagine that the networks at each of the stores has a key router and switch to connect to the network cloud; behind those routers and switches are the POS servers and the individual registers, kiosks, media players, and digital signs.
At the technical level, someone may see the problem as being a router failure. At a business level, however, someone may see the problem as being an
256 Part V: Real Life with Service Management
inability to process credit or sales. When a router goes down, the monitoring team not only sees red flags, but also knows which series of business processes is being affected, which helps the team determine the level of priority.
When IT knows the business impact, it can make the appropriate decision. A problem that might affect $100,000 worth of sales that day (if the registers go down, for example) gets top priority; it needs to be fixed no matter how much the repair costs. If, on the other hand, the issue might affect $20 worth of sales (a digital sign isn’t working, for example), expediting a fix for it isn’t worth a huge expense.
Don’t light my fire
Automated monitoring meant a change for the network engineering team, which was used to a firefighting mentality. If something went wrong, team members got involved. It was a bit of a shift to look at a dashboard that lit up and a sent e-mail to report that something was amiss. The alert e-mail didn’t necessarily say that the system had crashed — it might simply say that a database table increased by more than 10 percent in a single day — but it could predict a problem that needed investigation. The idea was to prevent the fire from coming.
This change affected many engineers’ personal measures of success, because no one would necessarily know that he’d averted a potential disaster. In fact, several engineers left the organization to move to areas where they could start building new networks.
Fort attributes this situation to a maturity curve, saying that some personality types may find it unrewarding to monitor smoke instead of fighting raging fires. He imagines it this way: On the left side of a firefighting icon are the data-center-minded people, who are very process-oriented types who believe in governance and don’t particularly care for fires. On the other side of the icon are the people who love fires, are always off to something new, and love to push boundaries. Fort believes that every organization going through service management will eventually move from one side of the icon to the other.
Chapter 22
Hospitality
In This Chapter
InterContinental Hotels Group
Do you simply want a clean bed to sleep in when you travel, or do you opt for luxurious accommodations with a health club, infinity swim-
ming pool, and Zen-style spa? Or do you insist on a green hotel with an environmental conscience? Whatever your preference, many of the largest hotel management companies operate multiple brands catering to a wide variety of market segments.
The competition in this industry is intense and has become even more challenging given the sharp decline in spending on leisure and business travel as both families and businesses tighten their budgets. Businesses in the hospitality industry use the diversity and quality of their services as a way to differentiate themselves from the competition. Although the quality of services such as housekeeping, restaurant meals, room service, and concierge assistance depends heavily on the performance of hotel staff, a lot of technical infrastructure is required in the background. The organization needs an agile and efficient infrastructure to focus on optimizing the customer experience. Guest profiles, housekeeping status, staffing reports, and other operations data must be available on demand to hotel management to ensure a positive experience for guests.
In addition, large hotel and resort companies employ many channels to serve customers, including the Web, call centers, travel partners, global distribution systems, travel agents, and other intermediaries. If you want your guests to keep coming back, they need on-demand access to room rates and availability from their preferred channels. Companies such as the one profiled
in this chapter have implemented a service oriented architecture (SOA) to deploy these services more effectively. You can’t deploy SOA, however, if you don’t have the techniques in place to manage and govern these services. Managing the business services of an SOA environment is an integral part of a holistic approach to service management. (For more information on SOA, see Chapter 6.)