Evaluation and Selection

Increasing project complexity, conflicting demands for resources, emerging development methodologies, technology integration, mergers and
acquisitions, and changing business needs are just some of the challenges that are forcing CIOs to develop innovative strategies for successful IT
implementation. Akhtar Pasha analyses some of steps that IT managers take while evaluating and selecting vendors

Vendor evaluation is an ongoing process that involves the objective monitoring and evaluation of a vendor’s performance regarding specific criteria such as quality or timeliness. The main objective of any vendor evaluation is to ensure a company’s product and service quality. This objective correlates with constraints such as the constant drive to bring down costs and maintain a competitive advantage. Information gathered from a company’s own experience with a vendor is perhaps the most reliable source for managing and improving existing vendor relations.

Any technology vendor doing business with Videocon must please two constituents, says Farhan Khan, General Manager-IT, Videocon India. A vendor has to satisfy him by, living up to service level agreements. Additionally a vendor must “be accessible across our operations. It’s one thing to make the CIO happy and it’s another to make customer happy.”

Setting the criteria

Criteria for selecting the best IT vendor can be a daunting task by itself. It begins with the IT manager using his own domain knowledge and following technology trends as well as experience from projects executed in the past. With this, the task of evaluating and selecting tech vendors and system integrators (SIs) becomes that much easier. Having said that let us examine some key steps adopted by IT managers and CIOs, and learn from their experience.

T G Dhandapani, CIO, TVS Motor, said, “We use matrices such as capability, flexibility, sustainability of the tech vendor and map it up with cost for selecting vendors. Suitability, adaptability, sustainability, simplicity and cost are the criteria used for evaluation. We then benchmark them on different parameters including success stories. We ask for a reference site from product companies so that we can talk to those customers and get valuable feedback.”

According to Nanaiah M C, General Manager—Systems and Processes, Kemwell Pvt Ltd., a year back he studied the computing requirement at Kemwell looking into both business and technical requirements of the company for its Swedish plant (it was a Pfizer plant) and found that Pfizer was running 3-4 ERP systems with different modules without proper integration on 5-6 Unix systems. “We looked at the number of users, database size and number of transactions to be supported and mapped the hardware requirement for present and future growth (supporting 170 users with a turnover business of Rs 200 crores) and arrived at the decision of going with IBM AIX servers. Since we had already had a fantastic experience working with SAP ERP at Kemwell India and had experience in implementing it, we thought of SAP for our Swedish plant as well,” he said. Additionally prior experience with vendors and global support including localization and availability in the market also play a significant role in deciding on a vendor and system integrator for a technology implementation. “We also look at the vendors’ product technology roadmap,” he added.

B G Shenoy, Head-Finance and IT, MTR Foods Ltd (a fully owned subsidiary of Orkla), said, “Learning from past IT implementations comes in handy for any CIO working on a new IT project. To stay focused on your business needs, you must develop and gather internal consensus on the requirements you expect a vendor to meet. Get approval on the requirements from all parties before you sit down with vendors. In doing so, you ensure that you have considered the critical factors, everyone agrees on the mission, and that the team is prepared to judge each vendor effectively. In addition, by disclosing these requirements to your vendors, you are in a position to direct the presentations and sales focus to the actual requirements of your company.”

Shenoy recommended, “Business can take the help of external consultants who can conduct a scoping study for sizing the IT infrastructure requirement and give feedback on the same. Alternatively, if budgets permit you to engage IT consultants such as Wipro Infotech, PwC, and KMPG who have proven experience in scoping, projects and execution you can hire an IT consultant. Additionally you should leverage your existing business partner [hardware/software or SIs] and establish a long term relationship as this helps drastically reduce the cost [hardware/software].” MTR Foods has not changed its SI, L&T Infotech since it engaged the SI back in 2003 for implementing SAP R/3. “One should not change the SI or technology partners just for the sake of change. We have been using SAP R/3 on Sun Solaris for the past five years and it is giving us 100% uptime, the OS is stable and the combination supports any number of transactions. So why disturb it?” said Shenoy.

Key criteria that most CIOs follow

• Pricing as it relates to value for money as well as the vendor’s flexibility in bargaining.
• Innovation is important, not just vis-à-vis few features but in terms of fresh approaches to solving problems. How quickly and creatively does the vendor respond to mishaps involving its product?
• Reliability and scalability are vital.

Studying a vendor’s performance

Unlike technology evaluation, selecting vendors is a tricky task because you are doing strategic IT planning for the next 3-5 years taking into consideration future growth, application and IT support. Hence, a business should choose a tech vendor with a clear roadmap for at least 3-5 years. Additionally a company should also study the financial health of the tech vendors as this would give it solid knowledge about how its prospective partner is doing, the kind of customers it is acquiring, and whether it has sufficient reserves to conduct R&D [to execute its roadmap] and more.

Khan explained, “When selecting a technology vendor there are a couple of parameters that you must consider. To begin with, look at some basic parameters to qualify vendors including scalability and seriousness. We study the vendor’s performance [financial data], roadmap for support, the profile of the vendor’s existing customers, vertical experience, customer satisfaction and best practices and certification for global governance, risk and compliance to name a few. Another important aspect is to ask the vendor for a ‘Proof of Concept’ so that you can see how your business applications and support infrastructure are likely to perform on different workloads and number of transactions. This would further help in selecting the right hardware platform and OS.”

Satish Syal, CIO, NIIT, suggested, “The tech evaluation for vendors and SIs should focus on business alignment, pain areas and benchmark business alignment with IT. Having external consultants can significantly cut down the time that you need to implement technologies. He however cautions that, at times, some consultants tend to be biased toward a particular technology or vendor and for this reason it is advisable to get a second opinion and use your own experience.”

Getting professional help

Over the past 18 months, the market has undergone radical changes in the way businesses evaluate technologies and vendors. Many businesses are taking professional help from consulting firms for end-to-end technology evaluation as well as vendor and SI selection. Ramanath L, General Manager-Consulting, Wipro Infotech, said, “Customers are taking the help of professional consulting firms to plan their business technology implementation. We have emerged as the end-to-end consulting firm in technology wherein we provide a complete IT strategy and roadmap for large businesses. We are handling 15 projects at any given point of time and these are typically of 2-3 months duration. Typically depending upon the size of the organization, large businesses are spending about Rs 20 lakhs to get their IT strategy and road-map done by an external consultant like us.”

Wipro Infotech is working on an interesting project, he continued, where it is working on a project named ‘Unique ID’ for the Ministry for Planning wherein all residents of India will have a unique identification number across India and this will be included in the 2011 census. “Similarly, another project that we are doing is for government employee pension wherein a small portion of the pension amount will be invested in equities and the project scope is to manage, monitor the funds and agencies,” added Ramanath.

Shenoy said, “We have an IT committee that consists of HCL Infinity, Wipro Infotech and Grover Infotech that recommends on projects. Having a distinguished panel on board helps us remain focused and prevent deviation from our business plans.”

Surendra Ramaiah, Practice Head-IT Strategy & Governance, Wipro Infotech talked about critical areas of IT implementation and vendor evaluation strategies saying that business requirements and future growth are planned for at least three years at a time and it has become a business exercise wherein business issues will be addressed through IT. Typically, a four-phase approach is taken in technology evaluation and end-to-end consulting services for IT projects—requirement analysis and RFR preparation; evolution of vendors, products and solutions; program management and quality assurance as well as review and monitoring of performance. [Please see the slide: Approach for technology selection]

Phase 1 [Refer box : Approach for Phase 1] has four parameters:

• Setting the objective encompasses identifying the team, both from IT and business, charting out the detailed project plan and kicking things off.
• IT enablement and gap analysis for business functions. This involves functional heads, senior management and top management, and key users. Identify infrastructure, resources for application and systems.
• Define each business process, detailed functional requirements from the technology/product solution; prepare a RFP [Request for Proposal] for relevant solution/hardware/software.
• Prepare a roadmap for implementation, time line, costs and resources.

In phase 2 [Refer box : RFP evaluation and selection] the RFP is raised and the top four or five vendors are identified. After that, the team analyzes proposals, shortlists vendors based on RFPs in terms of completeness of products and evaluates vendor performance. After this, the vendor makes a product demo to business process owners, the IT team and the consultants. The team also visits sites of reference customers, takes their feedback and compiles a report.

The backbone of an implementation

Nanaiah said, “We preferred Siemens because of their experience in executing big projects both globally and in India. Siemens also played a significant role in sizing the SAP ERP system, server hardware, messaging, OS and bandwidth requirements. Since they have solid vertical experience and the pharmaceutical industry happens to be a vertical in which they have such expertise, they can recommend the IT systems that are required to run ERP and messaging to support our five-year growth plan. In fact, they [Siemens] have a team in Sweden as well. Once we [IT team] select the vendor and system integrator, we justify the selection to the top management and the functional heads by having a joint meeting.”

Khan reiterated, “SIs are vital back-bone for any technology implementation and much of your success in implementing an IT project lies with them. For Videocon, our SI helped in sizing the SAP production server and did this based on the current number of transactions and scope to add more users and transactions over a period of five years, sizing the databases [over a 5 year period] and helped benchmark servers for the next three years with each server supporting at least 700 users.”

He cautioned, “A business should be demanding when it comes to extracting work from the SI. You should not give the SI full freedom during an implementation. Rather, tie them down with checks, controls, and periodic reviews during the entire implementation period. If you fail to do this, there would be incremental cost in implementation and a delay in the project [going live]. This could result in the costs going up and lead to loss of opportunity. I have personally witnessed a project delayed by 60-70 days because of lack of planning and review in a case where 15 days flat should have been enough. There are certain activities [during implementation] that can be simultaneously performed to shrink the implementation period.”

SLAs as a basis for negotiation

A critical project overrun can lead to escalation in project costs, opportunity loss or deviation from business goals. In such a case, you must prepare a timeline and impose a penalty on over-running projects. According to Gartner, an evaluation of solution provider audit involves more than just a one-time check for problems. Instead, it is a continuous improvement process. A performance audit is one of the ongoing measurement tools used by both the vendor and the client to measure the performance of activities against goals. For example if somebody has a goal of help desk calls being answered within 30 seconds 95% of the time, and that is not being achieved, then both parties should investigate or a penalty can be imposed. The service receiver and service provider would analyze why they are unable to meet the goal, determine the root cause, and recommend a fix to the problem. After implementing a recommendation, the need for a SLA arises.

When properly implemented, SLAs are an extremely powerful tool for evaluating performance. SLAs list the services and deliverables expected from an outsourcer, identify metrics for evaluating whether performance meets expectations, and use these metrics to define initial performance standards. Typical categories of metrics for SLA deliverables include volume of work effort, responsiveness, quality, and cost and efficiency. You choose specific metrics within these categories based on the type of work involved. Since SLAs are metrics based, they provide an objective report card on current and historical performance. Once you identify a problem by using SLA-derived metrics, you can quickly determine if subsequent corrections were effective.

Shenoy explained, “IT application integration is integral to the business and is useful if you have a strict SLA so that the vendor does not deviate from the contract. It is something like having a cap on them. You can use a SLA to impose a penalty if your business is fully depended on IT systems. At the same time, you can give something back if performance proves satisfactory. For example, we give them a ‘bonus’ if they complete the implementation [to our satisfaction] ahead of the time allocated to them. The bonus is in the form of an incentive as it acts as a motivating factor.”

Khan added that you could use SLA-based performance management for vendors and SIs constructively. This is all the more so, if you happen to outsource your IT infrastructure management to a third party. You can apply SLAs in a SLA based IT management service, on applications and on data management. It is important to know where to start these SLAs and get clarity on the process.

Working with vendors can be a challenging task for most businesses. If you execute the evaluation with a simple but methodical approach, the success of the tech implementation goes up by that much. So plan your business growth, map it up with technology, evaluate vendors and SIs thoroughly on various parameters and benchmark their solutions, prepare a time-line, stick to it and impose SLAs to execute the implementation.

akhtar.pasha@expressindia.com