Chapter 4: Engaging and Evaluating Vendors

Where the Rubber Meets the Road

Now that you’ve considered your business needs, created a project plan, and captured your specific requirements, you’re ready to start evaluating vendor solutions. Your team should focus on your needs and not necessarily the capabilities that each vendor may want to emphasize. It’s important not to get enamored with bells and whistles that don’t directly help your team and company meet the stated requirements.

More importantly, remember that you’re reviewing more than a software solution. You’re looking at each vendor’s complete set of services and approach to not only implement software but to help your company fuel adoption, gain optimal benefits, and continue to be successful for years.

Look for Transparency

This is worth repeating: There’s no perfect software solution. There’s no single system that will meet every one of your requirements in exactly the way you want. Your goal is to look for the best fit: the vendor solution and approach that comes closest to meeting your requirements.

The best vendors understand the power and benefits that come from having more transparent dealings and are not afraid to “just say no” when they can’t meet a particular need.

Warning: If any vendor says “yes” too often or to almost every question—consider it a red flag. They’re either misleading you, or they’re setting you up for a costly custom-coding project that may lead to a long implementation or, worse yet, failed deployment.

Software Evaluation Is Easier Than Ever

Before Software as a Solution (SaaS) became popular, it was more difficult to evaluate vendor software solutions. Buyers often had to check out long (and infrequently updated) analysts’ reports and weed out bias based on which vendors paid for good reviews. And, gone are the days when you must sit through multiple, long, and tedious sales engagements with onsite meetings and demos.

Not only are cloud-based SaaS solutions easy to use and deploy—they’re also easier to evaluate. For this guide, we assume you are looking at SaaS solutions but almost all outlined here applies to on-premises solutions as well.

Here are six simple steps you should take to evaluate software solutions.

1. Divide and conquer

As project manager, you don’t need to do all the heavy lifting. You can delegate to committee team members to help conduct the vendor evaluations. Using your requirements document, each assigned person can review and rank vendor capabilities.

2. Start with customer reviews

There may be dozens of vendors that provide solutions to your challenges. You can do web searches for top solutions, but we recommend going directly to independent customer review sites to search for solutions by product category. Just like Yelp can help you quickly zero in on the best restaurants or Amazon can help you find the best consumer product, customer review sites like G2 and TrustRadius are designed to help you quickly compare business-to-business software offerings.When referring to customer reviews, consider how many reviews are positive and how many are negative. If there are only a few positive or negative, then you may not place much weight on those reviews. However, a lot of positive or negative reviews is a good indicator regarding the vendor’s solution and how it performs.

To make the reviews easy and consistent, include additional high-level criteria for quicker assessment as well (see Vendor Customer Reviews template, also available in the software search project template).

Once you’ve narrowed the field to the top three or four solutions, you’re ready to do a deeper dive.

3. Research the vendor’s website

Before you invest time talking to vendors, make sure there are no deal-breakers that disqualify a given vendor from consideration based on your online evaluation. As an example, look for industries served or customer success stories that show the vendor has experience solving issues for companies like yours. For instance, Arena is designed to help companies with physical, discreet products comprised of hardware, software, and electronics. We’re not a good fit for, say, an oil refinery.

Armed with the feedback from customer reviews and overall vendor rankings, you need to get firsthand information from each vendor. You should start with a review of each vendor’s website material, product videos, and other resources provided.

4. Engage the vendor

There’s only so much information you can gather online, so now it’s time to engage the vendor’s sales team to get an in-depth understanding of each solution. You can contact them via phone, but it’s often easiest to make the request via online chat or “get demo” directly on the vendor website.

To make discovery and demo calls the most efficient and effective, you want to:

• Include your entire software evaluation team to ensure each team member can review the solution capabilities that affect their team
• Invite your executive sponsor
• Provide vendors in advance with a general statement of your business needs and environment as well as specifically what you would like to accomplish in the first meeting (and any subsequent). While vendors will probably confirm this information, it saves you time because the vendors are going to be more focused.
• Have the vendor clearly articulate what will and won’t happen in a session so everyone attends with the correct expectations and a known agenda. Will there be a demo at this meeting or next? Is this meeting the right place for a deep dive on features, technical details, or pricing?
• Record the meeting for anyone who cannot participate from your team. If the vendor set up the call, ask for the recording—you won’t have to relay information and possibly lose some of the nuanced details in the process.

As you meet with vendors, work through your requirements  to score each solution. Remember that you may need to update your requirements or the weighting as you learn about what is possible and how solutions meet needs. Go back to your business needs  when it gets confusing to ensure your requirements support the business needs.

5. Conduct vendor reference checks

Ask the vendor to provide two or three customer references that share a similar business model to yours (e.g., similar size, industry, product offerings). Interviewing references will help you determine whether the solution is truly the right fit for your organization.

It is important to consider the role of the references in implementing and using the technology. Ideally, the contacts should be power users who were part of the software selection committee, use the system as part of their daily job function, and understand the processes that the software should support.

Key questions to ask the vendor references:

• Sales process

Were you part of the software selection committee?

What drove your company to invest in the solution?

Why did you choose this solution over the others?

Did you have to acquire additional solutions, services, and/or training after the initial purchase?

Did you have to justify the purchase to stakeholders? If so, how did you build a financial case?

• Implementation process

Did the vendor provide implementation services or was it handled by a third party?

Was the implementation a fixed price or was it billed according to time and material?

What were the resource and time investments needed to implement the solution?

How did the software vendor handle any hiccups or surprises that occurred along the way?

Did you use any of the vendor’s training courses or materials as part of the implementation? If so, how would you rate these offerings?

Did the implementation live up to expectations set in the sales process?

• Solution capabilities

How often do you personally use the solution? Have you used similar systems in the past?

What do you like most about the solution?

Describe user adoption at your company

How does the system’s security model support your needs?

Do you feel confident the system will easily scale with your business as it evolves?

• Customer education and support

What has been your experience with the vendor’s handling of software issues, questions, or concerns?

How are upgrades performed? What IT resources (if any) are needed?

What level of communication do you have with the vendor regarding software enhancements or
upcoming releases?

Do you have access to training materials or educational resources (best practices, tech sessions, webinars, etc.)?

• Results

How many employees and business partners are currently using the system? What was the level of
effort to onboard them?

Have you observed any key performance indicator (KPI) improvements since implementing the system?

Does the solution meet or exceed your expectations?

6. Determine cost to configure and deploy

Once you’ve determined the vendor’s solution is a good fit, you’ll want to gather specific pricing information to help you determine not just the cost of the software, but the cost of the services and ongoing support and maintenance. Knowing the total cost of ownership (TCO) is important, so ask pointed questions to make sure you’re comparing apples to apples across vendors.

• Software

Find out the cost of the software: Is it based on users and, if so, are there different user roles that are priced accordingly?

Based on your demo and requirements, make sure you understand what software modules are included in the quote.

What are the options for payment? Most SaaS offerings provide monthly costs, but bill annually. See if there’s a discount for entering into a two- or three-year agreement. Most vendors will offer a compelling discount to do so and you’re not going to implement an enterprise system for only a year, so take advantage of a multiyear price agreement.

Are there any costs to upgrade to new versions or releases of the software? How are those deployed and at what frequency?

• Implementation

What resources are needed from the vendor and from your team? How much time is necessary to configure, test, and go live?

How are these services quoted—fixed-price packages, custom scoped statement of work, or time and materials?

• Training

What is the training approach? Is it a train-the-trainer? Is it live or recorded? How many people should go through training and at what cost? (You should also consider how much training is needed, which relates to overall usability of solutions.)

• Adoption and scalability
  Consider what happens AFTER you go live. Many companies implement software solutions based on current best practices, specific modules purchased, and existing business processes. However, no company stands still. To compete, you’re constantly changing your business processes to shrink overall development and time to market.

Ask each vendor whether they have a plan and process to help your company evolve and get the most from your application investment. As an example, check out how Arena looks at the entire customer lifecycle.

• Customer support

SaaS offerings normally include customer support in the cost of the software subscription itself, but you should confirm that is the case. Does the vendor have tiers of support or limitations to support provided as part of the SaaS subscription fee? Make sure you understand the normal business hours for engaging support whether this is done via email, online portals, and/or with live calls.

If the software is not SaaS, you may have additional annual maintenance fees and upgrade services to consider.

Justification

Once you’ve met with all the vendors and received quotes and service details, you can summarize this information to compare costs and implementation details. The next chapter offers some best practices to help you make a compelling case to senior leadership based on the value and benefits you expect to gain by using the software.