In Partnership with NORMADOC
Your purchasing center for standards regulations & technical and legal books

Managing Software System Vendors

Coster, Andy
3/25/2015

1. Overview.

Managing software system vendors can bring success to a software development program, by adding key skills and experience to the resource available to implement a development program, or even to support developed software through its use.  However, if poor attention is paid to the track record of the vendor, to the definition of contract requirements, and to establishing milestones and monitoring arrangements, the success can easily become a failure.  To tackle these important aspects of vendor management, a multi-step process can be employed:

2. Vendor selection

2.1 - Define scope of vendor relationship.

The vendor relationship could be a strategic long term one for your organization or just a “stop-gap” to get you over a resource shortage.  The method of working could be to YOUR processes, techniques and tools, or allow whole parts of the program to be managed and implemented by the vendor working to their own processes.  Enforcing your own processes will generally mean consistency of development, but the vendor may have better processes and tools and their experience could be based on a specific way of working that would add value to the development.  The scope of vendor relationship needs confirming before you move to the next step, since it will determine many detailed aspects of the Request for Quotation (RFQ).

2.2 - Prepare RFQ

The RFQ summarizes the scope and style of relationship and identifies known requirements for the development (or support) program.  It should specify timescale, delivery milestones, review activities, acceptance criteria and anticipated resources required.  It is possible to just specify the requirements for the software or support activities and this would fit a program that includes all or most of the program to be outsourced.  Collaborative programs however, require more thought and attention to reviews and management.  As a part of an RFQ consider the criteria in Section 4 that you will use for evaluation of tenders.  This can be used as a set of headings in the RFQ or more likely a questionnaire to accompany the RFQ that forms part of the evaluation of tenders.  The RFQ is sent to any organization you identify as having the capabilities (in principle) to be the software system vendor, especially any company you have (good) experience with, or has been recommended.  Company Internet sites will help to confirm the credentials for a vendor you are considering doing business with.  This can help with item 1 in section 4 especially.  The RFQ should also state the timescale for receipt of tenders and how questions are to be answered, e.g., by post, email, bidders conference, etc.

2.3 - Evaluate tenders

Once the tenders have been received, they should be sorted into a preference order.  To do this again the answered section 4 criteria can be used with some form of scoring per item, such as 1 point for conformance or satisfactory reply.  The questions could also be graded by importance with more points scored for those items critical to the program under consideration.  Either way you are looking for the best tender that is a fit for the scope, style and fit for the program.  It is likely there will be more than one organization that is a good fit, so a further evaluation may be necessary, including interviews or further questioning of specific aspects of the program.

2.4 - Select Vendor and agree Contractual terms

Once a vendor or vendors is identified, the selection of a preferred vendor is made, keeping other conformance vendors in dialogue, until a contractual agreement is confirmed with the preferred vendor.  At this point it is important to define what the vendor is to do, by when and for how much.  This may be different than what was originally asked for in the RFQ since it will reflect discussions with vendors and details that may not have been considered in the original RFQ.  Legal assistance is crucial to ensure a good basis for the program is defined.  The contract should also specify deliverables, milestones, review mechanism and acceptance criteria.  A change control process is fundamental for fixed price contracts, but even for a time and materials contract some way of reviewing progress and identifying cost increases is necessary to prevent costs running over the identified budget.  Some simplification is possible for resource hire contracts where control is retained in-house.

3. Vendor management

3.1 - Establish milestones, reporting, deliverables, risks and issues.

The contract should establish milestones and deliverables.  These need adding to a program plan, if not already done, and the first program management meeting held with the vendor.  At the meeting, formats for reporting, risks and issues formats should be documented.  A program organization chart or RACI (responsibilities, accountable, consulted and informed) matrix should be drawn up for the combined program team.

3.2 - Agree meetings schedule and periodicity

Also at this first meeting define a meetings schedule, scope and periodicity.  The main meeting will be a regular review of the program, but there may be additional senior management or governance review meetings to plan as well.

3.3 - Define audit plan (use Evaluation criteria for audit scope)

As a part of program planning, identify the points at which audits of the vendor will take place.  To qualify the organization in the RFQ step you may have already conducted an audit.  If however, you just used the answered evaluation criteria (section 4), it would be appropriate to conduct an audit as early as possible after contract award.  Other audits should be planned to suit key milestones or for example on an annual basis for longer term contracts.  Once the contract is in place the evaluation criteria and hence the topics for checking at audit, may need changing to reflect the scope of work.

3.4 - Set up steering committee to review progress and issues (Internally)

Internally it is good practice to set up a governance arrangement with key personal who will oversee the completion of the contract.  This could be frequent as the program starts and slow as the program gets underway, or set at regular intervals around the regular program management review meetings.

3.5 - Establish acceptance criteria

During the set-up of the contract, acceptance criteria should be defined, but once the contract starts this need developing through a defined acceptance process.  Any testing can be derived from the vendors internal testing, or developed separately from contract (technical) requirements.  An acceptance process should include; the scope of acceptance activities, the people and resources needed to conduct any testing, the definition of the tests themselves and a mechanism for treating any concerns or issues raised during acceptance.

3.6 - Implement contract (based on 1-5)

During the implementation of the contract there will be reviews and reporting, issues arising and risks managed.  Any unclear requirements may need change control applied.  Information about performance should be analyzed including from deliveries (timeliness, quality, and cost), audit recommendations, issues and risks.  Governance should aim to monitor the program against cost, time and quality, while keeping vendor relationships on an amicable level.

3.7 - Arrange orders and payments to vendor

Once a delivery or deliveries have been accepted there remains arranging payment to the vendor.  It is important that the payment is arranged after acceptance, unless you have agreed something else in the contract.

4. Vendor evaluation criteria

The following criteria can be used as a checklist for vendor evaluation and works both as a set of questions to ask in an RFQ and for a question set to be used during an on-site audit.

Software life cycle/ practices used by Vendor

Requirements definition and management

Design practices and documentation

Design review practices

Coding practices, standards and review mechanisms

Testing practices and tools usage including defect analysis

Configuration management practices and tool usage

Project Management approach, risk and issue management

Support and documentation arrangements

Communication and contract arrangements

Note:  These are the areas for evaluation, some of which may not be relevant for a specific development program, so some degree of tailoring is inevitable.

5.  Help from Standards

This approach uses ISO 9001 Quality System Requirements standard and the specific guidelines in ISO/IEC 90003 – Guide to using ISO 9001 for Software.   The latter draws heavily on ISO/IEC/IEEE 12207 Software Life Cycle processes to interpret ISO 9001 in a software process context. 

This article has glossed over a number of processes that are part of ISO/IEC/IEEE 12207 – Software life cycle processes, and in the guidelines of ISO/IEC 90003, concentrating on the main elements of vendor management.

These standards and related standards checklists available from SEPT provide a more complete coverage of the vendor management and should be consulted for a more stringent approach.  Any additional concerns identified for your program can be added to the evaluation criteria checklist.

In addition SEPT has defined a specific vendor version of the ISO/IEC 90003 checklist which can be used to augment the evaluation criteria defined in section 4.