Selecting COTS software under pressure (Forrester and Gartner)

This happens on many projects: everyone agrees that the core of a solution (e.g. workflow engine) will be purchased instead of built from scratch. This is an absolutely rational decision for most enterprise applications. The next question is which vendor to select.

The rational approach is to identify the core functional requirements and key quality attributes that the solution must exhibit. This is an absolute must; because without understanding what you need you will not be able to objectively judge your options. The criteria on which you should evaluate you COTS software candidates must be granular and quantifiable to the degree that makes sense. Each criteria element needs to be weighed against all other chosen consideration elements so that all weights add to a whole. E.g. In case of workflow COTS software the number of concurrent workflow processes that the solution must support has to be at least 400 and this criterion has a 15% importance in relation to other selection elements.

The next step is to conduct evaluation of the market space for available vendor solutions. A number of solutions will not meet the top selected criteria based on the marketing information from the vendor, but few will emerge as potential candidates. Since COTS software will be an integral part of the solution a targeted proof of concept should be implemented and results judged against the established criteria. The outcome of the test along with other evaluation criteria is used to make an objective and rational decision.

If a decision on the COTS software absolutely has to be made in a period of time that doesn’t allow a proof of concept, then you should buy the intelligence from a third party that has done COTS software evaluation for you. Both Gartner (Magic Quadrant) and Forrester (The Forrester Wave) provide analysis of various COTS products that fulfill a particular segment of business applications (e.g. BPM tools). Each vendor solution is evaluated against a wide range of criteria. These reports provide very valuable insight, but without objectively understanding the needs of the solution you’re architecting it’s not possible to fully utilize the insight provided by a third party.

In general a Gartner, Forrester, or another 3^rd party report should be used in any COTS software evaluation whether you’re pressed for time or not.

Constantin K.
Firebrand Architect™
www.SoftwareArchitectures.com

Criteria For Failure

Every project team asks themselves, implicitly or explicitly, what are our criteria for success? Depending on the range of the stakeholders involved you’ll get answers from the superficial (e.g. “users of all skill level can use software”) to the goal oriented (e.g. “identify the safest and most viable customers who need car insurance”). It’s easy to talk about success – especially non-quantifiable success.

The Firebrand Architect philosophy encourages you take a step further and pose a question: “What are our criteria for failure?” In other words – when we look back a year from now how would we know for sure we have failed to achieve our objective? Depending on your relationship with the group of the stakeholders you may have to be diplomatic as to how you pose the question.

Let’s work through an example. A project aims at providing insight into the inventory stored in many warehouses. The goal is to keep the inventory as low as possible while providing quick response to product demand. This project has a budget of $2,000,000 and will require a purchase of expensive COTS software ($500,000). A team of stakeholders will let the technical team decide which of the vendors to select, but know that a mature COTS software package has solved a similar problem in other organizations.

A year goes by, hardware has been purchased, COTS software installed and linked to a data warehouse, insight for a few categories of inventory can be obtained. Is this project a success or a failure?

One may say that since the software has been installed and connected to a data source in the production environment it’s a success. If that’s the only claim to fame, then this project is a failure. It doesn’t produce the insight that a business needs.

Using the same example as above, if a project produces some level of insight, but not all what was envisioned was it a failure or success? This is a harder question. If the paradigm and the model of operation through which the limited insight has been developed can be naturally expanded to gain insight in all envisioned categories, then the project is a success despite not delivering everything what was planned. If the paradigm doesn’t scale to cover additional level of insight, then the project is a failure.

Knowing the definition of success is important. Knowing the definition of failure is paramount.

Constantin K.
Firebrand Architect™
www.SoftwareArchitectures.com

Architecting With 3rd Party Components: "Trust, but verify"

A proverb: “Trust, but verify” should always resonate in software architect’s mind when a decision is made to use a component that the architect does not directly control. This component may be a COTS software piece or an enterprise library developed by someone else. This becomes especially important if that component is part of the project’s critical path (e.g. a communication infrastructure component).

For example, the enterprise may require every solution to use component Foo to obtain a security token. Or the enterprise may require component Bar to be invoked to pass through messages so that certain information can be collected for enterprise auditing purposes.

Unless you have previous experience architecting with that 3^rd party component take a safe path. Identify the elements of your solution that will rely on the 3^rd party component. Then thoroughly analyze the external component for each quality attribute requirement that is associated with the software architecture element that uses that component. If the documented performance, security, availability, and other quality attributes match the needs of your architecture proceed with caution.

Think about mitigation strategies (alternative implementation tactics) in case the 3^rd party component does not live up to the promised expectations. For example, can auditing functionality be done as a batch mode instead of real time? Can an alternative protocol be used for issuing security tokens?

As the implementation commences inspect the properties of the promised quality attributes with the intensity that is equal to the role that the 3^rd party component plays. If a 3^rd party component is part of the critical path of a project conduct intense quality attribute related tests prior to committing your architecture to it.

For designing with COTS software see this article.

Remember: “Trust, but verify”

Firebrand on duty: Constantin K.
www.SoftwareArchitectures.com