Quality

Software Quality

• Quality cannot be added as an afterthought.
• What is quality?
• Views of quality.

Commitment to quality pays off

• A landmark book “In Search of Excellence” identified key factors that set successful companies apart from less successful, commitment to quality was one of them.
• Success = profitability.
• This is not the only reason quality is important software pervades all aspects of life.

What is quality

• A principal objective of software engineering is to improve the quality of software products. But quality, like beauty is very much in the eye of the beholder.

A quality software product

• Users: Works and produces correct results, is available when needed.
• Testers: Compliance to requirements
• Customer: Is not too expensive.
• Marketing Manager: Looks good, has more features than the competitors product.
• Developers: Can be modified and enhanced to user needs.
• Quality is different things to different people. All are valid but they may not coincide.

Example

We have €100,000 to improve the quality of a product. How should it be divided between the following factors ?

• Ease of use.
• Performance (response time).
• Reliability .
• …

How does the answer change depending on the product ?

• An ATM.
• A language teaching software for children.
• A CASE tool.
• A computer game.
• A nuclear power station control software.

What is quality?

Components of quality

• From a measurement perspective we must be able to define quality in terms of specific software product attributes of interest to the user.
• That is we want to be able to measure the extent to which these attributes are present in our software products.
• We can then set targets for quality attributes in a measurable form

What we measure

• Processes – the collection of software-related activities.
• Products – any artefacts, deliverables or documents that result from a process activity.
• Resources – entities required by the software process.

Classification of attributes

• Internal Attributes
  – These can be measured by examining the object on its own, separate from its behaviour.
• External Attributes
  – These can be measured only with respect to how the object relates to its environment.

Components of measurement

What is Quality?

• Quality, simplistically, means that a product should meet its specification.
• This is problematical for software systems
  • Tension between customer quality requirements (efficiency, reliability, etc.) and developer quality requirements (maintainability, reusablity, etc.).
  • Some quality requirements are difficult to specify in an unambiguous way.
  • Software specifications are usually incomplete and often inconsistent. A software product may conform to its specification but users may not consider it to be of high-quality.

Software Fitness For Purpose

• The focus may be fitness for purpose rather than specification conformance.
  • Have programming and documentation standards been followed in the development process?
  • Has the software been properly tested?
  • Is the software sufficiently dependable to be put into use?
  • Is the performance of the software acceptable for normal use?
  • Is the software usable?
  • Is the software well-structured and understandable?

Software Quality Attributes

Quality Conflicts

• It is not possible for any system to be optimised for all of these attributes – for example, improving robustness may lead to loss of performance.
• The quality plan should therefore define the most important quality attributes for the software that is being developed.
• The plan should also include a definition of the quality assessment process, an agreed way of assessing whether some quality, such as maintainability or robustness, is present in the product.

Process and Product Quality

• The quality of a developed product is influenced by the quality of the production process.
• Process quality is particularly important in software development as some product quality attributes are hard to assessg. maintainability – need to use the software for a period of time.
• However, there is a very complex and poorly understood relationship between software processes and product quality.
  • The application of individual skills and experience is particularly important in software development;
  • External factors such as the novelty of an application or the need for an accelerated development schedule may impair product quality.

Process-based Quality

Process and Product Quality

• Process quality and product quality are closely related.
• A good process is usually required to produce a
  good product. Improving a process so that defects are avoided will lead to better products.
• For manufactured goods, process is the principal quality determinant.
• For design-based activity, other factors are also involved especially the capabilities of the designers

What does quality mean?

• A good definition of quality must let us measure quality in a meaningful way.
• Measurements will show if techniques really improve software, as well as how process quality affects product quality.
• We need to know how the quality we build in can affect the product’s use after delivery and if the investment of time and resources to assure high quality reap higher profits or larger market share.
• Is good software good business ?

Views of Software Quality

• The transcendental view sees quality as something that can be recognised but not defined.
• The user view sees quality as fitness for purpose.
• The manufacturing view sees quality as conformance to specification.
• The product view sees quality as tied to inherent characteristics of the product.
• The value-based view sees quality as dependent on the amount a customer is willing to pay for it.

Transcendental view

• Concerns innate excellence.
• The type of quality assessment we apply to novels, how do you express the qualities of a good book ? The practised reader develops a “good feeling” for this type of quality.
• So too with software the striven-for “recognition” is the transcendental definition of quality.

User view

• Concerns ‘fitness for use’.
• Whereas the transcendental view is ethereal, the user view is more concrete, grounded in product characteristics that meet the user’s needs.
• This view of quality evaluates the product in a task context and can thus be a highly personalised view.
  • Rreliability
  • Performance
  • Usability

Manufacturing View

• Concerns conformance to specification.
• The Manufacturing view focuses on product quality during production and after delivery.
  • Examines whether or not the product was constructed “right the first time,” in an effort to avoid the costs associated with rework during development and after delivery.
  • This process focus can lead to quality assessment that is virtually independent of the product itself.

Product View

• Whereas the user and manufacturing views examine the product from without, a product view of quality looks inside, considering the product’s inherent characteristics. Relates to attributes of the software.
• This approach is frequently adopted by software-metrics advocates, who assume that measuring and controlling internal product properties (internal quality indicators) will result in improved external product behaviour (quality in use).

Value-based view

• Deals with costs and profits. It concerns balancing time and cost against profit.
• Equating quality to what the customer is willing to pay for encourages trade-offs between cost and quality.

Agreement Is Needed

• Different views can be held by different groups involved in software development.
  • Customers or marketing groups typically have a user view, researchers a product view.
  • The production department a manufacturing view.
  • Software developers concentrate on the product and manufacturing view.
• If the difference in viewpoints is not made explicit, misunderstandings about quality created during project initiation are likely to resurface as (potentially) major problems during product acceptance.

Defects –based Quality Measures.

• Defect – A known error, fault or failure.
• A failure is something which is observed while the software is in operation.
• A fault is the part of a software document (code or other forms of documentation) that is the encoding of human error.
• Failures are caused by triggering one or more faults with specific inputs. Faults will only cause failures if they are triggered with a particular combination of inputs.
• A de facto standard measure of software quality is defect density.
• Two types of defect can be counted :
• Known defects that have been discovered through testing, inspection and other techniques;
• Latent defects that may be present in the system but have not been discovered.
• Defect density = number of known defects
                                       Product Size

Defect Density

• When using defect density for quality assurance purposes or to compare performance remember :
  • There is no consensus on what constitutes a defect. To make comparisons all parties must count the same way.
  • There is no consensus as to how to measure software size.
  • Defect density is derived from the process of finding defects. It may thus tell us more about this process than about the product quality itself.
• Even if the number of residual faults (those discovered after release) are known it can be hard to predict how the system will operate in practice. This is due to two key findings :
  • It is difficult to determine in advance the seriousness of a fault.
  • There is great variability in the way systems are used by different users, and users do not always use the system in the ways expected or intended. It is therefore difficult to predict which faults will lead to failures, or to predict which failures will occur often

Faults & Failures

• A small proportion of faults in a system can lead to most failures in a given time period; most faults in a system are benign and will not lead to failure in the same time period.
• It is possible to have a product with a large number of defects rarely failing.
• Finding and removing large numbers of faults may not improve reliability.

Reported Evidence

• Companies are reluctant to publish defect densities. Not withstanding the difficulty of determining the validity of figures there is some consensus.
• USA/Europe average is 5-10 per KLOC.
• Japan 4 per KLOC, may be misleading because only top companies reported.
• A defect density of below 2 per KLOC is considered good.

Other Defect-based Measures

Defect density is not the only useful defect-based quality measure :

Spoilage = time to fix post-release defects

cost to fix post-release defects x 100
total project cost

The drawback with using low defect rates as if they were synonymous with quality is that the presence of defects may not lead to subsequent system quality problems.

Software faults do not necessarily lead to software failures.

Modelling Software Quality

• Quality is a composite of many characteristics.
• Quality models describe the dependence between:
  • The quality component that we are interested in and the attributes that may be directly observed.
• Fixed quality model
  • McCall software quality model
  • ISO9126
• Define your own model
  • Appropriate quality attributes and suitable measures for their evaluation are decided together by developer and user.

McCall’s Quality Model

• Describes quality using a decompositional approach.
• Focus on the final product- usually executable code and identify key attributes of quality from the user’s perspective.
• Key attributes are called quality factors and are normally high-level external attributes –reliability, usability and maintainability.
• Model assumes quality factors are at too high a level to be meaningful or to be measured directly. These factors are further decomposed into lower level quality criteria

Quality Factors

• Correctness. The extent to which a program satisfies its specification and fulfills the customer’s mission objectives.
• Reliability. The extent to which a program can be expected to perform its intended function with required precision.
• The amount of computing resources and code required by a program to perform its function.
• The extent to which access to software or data by unauthorized persons can be controlled.
• The effort required to learn, operate, prepare input, and interpret output of a program.
• The effort required to locate and fix an error in a program. (This is a very limited definition.)
• The effort required to modify an operational program.
• Testability. The effort required to test a program to ensure that it performs its intended function.
• Portability Effort required to transfer the program from one hardware/software system environment to another
• Reusability Extent to which program (or parts) can be reused in other applications.
• Interoperability Effort required to couple one system to another

Factors that Affect Quality [McCall and Cavano1978]

Factors assess software from three distinct points of view:

• product operation (using it),
• product revision (changing it), and
• product transition (modifying it) to work in a different
  environment, i.e., “porting” it.

Quality attributes (McCall)

• Product operation
  • Correctness:      does it do what I want?
  • Reliability:         does it do it accurately all of the time?
  • Efficiency:         will it run on my hardware as well as it can?
  • Integrity:          is it secure?
  • Usability:          can I use it?
• Product revision
  • Maintainability:  can I fix it?
  • Testability:        can I test it?
  • Flexibility:         can I change it?
• Product transition
  • Portability:         will I be able to use it on another machine?
  • Reusability:        will I be able to reuse some of the software?
  • Interoperability: will I be able to interface it with another system?

Mc Call’s Quality Model

Mc Call’s Quality Model

• A quality factor represents a behavioral characteristic of the system.
• A quality criterion is an attribute of a quality factor that is related to software production and design.
• A quality metric is a measure that captures some aspect of a quality criterion.

What are software metrics?

• metric: a quantitative measure of the extent or degree to which software possesses and exhibits a certain characteristic, quality, property or attribute – software size
• unit of measure: a reference quantity in multiples or fractions in which any quantity of the same kind can be measured – KDSI;
• measurement: a number with an associated unit of measure which describes some aspect of software – 10,000 KDSI

McCall’s Quality Model

• A further level of decomposition is required in which the quality criteria are associated with a set of low-level directly measurable attributes

Factor           Reliability

Criteria         Error Tolerance

Metric           MTBF (mean time between failure)

Fixed Model Approach

• Assumes all important quality factors needed to monitor a project are the subset of those published in the model.
• To control and measure each factor, the model’s associated criteria and metrics are accepted and most importantly the proposed relationships among factors, criteria and metrics.
• Data collected is then used to determine the quality of the product.

ISO 9126

• Research the above quality model on the web.
• How does it define quality?
• What does it consider as quality factors?
• How does it compare to McCall’s Model?

Review Quality