Software Testing

Software Testing
Being a Software Test Professional, you must know a brief history of Software Engineering. Software Testing comes into picture in every phase of Software Engineering.

The software industry has evolved through 4 eras, 50’s –60’s, mid 60’s –late 70’s, mid 70’s- mid 80’s, and mid 80’s-present. Each era has its own distinctive characteristics, but over the years the software’s have increased in size and complexity. Several problems are common to almost all of the eras and are discussed below.


The Software Crisis dates back to the 1960’s when the primary reasons for this situation were less than acceptable software engineering practices. In the early stages of software there was a lot of interest in computers, a lot of code written but no established standards. Then in early 70’s a lot of computer programs started failing and people lost confidence and thus an industry crisis was declared. Various reasons leading to the crisis included:

- Hardware advances outpacing the ability to build software for this hardware.
- The ability to build in pace with the demands.
- Increasing dependency on software’s
- Struggle to build reliable and high quality software
- Poor design and inadequate resources.


This crisis though identified in the early years, exists to date and we have examples of software failures around the world. Software is basically considered a failure if the project is terminated because of costs or overrun schedules, if the project has experienced overruns in excess of 50% of the original or if the software results in client lawsuits. Some examples of failures include failure of Air traffic control systems, failure of medical software, and failure in telecommunication software. The primary reason for these failures other than those mentioned above is due to bad software engineering practices adopted. Some of the worst software practices include:

- No historical software-measurement data.
- Rejection of accurate cost estimates.
- Failure to use automated estimating and planning tools.
- Excessive, irrational schedule pressure and creep in user requirements.
- Failure to monitor progress and to perform risk management.
- Failure to use design reviews and code inspections.

To avoid these failures and thus improve the record, what is needed is a better understanding of the process, better estimation techniques for cost time and quality measures. But the question is, what is a process? Process transform inputs to outputs i.e. a product. A software process is a set of activities, methods and practices involving transformation that people use to develop and maintain software.

At present a large number of problems exist due to a chaotic software process and the occasional success depends on individual efforts. Therefore to be able to deliver successful software projects, a focus on the process is essential since a focus on the product alone is likely to miss the scalability issues, and improvements in the existing system. This focus would help in the predictability of outcomes, project trends, and project characteristics.

The process that has been defined and adopted needs to be managed well and thus process management comes into play. Process management is concerned with the knowledge and management of the software process, its technical aspects and also ensures that the processes are being followed as expected and improvements are shown.

From this we conclude that a set of defined processes can possibly save us from software project failures. But it is nonetheless important to note that the process alone cannot help us avoid all the problems, because with varying circumstances the need varies and the process has to be adaptive to these varying needs. Importance needs to be given to the human aspect of software development since that alone can have a lot of impact on the results, and effective cost and time estimations may go totally waste if the human resources are not planned and managed effectively. Secondly, the reasons mentioned related to the software engineering principles may be resolved when the needs are correctly identified. Correct identification would then make it easier to identify the best practices that can be applied because one process that might be suitable for one organization may not be most suitable for another.

Therefore to make a successful product a combination of Process and Technicalities will be required under the umbrella of a well-defined process.

Having talked about the Software process overall, it is important to identify and relate the role software testing plays not only in producing quality software but also maneuvering the overall process.

The computer society defines testing as follows: “Testing -- A verification method that applies a controlled set of conditions and stimuli for the purpose of finding errors. This is the most desirable method of verifying the functional and performance requirements. Test results are documented proof that requirements were met and can be repeated. The resulting data can be reviewed by all concerned for confirmation of capabilities.”

There may be many definitions of software testing and many which appeal to us from time to time, but its best to start by defining.

Software Development Life-cycle

SDLC is a systematic and orderly approach to solving problems related to software systems or in other words, we can say it is a structure imposed on the development of a software product.
Domain Analysis: This phase is very important. The more knowledgeable you are about the domain, less the work required. Another objective of this phase is to make the analysts who will later try to elicit and gather the requirements from the area experts or professionals. So, this phase is an important prelude to extracting and gathering the requirements.
Requirement Analysis: The most important task in creating a software product is extracting the requirements. Customers typically know what they want, but not what software should do, while incomplete, ambiguous or contradictory requirements are recognized by skilled and experienced software engineers.

Scope Analysis: Once the requirements are gathered from the client, an analysis of the scope of the development should be determined and clearly stated. This is often called a scope document.

Specification: It is the task of precisely describing the software to be written, possibly in a rigorous way. In practice, most successful specifications are written to understand and fine-tune applications that were already well-developed, although safety-critical software systems are often carefully specified prior to application development. Specifications are most important for external interfaces that must remain stable. A good way to determine whether the specifications are sufficiently precise is to have a third party review the documents making sure that the requirements are logically sound.

Software Architecture/Design: Architecture is concerned with making sure the software system will meet the requirements of the product, as well as ensuring that future requirements can be addressed. The architecture step also addresses interfaces between the software system and other software products, as well as the underlying hardware or the host operating system.

Coding: The design must be translated into a machine-readable form. The code generation step performs this task. If the design is performed in a detailed manner, code generation can be accomplished without much complication. Programming tools like compilers, interpreters, debuggers etc... are used to generate the code. Different high level programming languages like C, C++, Pascal, Java are used for coding. With respect to the type of application, the right programming language is chosen.

Testing: Once the code is generated, the application / software testing begins. Different testing methodologies are available to unravel the bugs that were committed during the previous phases. Different testing tools and methodologies are already available. Some companies build their own testing tools that are tailor made for their own development operations.
Implementation: After the code is appropriately tested and approved, it is made available for business use i.e. moved into production environment.

Documentation: An important task is documenting the internal design of software for the purpose of future maintenance and enhancement.
Software Training and Support: As a part of the deployment phase, it is very important to have training classes for the software users. Users will have lots of questions and software problems which leads to the next phase of software.

Maintenance: Maintaining and enhancing software to cope with newly discovered problems or new requirements can take far more time than the initial development of the software.

Solutions for the software development problems

1.Requirements should be clear, complete, detailed, cohesive, attainable, testable and must be agreed by all players. Use prototypes to help nail down requirements. In 'agile'-type environments, continuous close coordination with customers/end-users is necessary.
2. Stick to initial requirements as much as possible. If changes are necessary, they should be adequately reflected in related schedule changes. If possible, work closely with customers to manage changes. This will provide them a higher comfort level with their requirements decisions and minimize excessive changes later on.
3. Allow realistic and enough time to plan, design, test, bug fix, re-test, change, and document in the given schedule.
4. Make extensive use of group communication tools - groupware, wiki's, bug-tracking tools and change management tools, intranet capabilities, etc. Insure that information/documentation is available and up-to-date, promote teamwork and cooperation. Continuousally communicate with end-users if possible to clarify expectations.
5. Start testing early, re-test after fixes or changes, plan for adequate time for testing and bug-fixing. 'Early' testing ideally includes unit testing by developers and built-in testing and diagnostic capabilities.

When requirements are changing continuously
what should be our approach when requirements are changing continuously.
Work with end users and management early on to understand how requirements might change, so that alternate test plans and strategies can be worked out in advance. It is helpful if the application's initial design allows for some adaptability, so that later changes do not require re-doing the whole work from scratch. Below are some more points that might help:

Devote appropriate effort to risk analysis of changes, in order to minimize regression-testing needs.
In the project's initial stages, allow for some extra time to commensurate with probable changes.
Balance the effort put into setting up automated testing with the expected effort required to redo them to deal with changes.
Design some flexibility into test cases; this is not easily done; the best bet is to minimize the detail in the test cases, or set up only higher-level generic-type test plans.
Focus less on detailed test plans and test cases and more on ad-hoc testing with an understanding of the added risk this entails.
Try to design some flexibility into automated test scripts.
Focus initial automated testing on application aspects that are most likely to remain unchanged.
Ensure management and client understand scheduling impacts, inherent risks and costs of significant requirements changes. Then let management or the customers decide if the changes are acceptable.

Quality
IEEE defines the Quality as “the degree to which a system, component, or process meets specified requirements, and customer or user needs or expectations”.
and ISO define the Quality as "the totality of features and characteristics of a product or service that bear on its ability to satisfy specified or implied needs".

In other words we can say that “Meeting Customer Needs” is the key in these definitions.
Basically, Quality is a subjective term. E.g. according to finance department cost can be the major quality factor and for users usability can be basic quality factor. However, we can say that a software is quality a software if it is reasonably bug free, delivered on time, maintainable, reusable, testable etc.
IEEE defines the Quality as “the degree to which a system, component, or process meets specified requirements, and customer or user needs or expectations”.
and ISO define the Quality as "the totality of features and characteristics of a product or service that bear on its ability to satisfy specified or implied needs".

In other words we can say that “Meeting Customer Needs” is the key in these definitions.
Basically, Quality is a subjective term. E.g. according to finance department cost can be the major quality factor and for users usability can be basic quality factor. However, we can say that a software is quality a software if it is reasonably bug free, delivered on time, maintainable, reusable, testable etc.

Software Quality Assurance (SQA) is the set of methods used to improve internal and external qualities. SQA aims at preventing, identifying and removing defects throughout the development cycle as early as possible, as such reducing test and maintenance costs.
Software Testing shows the lack of external quality i.e. correctness, efficiency, reliability. External quality displays the visible symptoms when there are issues, but the roots are invisible internal quality attributes like program structure, complexity, coupling, testability, reusability, readability, maintainability etc.

Remember, throwing a lot of testing at the product can improve your external quality to an extent that it is satisfactory for your customers, but at a significant cost.
At the end I’ve an interesting question for you -- whether "the quality of your software makes your customers happy" or "the customers define the quality of your software"?

This is a topic of big debate. Many software engineers have different views about this. Let’s leave comment on this topic so that we can see the views of different professionals.) is the set of methods used to improve internal and external qualities. SQA aims at preventing, identifying and removing defects throughout the development cycle as early as possible, as such reducing test and maintenance costs.

Software Testing shows the lack of external quality i.e. correctness, efficiency, reliability. External quality displays the visible symptoms when there are issues, but the roots are invisible internal quality attributes like program structure, complexity, coupling, testability, reusability, readability, maintainability etc.

Remember, throwing a lot of testing at the product can improve your external quality to an extent that it is satisfactory for your customers, but at a significant cost.
At the end I’ve an interesting question for you -- whether "the quality of your software makes your customers happy" or "the customers define the quality of your software"?
This is a topic of big debate. Many software engineers have different views about this. Let’s leave comment on this topic so that we can see the views of different professionals.