Testing Tools: Different tools used for Testing

Depending on the size of the company, testing is done using various tools for each phase of testing.

Quality Management Tools:

Quality Center (9.2+) - Latest version available is 11. You can actually download a copy of it from HP's BTO software website. Most of the medium to big organizations do use HP's Quality Centre for writing and running tests
Note: For version 11 onwards this is more referenced as Application Life Cycle Management software - ALM
Microsoft Excel - for writing test cases

Microsoft Word - for writing test plans and test strategy document

Test Tools:
SOAP UI: for making web service requests and to verify the responses coming back
QTP: for automating the regression tests & create a regression suite that can be run continuously
Test Complete: for automating the regression suites
Load Runner: for load testing purposes
Win Runner: for automating the regression suites. Most of the places is now replaced by QTP
Fitnesse: to test individual methods or classes. Generally used to Unit test, or may be used for integration testing purposes. Mostly used by development in Agile development projects
Selenium: to perform integration or system test on web application using specific technologies. Can not run on all platforms/ applications. Mostly used in agile development projects

and so on..there's no end to this because of wide variety of projects and environment today's projects running on

Automation Tools - Different tools used for Automation

Automation is something that is becoming an important skill for any tester. If you have a minimum knowledge on how to automate an application & how to use the available automation tools would give a distinct quality for your profile.

Here in this blog, will try to explain what tools are in use in the present market and little bit about each of those tools.

GUI Layer Testing Tools:

Quick Test Professional (QTP) - by Hewlett-Packard (formerly owned by Mercury)

Test Complete (TC) - by Smart Bear (formerly Automated QA)

WinRunner - by Hewlett-Packard (formerly owned by Mercury)

IBM Rational Functional Tester - by IBM

Silk Test - by Borland

Note: This list contains only important or tools that are used by most of the companies. It doesn't include every tool that is in use.

Load/ Performance Testing Tools:

Load Runner - by Hewlett-Packard (formerly owned by Mercury)

IBM Rational performance Tester - by IBM

VisualStudio Load Test - Microsoft

Silk performer - by MicroFocus

GH Tester - by Greenhat

There's load of other load testing tools that you can find on internet.

References:

http://www.load-testing-tools.com/

http://www.testingfaqs.org/t-load.html

Test Cases: Plan/ Write/ Execute

Test case is the test documentation that a tester would write documenting the below:

• what functionality he's going to test, 
• what input he would require to give to execute this test, 
• in how many steps can he finish the step
• what is expected out of each step and finally document what is the Actual result of each step while executing.

Writing test cases is one of the typical responsibility that any tester would do all the time at his work. However writing test cases is done in chunks in different stages of SDLC. Test cases are planned, written and executed in different stages of application development. Test cases should be

Planned: when requirement documents are being prepared

Written: when requirement documents are signed off and when development is busy coding, should be peer reviewed, walk through'ed with all stake holders

Executed: when development is done with coding and when code is handed off to QA

What is a Test case Matrix:

Test cases could be written in either a spresdsheet or in Quality Centre like quality management tools. In most of the small companies, would go prepare a Test Case Matrix if they want save some money on these quality management tools.

Test case Matrix is nothing but a spreadsheet that has a list of tests and its results formatted in a column format for readability purposes. In some cases Test case matrix may have a set of requirements, set of test cases for each of those requirements and its results after the run, any defects if raised and a link to the test case running which caused that defect. A perfect relationship that one would see in Quality Centre like management tools can be replicated in Excel spreadsheet.

Example Template can be found here......

Testing Deliverables

When you perform testing, you deliver a quality product that is clean code (defect free code). However on top of this as QA team you would also deliver documents suggesting how did you go about testing, what all test cases you ran, are you happy with the application or not? & finally any lessons learned from this testing. Typically the documents expected to be delivered from QA are given below:

1. Manual Test case Matrix

2. Automation Test Scripts

3. Test Strategy/ Master Test Plan Document

4. Lessons Learnt Document

5. Sign-off Document

Note: These documents are stored in a file repository database so they can be referred to, if needed. Ex: Sometimes there's a bug found in production, so QA lead has a repository to go back to check for test cases that they wrote for testing that piece of functionality currently broke in production. If there were no tests written to test this particular scenario that failed in production, we will add a test to the regression test suite with High priority as this may repeat again

On top of this, you would be reporting the status of testing (may be in a weekly progress report call) using a document that you produce on the fly:

1. Defect Report

Note: Defect reports are generally used to track either test team or dev progress on the raised defects.

Testing Process - Steps Involved in Testing an Application

Whether it is waterfall or agile, typically testing starts right after development. If it is waterfall, testing will start after development is complete with coding. Where as in Agile, testing will start with only some x% of development is done. So obviously there's a lot of going back & forth in agile then waterfall. In both cases, to test an application a process needs to be followed. I tried to depict the process that a tester has to follow at different stages of development, in the following diagram.

Types of Testing

There's loads of different types of testing that're currently in use. Depending on the criticality of the application, various types of testing is performed on the application. But they can be classified in to 2 different types:

1. Functional Testing

2. Non-Functional Testing

1. Functional Testing: Testing an application or a system per the given requirements. In functional testing, tester would ensure that application behaves as per the requirements set in the requirement documents (System Req. Document, Software Req. Document, Business Req. Document etc). Functional testing comprises the following types of testing

• Unit Testing: Testing a piece of code. Could be a .net method or a java class or cobol/jcl program
• Integration Testing: Testing a set of programs or physical layers of application. or testing whether 2 or more programs work together or not?
• System Testing: Testing a full system. Here a tester would test all the modules of the system. Tester would make sure if all the programs/ application as a WHOLE works fine
• Regression Testing: Every time a new functionality is added, tester would need to make sure it doesn't effect the existing functionality. This type of testing is called Regression testing
• End-to-End Testing: Testing the full system flow. Meaning, testing from system-A thru system-z
• Ad-hoc Testing: Testing a one-off report or a job
• White box Testing: This is nothing but Unit/ Integration Testing. Testing the actual physical code. This is a developers job to perform white box testing
• Black-box Testing: Testing something without knowing what code lies inside it. This is nothing but System testing. This type of testing is done by QA.
• Accessibility Testing: Testing to make sure if your application is accessible by a variety of people with accessibility problems. Such as, visual impairments/ partially blind/ hearing impairments etc
• GUI Testing: Testing the user interface works per the requirements. This is to make sure things like layouts, forms, form controls, grid objects, buttons, text box, check box, labels etc are intact

2. Non-Functional Testing: Testing an application beyond the functionality. This is to test something other than what is written in the functional specification. This type of testing requires a different mind set to think out of the box as well. Non-Functional testing comprises the following types of testing

• Performance Testing: Testing an application to measure the performance by running scripts. Ex: Win-runner scripts can be created to simulate the transactions performed on an application & run to see the performance of the application meets the requirements set by the business.
• Load Testing: Testing an application to measure the maximum load that it can take. Same as performance testing, an automation engineer would create tests/ scripts to simulate the transactions performed on an application up to a certain level set by the business.
• Stress Testing: Testing an application beyond an operational capacity or level/ limit is called stress testing. Stress testing is mostly done until the application crashing point (or breaking point) to verify the stability of the application
• Fault Tolerance Testing: How an application would survive any real-time faults/ failures that could happen in production. Take database server down, network down, application server is down etc and run them back again to see if the application starts responding soon after the resources are available
• Disaster Recovery Testing: Testing an application's disaster recovery measures. A company may be using a system in India. If an earth quake in an ocean disrupted network, everything should be switched to backup/ contingency line to run business as usual. Testing such backup procedures could be called disaster recovery testing. This usually is a part of Operational testing
• Environment Testing: Testing the full environment where a given application runs. Example is testing - Database (creating, updating, deleting), Application servers (get it down, up, restart), Installing this full environmental changes just by looking at the documentation available etc
• Operational Testing: This is often referred to as Operational readiness testing as well. Testing to make sure everything is ready for operations team. This would include Fault tolerance testing, Disaster recovery testing, Environment testing, Installation testing etc
• Installation Testing: Testing the installation instructions set by the development. Just follow the step-by-step process listed in the documentation and make sure it doesn't miss anything. Every work item or step that need to be performed by a user to install a given application should be documented & well tested

QA Involvement in SDLC

Although QA acts & tries to run fast to complete testing at the very end phase of a development life cycle; QA involves from the beginning. As you can see form the below diagram, QA involves almost from the time when the business requirements are written. QA should provide their valuable input so as to improve the quality of the product. Any risks/ concerns/ suggestions should be raised before hand.

Involving QA from the beginning would give the team to prepare with a mind set that is required to test the product - WHICH IS THE MOST IMPORTANT THING, that should never be ignored

Testing Concepts - Phases of Development (SDLC)

What is Testing? When & where does testers involve in the whole process of development?

Testing is a verification process where an already developed product will be verified for any defects or faults. Testing is done on a product before its released for use by the customers. Testing is often referred to as "Quality Assurance". Depending on the industry type testing is done in different ways. However I would concentrate only on IT industry, here in this blog.

Tester should have an understanding of what process is followed in the industry & where does QA (Quality Assurance) gets involved in that process. Most industries follow a standard way of development. That standard is called SDLC. It refers to Software Development Life Cycle. You can find many articles on what is SDLC & how it works and so on. I'll try to explain it from a ground level, thinking that the user doesn't have basic understanding of testing process.

In a real world, this is how it works.

1. A shop owner who wants a systemic way of processing transactions in his shop. He knows only the scenarios that he wants to process using a system. Like a customer comes in to a shop, brings in goods, purchases at the till by paying for it using Cash/ CC/ DB/ Account etc

Shop Owner is CLIENT/ CUSTOMER

2. He calls for a tender process for converting these scenarios in to a System. Lets say a Company has been chosen for completing the work

This is Project Initiation - where a budget (estimated), time scale are agreed

3. Business Analysts/ Architects will speak to the customer/ client to get an understanding of all the possible scenarios that would ever occur in the shop. A document is written with all the notes taken from the client.

This is Project Planning

4. This Business requirements document is then used by implementation team to write System (and/or Software) requirements that are low-level requirements, that has information about how does system behave in a scenario given in the business requirements. And also information about how should the programs be written (pseudo programs etc) & how should they communicate to each other, all about integration of different programs/ modules and every little detail about the system

This is Project Design

5. Business, System & Software requirements would become the input for a team of developers, who can write programs to meet the criteria specified in the Design documents

This is called Implementation - as a part of this process developers should also make sure their programs work on their own (by writing unit tests) & work when clubbed together with other programs written by his team members (by writing integration tests)

6. Code is then built in to an exe (or something similar to that) file and given to Q.A. team for further testing & verifying that it works

This is called Verification/ Testing - as a part of this process QA team would test the system in many ways (Ex: End-to-End tests, Functional Tests, Non-Functional Tests, Regression tests, Negative tests, Installation tests, Operational tests etc)

7. If the code is doing what is supposed to do & if QA has certified that by signing off on the developed features of the system, then the system is ready for our shop owner to install & work away

This is called Production/ Release - where the system is handed off to the customer/ client

Below picture gives an understanding of how SDLC is formed from the above scenario.