Configuration Testing is a type of software testing, where we test whether the software works with the possible minimal configuration of hardware and performs with its normal efficiency.
Configuration testing is done to measure whether the software is able to perform in extreme situations also. For example, the software may be installed on a system with less hardware compatibility, or slow internet connection, or the absence of required addressable memory.
Now, in such cases, the objective of the testing is that the software is expected to run at its normal efficiency even when resources are not meeting all the requirements for the software.
There are two kinds of configuration testing, one is software configuration testing and the other is hardware configuration testing. We shall focus more on software configuration testing but also discuss hardware configuration testing.
Below diagram will make our idea clear:
As shown in the above diagram, the nodes, the database, the addressable memory, the network as well as the OS are all the configurations that the software should be configured with.
What problems may occur if configuration testing is not done?
- The software may produce bad results when there is any lack of peripherals or any resource gets suddenly disconnected. This is sometimes normal, that all the configurated elements may not remain connected to the software, and it's the responsibility of the software itself to handle those kinds of scenarios.
- The software may even break down or stop working completely. For example, software used for air-ticket booking may be interrupted due to lack of internet connection, but it has to retain its cookies once the connection is recovered, the user may proceed without being redirected to the starting page again.
- The software may not work if the operating system on which it is installed is changed. Therefore, testing for platform independencies is also a part of configuration testing.
- There is software that is installed in a distributed system, and so the software is designed to work collaboratively with all other nodes connected to it. Now, in such cases, if one node fails, the software installed on all other nodes must be able to work normally and also balance the system.
Methods of Penetration Testing
Here we shall see what are the different requirements which are to be fulfilled before the initiation of the configuration testing. They are below:
- The testing of all the resources such as hardware or software which will be connected to the software for carrying out the configuration testing. This is done to ensure that, all the entities are working properly individually, and thus while testing also, they would perform well.
- Since all the configurations cannot be tested at once. Therefore, they are listed based on their priority, and then the configuration testing process is initiated. This is done so that the most important configurations should work fine, while some others being ignorable.
- Checking all the connections. While all the connected hardware or the software may be performing well when tested individually, but lack of interconnection between them will provide a biased result or a wrong result which is not expected.
- Before initiation of the test, all the configuring entities which are malfunctioning in any sense must be removed.
- For keeping track of all the created scenarios of configuration testing, there is a matrix created. This matrix will helps track the configurations which have been tested and also the configurations which are the most and the least important. This matrix also helps in prioritizing the configurations and its scenarios.
Need For Active Testing
Below are the goals of configuration testing:
- To use an optimal configuration set-up and see whether the software performs well. This is the minimum-configuration point for the software.
- To measure the efficiency, performance, speed, etc of the software by intentionally modifying the hardware resources or sometimes misplacing the resources or connecting bad hardware.
- To validate the data output by the system or the software when the software is installed in a distributed system and all the parts are not well connected through a good internet connection.
- To see whether with the less configuration also the software is able to produce results, which are either normally accurate or almost accurate.
- To see and measure the stability of the system under the replacements of hardware, which means changing the configuration while the software is in running mode.
- To ensure that, geographically challenged locations also, do not hamper the stability of the system, which is totally dependent on the configurations of the hardware.
- To ensure that the errors and bugs can be reported and resolved when there is a change in configuration.
How to Perform Compliance Testing
In software configuration testing, the main concern is to check what are additional software or the operating system needed for the normal work of the software under test.
This is not only not the software that runs on the hardware directly, but it also runs on a designated operating system. And the hardware also runs on a kernel. The kernel actually runs on the hardware directly. Therefore, the software has to be configured with all other needed software to perform.
Let us take an example, of how an application software may be dependent on many other software to perform its operation.
Consider a CCTV application installed in a machine which integrated with IoT devices. Now the motive is to track CCTV footage and take appropriate action based on the real-world scenario.
For this to happen, we need a sensor node operating system installed on the cameras to run the camera as per the user's requirement, which is itself software. Next, we require the operating system which is also software. Subsequently, we need other software which will run on the user's smart devices.
Next, for the IoT devices also, software will be required. Therefore it can be seen that the main software which is installed on our system independent of so many other software.
This need for installation for different software for the main software to run is called software configuration. Now to carry out the test, all this software should be considered as configuration software.
So, in this testing, it is checked whether the configured software is performing normally in collaboration with the configuration software.
The below image will be the idea clear about what are the different software entities connected to the software.
What to be assured in software configuration testing?
There are many things that are to be assured in software configuration testing. This can be better explained with the help of an example:
Let us take an example where an e-commerce website is connected to a major data server and also a merchant website or payment gateway.
Now, there may be cases when the e-commerce web application cannot connect to the payment gateway. In that case, also, the user should be prompted to some option to check out with a purchase. This is the maintainability of a property, that failure of a software configuration should not let down the whole application process.
Therefore to conclude, the main purpose of software configuration testing, is about checking all the installation required and all the different versions of the plug-ins and add-ons that the software needs while test or run.
Defect in Testing
In hardware configuration testing, the main concern is to check what are the different types of hardware which are necessary for the software to run normally.
Hardware configuration testing has some major challenges because the hardware is limited elements are not expandable in any sense within a shorter period of time.
It is very clear that the software runs on the hardware. And the hardware resources required for the execution of the software may not be the only ones, but there can be collaboration or connections between different hardware to carry out a single task by the software.
Below diagram will make the idea about hardware configuration more clear:
Let us take an example, which will make the idea of hardware configuration more clear.
Consider, the same example of CCTV application software which is installed in our system. In our previous example, we have seen how the different hardwares uses different softwares. Now, in this case, we shall the hardwares itself.
So in our example, we CCTV cameras, different IoT devices, memory, etc.
Consider the scenario:
A place catches fire, and the cameras send real-time pictures of the fire.
Next, the heat sensors which are integrated with IoT will send its data to the software admin.
Next, the humidity sensor also senses the humidity and sends the data to the software admin.
If the admin now takes any actions then the robotic arm installed inside the building will be activated and will start to spray water to control the fire.
In this example, we have how different hardware is involved to make the administrator's software working.
Consider a different scenario:
All the sensors are working properly and are sending real-time data to the administrator. Now consider, the cache on the side of the robot arm is filled, and spooling cannot be carried out properly. In such cases, the administrator will be sending data for initiating actions, but the robot arm will ignore it.
This will happen because the instruction already in the cache is not removed and new data are not reaching the micro-controller for which the arm did not act as per the new instructions.
This is a memory address failure that could not be addressed by the administrator's software. A solution would be sending an interrupt signal which would halt the robot arm for a moment and the cache could be overwritten by the new information and restarted.
Therefore the purpose of the hardware configuration testing is to see whether the software is able to perform if there is a lack of hardware elements in the system.
Basis Path Testing
In order to perform a configuration test, whether software configuration test or hardware configuration test, we should create some scenarios where we remove some elements from the configuration or modify some elements.
Below we shall see how with a few steps we can perform the software configuration testing:
Let's take an example of an e-commerce web application. Here first we configure the application with the pre-requisites and test the application under normal conditions and record the performance. Then start modifying the configurations.
So, first, let start with software configuration. Let the software is performing in its recommended browser. For example Google Chrome. Now the application is made to run without its recommended plugins. After that, the application is opened using an outdated browser and the performance is tested.
This way, the application software has to face a lot of challenges with software compatibility.
Now let us see, about the hardware configuration. Let us take the CCTV application software itself. We remove a sensor from the experimental site and the other sensor is automatically activated and retains the data sending property through IoT.
Similarly, the robot arm for the water sprayer may not work for power failure, and so the other arm for CO2 release should be activated.
This way, the application software has to face a lot of challenges with hardware compatibility.
Configuration testing is a software testing type in which the configurations for the software, are tested. In this testing, it is checked what are the minimal configurations are required for the software.
Configuration testing is important, as the software will be used in a different environment. Some environments may be well equipped with all the resources that the software required, but in some remote areas and geographically challenged places, there may not be the availability of resources.
Configuration testing is one of the most important testings because here, all the resource-scenario is tested which reduces the risk of software failure in the future simply for different or lack of configurations.
When we speak about configurations, it means, network, Operating system, primary and secondary memory, servers, etc.