Demos, Tests and Databases

This might be too much for one entry, but here goes. The art of testing usually involves some form of data and some configuration of machines, operating systems and other items that constitute the 'system'. Over the years, one of the biggest challenges is to recreate that actual environment as the customer would experience it. And it is a thorn in the side of most test professionals. How does it relate to quality software?

It relates to quality in terms of quality are a measure of customer perception, in terms of real dollars spent and in terms of the relationship between testers and developers that is so important to efficient scheduling.

1. Hardware costs money, so the same hardware used to test the software is sometimes shared with the people doing demos. Usually, but not always these people are marketing people. Consider when it is for beta testing using customer data and multiple customers see it during demos.
2. It is common for the data to become hopelessly out of date in the test area. So that even though you have the hardware you need, you don't have the real data. So, one of two things happens. You report bugs, which quickly get bounced back by the developers, because you were using bad data. Or, you don't find any performance issues, because your data set doesn't accurately reflect the size of the data set.

You may have heard that live data has been used to test passports - particularly presidential candidates information. This is actually quite common activity. Many times I will be told that live data, live systems (with test data) are used, especially for performance testing. What is it doing to the performance of the actual system?

Or, what happens during a demo when competitors get to see your customers data? Or, what happens to the test schedule when you can't get the system you need? Or it got hopelessly reconfigured -- again, with invalid bug problems.

There are solutions to this -- never share systems between departments, set up rigorous procedures for sharing systems, create transparency with your customers if you are going to use their data -- get their permission and allow them to audit your use of it.