Firstly, look up ';Statistics';. That should put you on the right path.
If the thousands of test cases are for identical tests, then you basically count the number of each different answer and report that. You could also report the standard deviation (I'm not going to get into the theory here, but your calculator probably has the formula programmed into it, as well as all spreadsheets you might run into).
If all the test cases were different, for example, if you were testing a car and one test was for the quarter panel thickness, and another for the height of the tires, I'd record what was tested, and give the answer in a huge table. If there were several identical tests, I'd give the average and standard deviation for the answers to each set of identical tests.
Hopefully, that gave you the start of the answer. Depending on how far you wish to go with it, it is possible to take a course (or get a few degrees) in statistics. This type of problem regularly shows up in quality control, actuarial sciences, engineering, psychology, actually, it shows up just about everywhere.
Now, if you were trying to write up the actual tests, then it gets a little harder. Basically each test is carefully described as far as what it is testing, the procedure, expected results and how many times it must be performed under what conditions. If there are several different conditions, then we usually have a table of them in the procedure. That lets one procedure work for several cases.
On a complex problem, it can take weeks, or even months to adequately describe each and every test that must be performed. There is also a brancd of statistics called ';design of experiments'; which is used to help figure out how to put the tests together.
Along those lines, sometimes it is possible to reduce the number of test cases by testing some carefully chosen fraction of them. Looking up ';design of experiments'; and ';fractional factorial design'; would be a fun and interesting exercise.
Now, the interesting part here is that if you design the experiment correctly with a fractional factorial design, there is a correct way to analyze the data (often the programs which do the design also analyze the results for you) and you can get nearly as much information with fewer tests than if you did every single possible case.
I hope that gave you a start. There is a lot more to the science.How we will write test cases when thousand of test cases?
Well the first thing to do is see if you can reduce then number of test cases, and the post before mine takes that approach, actually as I look at it sort of does from the back end. If you can see a way to reduce the results into something smaller using that approach, then maybe you can be better at picking what tests to do in the first place.
But the truth is that if you are testing a complicated program or device you will have thousands of test cases. Depending on what kind of tests you are doing there might be ways of automating the tests.
If all that still doesn't apply, well there is hire/assign more people, hire/assign the right people, as in maybe you have people that know what tests need to be run, but aren't really fast on typing them up and formatting them. You might then get someone with that skill or maybe there might be some programs out there that can help you format test cases into some template. For sure you should have a template stating what to do and what to expect for a pass or fail.
No comments:
Post a Comment