The middle "boundary condition"
-
We all know when testing you try the minimum, maximum, something in the middle, one plus the max, one minus the minimum and maybe a couple interesting values.
When testing an administrative application, I ran into a critical failure at the system level when the domain name was exactly 26 characters long. The minimum was 3, the maximum (in this case) was 64 due to other system restrictions and I stumbled upon this problem after a previous version of the system had been tested and released containing this bug.
After investigating and discussing the bug with the developer, the system call was initially guessing and only allocating 26 characters, it would get a failure if the buffer wasn't big enough and from the failure code it would then know how much memory to allocate, but at that undocumented initial buffer guess boundary the initial conditional statements were a bit off, > rather than >=, so the system came to a crashing halt.
I have to say, as a tester, this type of thing is very frustrating. I really want to help my developers find the bugs before we release, but hidden magic numbers like this aren't usually found without exhaustive testing and we never have time for exhaustive testing.
-
Of course, you did add a test case to your unit testing so this will never happen again, right?
-
I was responsible for system integration testing and due to corporate politics, I couldn't add a unit test for the appropriate component. At the time, I doubt that it had *any* unit tests for the whole component.
I did the best I could, I added a system integration test for my component that overlapped with the test that the other component *should* have been doing from the start and included a note about this bug, but for all I know, the other developer could have just changed his magic number to 47.
-
this is the problem with black box testing. In a more complicated scenario there could be millions of possible magic numbers and the test script would have to run for a very long time to find such magic numbers, if the tester even suspected one existed.
Of course when the software goes public and a bug is reported from a customer they will be all over QA about it...
-
@snyd3282 said:
We all know when testing you try the minimum, maximum, something in the middle, one plus the max, one minus the minimum and maybe a couple interesting values.
When testing an administrative application, I ran into a critical failure at the system level when the domain name was exactly 26 characters long. The minimum was 3, the maximum (in this case) was 64 due to other system restrictions and I stumbled upon this problem after a previous version of the system had been tested and released containing this bug.
With a maximum of 64 characters, it seems a bit too close-fisted not to allocate 64 bytes in advance (or even 65, to catch those off-by-one bugs).
Other than that, without peeking into the source, it's impossible to tell which input data is "interesting". I once had to work on a program that had worked for months, but suddenly crashed on march 1st. As it turned out, the program internally converted a date to a string, then later the string back to a date; and it seems that somewhere in between there was a problem with Umlauts; the German word for march is "März".
-
@ammoQ said:
With a maximum of 64 characters, it seems a bit too close-fisted not to allocate 64 bytes in advance (or even 65, to catch those off-by-one bugs).
While I agree that saving a few bytes of memory doesn't justify starting with a too-small buffer. I don't think allocating an extra character (not byte, characters can take more than one of those) is the right way to go about avoiding off-by-one errors. You still have a bug, and the louder and noisier it blows up, the faster you can fix it. Preferably, even, before it makes it into production. You might need 65 characters if you're using C string format (NUL terminated).
Of course, I don't know what language this is. It might have built-in support for Unicode and strings, or include standard libraries for such support; in which case, why use a statically allocated character buffer anyway? It doesn't sound like an embedded system or one with severe resource restrictions, so you're probably better off using the tried-and-true code that's already written for such purposes. There won't be nearly so much chance for surprises like this, in that situation, and the semantics just seem clearer. Faster to write, easier to understand and maintain, and less bug prone; at the cost of a slightly larger footprint.
-
-
@joe.edwards@imaginuity.com said:
@snyd3282 said:
...The minimum was 3...
I guess w3.org is out of luck.
I think he meant the entire domain name: "w3.org" => 6 characters. Which makes me wonder if there is a three character domain: "a.b"
-
@savar said:
I think he meant the entire domain name: "w3.org" => 6 characters. Which makes me wonder if there is a three character domain: "a.b"
Not on the interweb, but there's nothing stopping you creating one locally.
-
@savar said:
@joe.edwards@imaginuity.com said:
@snyd3282 said:
...The minimum was 3...
I guess w3.org is out of luck.
I think he meant the entire domain name: "w3.org" => 6 characters. Which makes me wonder if there is a three character domain: "a.b"
The official TLD registry says no.
However, feel free to create your own alternate dns system.
-
That is yet another whole WTF. Many customers of ours did implement their own alternate internal dns systems.
-
In this case, the langueage was C and Unicode was a definate possibility. There could also be thousands of these calls in some cases so allocating the maximum space would be a waste.
