Bug in the wild - Google
This morning, while working on an article for data-driven testing with IBM Rational Functional Tester, I found a bug with the Google Web Search feature: Book Search.
The Book Search feature works like this, you enter the search text books about /something/ and the first Google result returned should be Book results for books about /something/.
In my article, I wanted to show an example of converting a record and playback script into a data-driven script. After recording a script that executed several of the Google Web Search features, I played it back to make sure it worked. It failed the first time I ran it. I then proceeded to test the feature by hand, clicking search repeatedly with the same search text in the box (my search text was books about software testing). It failed roughly fifty percent of the time.
My guess is the problem is a deployment problem, not a code problem. I'm guessing Google has a lot of servers running search and a couple of them just don't have the right code or the feature enabled. I don't know. Either way, this seems like something easy to catch. I found this with the lowest form of automated testing life - a record and playback script. I found it the first time I ran the script. I found it with only one value to test the feature. The total cost for me to create and run the test was about 15 minutes (for more then one feature). If I had been using an open source tool like Watir I might have found it even faster, without the additional cost of the enterprise tool.
Why didn't Google catch this? Even if it is a deployment problem, it's not like they can't test this feature once they've deployed. A test like this doesn't change any data in the system.
The Book Search feature works like this, you enter the search text books about /something/ and the first Google result returned should be Book results for books about /something/.
In my article, I wanted to show an example of converting a record and playback script into a data-driven script. After recording a script that executed several of the Google Web Search features, I played it back to make sure it worked. It failed the first time I ran it. I then proceeded to test the feature by hand, clicking search repeatedly with the same search text in the box (my search text was books about software testing). It failed roughly fifty percent of the time.
My guess is the problem is a deployment problem, not a code problem. I'm guessing Google has a lot of servers running search and a couple of them just don't have the right code or the feature enabled. I don't know. Either way, this seems like something easy to catch. I found this with the lowest form of automated testing life - a record and playback script. I found it the first time I ran the script. I found it with only one value to test the feature. The total cost for me to create and run the test was about 15 minutes (for more then one feature). If I had been using an open source tool like Watir I might have found it even faster, without the additional cost of the enterprise tool.
Why didn't Google catch this? Even if it is a deployment problem, it's not like they can't test this feature once they've deployed. A test like this doesn't change any data in the system.