Posts in Testing Techniques
Visualize XML then Blink Test
XML format is a very popular data container. Programs read, generate, convert, and exchange data in XML format.
XML data chunks can be uploaded from application's GUI, or can be generated by a client part and then transmitted to a server. XML data input can be effectively used when an application has unstable GUI or has no GUI at all.

[sourcecode language="XML"]
<pre><?xml version="1.0" encoding="utf-8"?>
<transactions>
<debit> <amount>10.0</amount></debit>
<credit> <amount>55.0</amount> </credit>
<debit> <amount>120.0</amount> </debit>
<debit> <amount>25.0</amount> </debit>
<credit> <amount>5.0</amount> </credit>
<credil> <amount>15.0</amount> </credil>
<credit> <amount>15.0</amount> </credit>
<debit> <amount>60.0</amount> </debit>
<debit> <amount>50.05</amount> </debit>
<debit> <amount>75.0</amount> </debit>
</transactions></pre>
[/sourcecode]

...But, in testing, XML data verification task means time-consuming process of going through countless, similar-looking text lines, with a high chance to get bored and miss an inconsistency.

Sample Task


Verify “transactions” xml file presented above.
Only “debit” or “credit” records are valid. Any other entries must be located and reported.
Additionally, identify and report all debit records with debit amount greater than 50.00.

As you can see, even going through just 10 records takes a significant time and might become really boring task.
With a 100 of records it's pretty much a brain-dead work to do.

How can we transform it to re-enable brain-powered testing?


Blink Testing technique, as described by James Bach, would be a perfect fit to engage rapid pattern recognition capabilities of a human brain. However, applying it directly to a textual source of an XML file may lack of efficiency due to complexity of a pattern.

Transforming XML text into a web-page looking table (HTML table) with inconsistencies color-coded, gives an instant productivity boost for blink testing.

transactions

 

XML visualization technique


XML transformation, presented above, is performed with help of XSL (Extensible Stylesheet Language) script.
Creation of XSL scripts does not require installing any IDE - you can use just a text editor, like Notepad. You would need minimal programming experience to start creating your own scripts, and you can easily modify existing templates.

XML source sample and its visualization were taken from my article "XML verification example".
You can also find there XSL source code template and links to other XML/XSL related articles.
Identifying boundaries
Boundary value analysis is one of the most popularized testing techniques out there, and rightly so. It yields results. Most people think of a simple example when they think of boundary value analysis; like a field with a maximum character limit of 20 characters and visible character limit of 10 characters. We know that the field has three explicit boundaries: 0, 10, and 20 characters.

What's often more interesting to me, are the other boundaries related to that field. For example, it's likely that whatever value I enter is going to get stored somewhere. The variable where it's stored has boundaries based on it's datatype. If it's stored to a database, the field has boundaries. If it's stored to a file, the input and output programs that read and write the value expect certain boundaries. If it's ever rendered on the screen, it's rendered based on certain assumptions about it's boundaries. And certainly any processing done using the value includes boundaries (whether the programmer was thinking about them or not).

All of this leads to some interesting analysis of what we really mean when we say we're doing boundary value analysis. We're really looking at all of those various places where a boundary might exist. We're also looking at the potential interations of those boundaries with other variables within the system, and often we won't know what those are right away. Recently, I tried to illustrate some of the complexities of this in an example I did using Microsoft Word. In that example, I tired to model the boundaries related to bullets and numbering.

Here are some of the high-level things I think about when doing this type of analysis:

  • What boundaries can I see (explicit)?

  • What boundaries can't I see (implicit)?

  • What boundaries exist around persistence?

  • What boundaries exist around business processes and interaction?

  • What boundaries are shared with other variables?

  • What boundaries can be modified or configured (meaning they're variable, not static)?

  • What boundaries exist in relation to time (get reset, race conditions, etc...)?

  • What external boundaries influence this boundary (like the size of a disk or size of a piece of printed paper)?

Look for opportunities to shortcut a process
When I'm testing applications, I keep an eye out for opportunities to shortcut a process. For example, if you got to book a hotel room on Expedia, you very likely do the following:

  • Enter an arrival date and end date

  • Enter a city or address

  • Change the default number of rooms and/or travelers


When you click search, you get a set of results back and all of them will have some indication or price and availability. I suspect that this is the "normal" process for booking on Expedia. However, you can also "repeat a trip" using Expedia. When you repeat a trip from your past itineraries, it defaults the hotel selection to the same one from the trip. You just select new dates.

Recently, I did a normal search for a hotel, and Expedia told me that the hotel was unavailable for my dates. When I repeated a previous trip for the same hotel, suddenly those dates were available and I could book. I found that interesting. Apparently, there are different rules that fire in each process flow.

When I'm evaluating what to test in an application, I look for those opportunities where I can shortcut one process with another. It's not uncommon to find differences in rules, available data, or messaging inconsistencies when doing so.
Changing the testing culture in your organization
Many of you may have already watched the Google video about testing on the toilet, so forgive me if I bore you, but I truly found this video inspirational.

Its a story on changing the testing culture within an organization.

The key points that struck me where:

  • The testers had a high profile stakeholder who championed and explored their cause.

  • They didn't try to sell testing themselves, they looked for volunteers from all over organization to come up with ideas to change the culture

  • They didn't try to apply a 'one approach fix all' to all groups. Instead new staff were trained up during orientation, and existing staff were trained through free talks

  • When one approach didn't work, they changed it and tried something different. They never gave up.


There is much more in this entertaining video here's link to watch it yourself. Its about 3o mins in length.
Trick for clarifying a test charter
Sometimes when I ask someone what their test charter is, I get a paragraph in response. That's not bad, but I find that it often leads to a poorly understood scope definition, which leads to lack of focus while testing, which leads to a charter that runs way too long or feels unproductive. I have a trick I use to help simplify the mission of the charter when this happens.

Try using the following template:
"My mission is to test <insert risk here> for <insert coverage here>."

Some examples:

  • My mission is to test for various boundary errors for Microsoft Word's bullets and numbering feature.

  • My mission is to test for accurate error messaging pop-ups for Ford Motor Vehicle's Build and Price website.

  • My mission is to test for SQL injection vulnerabilities for application login and administration screens.

  • etc...


You might then still use the original paragraph to help detail out the charter, but getting a clear and concise mission helps me better estimate how much time I'll need to test, and maintain better focus while testing.