If you're not familiar with them already, take some time to checkout the free cheat sheets on the Added Bytes website - they are a set of quick references for a variety of languages and technologies. I use the Regular Expression, Ruby, and SQL cheat sheets quite a bit. I'd like to use the Subversion and Rails sheets more, but just haven't had the opportunity.

I'm surprised I haven't posted about soapUI yet given how often I use it. It's one of the first web service testing tools I ever used, and I haven't had much need to look elsewhere. soapUI is great for both functional testing and performance testing. They also have a great service mocking feature.

I've never used the pro version, so I can't speak to those features or the support that comes with it. But I will tell you this, the first (and only) time I had an issue with the tool I contacted the developers and they had a build for me the next day with a fix. I was amazed at the turnaround. I'm sure not all fixes can happen that fast, but it still says something about their passion for the tool and the people that use it.

You can checkout the full feature set here: http://www.soapui.org/features.html

Stoyan Stefanov recently posted a fantastic set of slides on pitching better website performance. The slides look at different websites and how performance affected the core business. It's a cool idea. I think material like this is useful for a variety of reasons, but most of all because it provides insights into how I can better transform my technical work into something that talks about business value. Each slide is a mini case study on looking at a company and translating performance into a business driver important to them. As testers, we need to be good at that. The more exposure we get to materials like this, the easier it becomes to recognize similar opportunities in our own organizations.

Boundary value analysis is one of the most popularized testing techniques out there, and rightly so. It yields results. Most people think of a simple example when they think of boundary value analysis; like a field with a maximum character limit of 20 characters and visible character limit of 10 characters. We know that the field has three explicit boundaries: 0, 10, and 20 characters.

What's often more interesting to me, are the other boundaries related to that field. For example, it's likely that whatever value I enter is going to get stored somewhere. The variable where it's stored has boundaries based on it's datatype. If it's stored to a database, the field has boundaries. If it's stored to a file, the input and output programs that read and write the value expect certain boundaries. If it's ever rendered on the screen, it's rendered based on certain assumptions about it's boundaries. And certainly any processing done using the value includes boundaries (whether the programmer was thinking about them or not).

All of this leads to some interesting analysis of what we really mean when we say we're doing boundary value analysis. We're really looking at all of those various places where a boundary might exist. We're also looking at the potential interations of those boundaries with other variables within the system, and often we won't know what those are right away. Recently, I tried to illustrate some of the complexities of this in an example I did using Microsoft Word. In that example, I tired to model the boundaries related to bullets and numbering.

Here are some of the high-level things I think about when doing this type of analysis:

• What boundaries can I see (explicit)?


• What boundaries can't I see (implicit)?


• What boundaries exist around persistence?


• What boundaries exist around business processes and interaction?


• What boundaries are shared with other variables?


• What boundaries can be modified or configured (meaning they're variable, not static)?


• What boundaries exist in relation to time (get reset, race conditions, etc...)?


• What external boundaries influence this boundary (like the size of a disk or size of a piece of printed paper)?

With most project teams I've worked with, I've found that when I say the words "stress testing" people really hear the words "load testing." To me this is rather odd, but when I step back and think about it, I guess it makes sense. Most applications I've worked on have been web applications, and in that world load is often one of the biggest factors related to stress.

However, to help remind myself that there are other forms of application stress, I use the following checklist when I evaluate what stress testing I want to do. For each of these, I look at each piece of the architecture (desktop/browser, database, service/server, etc...) and in turn ask myself, "What would happen if...?"

• heavy load:
  
  
  
  • large number of users
  
  
  • large volumes of data
  
  
  • large number of processes
  
  
  • large number of transactions
  
  
  
  


• constrained resources
  
  
  
  • not enough memory, processing power, or disk space
  
  
  • not enough bandwidth
  
  
  • not enough queues (or even shallow queue depth)
  
  
  • not enough licenses
  
  
  • inability to access files, servers, or services
  
  
  
  


• unfriendly inputs
  
  
  
  • invalid file types and/or data types
  
  
  • fuzz testing
  
  
  • very large (or very small) inputs
  
  
  
  


• shared resources
  
  
  
  • crash or stress other applications in a shared server space or that use a shared device/service
  
  
  
  


• timing
  
  
  
  • constant or steady load for long periods of time (endurance)
  
  
  • intermittent loads with long pauses in between transactions (looking for timeouts or parts of the system that go to sleep or get deprioritized with inactivity)
  
  
  • concurrency (trying to force potential race conditions, file/database locking, etc...)