Mike Kelly’s blog

Cem and I have announced that we are suspending activity on the Open Certification project to focus our efforts on the AST Black-Box Software Testing (BBST) courses and working to develop a skills-based certification based on that material.

The original Open Certification project was focused on providing an open alternative to question-based certification exams, focusing the following advantages over current industry certifications:
1. The large pool of questions will be public, with references. They will form a study guide.
2. The pool is not derived from a single (antiquated) view of software testing. Different people with different viewpoints can add their own questions to the pool. If they have well-documented questions/answers, the questions will be accepted and can be included in a customizable exam.
3. The exam can be run any time, anywhere. Instead of relying on a certificate, an employer can ask a candidate to retake the test and then discuss the candidate’s answers with her. The discussion will be more informative than any number of multiple-false answers.
4. The exam is free.

The open certification was intended to serve as a bridge between the current certifications and skill-based exams that might someday be available. AST’s current implementation of the BBST courses provide a balance between quizzes and exams along with interactive reviews of debate and work-products by peers and instructors. The current implementation of the course provides a skills-based evaluation.

We want to focus our efforts on the efforts that best support our original goals, and we feel that the skills-based evaluations that take place in the BBST courses are the best current alternative to existing certifications. In support of that focus, we are doing the following:
• Transitioning the existing Open Certification question server technology to support the BBST courses
• Transitioning the existing Open Certification vision for creating a pool of open and available content for study material to the AST Basic Concepts SIG. The Basic Concepts SIG is an effort to create a reference for testing terminology structured in the format of the Oxford English Dictionary or the Blacks Law Dictionary and ask, what are the many ways that this word is used and what are good examples of each usage? We want to mirror the field in its diversity, rather than impose a false uniformity.
• Transition existing energy for question development to developing questions and course materials in support of the AST implementation of the BBST courses.

If you have any questions about the Open Certification project, please contact Cem or I at kaner@kaner.com and mike@michaeldkelly.com.

I have a recommendation from an additional application tour from Cem Kaner.

“Testers should be touring the market context as well as the product itself. For example, a lot of customer expectations are best discovered from regulations, competitive products, magazine articles or books about this class of product.”

I like it.

I’m trying to figure out a new mnemonic that works with the new letter “M.” I’m thinking FCC CUTS VIDS could become FCC VICDUM TSS. Perhaps I can remember that some fictitious company “TSS” was a victim of the FCC.

I guess I’ll have to keep thinking about it.

The third and final article in my SearchSoftwareQuality.com Testing for Performance series posted. The complete series can be found here:

• Testing for performance, part 1: Assess the problem space
• Testing for performance, part 2: Build out the test assets
• Testing for performance, part 3: Provide information

“Instead of focusing on training for a complex task, how can you make the task easier to perform?” -Influencer

Among other things, I’ve been leading a performance test team for the last couple years now. It’s been an uphill battle trying to find and develop the right talent, tame the performance testing environments, and deliver consistent results for the application teams we support. I now think I have one of the best groups of performance testing talent in the Midwest, but there is still so much we don’t know! It’s not for lack of trying, but the problem with performance testing (like anything else) is that you can never know it all: databases, protocols, mainframes, web servers, application servers, load balancers, driver upgrades, data center moves, AJAX, programming languages, performance test tools, performance testing methodology, changing user populations, coordinating deployments and monitoring, and oh yea, application functionality that just won’t stay the same from month to month.

Even for the best and the brightest, the problem of learning becomes overwhelming.

Last year I read Influencer: The power to change anything. It wasn’t as good as Crucial Conversations (by the same authors), but it was still a good read. The quote at the start of this post really hit home for me. It got me thinking about the learning problem for complex tasks. How can I structure the work to make it easier, instead of structuring the training to address the most difficult tasks? It’s a great question. It challenges you to recognize that sometimes, we don’t simplify the things that we do, simply because we already know how to do them. Then sometimes go about training others in what we know how to do, without ever asking if that’s the best way to do it.

Tool vendors understand this principle (I think). They develop tools that make it easier to perform our tasks. If a tool is well designed, it takes something complex and simplifies it. Many of the tools I’ve seen for performance testing, web service testing, and exploratory testing do this. I also think checklists (formal, heuristic, or taxonomies), simple scripting (Ruby, macros, or AutoIT), and simple lean process principles can play a large role in simplifying the work to make it easier.

On a cautionary note, I’m not saying better tools are a substitute for training. I remember Goranka Bjedov once saying, “A fool with a tool is a faster fool.” I’m not suggesting you don’t need to continue to train. I am suggesting that when you think about the never ending problem on training people for complex tasks (which testing is) that simplifying the tasks can be one way to help cope with the complexity.

As we continue to grow and learn as a team, I’m planning to remain focused not only on how we can improve our training but also on understanding how we can change the work to make some of the more complex tasks easier to perform. Right now I’m big on checklists, I’ve always been big on scripting (I like to go back to my roots), and I think we can do more if we start looking at what tools we use, how we manage the work, and what processes we currently use. (Note, I’m not talking about the documented processes, but the actual processes - how the work happens today.)

How do you deal with this on your teams? How do you make complex tasks easier to perform?

Another long day at the office, a crashed hard-drive at home, the wrong version of Windows Vista media (who knew you had to order the 64-bit version after you opened the version you purchased?), and estimates for a new driveway stuck in the door when you get home… It’s all ok now. I’m smiling thanks to Geordie Keitt’s new song, Black-Box Testing.

Mr Keitt, thank you vey much!