QA from Point 0

Lots of startups begin developing their products without the involvement of QA in the process. After a while, the time comes (usually when the product needs to be delivered to the customer) when a QA process becomes necessary. Together with the product and its release notes, the customer often demands documentation of the QA process (ATP, STP, STD).

At that point, the company will hire a QA leader, who may work alone at first but will often eventually expand the QA team. The leader must be capable of managing the QA system and implementing QA methodology in the product’s development process.

I would like to suggest some helpful points for the new QA leader who needs to build a QA process from point 0:

·         Learn the product
Depending on the product’s complexity, take 1-4 weeks to study the product. A QA leader should know the product better than the developers (he/she is often the only one who has all the pieces of the puzzle).
Read the user guide and official release notes.

Enlist the help of key people in the product chain to help you understand the product better (Product Owner, CTO, System Architect, Developers, Integration Engineer, Technical Writer and others).

·         Write a basic sanity scenario
Usually the best way to do this is by using the marketing team’s demo. The demo will provide a quick POC for the use of QA, and will inevitably be the first test you will run anyway.

·         Regression backlog

Produce a backlog from the regression. Find out which modules constitute the core of the product and start with that. Remember, it is better to have one module in the plan than two with no coverage.

·         Don't collect debts
Write tests for all the new features so that you will need to write regressions only for older features; otherwise the regression backlog will get bigger and bigger.

·         Be involved in the design
When you get involved in the design, you can prevent the occurrence of bugs. Then, when you prove your worth in this area, you can start talking about TDD .

·         Automation
Try to automate the basic flows of the product. This will make the sanity and regression processes much faster and easier and will free up time for you to explore and cover the new features.

If most of your product’s logic is on the server side (which is the case for most products), it is much faster and more stable to start automating the server side rather than the UI.
From my personal experience, although everyone agrees automation is necessary, in many companies the developer will try to keep the code to him/herself. Solving this matter, in my opinion, involves teaching development teams to trust and even depend on the QA testing process.

One small tip: If you find yourself in a situation like that, contrary to what I mentioned above, begin the automation from the UI. It can be used as a POC for you and will provide the way forward to the server side automation.

And if you decide to go with UI automation, try to identify objects by their IDs (if possible).

Testing Survey

I support, do you?
You want more info - click here.
You want to help - click here.
Subscribe to get notified when the survey goes live - click here.

JIRA Tips

Like other software companies, we use an issue tracker. There are lots of trackers out there and our company works with JIRA, one of the most common tools in use today. JIRA is modern and intuitive, and even offers an annual practical joke.

Due to recent requests by people from within and without my company to explain how I use JIRA in the testing environment and for the tracking of version status, I will offer a few tips and links that may help you start using JIRA like I do.

Filters and (JQL)

Filters are the basis for a lot of the analysis work you can do in JIRA. They are saved queries that can be run in basic mode (form based) or advanced mode (JQL).

JIRA has its own query language - JQL, don’t be afraid to use it. Its intuitive nature will be much appreciated, especially if you are familiar with different kinds of queries.

In addition, when you become familiar with the JQL mode, you’ll find it much easier and faster to use than the basic mode.

Options for filters include managing them, sharing them with others, keeping them private, cloning them (which can save lots of time), and more.

Regression

A useful checkbox field can be added to each bug to indicate if it is a regression. The quality policy in my company is not to approve a release if a regression exists and every regression must be corrected immediately.

Dashboards and Gadgets

In JIRA, you can create as many dashboards as you want and configure them to suit your requirements. You can also share your dashboards with others or keep them private.

o  “Recently Created Chart”

In my company, we use this chart together with the filter of all bugs found in the tracked version that have a priority higher than major or are marked with “regression”. I expect that as we move closer to the release date, the amount of active bugs will decrease and the columns will appear in green (meaning the bugs were fixed).

o  Tracking a risk area in your project

You can use the pie graph in JIRA to locate problems in the project. For example, you can create a pie graph with statistical information indicating regression. This will help you discover if your project has a problem with its work processes.

As another example, you can use a statistical graph for different system components to determine where most of the bugs in the system occur and why.

Hierarchical trees

Truth be told, this is one of the most important features that JIRA lacks. However, you can work around this by using epics and user stories. In addition, make sure bugs are linked to epics.

Although this may not provide a perfect solution, it does a good job of covering the issue and helps gather all the bugs in one place.

Another option is to use a Structure plugin.

Hierarchical trees

Truth be told, this is one of the most important features that JIRA lacks. However, you can work around this by using epics and user stories. In addition, make sure bugs are linked to epics.

Although this may not provide a perfect solution, it does a good job of covering the issue and helps gather all the bugs in one place.

Another option is to use a Structure plugin.

Plugins

Here are a few plugins you may want to check out:

o  Agile (older name: Green Hopper) - A superb tool for agile teams.

o  Capture (older name: Bonfire) - Great for exploratory testing and the latest word in software testing. If you use this tool, make sure to read about the special annotation in the notes .

o  Zephyr for JIRA - If you can live without hierarchical trees, you may find this tool useful when it comes to managing your manual tests.

Test managing tool within the budget limit

At the beginning of our project (as with other projects) we wrote and managed our test in Excel files. It was great at the beginning, when we had small test sets that were all run manually by the same person who represented the entire QA team.

As time passed and our test sets grew , the number of our test cases increased to more than 500, some of which were run manually and some of which were automated on different platforms. Our team grew as well and it became impossible to manage everything via  Excel files. Producing test documents such as ATP, STP and STR became a nightmare and the chaos grew in leaps and bounds.

We needed a test management tool for our project - a tool that would integrate with the different frameworks we already worked with and wouldn’t upset the entire system. This tool would help us control the version quality development and status and provide us with an option to communicate the relevant information to the stakeholders (managers, customers, others) within minutes.

We didn’t want to spend too much of our project’s budget on the tool, since we needed the budget for the development of automation and to support the  manpower that comprised our team. In addition,  if the tool was simple, we wouldn’t have to spend more of our budget on  training the tester to work with the new tool.

Our criteria included the following:

·      Write and organize test cases according to our module and testing phase.

·      Plan test sets and execute them.

·      Tractability between the requirement, tests and bugs.

·      Generate reports.

·      Support custom dashboards to control project quality.

·      Integration with JIRA as our bug tracker.

·      Integration with the different frameworks we were using (QTP, Telerik Test Studio, Soap UI, Jenkins, etc.).

·      And the bottom line: price.

We tried five management tools:

·      Quality Center

·      PractiTest

·      Zephyr

·      Zephyr for JIRA

·      Xstudio

View Evaluation Document

In the end, we decided to go with PractiTest. Besides the fair price and trivial functionality we were looking for, which constituted our basic criteria, we liked the idea of the Filters. This feature presented out-of-the-box thinking of how to organize and reuse test cases via hierarchical filters, instead of the classic hierarchical folders. The filter is a superb implementation of the Agile theory in the software testing world.

And there was also one more small advantage -  the satisfactory patriotic feeling I experienced after the purchase was approved, since being an Israeli,  I of course like to support our home-grown industries.

Communication via “The Triangle” (notes from a customer communications workshop)

Yesterday I attended a customer communications workshop conducted by Barry Katz . The main focus was on proper communication with clients, but since the audience included personnel with different roles in the same company and some of our customers are employed in the company itself, our discussions tended to gravitate towards intracompany communication.

Throughout the workshop, the issue of a project management triangle was raised over and over again - the need to communicate and find a balance among the three different forces and various stakeholders in a particular project.

We all know the famous project management triangle, where each vertex represents Scope, Time and Cost, with Quality in the middle. Each stakeholder pulls the vertex in his/her direction, while the QA team is smack in the middle, between the development, management and customer. During the workshop, we learned how to facilitate proper communication among the different branches.

Must it always be a win-lose situation? How do you parallel your expectations with those of the other side, and how do you enable each side to feel it got the most out of the situation (we may call this a win-win, but sometimes the idea is simply to make sure no one loses).

There is no magic to communication. It’s all about the way you think, which leads to the way you view a situation, feel about it, and how you act based on those thoughts and feelings. And after you act, there is the inevitable assessment of the results of your actions.

The key is to begin thinking like a winner, with self-respect and confidence, as well as respect and empathy for the other side.

In light of the above, here is a brief list of tips that may help create a more pleasant communication environment:

·      The obvious needs to be stated - Everyone knows quality is important, but does everyone know what the impact of an action will be on the quality of the product? Make sure that the issues that are obvious to you are stated and made clear to all stakeholders.

·      Propose options that let everyone feel in control - Explain the different options available for action and the results of each action. Help the other side feel it too has control over the situation without detracting from your own control. For example, as part of a QA team, you can provide several options for periods of testing and the expected result of each period. A shorter period of testing would have lower coverage with a bigger risk, while a longer period of testing would have a higher coverage with a lower risk. Using this example, you may want to explain Bach’s “good enough” principle  and decide together with the development team and management how your company defines “good enough”.

·      Motivate people through free will - Remember, we are all on the same side and we all want our product to be top quality. In an environment where we need to work together, asking someone to do something politely (even if it’s on their task list) is always better than ordering them to do it.

·      Motivate people through personal relationships - Try using more personal communication. As part of a QA team, it would be a good idea to maintain a good relationship with the development team, for example, and show the more human side of you when sharing small talk and coffee. During these friendly moments, you can also ask (not demand from) the developer if he/she would be willing to take a look at a bug you discovered. You will probably be pleasantly surprised at his/her willingness to help.

·      Maintain a quiet environment - Try lowering the decibels when it comes to bugs. Give the developer the peace of mind he/she needs when tackling a problem that needs to be solved.

Have any other good tips or comments about how to better communicate with development, management and customers? Please feel free to share them with us!

Notes from the SIGIT seminar: Exploring Exploratory Testing with Lee Copeland

After participating in the seminar of Lee Copeland on Exploratory Testing, which took place at the SIGIT conference on July 23, 2013, I pondered how it would be best to document my notes and decided in the end to divide the post into two parts. The first part will include new thoughts and ideas Copeland spoke about (he did not originate all of them), which I intend to adopt and pay more attention to in my daily work. The second part will point out the way I intend to implement these ideas.

Part 1: Exploratory cheese via improvisation

The main gist of exploratory testing is that the analysis, design and test execution are performed simultaneously, so that the exploratory testing coincides with agile testing. Copeland mentioned that the problem with the V Model is that we often start planning the test at a time when we know very little about the system.

To emphasize this point, Copeland introduced us to a short game, as James Bach suggested, known as “twenty questions”. After we finally figured out (using more than twenty questions) that Copeland was thinking about “cheese”, we all agreed that if we had written down our questions in advance, we would not have succeeded at all in divulging the answer.

This demonstration did the job of convincing us that exploratory testing was indeed a super concept.

Copeland, however, proceeded to warn us that there are some cons to exploratory testing:

·           Our daily experiences can cause us to be “blind” to our reality.

·           Exploratory testing may be a novel method, but it may not always be the way to go. Yes, it should be included in our toolbox, but we need to have other tools in that box as well.

In concluding this section, I would like to point out that Copeland asked us to associate exploratory testing with improvisation, recalling the top-rated TV show “Whose line is it anyway”.

Part 2: Schedule the session and do the test

(Note that since my team works with the Jira application, this part is based on working with that application. Please feel free to apply the information in this part to any other tool you are using.)

During the seminar, I especially liked the idea of scheduling exploratory testing, just as we do with other iterations. This idea will become a task in our Jira program (and may also be used to log out at the end of the process).

Copeland spoke of Bach’s idea of performing exploratory testing in sessions (SBTM). Jira employs a great tool for that called “Bonfire”. 

Due to my military service as an officer in the IDF, Copland and I happen to share a great love for mnemonics. He used several throughout the seminar, by I am determined to adopt two particular ones in my daily routines.

1.   SF DPT (San Francisco Depot) - make sure all items are covered.

For exploratory testing, it is a good idea to use a checklist of objectives that includes the following items: Structure, Functions, Data, Platform, Operations and Time. (Note that the boundaries between these items need not be so sharp.)

In Jira Bonfire, this session should be in an additional information section.

2.   During the session, use PROF mnemonics.

Past: What was done? Compile notes during the session, using the Bonfire extension to record what you have done.

Results: What were your findings? In Jira, defected and screen shots are automatically added to the Bonfire session if you opened the issues using the Bonfire extension.

Obstacles: What slowed down or blocked our session?

Forward thinking: Where do we go from here?

Feelings: How do we feel about what happened? Since I am a red-headed Israeli, the process will certainly touch my heart and cause me to experience feelings and emotions concerning the testing activity; but this is something I will have to cover in a separate post.

At the end of this part, we spoke about the outputs of our activity. At the end of the day, we all have to report something to someone. The report should be generated based on the Bonfire session and added to the ticket of the session as a document.

The report should include the following: charter, metrics, charter opportunity for other sessions, notes, and issues raised during the session.

Part 3: Concepts that require some follow-up posting

• Testing and emotions
• Is the waterfall as quite as it sounds?
• Does an infant born as a tester? (ad hoc testing thoughts)

How you know you are a tester in your soul?

You taking time for the coffee machine

You look for typo on the cereal box

You click all buttons of every electric machine you have in your house (including the washing machine and the microwave)

You tried every combination of two buttons in the remote 

You take screen shot when you surf the net

You locked your email and/or facebook account at list once when you purposely tried to typed wrong user name and password 

You try to order more movie tickets then the actual seats in the cinema hall

You reopen a box after you put something in, just to make sure it still look the same

You have no routine because you are doing everything different each time