Requirement Identifiers Reqs Tested REQ1
UC
1.1 REQ1
UC
1.2 REQ1
UC
1.3 REQ1
UC
2.1 REQ1
UC
2.2 REQ1
UC
2.3.1 REQ1
UC
2.3.2 REQ1
UC
2.3.3 REQ1
UC
2.4 REQ1
UC
3.1 REQ1
UC
3.2 REQ1
TECH
1.1 REQ1
TECH
1.2 REQ1
TECH
1.3
Test Cases 321 3 2 3 1 1 1 1 1 1 2 3 1 1 1
Tested Implicitly 77
1.1.1 1 x
1.1.2 2 x x
1.1.3 2 x x
1.1.4 1 x
1.1.5 2 x x
1.1.6 1 x
1.1.7 1 x
1.2.1 2 x x
1.2.2 2 x x
1.2.3 2 x x
1.3.1 1 x
1.3.2 1 x
1.3.3 1 x
1.3.4 1 x
1.3.5 1 x
etc…
5.6.2 1 x
A traceability matrix is a document, usually in the form of a table, that correlates any two baselined documents that require a many to many relationship to determine the completeness of the relationship. It is often used with high-level requirements (these often consist of marketing requirements) and detailed requirements of the software product to the matching parts of high-level design, detailed design, test plan, and test cases.
For instance a requirements traceability matrix is used to check to see if the current project requirements are being met, and to help in the creation of a Request for Proposal, various deliverable documents, and project plan tasks.[1]
Common usage is to take the identifier for each of the items of one document and place them in the left column. The identifiers for the other document are placed across the top row. When an item in the left column is related to an item across the top, a mark is placed in the intersecting cell. The number of relationships are added up for each row and each column. This value indicates the mapping of the two items. Zero values indicate that no relationship exists. It must be determined if one must be made. Large values imply that the relationship is too complex and should be simplified.
To ease the creation of traceability matrices, it is advisable to add the relationships to the source documents for both backward traceability and forward traceability. In other words, when an item is changed in one baselined document, it's easy to see what needs to be changed in the other.
Forward Traceability and Backward Traceability.
E.g. If a requirement is changed then related test case need to be changed.
Forward traceability means mapping requirements with test cases whereas backward traceability means mapping testcases with requirements.
Mapping take place from from requirements to end products is Forward Traceability
Mapping take place from end product back to requirements is Backward Traceability
Using both the Forward and Backward Traceability is called Bidirectional Traceability. When the requirements are managed well to testcases and testcases to defects and vice versa. This helps nothing is missed in testing process....Bidirectional traceability needs to be implemented both forward and backward i.e. from requirements to end
Friday, December 11, 2009
Test Coverage Matrix
Test coverage matrix is a checklist which ensures that the functionality of the given screen(unit) is checked in all possible combinations (positive and negative) which have not been covered in test cases. Test coverage matrix is usually prepared for a screen having large number of controls (textboxes dropdowns buttons etc) usually test coverage matrix is prepared in a spread sheet having all the controls (textboxes dropdowns buttons etc) in the columns and then all possible entries in those fields in the rows with an ''yes'' or ''no'' in the rows against the controls listed in the columns. For example consider a ''login'' screen wherein we have ''username'' and ''password" textfields.
While preparing test coverage matrix the first column will be ''s.no'' and the second will be ''username" and ''password" will be the third field followed by ''ok'' and ''cancel'' button. Then in the first row for s.no 1 enter ''yes'' for both ''user name'' and ''password'' columns ''yes'' implying that a value is entered in that field. In the second row enter ''yes'' and ''no'' and in the third row ''no'' and 'yes'' and so on.
The complexity increases with the number of controls in the screen. Each of the row is considered as one condition and executed while testing. This is how we prepare test coverage matrix.
Sr. No. Username Password Ok Cancel
1 Yes Yes
2 Yes No
3 No No
4
5
6
7
While preparing test coverage matrix the first column will be ''s.no'' and the second will be ''username" and ''password" will be the third field followed by ''ok'' and ''cancel'' button. Then in the first row for s.no 1 enter ''yes'' for both ''user name'' and ''password'' columns ''yes'' implying that a value is entered in that field. In the second row enter ''yes'' and ''no'' and in the third row ''no'' and 'yes'' and so on.
The complexity increases with the number of controls in the screen. Each of the row is considered as one condition and executed while testing. This is how we prepare test coverage matrix.
Sr. No. Username Password Ok Cancel
1 Yes Yes
2 Yes No
3 No No
4
5
6
7
Wednesday, December 9, 2009
Test Matrix
What is a Test Matrix?
Matrices provide an easy structure for testing common issues. A common issue is an issue that repeats itself from project to project. Testing a common issue should be relevant to the project itself.
Examples for common issues are:
Fields (integer, dates, time, etc)
File names
Printing
Saving a file
Deleting a file
Sending a file
Login process
UI issues
Other
Why is it important to learn "Test Matrix"?
There are several reasons why to learn and to do test matrices.
It can reduce working time - once you have a matrix for a specific issue, it will take you less time to test it or to think how to test it.
It is logical and testing challenging - It is a challenge to make your own matrix and to find common issues that are repeatable from project to project.
In future projects it will save you time and it can give your more time to handle more complex issues.
It’s fun (for those who have a testing mania like the writer).
How to Do a Test Matrix?
The algorithm for creating a test matrix is:
Find an issue that repeats itself from project to project
Think of tests that you routinely perform on this issue
Sort all the tests and put them in a matrix
Example
Let’s build a matrix for integer field. First, we will think and write all the tests that we can perform on an integer field:
0
Valid value
Lower boundary – 1
Lower boundary
Lower boundary + 1
Upper boundary – 1
Upper boundary
Upper boundary + 1
Nothing
Negative value
Special chars: < > ? , . / ; : ‘ “ [ ] { } \ | + = _ - ( ) * & ^ % $ # @ ! ~ `
Uppercase chars
Lowercase chars
Spaces
Leading spaces before the value
Value follows with spaces
Length lower boundary – 1
Length lower boundary
Length lower boundary + 1
Length upper boundary – 1
Length upper boundary
Length upper boundary + 1
Mix of digits, chars and spaces
…
Now we will insert them into a generic matrix:
Integer field matrix
Field Name
Cases
0
Valid value
Lower boundary – 1
Lower boundary
Lower boundary + 1
Upper boundary – 1
Upper boundary
Upper boundary + 1
Nothing
Negative value
Special chars: < > ? , . / ; : ‘ “ [ ] { } \ | + = _ - ( ) * & ^ % $ # @ ! ~ `
Uppercase chars
Lowercase chars
Spaces
Leading spaces before the value
Value follows with spaces
Length lower boundary – 1
Length lower boundary
Length lower boundary + 1
Length upper boundary – 1
Length upper boundary
Length upper boundary + 1
Mixed of digits, chars and spaces
…
Now let use it for a project that contains 2 integer fields: “Age”, “Price”. For each field we will mark those cases we want to test.
Integer field matrix
Field Name Age Price
Cases
0 X X
Valid value X
Lower boundary – 1 X
Lower boundary X
Lower boundary + 1
Upper boundary – 1
Upper boundary X
Upper boundary + 1 X
Nothing
Negative value X
Special chars: < > ? , . / ; : ‘ “ [ ] { } \ | + = _ - ( ) * & ^ % $ # @ ! ~ ` X X
Uppercase chars
Lowercase chars
Spaces X
Leading spaces before the value
Value follows with spaces
Length lower boundary – 1 X
Length lower boundary X
Length lower boundary + 1
Length upper boundary – 1 X X
Length upper boundary X
Length upper boundary + 1
Mixed of digits, chars and spaces X
…
This method gives you the power to change in each test cycle the cases that need to be tested for those fields, without investing many efforts on changing the STD document.
Practice 1
Create a matrix for saving a file. Try to solve it before continue reading.
Practice 1 - answer
Save a new file
Save a file with an existing file name
Save in another format
Save a file to a full disk
Save a file to a write protected disk
Save a file to a remote disk
Save a large file and during the saving process print the file
…
Practice 2
Create a matrix for a date field. Try to solve it before continue reading.
Practice 2 - answer
Insert chars
Insert numbers
Insert day/month 0
Insert day 32
Insert month 13
Insert year 90 (means 1990, 2090 ???)
Insert other format of dates:
24/12/1978 and 12/24/1978
Insert 16/9/2006 and 16.9.2006
…
Practice 3
Create a matrix for login a system: username and password. Try to solve it before continue reading.
Practice 3 - answer
Correct username and wrong password
Wrong username and correct password
Wrong username and wrong password
Correct username and correct password
Correct username and password like ‘select 1’
Uppercase and lowercase
…
Practice 4
Create a matrix for a char field with length x. Try to solve it before continue reading.
Practice 4 - answer
Similar to the integer example
Practice 5
Create a matrix for deleting a file. Try to solve it before continue reading.
Practice 5 - answer
Delete a file
Delete a very large file
Delete an empty file
Delete an empty folder
Delete a folder with many files
Delete an open file
Delete a file while sending other files to the printer
…
Practice 6
Create a matrix for testing an email field. Try to solve it before continue reading.
Practice 6 - answer
Insert a valid mail: a.a@a.com, a@123.co.il
Insert an invalid mail format
Insert chars
Insert numbers
Insert a very long email
…
Practice 7
Create a matrix for testing a new screen that will be insert into the application. Try to solve it before continue reading.
Practice 7 - answer
Test typo in screen title, buttons, fields and tables.
Test that keyboard shortcuts work well when buttons are enable and lock when buttons are disabled.
Check sorting at least 3 times on each column.
If you have a search criteria, combine a search with several fields using a method named "all pairs".
Create a test where the search is bring you one row, 2 rows, more then 2 rows and 0 rows.
Create a test for clear the search (brings all rows).
Create a search where in string/text field you are trying to insert a word contains the sign ' - if the programmer didn't handle it and you are working with XML files or with databases, you will get an error.
If you can search by "starting with" or "contains with" test spaces and '.
Test that each button is working as expected.
Test the paging feature if you screen support in regular view and in searching results - sometimes there is a bug that paging in search result will bring the entire rows.
Ttest the "yes", "no", "cancel" options in message box.
Try to active/inactive a radio button and a check box field.
Test that mandatory fields are really mandatory and that the message about it is correct.
Test the close window button.
Matrices provide an easy structure for testing common issues. A common issue is an issue that repeats itself from project to project. Testing a common issue should be relevant to the project itself.
Examples for common issues are:
Fields (integer, dates, time, etc)
File names
Printing
Saving a file
Deleting a file
Sending a file
Login process
UI issues
Other
Why is it important to learn "Test Matrix"?
There are several reasons why to learn and to do test matrices.
It can reduce working time - once you have a matrix for a specific issue, it will take you less time to test it or to think how to test it.
It is logical and testing challenging - It is a challenge to make your own matrix and to find common issues that are repeatable from project to project.
In future projects it will save you time and it can give your more time to handle more complex issues.
It’s fun (for those who have a testing mania like the writer).
How to Do a Test Matrix?
The algorithm for creating a test matrix is:
Find an issue that repeats itself from project to project
Think of tests that you routinely perform on this issue
Sort all the tests and put them in a matrix
Example
Let’s build a matrix for integer field. First, we will think and write all the tests that we can perform on an integer field:
0
Valid value
Lower boundary – 1
Lower boundary
Lower boundary + 1
Upper boundary – 1
Upper boundary
Upper boundary + 1
Nothing
Negative value
Special chars: < > ? , . / ; : ‘ “ [ ] { } \ | + = _ - ( ) * & ^ % $ # @ ! ~ `
Uppercase chars
Lowercase chars
Spaces
Leading spaces before the value
Value follows with spaces
Length lower boundary – 1
Length lower boundary
Length lower boundary + 1
Length upper boundary – 1
Length upper boundary
Length upper boundary + 1
Mix of digits, chars and spaces
…
Now we will insert them into a generic matrix:
Integer field matrix
Field Name
Cases
0
Valid value
Lower boundary – 1
Lower boundary
Lower boundary + 1
Upper boundary – 1
Upper boundary
Upper boundary + 1
Nothing
Negative value
Special chars: < > ? , . / ; : ‘ “ [ ] { } \ | + = _ - ( ) * & ^ % $ # @ ! ~ `
Uppercase chars
Lowercase chars
Spaces
Leading spaces before the value
Value follows with spaces
Length lower boundary – 1
Length lower boundary
Length lower boundary + 1
Length upper boundary – 1
Length upper boundary
Length upper boundary + 1
Mixed of digits, chars and spaces
…
Now let use it for a project that contains 2 integer fields: “Age”, “Price”. For each field we will mark those cases we want to test.
Integer field matrix
Field Name Age Price
Cases
0 X X
Valid value X
Lower boundary – 1 X
Lower boundary X
Lower boundary + 1
Upper boundary – 1
Upper boundary X
Upper boundary + 1 X
Nothing
Negative value X
Special chars: < > ? , . / ; : ‘ “ [ ] { } \ | + = _ - ( ) * & ^ % $ # @ ! ~ ` X X
Uppercase chars
Lowercase chars
Spaces X
Leading spaces before the value
Value follows with spaces
Length lower boundary – 1 X
Length lower boundary X
Length lower boundary + 1
Length upper boundary – 1 X X
Length upper boundary X
Length upper boundary + 1
Mixed of digits, chars and spaces X
…
This method gives you the power to change in each test cycle the cases that need to be tested for those fields, without investing many efforts on changing the STD document.
Practice 1
Create a matrix for saving a file. Try to solve it before continue reading.
Practice 1 - answer
Save a new file
Save a file with an existing file name
Save in another format
Save a file to a full disk
Save a file to a write protected disk
Save a file to a remote disk
Save a large file and during the saving process print the file
…
Practice 2
Create a matrix for a date field. Try to solve it before continue reading.
Practice 2 - answer
Insert chars
Insert numbers
Insert day/month 0
Insert day 32
Insert month 13
Insert year 90 (means 1990, 2090 ???)
Insert other format of dates:
24/12/1978 and 12/24/1978
Insert 16/9/2006 and 16.9.2006
…
Practice 3
Create a matrix for login a system: username and password. Try to solve it before continue reading.
Practice 3 - answer
Correct username and wrong password
Wrong username and correct password
Wrong username and wrong password
Correct username and correct password
Correct username and password like ‘select 1’
Uppercase and lowercase
…
Practice 4
Create a matrix for a char field with length x. Try to solve it before continue reading.
Practice 4 - answer
Similar to the integer example
Practice 5
Create a matrix for deleting a file. Try to solve it before continue reading.
Practice 5 - answer
Delete a file
Delete a very large file
Delete an empty file
Delete an empty folder
Delete a folder with many files
Delete an open file
Delete a file while sending other files to the printer
…
Practice 6
Create a matrix for testing an email field. Try to solve it before continue reading.
Practice 6 - answer
Insert a valid mail: a.a@a.com, a@123.co.il
Insert an invalid mail format
Insert chars
Insert numbers
Insert a very long email
…
Practice 7
Create a matrix for testing a new screen that will be insert into the application. Try to solve it before continue reading.
Practice 7 - answer
Test typo in screen title, buttons, fields and tables.
Test that keyboard shortcuts work well when buttons are enable and lock when buttons are disabled.
Check sorting at least 3 times on each column.
If you have a search criteria, combine a search with several fields using a method named "all pairs".
Create a test where the search is bring you one row, 2 rows, more then 2 rows and 0 rows.
Create a test for clear the search (brings all rows).
Create a search where in string/text field you are trying to insert a word contains the sign ' - if the programmer didn't handle it and you are working with XML files or with databases, you will get an error.
If you can search by "starting with" or "contains with" test spaces and '.
Test that each button is working as expected.
Test the paging feature if you screen support in regular view and in searching results - sometimes there is a bug that paging in search result will bring the entire rows.
Ttest the "yes", "no", "cancel" options in message box.
Try to active/inactive a radio button and a check box field.
Test that mandatory fields are really mandatory and that the message about it is correct.
Test the close window button.
Thursday, December 3, 2009
Types of Reviews
Types of Reviews
--------------------------------------------------------------------------------
Based on purpose reviews are classified as
1. Technical Review Review of work products Requirements Design
Code Test Plans etc.
2. Non-technical Review Review of status (e.g. PMRs) Project Plan
SCM Plan etc. - ensuring process compliance and process adequacy
Based on extent of formality reviews are classified as
a. Informal Review - One on one with peers no agenda or preparation
time. Informal reviews do not have the rigor of a formal review. The reviews may
not be planned or metrics may not be captured to measure review effectiveness.
They are typically used to confirm understanding test ideas brainstorm etc.
b. Semiformal Review - Author walks the participants through work
product defects are pointed out by participants solutions may be discussed for
defects found. This could be a one person review also. Semiformal / Informal
reviews shall be considered for agile based projects.
c. Formal Review - Meeting is planned in advance facilitated by a
moderator follows a proper process participants come prepared for the meeting.
Reviewers are selected based on skill and knowledge
--------------------------------------------------------------------------------
Based on purpose reviews are classified as
1. Technical Review Review of work products Requirements Design
Code Test Plans etc.
2. Non-technical Review Review of status (e.g. PMRs) Project Plan
SCM Plan etc. - ensuring process compliance and process adequacy
Based on extent of formality reviews are classified as
a. Informal Review - One on one with peers no agenda or preparation
time. Informal reviews do not have the rigor of a formal review. The reviews may
not be planned or metrics may not be captured to measure review effectiveness.
They are typically used to confirm understanding test ideas brainstorm etc.
b. Semiformal Review - Author walks the participants through work
product defects are pointed out by participants solutions may be discussed for
defects found. This could be a one person review also. Semiformal / Informal
reviews shall be considered for agile based projects.
c. Formal Review - Meeting is planned in advance facilitated by a
moderator follows a proper process participants come prepared for the meeting.
Reviewers are selected based on skill and knowledge
Subscribe to:
Posts (Atom)
