Final Project Testing

Universities rarely have a chance to thoroughly test homework assignments, yet testing is one of the most critical skills for a software engineer. For your final projects, each project will be tested by members of another team. Testing assignments:

Team 1 will test Team 2
Team 2 will test Team 3
etc.
Team n-1 will test Test 1

Node teams will test other Node teams, JavaScript teams will test other JavaScript teams.

Test Script

A test script is a plan for how to test a program. This type of testing, with a specific script that a human could follow, is one way to do acceptance testing. In our software engineering courses, we have stressed unit testing. Unit tests can ensure that specific functions perform correctly, but they can't ensure the system will meet the client's requirements. The equivalent of an acceptance test is often done by the grader or instructor.

The test script you will create is related to the project rubric, because each item of the rubric must be tested. As an example, here's a script I provided to the grader for Unit #3, the concentration game:


Run program in browser

    - Ensure instructions show when program loads
    - Ensure 3 section layout (game in middle, player's cards on either side)
    - Since cards are all visible
        - Select matching cards, ensure shows up for player 1
        - Select a second match, ensure shows up for player 1 again
        - Select cards that do not match, ensure now player 2 turn
        - Select matching cards, ensure shows up for player 2
        - Try to select same card for both card 1 and card 2, ensure error message is shown
        - Now just play the game quickly so that player 1 wins (can just select all matches)
        - Play the game quickly so player 2 wins (no restart button, just reload page)
        - Play the game so it is a draw

Now edit the file so that cards display the backs and only flip over when clicked or won.

    - When game starts, back of all cards is shown
    - I click the cards in order (e.g., starting along row 1), rather than randomly 
	  (just to be systematic)
    - Ensure that cards do not turn over until the next player selects their first card
    - As soon as I detect a match, ensure the card gets assigned to player (already done above) 
	  AND cards no longer turn over (i.e., cards that are won continue to be displayed). 
    - Since other functionality was already tested, no need to continue this game to the end.

This script relates to, but is not the same as, the Unit #3 rubric:


This assignment is worth 56 points. Game play requirements:

	(6) Reasonable game board layout with 3 sections
	(4) Show instructions when program loads (onload).
	(4) Start screen shows only backs of all cards
	(4) When a card is clicked, the card image is displayed
	(4) If the same card is clicked twice, an error displays and the player must 
		choose a different card
	If 2 matching cards are turned over:
		
			(4) The player whose turn it is "wins" the prize (it's displayed in sidebar)
			(4) Both cards remain visible
			(8) If these are the last available cards, a message displays that 
				the game is over, and who won (could be a tie).
			(4) Otherwise, it remains that player's turn
		
	If the 2 cards do not match:
		
			(4) the cards will be turned back over (i.e., 
				backs of cards displayed), when the next player selects their first
				card. WHY wait? If you turn them over immediately when clicked, users may 
				not have time to see both cards.
			(4) game play moves to the next player

Test Report

Each member of each team will create their own script for testing another team's project (testing assignments listed above). The script AND a list of bugs (i.e., the Test Report) will be submitted to both the tested team AND on Canvas. This ensures that each project will be tested by at least two other students.

Rubrics

Remember that test scripts should cover all functional items of the rubric (e.g., "Way to list items"). The script does not need to include non-functional items (e.g., "Deployed on Heroku" and "Using an API").

Node option 1. The rubric includes:

(55) Basic Requirements completed
(10) New technology incorporated. The "About" page should list the new technology. The technology will relate to a number of specific features that you should include in your test script. For example, to test a login facility you should try to login in with an incorrect password, log out, try to access pages without logging in, etc. You should also test creating an account and ensuring the functionality makes sense (e.g., if editing an item is included, only the "owner" of that item should be able to do it). For thorough testing, it may be necessary to create more than one account.

Node option 2. The rubric includes:

(55) Basic Requirements completed
(30) Design per your rubric is complete
(10) Design is responsive (tested by altering screen size of browser)

Because the "About" page may not have the entire rubric, one person from each option 2 team should email their approved rubric (from week 1) to all members of the testing team.

JavaScript. The rubric includes:

(10) Visually appealing method to navigate from one challenge to the next
(5) State maintained using local storage
(5) Use of audio
(40) Challenge #2 game
(20) Challenge #3 activity

Note that Same Game is also included as a challenge, but this was tested in week 1 by the grader, so no need to re-test (although you might need to play the game, as it's one of the 3 challenges).

As with Node option 2, the testing team will need the rubrics for challenge #2 and #3. One person from each JS team should email their approved rubrics (pdf from week 2) to all members of the testing team. If I requested any changes, be sure to update your pdf before sending.

Ease of grading. You may recall that both Concentration and Same Game had options which would make it easier to grade (e.g., use of 2 colors for Same Game to test end-of-game). If your JS navigation requires the user to "win" or achieve a high score for Same Game, OR if you have an "ease of testing" option for one of your other challenges (e.g., Checkers), please do TWO posts on CS-CONNECT. One will be with the testing option set, one will be without. The titles should indicate that, e.g., Team n Testing and Team n Fun Challenge! I believe there are 3-4 teams who will be in this situation.

Test Report Format

The first part of the test report is the script. In a production environment, the script would be created first, then the program would be tested by following the script steps - sometimes by a different person, and sometimes more than once, if bugs are detected/fixed. For our purposes, I expect that you will create the script while you are testing (e.g., have a text editor open and type of brief description of each step). Because of this, the script may not be optimized (e.g., you may test a page such as Add Item, move to a different page, then realize a rubric item was missed... so return to Add Item).

Example of a partial script related to bartering program:


Test Script created by Cyndi:
Logged onto program 
Viewed list of items
Clicked item to see details
Clicked home button to view list of items
Clicked item to see details
Clicked "Want this item" to barter
Tried to submit with no information
Saw error message
Entered information
Clicked submit
BUG - noticed that the item was still listed as available
....

The second part of the test report is just a list of bugs - a brief list of issues identified during testing, such as:


List of Bugs:
Item is not removed after being bartered
After logging out, I could still access the item detail page
...

NOTE: For simplicity, put the test script and list of bugs in the same text file. If no bugs are found, include "List of Bugs" and state that you couldn't find any.

Test Report Submission

Each person should submit their test report to the appropriate area on Canvas.

In addition: the test report should be emailed to the team whose project you are evaluating. In case you aren't aware, you can go to inside.mines.edu and click the People radio button in the Search box to find other students' addresses (must click People for each search).

You will receive 10 points for performing the test and submitting the report. The grader will review briefly, to ensure the report has adequate length and detail. Team members' scripts should vary, as it would be highly suspicious if team members followed exactly the same path. We therefore expect that team members might find different issues, and some might find more than others. If a students finds no bugs, however, when their teammates find multiple, we might review the script in more detail to see whether the testing was really thorough (in other words, please do take this seriously).