-
Notifications
You must be signed in to change notification settings - Fork 1
TEST2
For the Test2 part of project phase 4 we decided to do the usability study and the reliability testing.
This usability study evaluates MetroBike, an android application, which provides bike+bus combined directions from a starting point to a destination for bikers.
The purpose of this usability test was to ascertain whether the app’s structure was easy for potential customers to understand and navigate.
We have asked the three participants to sign the consent form.
In our usability testing, we let the 3 participants to try to finish the following three tasks:
Task 1: Find a Route. (Find the bike+bus combined directions from “University of Washington” to “Gas Works Park”)
Task 2: Get Directions. (Choose a route and check its direction list)
Task 3: Follow the Navigation. (Follow the step-by-step instructions)
Problems discovered by development team:
-
Need to redesign the detail page to make it user-friendlier interface.
-
Performance issue: we may need to add a cancel button when searching route box pops up.
-
Duplicate routes in the result page.
-
Need to change the color and styles of step-by-step instructions in the Navigation page to make it more striking. As well, the layout for the instructions should changed to avoid overlapping.
Problems fixed in this phrase:
-
Redesign all the button and some pages to become more user-friendly.
-
Changed the color and styles on step-by-step instructions.
-
Add a cancel button while the app is searching for route.
All the results and surveys of the three participants will be sent to the stuff directly.
We implemented 3 of the possible six items. They are:
- Substantial Logging
- Subsystems Crippled
- Load Tests
In order to verify that this was completed you should be able to go into the backend code. From here if you search for System.out.println, you should see substantially more printlns than previously. Some methods will have more printlns than others because top level methods will end up calling smaller helper methods, which in turn will output more information. The Utility class in the backend package is a good example of this. If you want a better idea of how many printlns were added, most of the logging code was added on June 1st with the commit title "Logging for the XXX package."
As a note, the reason why we primarily used System.out.println and System.err.println in the backend (rather than Log.w/d/v/i/etc...) was that we designed our backend code to be as close to pure Java as possible. This allowed us to test our code in a more straightforward way and makes our backend code more portable.
In order to implement this we had to do half of the integration testing option. What is meant by this is we had to modify the architecture of our software in order to create a stub class that could be used to simulate the Google Maps APIs. What we decided to test was how our system would respond if the Internet connection was down and our app was unable to contact Google Maps. The way this test works is the code makes a request to our stub class (as opposed to the actual Google Maps APIS) and our stub class throws an IOException. This exception is handled by the upper levels of the MetroBike system, which returns a DirectionStatus.CONNECTION_ERROR enum value that lets the UI know something went wrong. The test checks for this enum value to be returned.
The exact test is in the "com.HuskySoft.metrobike.backend.test" package in the DirectionsRequestTest.java class. The test name is "testStubDoRequestCase3ConnectionFailedThrowIOException."
This test runs when the Junit tests are run.
In order to implement this also used the stub class. It is a single test as well and involves returning a 58,000 character JSON response string back to the MetroBike code (The Java compiler can't handle Strings bigger than 2^16 characters at compile time). Our code handles this like any normal response, it just takes a little bit longer. The reason for testing this was to make sure nothing crashed as well as stress testing being a generally good test practice.
The exact test is in the "com.HuskySoft.metrobike.backend.test" package in the DirectionsRequestTest.java class. The test name is "testStubDoRequestCase2StressTest."
There were four total tests utilizing the stub that stressed aspects of the MetroBike system that could not be reached using a request to the Google Maps APIs. These can be found in the DirectionsRequestTest class with the other ones (They should in fact be right next to each other).
As we mentioned to our customer TA in our Friday meetings, we ended up partially hitting a few of the Test2 System/Integration tests in order to implement the ones described in here.
Our project doesn't run in an Android emulator because of Google Play support issues, making testing our app's UI difficult. In order to test some portion of our UI, we used the Android Monkey system to send random input to our app. The app fared well, holding up under the tests below. One ad-hoc test (not shown below) even held up to 200,000 injected events.
/path/to/adt-bundle/sdk/platform-tools/adb shell monkey -p com.HuskySoft.metrobike --pct-syskeys 0 --pct-majornav 0 -s 600 2000
/path/to/adt-bundle/sdk/platform-tools/adb shell monkey -p com.HuskySoft.metrobike --pct-syskeys 5 --pct-majornav 5 -s 425 1000
/path/to/adt-bundle/sdk/platform-tools/adb shell monkey -p com.HuskySoft.metrobike --pct-appswitch 20 --pct-syskeys 0 --pct-majornav 0 -s 208 2300
/path/to/adt-bundle/sdk/platform-tools/adb shell monkey -p com.HuskySoft.metrobike --pct-appswitch 0 --pct-syskeys 0 --pct-majornav 0 -s 2332 2300