Personal Honours Project - Mk. 1 Prototypes - Make or break(!)

To test the experience, I enlisted 8 people with various degrees of familiarity with the coding language JavaScript - this ranged from complete beginners to more experienced users. I asked participants to read the instruction sheet I had provided for them and then took a back seat to observe the participant, takes notes and run the experience i.e. making the bubbles appear at each choice for the user to pick from. If the testers were really struggling, I acted as the help feature too in a social sense - if the user got stuck they could reach out to their peers for assistance via communication channels via the app.

Some of the testers were shown the raw code for the program we were creating to give the tester an idea of what we would be creating.

Tester 1 - didn't show code before beginning, familiar with JavaScript

Tester 1 found the experience confusing to begin with and commented that she felt the experience would be too difficult for beginners. However, they liked that the experience made them revisit knowledge established earlier in the experience, and the final question of the experience created a good learning opportunity. The tester also liked that the code was actually teaching you to do something.

Things to consider - difficulty level, using more colour and less text.

 

Tester 2 - showed code before beginning, some experience with JavaScript

Tester 2 said they found some sentences hard to understand and was having difficulty remembering what the different things did in JavaScript. They suggested incorporating a dictionary or some sort of points of reference for the users to have while learning. Tester 2 also suggested that at the end of each task at the hubs, the app should tell you why that's the best course of action - this will create more learning opportunities and build the confidence of the learner. Overall, they enjoyed the experience but found it difficult; they agreed though that it successfully broken down code structure to make it more manageable.

Things to consider - creating a reference sheet or dictionary, explaining answers to learners, categorising different aspects of JavaScript by what they do - using colour again to create patterns(?). More emphasis on encouraging users to tap and flip the bubbles to see the code.

Tester 3 - showed code before beginning, no experience with JavaScript

Tester 3 really struggled with the task but this was unsurprising considering the first two participants had commented on difficulty and the tester had no experience with JavaScript. She relied on the help feature a lot (i.e. me). She completed the task but said she did enjoy the experience. However she did find it slightly overwhelming to begin with.

Things to consider - creating a reference sheet, especially for beginners that clearly explains what everything does, an introduction to the experience and the way of teaching.

 

Tester 4 - didn't show the code before beginning, no experience with JavaScript

Tester 4 found that the hints were very helpful in narrowing down the answer, and again commented that the experience should do more to encourage users to tap the bubbles to view the code. They suggested that when you select the right bubble it should show you the code so that you still view it in some way. The tester enjoyed the experience and definitely felt like they had learnt something during the experience.

Things to consider - make the tap to see code more prominent, show the code when you select the right action, add definitions of JavaScript to the hints.

Tester 5 - showed code before beginning, no experience with JavaScript

Tester 5 enjoyed the experience overall but did find it challenging to begin with. They did comment that it made sense by the time they reached the bottom of the experience and that the things like the hints made everything easier to understand.

"Code is too daunting."

They ultimately felt like they learnt something and they liked working on their own through out the tasks to make use of their problem solving skills.

Things to consider - adding code to an off screen editor as you go that can be accessed at anytime/at the end of the experience to see how the code all fits together, questions with logic operators could be clearer and an option to toggle the help features on and off in an effort to build user's confidence.

 

Tester 6 - showed code, some experience with JavaScript

Tester 6 liked the pseudocode approach of breaking down the code and said that this worked well. They enjoyed the experience overall and managed to complete the task with minor difficulty. Their only problem with the experience was that they felt some users might constantly swipe away hints and tips if the users are receiving them every few minutes.

Things to consider - hints only if prompted or wrong action chosen too many times, user might get bored with constant reasons why x does y.

Tester 7 - didn't show the code before beginning, experienced coder

Tester 7 was the most experienced coder out of the group and completed the task relatively easily. However they did comment that some questions needed to be clearer. They did praise the pseudocode approach and said that the experience successfully broke down the coding structure. Tester 7 did comment that I need to make more emphasis on explaining every aspect of the experience, especially the differences between pre-defined functions (Math.random()) and user generation functions (compare()). Tester 7 also made sure that I had considered the order of learning - should this topic be introduced before this and so on.

Things to consider - wording of questions, making sure to talk through everything with the learners and consider the order of learning

 

Tester 8 - didn't show the code before beginning, limited skill

Tester 8 was the last person I tested the experience on. Tester 8 said that he needed something like this for learning coding initially - something that makes it more manageable to digest.

Things to consider - draw more attention to the names of variables and functions established through out the process so that you can refer back to them, just need to be more consistent with sentences. 

Overall, this was very worthwhile and gave me lots of things to consider for my experience.