Some of the testers were shown the raw code for the program we were creating to give the tester an idea of what we would be creating.
Tester 1 found the experience confusing to begin with and commented that she felt the experience would be too difficult for beginners. However, they liked that the experience made them revisit knowledge established earlier in the experience, and the final question of the experience created a good learning opportunity. The tester also liked that the code was actually teaching you to do something.
Things to consider - difficulty level, using more colour and less text.
Things to consider - creating a reference sheet, especially for beginners that clearly explains what everything does, an introduction to the experience and the way of teaching.
Tester 4 found that the hints were very helpful in narrowing down the answer, and again commented that the experience should do more to encourage users to tap the bubbles to view the code. They suggested that when you select the right bubble it should show you the code so that you still view it in some way. The tester enjoyed the experience and definitely felt like they had learnt something during the experience.
Tester 5 enjoyed the experience overall but did find it challenging to begin with. They did comment that it made sense by the time they reached the bottom of the experience and that the things like the hints made everything easier to understand.
"Code is too daunting."
They ultimately felt like they learnt something and they liked working on their own through out the tasks to make use of their problem solving skills.
Things to consider - adding code to an off screen editor as you go that can be accessed at anytime/at the end of the experience to see how the code all fits together, questions with logic operators could be clearer and an option to toggle the help features on and off in an effort to build user's confidence.
Tester 6 liked the pseudocode approach of breaking down the code and said that this worked well. They enjoyed the experience overall and managed to complete the task with minor difficulty. Their only problem with the experience was that they felt some users might constantly swipe away hints and tips if the users are receiving them every few minutes.
Things to consider - hints only if prompted or wrong action chosen too many times, user might get bored with constant reasons why x does y.
Tester 7 - didn't show the code before beginning, experienced coder
Tester 7 was the most experienced coder out of the group and completed the task relatively easily. However they did comment that some questions needed to be clearer. They did praise the pseudocode approach and said that the experience successfully broke down the coding structure. Tester 7 did comment that I need to make more emphasis on explaining every aspect of the experience, especially the differences between pre-defined functions (Math.random()) and user generation functions (compare()). Tester 7 also made sure that I had considered the order of learning - should this topic be introduced before this and so on.
Things to consider - wording of questions, making sure to talk through everything with the learners and consider the order of learning
Tester 8 - didn't show the code before beginning, limited skill
Tester 8 was the last person I tested the experience on. Tester 8 said that he needed something like this for learning coding initially - something that makes it more manageable to digest.
Things to consider - draw more attention to the names of variables and functions established through out the process so that you can refer back to them, just need to be more consistent with sentences.
Overall, this was very worthwhile and gave me lots of things to consider for my experience.