IBM Watson + WSA — Game Jam

Back in November (2017) Games Design & Art students worked with IBM and their speech to text system Watson to develop a game build on this technology. The project was supported and run by IBM, and formed part of the JAM weeks we run in year 2, IBM hired two of our own recent Graduates Jess Castle and Millie Ng to support the Jam and tutor the student group.

Tuesday 13th November

Today marked the start of the IBM games jam where a great mix of second year Games Design and Art students and a third year student from Graphic Arts met with one goal in mind, to make awesome stuff. We were lucky to have Jon McNamara from IBM come in who outlined what was expected from the jam, some ideas to get the group motivated alongside some links to tutorials for Watson and its Speech-to-text function and Node-Red.

The process began through a research stage where the students used the links to the tutorials in order to get to grips with Watson and Node-Red. This meant that there was a better understanding of how it all worked together which would help inform ideas and consider the possibilities in the later development stages.

An hour or so was spent working on this before the next step in the development journey, the brainstorming stage. Everyone grabbed a wad of large paper and began throwing around ideas and narratives that could benefit from the Watson Speech-to-text function. Like any idea generation stage, it started off slow but once the ball started rolling, more ideas flourished from this including narratives that included underwater environments, Antarctic temperatures and killer penguins.

One of the more popular genres was the horror genre paired with puzzles which is where the first solid idea came from. The plot follows the protagonist, a delivery driver, who turns up to a research facility in Antarctica to drop off resources for the team but no one’s there. After contacting Head Quarters, they too have no idea what’s going on and ask you to try and locate them. As you progress, the door locks behind you forcing you to move forwards where you later come into contact with an AI system who you work with throughout the game. This interaction with the AI is where the speech-to-text function would come in from Watson.

One of the students, George, developed the idea of a core mechanic which used tone of voice in order to determine interactions, outcomes depending on the player’s response and the team started throwing around ideas for the setting of the game; one of the favourites discussed was the idea of running a bank. As the idea fleshed out, ‘actions’ and random ‘events’ that could occur through the game were added which would make the player need to respond accordingly. Depending on choice and tone of voice, this would affect aspects such as staff morale, public morale, security, promotions and the amount of money your bank makes over four quarters.

The next step in the process was to make a paper prototype to test out the formulas that were created to work out the morale percentages, security upgrades, advertising, hiring and firing and promotion to see if the numbers worked. There were also event cards and an action list which could have positive or negative effects on the game which were tested to see if they were too harsh or too easy. Through play testing it as a board game, adjustments were made in order to make sure the game was actually playable, fair yet challenging and fun to play. Judging by the reactions of Jacob, the chosen player, and the rest of the group watching, many a laugh was had and at times it was tense as drastic choices had to be made so all the criteria seemed to be met.

Now that the preliminary testing phase had been completed, tomorrows plan was discussed and this was where the team split into Artists and Coders and tasks were delegated. The Artists will be designing assets whilst the Coders will be designing and implementing the prototype created today and checking everything works smoothly.

Wednesday 15th November

Today consisted of working in the two teams to piece together the digital prototype. The Coding team designed, planned out and discussed the best way to go about implementing the Bank Game into Unity and implementing Watson. On the flip side, the Art team were working together to design characters, looking into textures, colours and producing environment designs. Everyone worked well together to pick up tasks so that the workload was shared and this meant that the deadline should be met in good time on Thursday!

At three o’clock, a meeting was held to catch up on where everyone was and how it was all looking in terms of meeting tomorrow’s deadline.

The Art Team had managed to complete and digitise the main character (the Bank Assistant), a variety of UI had been drafted up ready to be digitised, the Lobby was near completion and the second environment was also hopefully going to be completed by the end of the day. Overall, the Art Team were on it and everything seemed to be falling into place.

The Coding Team, again, were on it and had everything in place ready to go by the end of the day and were just looking into how Watson could be worked into Unity. There has been a slight issue with Watson’s credentials when trying to use it in Unity which everyone had been trying to get their head around but hopefully it’ll be all ready before the deadline.

Long story short, everyone seems to be on track with their tasks and tomorrow should be a matter of adding all of the art assets in, implementing Watson into Unity and testing which is super cool and exciting!

Thursday 16th November

At the beginning of the day, the teams got together to recap where they were all at after yesterday as some people had chosen to work on their tasks at home. The Art Team had pushed ahead and completed the character design of the Robber, who features in some of the random events, as well as worked on the two base emotions that they have for now which are sad and happy. The first environment was completely finished and the second one was having some finishing touches implemented such as textures and fine details. The UI designs had been finished overnight too meaning that all that was left was to drop them into the Unity project and work on placement.

In terms of Watson, George had been working from home on how to implement it into Unity without throwing up errors which was great. The slight issue was that this work was locked to his computer system at home meaning he’d have to redo this for the team to use in the studio. Therefore, the task for him today will be to replicate this on the computers in the studios so it’s available to then drop all of the artwork into Unity and get Watson co-operating.

Today was mainly about finishing up assets, testing code and piecing everything together in order to create the game. Over the next week or so, internal testing was set to ensure that everything works and that Watson’s speech-to-text function was implemented and is working as expected.

Overall, the Jam has been an eye-opener for everyone involved in terms of using and learning the software available, meeting tight deadlines and completing the task as a team within the time frame and thus preparing them for the real world!