Ubiquitous Ambient Gaming - Evaluations
From EQUIS Lab Wiki
Contents |
First evaluation of the game outside
May 31st, 2010
Goal
The test was done to have a first feedback on the game played outside (is it a good game, is it feasible, do people have fun...). We also tested the audio localization using the GPS, compass and gyroscope.
The test was done with 2 subjects. The subjects were explained the rules of the game.
Evaluation
Game
- Both players had fun,
- they were able to catch animals and bring them to their shelters, using both the sound and the map.
- Only one player was hunted by the tiger, but he managed to keep it away by walking at a sustained pace.
- The game seems a little bit long (more than 1/2h), they would have preferred a 1/2h game, or a game this length but with somebody else to play with.
- NB : At first, both players tended to follow the path on the real map, without crossing on the grass, but they lost this habit after a few minutes.
Sounds
- A sound far away or very close is hard to localize, and the player tends to use the map to help:
- far away, the sound is too fade to tell exactly which direction;
- too close, the player can't distinguish loud and louder (it would be interesting to add information to the sound when the animal is close, like footsteps);
- the volume curve according to the distance will have to be improved.
- The sound of the animals and of the tiger were good and recognizable.
- The sound of the shelters was harder to localize.
- The sounds of an animal being caught or an animal being dropped at a shelter were frequently missed, giving some confusion to the player, for example :
- the player catches a dog without noticing it and hears another dog not very far, so he mistakes the other dog with the one he just caught;
- the player goes over a cage several times, thinking he didn’t drop the animal, when he already did (he has to check on the screen to confirm the drop).
The detailed results of Users' evaluation are available on the page Ubiquitous Ambient Gaming - First evaluation of the game outside.
Evaluation of the visual interfaces of the game
May 18th, 2010
Goal
The game is developed in order to present the potential of ambient audio, and to get feedback on the subjective feelings of the users concerning ambient audio. For the user to assess the possibilities of ambient audio, the game must be easy to play, the different interfaces must be easy to understand and to manipulate.
That is why, one of my first evaluations focus on the visual interfaces and interactions and not on the audio interactions. Since the game was created from scratch, the choice was made to first implement it with very simple visual interfaces that would contain what we considered useful to know, with not too much concern on the appearance. This allows users to play the game fully, to understand it and to notice what information they want to know while playing, and the best way to get informed.
.png)
.png)
Evaluation
Sequence
- Explain the rules to the user
- Explain the possible interactions
- Have him play a while with the game (with him being able to see the screen, since we are not evaluating the audio)
- Note where the user is having difficulties
- Ask about the global impression, the names, the disposition
Results
- Transition between interfaces:
- the red cross button was not understood as a way to end the game, but as a way to close the information interface
- the information interface was perceived as a sub-interface, like a window opening on the map
- Supplies:
- the users had difficulties knowing when they could buy tools and when they couldn’t
- the price of each supply is indicated nowhere
- the user should be informed when he can buy a supply, even if he is on the map interface
- the information is indicated twice in 2 different ways : icons or buttons with text. It is confusing.
- Miscellaneous:
- there is not enough information on the map (numbers of animals remaining, how much money the player has), and the info interfaces is too crowded.
- the map has too much color, it should be replaced by a simpler map, maybe with a bigger zoom
The detailed results of Users' evaluation are available on the page Ubiquitous Ambient Gaming - Evaluation of the visual interfaces of the game.
First evaluation of sound orientation and differentiation front/back
April 26-27th, 2010
A first test was done on the spatial orientation thanks to the ambient audio. I also compared 2 ways of differentiating the back and front sounds.
The users were asked to catch 2 animals among 4 without looking at the screen ( they moved thanks to the arrows). They were asked to talk out loud so that I could collect information on their perception of the sounds. The users played twice, each time with a different way of differentiating the front and back sounds. They didn’t know what difference there was between the two tests.
Parameters
- Four animals (2 cats and 2 dogs) are moving in an area of 400m x 400m.
- The player is placed in the middle of the area.
- The player can move thanks to the 4 arrows :
- left / right : the user stands still but turn around himself, one degree at a time
- up / down : the user moves forward / backward of 1m following his bearing
- Two ways of differentiate the back and the front were tested in a different order :
- the sound stops when the animal passes a certain angle behind the bearing of the user : 120 < teta < 240
- the sound fades progressively (linearly) when the angle between the animal and the player’s bearing gets close to 180 degrees : 110 < teta < 180 and 250 > teta > 180
- The animals don’t make a sound all the time, there are delays between their sound. The delay varies among the animals but it is constant for one animal. For the test, the maximum delay is 2s.
Results
- Way to differentiate front and back :
- Two users preferred the progressing fading and the other preferred the abrupt stopping of the sound. The abrupt stopping of the sound does not match the intuitive perception (if a sound stops, it means that the animal disappeared), but neither does the progressing fade (it is assimilated to the animal moving away). In each case, the user chose by default, because he didn’t like the other way, not because he liked the way he chose.
- They all felt several times that the sound was coming from behind when in fact it was coming from the front.
- It seems that either way, it would need something else to make the difference between front and back more natural.
- One of the users, said he would have preferred that there is no difference between front and back, and he would have determined by moving around from where the sound was coming (it is to notice that this user used a lot more the forward/backward than the turn arrows).
- As for the sound orientation :
- The users felt the difference between left/right and the variation of volume.
- They also established the link between the variation of volume and the distance, but they both felt it wasn’t completely natural and they sometimes would not know exactly how close they were from an animal, especially when they were close to it. It seems that the variation of volume, especially for something close, must be noticeable and recognizable.
- Remarks on the sounds of the animals :
- A big delay between 2 sounds emitted by an animal forces the player to stop and wait for the animal to make a sound again, which can be good for the game-play but might encourage the player to use the screen instead of the sound (a delay of 2s seems too important).
- Having always the same delay for an animal, or even the same sound, is not a natural way, and it can get annoying for the player.
- Moreover, when two animals of a same type are emitting a sound, even if they are not doing it at the same time, it can be hard for a user to focus on only one animal, he can get mixed up. It seems that, when the animals have the same sound, the user can’t apply the “cocktail effect” which allows him to focus on one sound among a lot of sounds.
=> The results tend to show that a player would learn very quickly how to use efficiently the sound to orientate himself (both users moved quicker the second time and with equal or more accuracy). Moreover, the players were pleased to discover that they could catch an animal using only the sound orientation.
The detailed results of Users' evaluation are available on the page Ubiquitous Ambient Gaming - First evaluation of sound orientation and differentiation front/back.
Evaluation of the game Zoo breakout
April 26-27th, 2010
An informal evaluation on the fun of the game has been done. Several persons read the rules of the game and gave us feedback on whether they thought this game would be fun. They were asked to give their opinion on the game, what they liked, what they disliked, what they would change.
Results
This evaluation has been presented to 3 persons. This is a summary of what seemed the most important.
- Changes proposed :
- Ability to see / hear the animals the player can’t catch to remember him of their existence (the player would have a global view of the state of the game) and to push him to buy tools to catch them.
- Things to precise :
- How does a player match a cage with a type of animal?
- How does a player find a cage’s location?
- We could be confronted to a “How weird I look” factor : people would be concerned about what the people in the street might think of them running around, stopping, going the other way. This might be a brake to the fun of the game.
- Animals chosen : mostly African animals, both users mentioned monkeys and elephants first.
The detailed results of Users' feedback are available on the page Ubiquitous Ambient Gaming - Evaluation of the game Growl Patrol.
Evaluation of the game Logical circuit
April 14th, 2010
I evaluated the fun of the game Logical circuit thanks to a paper prototype of the game. The ambient sound part of the game was not tested.
The game was transformed in a board game and two users played against each other.
Results
Both users said the game was fun, but some modifications had to be made, such as making the game harder to win by making the circuits more complex, modifying the board so that they won't have only one way to get to a power source. The improvements that the game would need were :
- Being very careful when creating the circuits so that they wouldn't be too simple to solve.
- Making the circuits more complex, either by having more types of nodes, or by making nodes with several outputs.
- Being able to see both circuits, and since the circuits are powered by the same power sources, have them merged at the power sources so that it would be easier to read.
Both the players also mentioned the risk that logical circuit would not appeal to non scientific players.
After this evaluation, we realized that the idea of logical circuit was too complex and that finding circuits of same difficulty, not too complex and not too easy, was going to be difficult.
Moreover, it appeared that if the power sources are fix in space, the ambient audio would only be used at the beginning of the game to find the power sources, and then the players would rely on their memory to find them again.
In conclusion, we decided to rethink the game before going further.
The detailed results of the evaluation are available on the page Ubiquitous Ambient Gaming - Paper prototype.
