|
|
A Comparison of Input Methods for 2D Fixed Shooter GamesProgramming Languages used: Java A Comparison of Input Methods for 2D Fixed Shooter Games
Abstract - Twelve participants completed a study comparing a keyboard and the Sony Dualshock 3 controller as input methods for a 2D fixed shooter game developed for this study, called Space Shooter. Two firing modes, continuous and discrete, were also compared. It was found that the controller performed 13% better than the keyboard in continuous firing mode, while the keyboard performed 9% better than the controller in discrete firing mode. The continuous firing mode performed 69% better than the discrete firing mode on the keyboard and 108% better on the controller.
I. INTRODUCTIONAs the gaming industry continues to grow, classic arcade style 2D fixed shooter games like Galaga and Space Invaders are making a resurgence in modern gaming culture. These popular classic games are often re-released on current gaming systems. As the controllers for these systems evolve, the games evolve with them. An example of this evolution is Space Invaders Groove Coaster, a Space Invader-like groove game for the Apple iOS, which uses one-finger touch control. In the development and adaptation of these games to fit modern gaming systems and controllers, performance and player experience are important factors. In a mobile device context, users could use the touch screen or the accelerometer to control their ship's movements, but which input method provides better performance? And which method do players prefer? These questions can be answered by performing a comparative evaluation of input methods. Those answers can then shape the direction of the future of 2D fixed shooter games.
A. Related WorkCummings [1] provides an historical overview of the evolution of game controllers over time. He describes how early game controllers were not usually designed for games but taken from whatever set of controls were available to the designer at the time, typically digital switches and knobs. He continues to describe how controllers shifted into being designed for games, and the effect that different controllers have on the player experience. In terms of motion sensing and touch screens, he cites immersion as a significant factor produced by these types of controllers. With regards to our experiment we are testing the player experience between two input devices: one that was not designed for playing games (the keyboard), and one that was designed for playing games (the controller). Gerling et al. [2] performed a study where they examined the impact of the gaming platform on player experience in FPS (first-person shooter) games. They tested users on a computer (using a keyboard and mouse) and on a console (using a gamepad). They assigned some users to play on the system they were most comfortable with, and others to play on the system they were least comfortable with. They found that player experience is only slightly affected by controller hardware. This study is similar to our own in terms of controller hardware, but differs in the hypothesis being tested, with our experiment evaluating player performance. Natapov et al. [3] performed an evaluation of video game controllers for point-select tasks. They tested a computer mouse against a Wiimote and Wii Classic Controller for selecting a point on a screen. They found that the mouse performed better than the controllers but that users preferred using the Wiimote. Our experiment is also evaluating user performance between controllers but in the context of a 2D fixed shooter game. Skalski et al. [4] performed two studies that examined the enjoyment of players using natural game controllers (such as a steering wheel) and spatial elements vs. traditional gamepad controllers. They found that more enjoyment was produced when players used natural controllers. Our experiment does not deal with natural controllers, but is performing an evaluation of different controller inputs. Thorpe et al. [5] give a brief overview of game controllers and performed an experiment to test how players received alternative input methods. They found that player immersion and fun increased with the use of alternative input methods when the player perceived the input method to fit the game play. Our experiment also evaluates player preference of input method through a questionnaire.
B. OverviewThis user study compared two input methods (keyboard and controller), and two firing modes (continuous and discrete). The study observed player performance using four possible combinations of input methods and firing modes. The main goal was to determine if one of the input methods performed better with respect to score and survival time. The secondary goal was to determine if there was a performance difference based on the firing mode in combination with input method. Participants were tested using a 2D fixed shooter game called Space Shooter. The game utilized a keyboard and the Sony Dualshock 3 controller as input methods. Participants were given five lives per input method and firing mode combination. The objective of the game was to survive as long as possible while killing as many enemies as possible with the fewest number of bullets. These statistics were recorded as survival time, score, and accuracy.
II. METHOD
A. ParticipantsThe experiment included 12 voluntary participants recruited from those readily available to the researcher. Ten participants were male and two female. Ages ranged from 19 to 53 years. All of the participants have played a game similar to Space Shooter before. The participants were not given incentives or compensation.
B. ApparatusThe hardware was a Clevo P170HM laptop running Windows 7. The built-in keyboard on the laptop was used (see Figure 1). The controller was a Sony Dualshock 3 (see Figure 2). The game software was developed in Java. Two 3rd party programs, ControlMK and DS3 Tool, were used to translate button presses on the Dualshock 3 into keystrokes.
 Fig. 1. The Clevo P170HM keyboard.
 Fig. 2. The Sony Dualshock 3 controller. For this experiment a custom game called Space Shooter was developed. Space Shooter is a 2D fixed shooter game set in outer space (see Figure 3). The game implements two different firing modes: discrete and continuous. Discrete firing requires the player to press and release the fire button to fire a bullet. Continuous firing allows the player to hold the fire button down to fire bullets at a constant rate. The player controls a spaceship that moves left or right at the bottom edge of the screen. Enemy spaceships enter at the top of the screen and move downward while also moving left and right. The enemy spaceships will also fire bullets toward the player's spaceship. The longer the player survives, the faster the enemy spaceships and bullets will move. This change in speed resets between lives.
 Fig. 3. Space Shooter game level. The goal of the game is to avoid being hit by enemy spaceships and enemy bullets while hitting as many enemies as possible with the player's own bullets. Each participant was given five lives for each of the four combinations of input method and firing mode, making a total of 20 lives. A life is lost upon collision with an enemy bullet or spaceship. The application began with a setup activity where the participant entered the following: the provided participant code, number of lives, input method, and firing mode (see Figure 4). The counter-balancing group the participant belonged to was recorded manually. The firing mode option changed how the player fired bullets in the game. After colliding with an enemy bullet or spaceship there was a five second cool-down period before the next life began. After five lives were consumed the player was shown the score screen which displayed: best score, best time, high score, shots fired, shots hit, and accuracy (see Figure 5). For the keyboard input method the player used the left and right arrow keys to control their spaceship and the spacebar to fire. For the controller input method the player used the left and right buttons on the D-pad to move and the X button to fire.
C. ProcedureThe experiment was conducted in various environments which all had medium light levels and low background noise (see Figure 6). The participants were seated in front of a laptop on a desk height table. They were allowed to have the laptop on their lap if they preferred. Participants were shown the game and allowed to play a practice round if they chose to.
 Fig. 4. The setup activity.
 Fig. 5. The results screen. Participants were then walked through the setup procedure as consent for participation in the experiment. Each participant took approximately 15 minutes to complete the experiment. Afterwards, participants were asked to take a questionnaire to provide qualitative feedback. Participants were asked about prior experience, input method preference, and other relevant feedback about the experiment.
 Fig. 6. participant performing the experiment.
D. DesignThe experiment was a 2 × 2 × 5 within-subjects design. There were three independent variables: input method (keyboard, controller), firing mode (continuous, discrete), and life (1, 2, 3, 4, 5). There were also three dependent variables: survival time, score, and accuracy. Survival time measured how long the participant survived each life. The score measured how many enemies were killed. Accuracy measured how many bullets fired by the player actually hit the enemies. Participants were divided into four groups to counterbalance the order of input methods and firing modes to offset learning effects. The total number of test rounds was 240 (12 participants × 2 input methods × 2 firing modes × 5 lives).
III. RESULTS AND DISCUSSIONThe effect of group (order of testing) was not statistically significant for score (F3,8 = 0.717, ns), survival time (F3,8 = 0.881, ns), and accuracy (F3,8 = 1.031, ns). Thus it can be concluded that counterbalancing had the desired effect of offsetting any learning effect due to the order of testing.
A. ScoreScore is based on the number of enemy spaceships destroyed: score = enemies destroyed × 100. The grand mean score over 240 rounds was 4427.5. The mean score using the keyboard with continuous firing was 5418.3. The mean score using the keyboard with discrete firing was 3211.7. The mean score using the controller with continuous firing was 6136.7. The mean score using the controller with discrete firing was less than half as much at 2943.3 (see Figure 7). The effect of input method and firing mode combinations was found to be statistically significant (F3,8 = 6.658, p < .002). Upon further analysis it was found that while keeping the firing mode constant (either continuous or discrete) the effect of input method was not statistically significant (F3,8 = 1.245, ns) (continuous) (F3,8 = 0.227, ns) (discrete). In contrast, while keeping the input method constant (either keyboard or controller) the effect of firing mode was statistically significant (F3,8 = 6.030, p < .05) (keyboard) (F3,8 = 10.247, p < .01) (controller).
 Fig. 7. Mean score based on firing mode and input method. Error bars indicate ± 1 SD. The mean scores show that for continuous firing the controller has a small advantage over the keyboard while the opposite is true for discrete firing. The mean scores also show that continuous firing has a large advantage over discrete firing for both input methods. The highest score of 23900 was attained using the continuous firing mode on the controller. The top score of 12200 attained with the discrete firing mode occurred with the keyboard. Both input methods shared the lowest score of zero, the controller attaining one zero and the keyboard attaining two zeros both with discrete firing. The controller outperformed the keyboard by 13% in terms of score for continuous firing mode. In contrast the keyboard performed 9% better than the controller in discrete firing mode. The continuous firing mode on the keyboard was 69% better than the discrete firing mode. On the controller the continuous firing mode was 108% better at attaining a higher score. It was observed that some participants had difficulty moving while repeatedly firing in discrete mode. This lack of coordination could contribute to the difference observed between the scores achieved in the different firing modes. It was also observed that while participants were able to fire bullets at the same rate in both continuous and discrete firing modes, they were able to maintain that rate of firing for longer in continuous firing mode which would contribute to the higher mean scores on both input methods.
B. Survival timeThe grand mean survival time over 240 rounds was 23.9 seconds. The mean survival time using the keyboard with continuous firing was 25.3 seconds. The mean survival time using the keyboard with discrete firing was 22 seconds. The mean survival time using the controller with continuous firing was 27.4 seconds. The mean survival time using the controller with discrete firing was 21.1 seconds (see Figure 8). The effect of input method and firing mode combinations was found to be statistically significant (F3,8 = 4.302, p < .05).
 Fig. 8. Mean survival time in seconds based on input method and firing mode. Error bars indicate ± 1 SD. Upon further analysis it was found that while keeping firing mode constant, the effect of input method was not statistically significant (F3,8 = 1.900, ns) (continuous) (F3,8 = 0.306, ns) (discrete). While keeping the input method fixed to the keyboard the effect of firing mode on survival time was not significant (F3,8 = 2.094, ns). However, with the input method fixed as the controller, the effect of firing mode on survival time was significant (F3,8 = 9.703, p < .01). The mean survival times show a trend similar to that found in the scores. The controller has a slightly better mean survival time in continuous mode, but the keyboard has a slightly better survival time in discrete mode. The best survival time of 57 seconds was attained in continuous firing mode on the keyboard. The best time attained on the controller was 55 seconds in continuous firing mode. The lowest survival time of four seconds was attained on the keyboard in discrete firing mode. The lowest survival time attained on the controller was five seconds in continuous mode. It was observed that all players were not very concerned about getting a higher score or hitting enemies with bullets; most players focused on surviving. The average survival times for all test conditions ranged from 21.1 seconds to 27.4 seconds. These averages are much closer together than the averages for score, which range from 2943.3 to 6136.7. The greater consistency observed in survival time likely correlates to the priority players placed on surviving.
C. AccuracyAccuracy was calculated as a mean accuracy over five lives in the same firing mode with the same input method. Accuracy was calculated as the percentage of the bullets fired that hit enemies. The grand mean accuracy was 31.2%. The mean accuracy using the keyboard in continuous mode was 28.8%. The mean accuracy using the keyboard in discrete mode was 33.0%. The mean accuracy using the controller in continuous mode was 28.3%. The mean accuracy using the controller in discrete mode was 34.7% (see Figure 9).
 Fig. 9. Mean accuracy as a percentage of shots hit for input method and firing mode combinations. Error bars indicate ± 1 SD. The effect of input method on accuracy was not statistically significant (F3,8 = 0.089, ns). The effect of firing mode, however, was found to be statistically significant (F3,8 = 11.497, p < .01). The mean accuracies show a reverse of the trend seen in score and survival time. The controller in discrete mode had the highest mean accuracy, followed by the keyboard in discrete mode, keyboard in continuous mode, and lastly, controller in continuous mode. The highest accuracy of 48.5% was attained using the controller in discrete mode while the lowest accuracy of 6.67% was attained by the keyboard in discrete mode. The lower accuracy observed in the continuous firing mode is likely the result of the increased rate of sustainable fire that participants were able to maintain. It was observed that of the three independent variables, accuracy was the least important to all but one participant. Most of the participants held the fire button down constantly in the continuous firing mode.
D. Participant FeedbackBased on the questionnaire it was found that nine of the participants preferred the controller while only three of the participants preferred the keyboard. Almost all of the participants preferring the controller cited comfort as the primary reason. All of the participants who preferred the keyboard cited experience as the primary reason. Based on these results, those participants who prefer the keyboard place a high value on their experience using the input method, compared to those that prefer the controller who place a high value on comfort. Comfort can be associated with experience, however. Only one participant cited a game related reason for their preference of the controller, being that they felt it provided better control. Nine of the participants preferred the continuous firing mode compared to three participants that preferred the discrete firing mode. Most of the participants that preferred the continuous mode cited the ability to focus on moving as the primary reason with easiness and enjoyment also cited twice. The reasons provided for preferring the discrete firing mode were accuracy and engagement. Only one participant provided a reason for preference of the discrete firing mode that was not directly related to a dependent variable. The other 11 participants cited the ability to kill enemies, attain high accuracy, or survive longer as reasons for their preference of firing mode. The only reason provided for preferring the discrete mode related to a dependent variable was accuracy. In contrast none of the participants cited accuracy as a reason for preferring the continuous firing mode. Instead, those that preferred continuous firing cited the ability to focus on moving (staying alive longer) as the reason for their preference. It is clear that the discrete firing mode is the preferred mode for attaining higher accuracies, while the continuous mode is the preference for longer survival times. Three participants cited exhaustion problems associated with repeatedly pressing the fire button in discrete mode. This problem could have contributed towards the lower score and survival times obtained.
IV. CONCLUSIONTwo input methods, keyboard and controller, were compared along with two firing modes, continuous and discrete. The experiment utilized Space Shooter, a 2D fixed shooter game developed for this experiment. The mean score for the keyboard in continuous firing mode was 5418.3. The mean score for the keyboard in discrete firing mode was 3211.7. The mean score for the controller in continuous firing mode was 6136.7. The mean score for the controller in discrete firing mode was 2943.3. The difference was substantial between firing modes, with the mean score more than doubling on the controller, and nearly doubling on the keyboard, with the higher value going to the continuous firing mode. Similar results were observed in the mean survival times, though not as extreme. It can be concluded that the controller using continuous firing offers the best performance in terms of score and survival time, with the controller in discrete firing mode offering the worst performances. The opposite effect was found for accuracy, with the controller in discrete mode offering the best accuracy and the controller in continuous mode offering the worst accuracy. Based on participant feedback, 75% of the participants preferred the controller over the keyboard. Seventy-five percent of the participants preferred the continuous firing mode over the discrete mode. The participant preference for input method and firing mode combinations is as follows: 16.7% preferred keyboard with continuous, 16.7% preferred keyboard with discrete, 50% preferred controller with continuous, and 16.7% preferred controller with discrete. Despite the fact that the controller with continuous firing outperformed the other combinations in both score and survival time, 50% of the participants still prefer other input method and firing mode combinations.
REFERENCES
|
||||||||||
© Copyright 2015-2024 Soullfire. |
|||||||||||