GCD Summary1
From EQUIS Lab Wiki
one way to economize on bandwidth in single user head mounted displays is to put high res info only where the user is currently looking
summarizes results from 6 studies investigating spatial, resolutional and temporal parameters affecting perception and perfomance in eye-contingent mult-resolutional displays.
most of the resources that are used to produce a large, high-resolution display are wasted, especially with single-user displays
the human visual system can only resolve detailed information within a very small area at the center of vision, with resolution dropping off rapidly as one moves into the visual periphery
one can save image bandwidth by producing images in which higher spatial frequency information is only encoded at the center of vision, leaving only lower frequency information in the periphery.
this requires that one know at any given moment where in the image the viewer is looking so that high frequency information from that location in the image can be made available to the viewer. This means that the image must be dynamically updated based on real-time gaze direction information. We refer to this as a Gaze Contingent Multiresolutional Display (GCMRD),
The simplest type of multiresolutional display, of course, is to have only two regions with different resolutions; more complex displays could reduce the resolution in steps at more visually eccentric locations, or smoothly degrade the image from the center of vision outward, attempting to match the spatial resolution function of the retina.
Two studies have found a functional tradeoff between size of the high-resolution window at the center of vision and the level of peripheral degradation
Specifically, the more degraded the periphery is, the larger the window of high resolution must be to achieve equivalent performance, whether measured in terms of recognition memory OR visual search times
Shioiri and Ikeda found that "useful resolution was substantially less than available resolution" that is, measures of peripheral acuity appear to over-predict viewers' ability to use information from the periphery.
Two other studies investigated the effects of varying the type of information available outside the high-resolution window
Wampers et al. found that the most disruptive form of peripheral degradation is that in which there is only low frequency information--a periphery with only high-frequency information was less disruptive to both search times and eye movement measures
the other 2 studies used more realistic images and involved eye tracking but did not independently manipulate window size and peripheral degradation making it impossible to determine the interaction of these factors.
All experiments used a set of 15 complex, monochromatic, photographic scenes as stimuli. Images were
transformed such that a circular, high-resolution region was surrounded by a degraded peripheral region
first experiment
In a gaze contingent multi-resolutional display ("moving window"), the image on the screen can be changed either continuously (updated on each frame based on current eye position) or only at the end of every saccadic eye movement, keeping the image stable during eye fixations even when the actual position of the eyes drift somewhat. they opted for the second approach, updating the image only at the end of each saccade, in order to reduce the likelihood of actual perception of the image changes taking place.
Image changes made during a saccadic eye movement go unnoticed because the visual system has a temporarily higher perceptual threshold, or "saccadic suppression". However, if the change takes place very long after the eyes have stopped moving, it will likely be noticed. As a precursor to the moving window studies, they performed a study to determine the earliest time following an eye movement that a change to the display, of the type planned, could be detected.
In this experiment a high-resolution version of a scene was initially displayed on the screen. However, at the beginning of selected saccades, this image was replace by a highly degraded version of the same picture. they then changed the degraded version back to the high-resolution version after a variable number of milliseconds. Participants were asked to press a button if they detected a flicker or blurred image.
The results showed that for an image change to go undetected, it must be started within 5 ms after the end of the eye movement.
studies 2 and 3
the next question concerned the perceptibility of peripheral degradation in a multi-resolutional display.
Two studies were conducted toi nvestigate the detection of peripheral image degradation with a stationary window briefly flashed at the center of vision for 150 ms
the participant's only task was to push a button if he/she detected peripheral degradation,
in the first study, the window sizes ranged from 2 degrees to 5 degrees radius and there were 4 levels of peripheral degredation. the least level of degredation went unnoticed even at the smallest window size (2 degrees) the opposite was true at the highest level -- was detectable even at largest window
in intermediate levels, window size made a difference, larger windows led to less detection.
the second experiment used the same degredation with different window sizes (1.6-4.1 degrees) and added 2 window edge sharpness conditions.. smooth and sharp.
edge type had no effect on detectability, and all other results were the same except they found that moderate degredation was seldom detected at the largest window size (4.1 degrees).
study 4
how performance and eye movement behavior in natural tasks would be affected by use of a gaze contingent, multi-resolutional display.
within 5 ms of the end of each eye movement the image on the screen was updated, selecting from among 330 versions of the image to center the high-resolution window on the viewer's fovea. Participants viewed a picture for varying times up to 20 sec. The window sizes and degradation levels were a subset of those used in the other detection studies. Participants' tasks were to search for an object in the scene, or to remember the scene.
search times were significantly slower in the smallest window condition (1.6 ° ) than in the largest window condition (4.1°), which did not differ from the control baseline. In contrast to the detection results, however, search times were less affected by the level of peripheral degradation. Time spent in each eye fixation (fixation durations) produced a similar pattern of results. On the other hand, both window size and peripheral degradation level significantly affected the mean distance traveled by the eyes in saccadic movements (saccade length). Specifically, saccade lengths were shorter in the smallest window condition than in the largest window condition, which did not differ from the control baseline. The smaller the window and the greater the peripheral degradation, the greater the tendency for the eyes to be sent to locations within the highresolution window. Overall, a window size of roughly 4 ° radius with moderate peripheral degradation resulted in eye movements and search performance that were quite comparable to the situation in which there was no window at all.\
5th study
investigated whether detection of peripheral degradation is reduced when the viewer is engaged in realistic, ongoing tasks such as visual search
participants were engaged in the same tasks as in the moving window experiment (i.e., search and memory), but the window of high resolution with peripheral degradation appeared only occasionally on single fixations, with a full high-resolution image present at all other times. Participants were given a secondary task, which was to press a button any time they noticed peripheral degradation. Window sizes and degradation levels were the same as in the moving window study.
the results were virtually identical to those of the previous detection studies.
this was surprising because in those studies, participants' only task was to detect degradation, whereas in this study their primary task was presumably much more demanding -- search for an object or conduct a detailed inspection of the image in preparation for a memory test
6th study
investigated how quickly the gaze contingent multi-resolutional display must be updated at the end of each eye movement in order to maintain natural task performance and eye movements
since they had a very high quality head tracker and precomputed images stored in a very large image memory, they assumed that most other head mounted systems would be slower and wanted to know how slow it can be without adversely affecting performance and eye movements.
used the same tasks and measures as the moving window study, but with two degradation levels, two window sizes (1.6 ° and 4.1 °) and three image update deadlines (5,15, and 45 ms following the end of a saccade).
The search time results showed no effect of updating deadline, though there were strong effects for both window size and peripheral degradation level. As in the moving window experiment, a window size of 1.6 ° produced longer search times than a window size of 4.1 ° radius, which was very similar to the control condition. However, contrary to our previous results, low degradation level also produced longer search times than high, which was also similar to the control.
attribute this difference in results to the greater power of our current study because we included twice as many participants as the previous study.
The results indicated that a 45 ms delay in updating a gaze contingent multi-resolutional display may be too long, especially with the window sizes used in this study. viewers' average fixation times were reliably longer for 45 ms delays than for either 5 or 15 ms delays, which did not differ from each other.
The results for window size and degradation level were consistent with the earlier moving window study
