View source

one way to economize on bandwidth in single user head mounted displays is to put high res info
only where the user is currently looking

summarizes results from 6 studies investigating spatial, resolutional and temporal parameters 
affecting perception and perfomance in eye-contingent mult-resolutional displays.

most of the resources that are used to produce a large, high-resolution display are
wasted, especially with single-user displays

the human visual system can only resolve detailed information within a very small area at the center of
vision, with resolution dropping off rapidly as one moves into the visual periphery

one can save image bandwidth by producing images in which higher spatial frequency information is only encoded at the
center of vision, leaving only lower frequency information in the periphery.

this requires that one know at any given moment where in the image the viewer is looking so that
high frequency information from that location in the image can be made available to the viewer. 
This means that the image must be dynamically updated based on real-time gaze direction
information. We refer to this as a Gaze Contingent Multiresolutional Display (GCMRD),

The simplest type of multiresolutional display, of course, is to have only two regions with 
different resolutions; more complex displays could reduce the resolution in steps at more visually
eccentric locations, or smoothly degrade the image from the center of vision outward, attempting
to match the spatial resolution function of the retina.

Two studies have found a functional tradeoff between size of the high-resolution window at
the center of vision and the level of peripheral degradation

Specifically, the more degraded the periphery is, the larger the window of high resolution must 
be to achieve equivalent performance, whether measured in terms of recognition memory OR visual 
search times

Shioiri and Ikeda found that "useful resolution was substantially less than available
resolution" that is, measures of peripheral acuity appear to over-predict viewers' ability 
to use information from the periphery.

Two other studies investigated the effects of varying the type of information available outside 
the high-resolution window

Wampers et al. found that the most disruptive form of peripheral degradation is that in which there
is only low frequency information--a periphery with only high-frequency information was less disruptive
to both search times and eye movement measures

the other 2 studies used more realistic images and involved eye tracking but did not independently 
manipulate window size and peripheral degradation making it impossible to determine
the interaction of these factors.


All experiments used a set of 15 complex, monochromatic, photographic scenes as stimuli. Images were
transformed such that a circular, high-resolution region was surrounded by a degraded peripheral region

first experiment
----------------
In a gaze contingent multi-resolutional display ("moving window"), the image on the screen can be changed either
continuously (updated on each frame based on current eye position) or only at the end of every saccadic eye movement,
keeping the image stable during eye fixations even when the actual position of the eyes drift somewhat.
they opted for the second approach, updating the image only at the end of each saccade, in order to reduce 
the likelihood of actual perception of the image changes taking place.

Image changes made during a saccadic eye movement go unnoticed because the visual system has a temporarily
higher perceptual threshold, or "saccadic suppression". However, if the change takes place very long after
the eyes have stopped moving, it will likely be noticed. As a precursor to the moving window studies,
they performed a study to determine the earliest time following an eye movement that a change to the 
display, of the type planned, could be detected.

In this experiment a high-resolution version of a scene was initially displayed on the screen. 
However, at the beginning of selected saccades, this image was replace by a highly degraded
version of the same picture. they then changed the degraded version back to the high-resolution
version after a variable number of milliseconds. Participants were asked to press a button if 
they detected a flicker or blurred image.

The results showed that for an image change to go undetected, it must be started within 5 ms 
after the end of the eye movement.

studies 2 and 3
------------------
the next question concerned the perceptibility of peripheral degradation in a multi-resolutional display.

Two studies were conducted toi nvestigate the detection of peripheral image degradation with a
stationary window briefly flashed at the center of vision for 150 ms

the participant's only task was to push a button if he/she detected peripheral degradation,

in the first study, the window sizes ranged from 2 degrees to 5 degrees radius and there
were 4 levels of peripheral degredation.
the least level of degredation went unnoticed even at the smallest window size (2 degrees)
the opposite was true at the highest level -- was detectable even at largest window

in intermediate levels, window size made a difference, larger windows led to less detection.

the second experiment used the same degredation with different window sizes (1.6-4.1 degrees)
and added 2 window edge sharpness conditions.. smooth and sharp.

edge type had no effect on detectability, and all other results were the same except they found 
that moderate degredation was seldom detected at the largest window size (4.1 degrees).

study 4
-------
how performance and eye movement behavior in natural tasks would be affected by use of a gaze
contingent, multi-resolutional display.

within 5 ms of the end of each eye movement the image on the screen was updated, selecting from 
among 330 versions of the image to center the high-resolution window on the viewer's fovea.
Participants viewed a picture for varying times up to 20 sec. The window sizes and degradation 
levels were a subset of those used in the other detection studies. Participants' tasks were
to search for an object in the scene, or to remember the scene.

search times were significantly slower in the smallest window condition (1.6 Â° ) than in the 
largest window condition (4.1Â°), which did not differ from the control baseline.
In contrast to the detection results, however, search times were less affected by the level 
of peripheral degradation. Time spent in each eye fixation (fixation durations) produced a 
similar pattern of results. On the other hand, both window size and peripheral degradation 
level significantly affected the mean distance traveled by the eyes in saccadic movements
(saccade length). Specifically, saccade lengths were shorter in the smallest window condition
than in the largest window condition, which did not differ from the control baseline. The smaller the
window and the greater the peripheral degradation, the greater the tendency for the eyes to be 
sent to locations within the highresolution window. Overall, a window size of roughly 4 Â° radius
with moderate peripheral degradation resulted in eye movements and search performance that were 
quite comparable to the situation in which there was no window at all.\

5th study
---------
investigated whether detection of peripheral degradation is reduced when the viewer is engaged 
in realistic, ongoing tasks such as visual search

participants were engaged in the same tasks as in the moving window experiment
(i.e., search and memory), but the window of high resolution with peripheral degradation appeared
only occasionally on single fixations, with a full high-resolution image present at all other
times. Participants were given a secondary task, which was to press a button any time they noticed
peripheral degradation. Window sizes and degradation levels were the same as in the moving
window study.

the results were virtually identical to those of the previous detection studies.

this was surprising because in those studies, participants' only task was to detect degradation,
whereas in this study their primary task was presumably much more demanding --
search for an object or conduct a detailed inspection of the image in preparation for a memory test

6th study
---------
investigated how quickly the gaze contingent multi-resolutional display must be updated at the end
of each eye movement in order to maintain natural task performance and eye movements

since they had a very high quality head tracker and precomputed images stored in a very large image
memory, they assumed that most other head mounted systems would be slower and wanted to know
how slow it can be without adversely affecting performance and eye movements.

used the same tasks and measures as the moving window study, but with two degradation levels, two
window sizes (1.6 Â° and 4.1 Â°) and three image update deadlines (5,15, and 45 ms following the end 
of a saccade).

The search time results showed no effect of updating deadline, though there were
strong effects for both window size and peripheral degradation
level. As in the moving window experiment, a window size of
1.6 Â° produced longer search times than a window size of 4.1 Â°
radius, which was very similar to the control condition. However,
contrary to our previous results, low degradation level also produced
longer search times than high, which was also similar to the
control.

attribute this difference in results to the greater power
of our current study because we included twice as many
participants as the previous study.

The results indicated that a 45 ms delay in updating a gaze contingent multi-resolutional
display may be too long, especially with the window sizes used in
this study. viewers' average fixation times were reliably longer for 45 ms delays than for 
either 5 or 15 ms delays, which did not differ from each other. 

The results for window size and degradation level were consistent with the earlier
moving window study