Search

This chapter provides a high-level overview of fifteen years of augmented reality research that was sponsored by the U.S. Office of Naval Research (ONR). The research was conducted at Columbia University and the U.S. Naval Research Laboratory (NRL) between 1991 and 2005 and supported in the later years by a number of university and industrial research laboratories. It laid the groundwork for the development of many commercial mobile augmented reality (AR) applications that are currently available for smartphones and tablets. Furthermore, it helped shape a number of ongoing research activities in mobile AR., Published in Expanding the Frontiers of Visual Analytics and Visualization, p431-448, 2012. Prepared in collaboration with the National Science Foundation, Washington, D.C., Coulumbia, Univ, New York, NY, University College London, UK, and Mississippi State University, Mississippi State, MS.

The full power of mobile augmented and virtual reality systems is realized when these systems are connected to one another, to immersive virtual environments and to remote information servers. Connections are usually made through wireless networks. However, wireless networks cannot guarantee connectivity and their bandwidth can be highly constrained. This paper presents a robust event-based data distribution mechanism for mobile augmented reality and virtual environments. It is based on replicated databases, pluggable networking protocols, and communication channels. The mechanism is demonstrated in the Battlefield Augmented Reality System (BARS) situation awareness system, composed of several mobile augmented reality systems, immersive and desktop-based virtual reality systems, a 2D mapbased multimodal system, handheld PCs, and other sources of information such as external data servers., Published in the Journal of Presence, v13 n2 p211-221, April 2004.

We describe a first experiment in evaluating the system capabilities of the Battlefield Augmented Reality System, an interactive system designed to present military information to dismounted warfighters. We describe not just the current experiment, but a methodology of both system evaluation and user performance measurement in the system, and show how both types of tests will be useful in system development. We summarize results in a perceptual experiment being used to inform system design, and discuss ongoing and future experiments to which the work described herein leads., Presented at the NIST Workshop on Performance Metrics for Intelligent Systems held in Gaithersburg, MD on 24-26 August 2004. Published in the Proceedings of the NIST Workshop on Performance Metrics for Intelligent Systems, August 2004. The original document contains color images.

Mobile augmented reality (AR) is a method for providing a "head up display" to individual dismounted users. A user wears a miniaturized computer system, tracking sensors, and a see-through graphics display. The system superimposes three-dimensional spatially registered graphics and sounds onto the user's perception of the real world. Because information can be presented in a head up and hands free way, it has the potential to revolutionize the way in which information is presented to individuals. A mobile AR system can insert friendly, neutral, and enemy computer-generated forces (CGFs) into the real world for training and mission rehearsal applications. The CGFs are drawn realistically and properly occluded with respect to the real world. The behaviors of the CGFs are generated from two Semi-Automated Forces (SAF) systems: JointSAF and OneSAF. The AR user appears as an individual combatant entity in the SAF system. The AR user's position and orientation are fed to the SAF system, and the state of the SAF entities is reflected in the AR display. The SAF entities react to the AR user just as they do any other individual combatant entity, and the AR user interacts with the CGFs in real time. In this paper, we document the development of a prototype mobile AR system for embedded training and its usage in MOUT-like situations. We discuss the tradeoffs of the components of the hardware (tracking technologies, display technologies, computing technologies) and the software (networking, SAF systems, CGF generation, model construction), and we describe the lessons that have been learned from implementing several scenarios., Published in the Proceedings of the Interservice/Industry Training, Simulation, and Education Conference (I/ITSEC 2004) held in Orlando, FL on 6-9 December 2004. The original document contains color images.

Uncertain temperature-salinity depth profile information results in inaccurate prediction of the curvature of the acoustic rays. Uncertain bathymetry (knowledge of the ocean bottom) includes misregistered maps, data gaps in maps, temporal changes to the ocean bottom, and limited knowledge of the mineral content of the ocean bottom. These factors again limit the accuracy of position estimations based on the acoustic returns. Finally, local oceanic effects (e.g., internal waves, thermal currents, fine-structure) also impact the acoustic return and thus add to the uncertainty surrounding the target's position. The resulting uncertainty has four dimensions: latitude, longitude, depth, and signal strength. The target state prediction process (i.e. determining the position, heading, etc. of a target) involves data that spans multiple dimensions caused by the uncertainties discussed above. We investigate how to represent the resulting multivariate information and multi-dimensional uncertainty by developing and applying candidate visual techniques. At this time, our primary focus is to (1) develop the statistical characterizations for the environmental uncertainty (described only briefly in this paper) and (2) develop a visual method for each characterization. We received feedback on the applicability of our techniques from domain experts. We used this in conjunction with previous results to compile a set of development guidelines (some obvious, others not). We first review a select number of applicable visualization techniques. We then describe the investigations for representing the bathymetric information in Section 3 and the target state estimation uncertainty in Section 4. Section 5 describes briefly the display-system architecture we developed to demonstrate the visual candidates. Finally, in Section 6 we summarize the results and compile a list of guidelines for developing visualization techniques for multidimensional uncertainty and multi-variate information., Published in Institute of Electical and Electronics Engineers (IEEE) Computer Graphics and Applications, v24 n5 p56-65, Sep-Oct 2004.The original document contains color images.

The Battlefield Augmented Reality System (BARS) is a mobile augmented reality system that displays head-up battlefield intelligence information to a dismounted warrior. BARS consists of a wearable computer, a wireless network, and a tracked see-through Head Mounted Display (HMD). The computer generates graphics that, from the user's perspective, appear to exist in the surrounding environment. For example, a building could be augmented to show its name, a plan of its interior, icons to represent reported hazard locations, and the names of adjacent streets. The full power of mobile augmented reality systems is realized when these systems are connected to one another, to immersive virtual environments, and to remote information servers. These connections are made through wireless devices that cannot guarantee connectivity and may have highly constrained bandwidth. Based on these constraints, we present a robust event-based data distribution mechanism for mobile augmented reality and virtual environments. It is based on replicated databases, pluggable networking protocols, and communication channels. For use in simulation and training exercises, we have been working with U.S. Army RDECOM to create an interface between this data distribution mechanism and a Semi-Automated Forces (SAF) system. With this interface, the BARS user appears as a dismounted warrior in the SAF system-the BARS user's position and orientation are fed to the SAF system, and the state from the SAF system is sent back to the BARS user's display. With this SAF interface, BARS becomes a training system that works in a real location (as compared to a virtual reality simulation) to make simulated forces appear to exist in and interact with the real world., Proceedings of the Interservice / Industry Training, Simulation, & Education Conference "I/ITSEC '03", Orlando, FL, December 1-4, 2003. The original document contains color images.

Many future military operations are expected to occur in urban environments. These complex, 3D battlefields introduce many challenges to the dismounted warfighter. Better situational awareness is required for effective operation in urban environments. However, delivering this information to the dismounted warfighter is extremely difficult. For example, maps draw a user's attention away from the environment and cannot directly represent the three-dimensional nature of the terrain. To overcome these difficulties, we are developing the Battlefield Augmented Reality System (BARS). The system consists of a wearable computer, a wireless network system, and a tracked see-through head-mounted display (HMD). The computer generates graphics that, from the user's perspective, appear to be aligned with the actual environment. For example, a building could be augmented to show its name, a plan of its interior, icons to represent reported sniper locations, and the names of adjacent streets. This paper surveys the current state of development of BARS and describes ongoing research efforts. We describe four major research areas. The first is the development of an effective, efficient user interface for displaying data and processing user inputs. The second is the capability for collaboration between multiple BARS users and other systems. Third, we describe the current hardware for both a mobile and indoor prototype system. Finally, we describe initial efforts to formally evaluate the capabilities of the system from a user's perspective through scenario analysis. We also will discuss the use of the BARS system in STRICOM's Embedded Training initiative., Presented at the Interservice/Industry Training, Simulation and Education Conference (I/ITSEC-02) held in Orlando, Florida on 2-5 December 2002. Published in the Proceedings of the Interservice / Industry Training, Simulation, and Education Conference, p1-8, 2002. The original document contains color images.

The acoustic lens is a high-resolution, forward-looking sonar for three-dimensional (3-D) underwater imaging. In this paper, we discuss processing the lens data for recreating and visualizing the scene. Acoustical imaging, compared to optical imaging, is sparse and low resolution. To achieve higher resolution, we obtain a denser sample by mounting the lens on a moving platform and passing over the scene. This introduces the problem of data fusion from multiple overlapping views for scene formation, which we discuss. We also discuss the improvements in object reconstruction by combining data from several passes over an object. We present algorithms for pass registration and show that this process can be done with enough accuracy to improve the image and provide greater detail about the object. The results of in-water experiments show the degree to which size and shape can be obtained under (nearly) ideal conditions., Published in the Journal of IEEE Transactions on Image Processing, v7 n1 p91-99, January 1998.

Both the National Research Council (NRC) report “Virtual Reality: Scientific and Technological Challenges” and the more recent NRC report “Modeling and Simulation: Linking Entertainment and Defense” discuss the need for nontraditional degree programs— programs that focus more closely on the issues of how we develop the software and content for networked virtual environments (VEs). Both reports point out that the more effective VE developer is not just a computer scientist, electrical engineer, or human factors specialist, but rather a scientist who sits in-between, a scientist whose education melds the precise parts required for developing VEs. To produce such graduates requires new degree programs. The Naval Postgraduate School (NPS) has developed one such program, the Modeling, Virtual Environments, and Simulation (Moves) degree program (http://www.moves.nps.navy.mil/). We present the composition of that degree program and its relationship to the research our students can then handle at the end of that program.

In spite of the large amount of hype that accompanied virtual reality (VR) earlier this decade, the field has produced only a few examples of demonstrably useful systems. Systems must be fielded and validated to show that VR is useful for purposes other than academic research. This paper discusses two VR systems developed at the Naval Research Laboratory that have received validation by statistical analysis or by user acceptance. One system focuses on experiments in shipboard firefighting to verify the effectiveness of VR as a mission planning tool. Using trained U.S. Navy firefighters, we conducted a shipboard experiment. The VR-trained firefighters performed significantly better on both navigation and firefighting tasks. The second system involved developing an application using NRL's VR Responsive Workbench to provide situational awareness inside a U.S. Marine Corps combat operations center (COC). This system has been called a major advance that is likely to eliminate paper maps in the COC., Presented at Virtual Systems and Multimedia (VSMM) '97 held in Geneva, Switzerland, Sep 1997.

Long, towed hydrophone arrays are useful in a sonar context as high-gain, high-resolution receivers. However, conventional beamforming assumes a straight, horizontal aperture. Towed arrays undergo dynamic shape changes due to ship track instabilities and to subsurface currents. This report documents a constructive algorithm for determining array shape as well as the results of using this algorithm to determine array shapes from the Spring 1979 FREDDEX experiment. It is shown that the algorithm works successfully on this data set and indicates that array deformation for FREDDEX was minimal. A typical value for the standard deviation of the array was 0.2 m. The lowest spatial mode (the bow shape) predominated and cycled in about 16 min. Acoustic degradation was evaluated for several frequencies and was always well below one dB. Increased sidelobes were apparent, but they were still 35 dB below the maximum response. (Author)

The Neil Brown/W.H.O.I. CTD Microprofiler has been extensively used by a variety of NRL researchers in at-sea experiments. This paper reports on software for both real-time data acquisition and display and for processing collected data. (Author)