Virtual Reality and Physically-Based Simulation WS 14/15
Virtual Reality (VR) is a research area at the intersection of computer graphics, physically-based simulation, and 3D human-computer interaction (HCI). VR is concerned with novel interaction devices and interaction metaphors in virtual environments (in particular intuitive and direct metaphors), immersion and presence, and real-time rendering. Another important topic is physically-based simulation in real-time, which tries to simulate real-world phenomena such as fire, cloth, the behavior of rigid objects when colliding with each other, fluids, or objects made of deformable material.
Over the past one or two decades, VR has established itself as an important tool in several industries, such as manufacturing (e.g., automotive, airspace, ship building), architecture, and pharmaceutical industries. In addition, a lot of the techniques and solutions developed in the area of VR can be applied directly to computer games.
In this course, we will first look at the fundamental methods, and then go on to more advanced algorithms that are needed to build complex and full-fledged VR systems or real-time computer games. Example topics are object behavior, acoustic rendering, haptics, and collision detection.
The assignments will be mostly practical ones, based on the cross-platform
VR system
InstantReality.
Participants can choose the programming language from
Java, Javascript, and C++.
You are encouraged to work on assignments in small teams.
Some of the topics to be covered (tentatively):
- Introduction, basic notions of VR, a few applications
- VR devices: displays, tracking, input devices, software design
- Stereo rendering
- Error correction: tracking correction, filtering
- Techniques for real-time rendering
- Fundamental immersive interaction techniques: gesture recognition, navigation, selection, grasping, menues in 3D
- Complex immersive interaction techniques: world-in-miniature, action-at-a-distance, magic lens, etc.
- Collision detection
- Force feedback: rendering forces
- Acoustic rendering
- Particle systems
- Spring-mass systems
Note: this list is just tentative and subject to change during the semester.
News
Slides
The following table contains all the topics that were covered in this class, the accompanying slides, exercise sheets, and frameworks for solving the programming exeercises. (This table will be filled week by week.)
Week | Topics |
---|---|
1. |
Introduction (definitions, immersion, fidelity, presence, history, Milgram's continuum), |
2. | Scenegraphs: immediate / retained mode, semantics of nodes and edges, hierarchical transformations, shared geometry, multi-threaded scenegraphs, distributed rendering. |
3. | The concept of fields-&-routes, introduction to VRML/X3D, indexed face set et al., grouping, re-using scene sub-graphs by DEF/USE, behavior graph, events and routes, behavior nodes, event cascades |
4. | Stereo rendering 1: depth cues, human visual system and stereopsis, stereo projection, issues with stereo rendering (convergence-focus incongruity, stereo violation image multiplexing (e.g., shuttering, infitec, etc.), |
5. | Stereo rendering 2: immersive projection technologies, autostereograms, projetion issues for co-located multiple users, rendering on HMDs, low and full persistence, space-time diagrams, smearing and judder and color fringes. |
6. | Real-time rendering: sources of latency, view-independent rendering, level-of-detail techniques, predictive LOD selection, progressive meshes, |
7. |
Real-time rendering 2:
state sorting using the online
sorting buffer algorithm,
latencies with conventional rendering,
image warping, Input Devices: degrees of freedom, multimodality, virtual trackball, directness continuum, isotonic vs isometric, tracking, data gloves, locomotion devices |
8. | Interaction metaphors 1: Universal Interaction Tasks, design of user interfaces, gesture recognition, the taxonomy, navigation metaphors (point-and-fly et al.), |
9. |
Interaction metaphors 2:
magic mirror, 3D navidget,
user models (power law of practice, Hick's law,
Fitts's law), selection techniques, go-go, ray-based techniques, control-display
ratio,
cone-based techniques, flexible pointer, bent ray,
iSith,
balloon selection,
action-at-a-distance, image plane interaction, proprioceptive interaction,
world-in-miniature, voodoo dolls, magic lenses,
marking menus,
|
10. | Particle systems: dynamics/kinematics, Euler integration, phase space, definition of particle systems, physical effects, non-physical effects, collision handling, rendering, rendering transparent objects, procedural modeling of plants, |
11. | Mass-spring systems: Newton's Laws, single spring-damper, explicit Euler integration, instability and error accumulation with explicit Euler integration, Runge-Kutta, Verlet integration, constraints, implicit integration, tangent stiffness matrix, comparison to explicit integration, mesh creation for volumetric objects, consistent collision response for volumetric mass-spring systems. |
Here are some examples and demos of VRML, some of which I have presented in class. You can use them, for instance, to test your VRML client/browser setup.
Literature
- Ralf Dörner, Wolfgang Broll, Paul Grimm, Bernhard Jung (Hrsg.): Virtual und Augmented Reality (VR/AR), Grundlagen und Methoden der Virtuellen und Augmentierten Realität. Springer 2013. You can read the e-book from within the university's network.
- Kay M. Stanney (Ed.): Handbook of Virtual Environments. Lawrence Erlbaum Associates, 2002
- William R. Sherman, Alan B. Craig: Understanding Virtual Reality. Morgan Kaufmann.
- Don Brutzman, Leonard Daly: X3D: Extensible 3D Graphics for Web Authors. Morgan Kaufmann, 2007.
- Daniel Fleisch: A Student's Guide to Vectors and Tensors. Cambridge University Press
- Kenny Erleben et al.: Physics Based Animation. Charles River Media, 2005.
- Mario Gutiérrez, Frédéric Vexo, Daniel Thalmann: Stepping into Virtual Reality. Springer, 2008. You can read the e-book from within the university's network.
- Anthony Steed, Manuel Oliveira: Networked Graphics: Building Networked Games and Virtual Environments. Morgan Kaufman, 2009. From this book, only chapters 7, 10, and 11 are relevant to this course.
Warning: these text books can only give you a general introduction to the field of VR! Most of the topics taught in class will not be covered by any of these text books directly -- in fact, AFAIK there are no text books that cover these topics. Therefore, I recommend to attend class.
If you are thinking of buying some of these books, then I suggest to consider buying a used copy of them -- very often, you can find them at a fraction of the price of a new copy. The following are two very good internet sites for finding inexpensive used copies of books: Abebooks and BookButler.
Assignments
In order to acquire the "Schein", you will have to do a number of assignments. These will be mostly small practical programming assignments (based on a freely available VR system).
Online Literature and Resources on the Internet
- The original article praising the panorama in Blackwood's Edinburgh Magazine (vol. 15, 1824)
- Interview with Jaron Lanier (it was part of the program "Druckfrisch" and aired on 5.10.2014 on the German ARD)
- In this category, you'll find links concerning X3D/VRML
- Tools that can "play" X3D- / VRML97 files:
- InstantReality; on their homepage, you can also find a lot of examples and tutorials.
- FreeWRL (for Linux and OSX).
- Cortona; this is a commercial browser plugin.
- Introduction to VRML / X3D:
- A "Cheat Sheet" for VRML.
- SIGGRAPH 2008 Class Notes: Don't be a WIMP; These course notes not only discuss some post-WIMP interaction techniques, but also explain how to use some of the more advanced features of InstantReality, such as different stereo rendering modi, special interaction devices, clustering, scripting, animations, etc. (Source)
- The Annotated VRML97 Reference Manual (Source)
- The VRML Primer and Tutorial; those chapters that deal with HTML are not relevant for this course. In addition, the chapter on tools is outdated; but otherwise it is still a veritable introduction for VRML novices. (Source)
-
The most important documents on the X3D standard:
- Architecture and base components (i.e., the specification of nodes and profiles)
- Scene access interface (SAI)
- X3D language bindings for Javascript (ECMAScript)
- X3D language bindings for Java
- Examples for X3D / VRML:
- The examples from the lecture (and many more!)
- More examples are on the homepage of InstantReality;
- And on the homepage of the X3D book by Don Brutzman and Leonard Daly;
- A meta web page with lots of links to archives containing more examples.
- A large collection of materials,
both in VRML- as well as in XML encoding.
And here it is again as a ZIP archive. - Links on socalled authoring tools; for VRML/X3D; if you want, you are welcome to use them. But if you have a decent programming editor (preferrably in ASCII), then you are probably more efficient with your editor.
- A handy little tool for Calculating VRML Viewpoints.
- Here are the -- not quite serious -- Cartoon Laws of Physics ;-) .
- Literature on stereoscopic ("3D") rendering:
- Implementing Stereoscopic 3D in Your Applications by Samuel Gateau and Steve Nash from NVidia at GPU Technology Conference 2010 (Source)
- Rendering 3D Anaglyph in OpenGL by Animesh Mishra, 2011 (Source)
- More literature on the topic of user interface design:
- The Siggraph 2001 tutorial Advanced Topics in 3D User Interface Design
- An entertaining and very insightful Quiz on Fitts' Law
- A survey where Fitts' law has been used in the UI design of Office 2007: Giving You Fitts
- Literature on particle and on spring-mass systems:
- William T. Reeves: Particle Systems - A Technique for Modeling a Class of Fuzzy Objects;
- More advanced particle systems (e.g., n-body systems) are explained in Real-Time Particle Systems on the GPU in Dynamic Environments by Shannon Drone (Source)
- Georgii, Westermann: Mass-Spring Systems on the GPU, Simulation Practice and Theory 2005. (Source)
- Literature on physically-based simulation in general:
- Introductory Vector Calculus by Norman Wittels (Source)
- Siggraph 2008 course notes: Real Time Physics
- Siggraph 2001 course notes: Physically Based Modeling
- Survey paper over the field of deformation simulation methods: Physically Based Deformable Models in Computer Graphics by Andrew Nealen, Mathias Muller, Richard Keiser, Eddy Boxerman and Mark Carlson, EG 2005
- A bit on the history of Virtual and Mixed Reality:
- An article from The Verge: The Rise and Fall and Rise of Virtual Reality by Adi Robertson and Michael Zelenko (Source, in particular here)
- Highlights from an interview with at The Verge: Digital Natives, A conversation between virtual reality visionaries Jaron Lanier and Kevin Kelly by Casey Newton (Source, in particular here)
- The Ultimate Display by Donald Sutherland
- The short novel
Gegen den Strich
by Joris-Karl Huysmans.
(Sorry, I have it in German only) Denkanregung: was hat das mit VR zu tun?
Last modified: Thu Jan 29 00:10:49 CET 2015