Sketch-based Interfaces
Dynamic Deep Octree for High-resolution Volumetric Painting in Virtual Reality
Abstract
With virtual reality, digital painting on 2D canvas is now being extended to 3D space. In this paper, we generalize the 2D pixel canvas to a 3D voxel canvas to allow artists to synthesize volumetric color fields. We develop a deep and dynamic octree-based painting and rendering system using both CPU and GPU to take advantage of the characteristics of both processors (CPU for octree modeling and GPU for volume rendering). On the CPU-side, we dynamically adjust an octree and incrementally update the octree to GPU with low latency without compromising the frame rates of the rendering. Our octree is balanced and uses a novel 3-neighbor connectivity for format simplicity and efficient storage, while allowing constant neighbor access time in ray casting. To further reduce the GPU-side 3-neighbor computations, we precompute a culling mask in CPU and upload it to GPU. Finally, we analyze the numerical error-propagation in ray casting through high resolution octree and present a theoretical error bound.
Supporting Information
| Filename | Description |
|---|---|
| cgf13558-sup-0001-S1.mp4194.2 MB | Supplement Material |
Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.
References
- Benson D., Davis J.: Octree textures. ACM Transactions on Graphics (TOG) 21, 3 (July 2002), 785–790. 3
- Boada I., Navazo I., Scopigno R.: Multiresolution volume visualization with a texture-based octree. The Visual Computer 17, 3 (May 2001), 185–197. 3
- Baxter W. V., Wendt J., Lin M. C.: IMPaSTo: A realistic, interactive model for paint. In Proceedings of the International Symposium on Non-Photorealistic Animation and Rendering (NPAR) (June 2004), Spencer S. N., (Ed.), pp. 45–56. 10
- Chen J., Bautembach D., Izadi S.: Scalable real-time volumetric surface reconstruction. ACM Transactions on Graphics (TOG) 32, 4 (July 2013), 113: 1–113:16. 3
- Chen Z., Kim B., Ito D., Wang H.: Wetbrush: Gpu-based 3d painting simulation at the bristle level. ACM Transactions on Graphics (TOG) 34, 6 (Oct. 2015), 200: 1–200:11. 10
- Crassin C., Neyret F., Lefebvre S., Eisemann E.: Gigavoxels: Ray-guided streaming for efficient and detailed voxel rendering. In ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (I3D) (Feb. 2009). 3
- Crassin C., Neyret F., Sainz M., Green S., Eisemann E.: Interactive indirect illumination using voxel cone tracing. Computer Graphics Forum (Proceedings of Pacific Graphics 2011) 30, 7 (Sep. 2011). 3, 10
- DeBry D. G., Gibbs J., Petty D. D., Robins N.: Painting and rendering textures on unparameterized models. ACM Transactions on Graphics (TOG) 21, 3 (July 2002), 763–768. 3
- DiVerdi S.: A Brush Stroke Synthesis Toolbox. Springer London, London, 2013, pp. 23–44. 10
- Dado B., Kol T. R., Bauszat P., Thiery J.-M., Eisemann E.: Geometry and attribute compression for voxel scenes. Computer Graphics Forum 35, 2 (2016), 397–407. 3
- Dolonius D., Sintorn E., Kämpe V., Assarsson U.: Compressing color data for voxelized surface geometry. IEEE Transactions on Visualization and Computer Graphics (TVCG) (2018). 3
- Engel K., Hadwiger M., Kniss J., Rezk-Salama C., Weiskopf D.: Real-time volume graphics. CRC Press, 2006. 3
10.1201/b10629 Google Scholar
- Frisken S. F., Perry R. N., Rockwood A. P., Jones T. R.: Adaptively sampled distance fields: A general representation of shape for computer graphics. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (2000), SIGGRAPH ‘00, ACM Press, pp. 249–254. 3
- Gobbetti E., Marton F., Guitián J. A. I.: A single-pass gpu ray casting framework for interactive out-of-core rendering of massive volumetric datasets. The Visual Computer 24, 7–9 (2008), 797–806. 3, 5
- Google: Tilt brush by google. https://www.tiltbrush.com/, 2015. 2
- Hadwiger M., Beyer J., Jeong W., Pfister H.: Interactive volume exploration of petascale microscopy data streams using a visualization-driven virtual memory approach. IEEE Transactions on Visualization and Computer Graphics 18, 12 (Dec 2012). 3
- Havran V., B ittner J., Žára J.: Ray tracing with rope trees. In 14th Spring Conference on Computer Graphics (1998), pp. 130–140. 3, 5
- Hadwiger M., Ljung P., Salama C. R., Ropinski T.: Advanced illumination techniques for gpu-based volume raycasting. In ACM SIGGRAPH 2009 Courses (2009), SIGGRAPH ‘09, ACM, pp. 2: 1–2:166. 3
- Hoetzlein R. K.: Gvdb: Raytracing sparse voxel database structures on the gpu. In Proceedings of High Performance Graphics (2016), HPG ‘16, Eurographics Association, pp. 109–117. 3, 10
- Hakkinen J., Vuori T., Paakka M.: Postural stability and sickness symptoms after hmd use. In IEEE International Conference on Systems, Man and Cybernetics (2002), Vol. 1, pp. 147–152. 5
10.1109/ICSMC.2002.1167964 Google Scholar
- Hornung A., Wurm K. M., Bennewitz M., Stachniss C., Burgard W.: Octomap: An efficient probabilistic 3d mapping framework based on octrees. Autonomous Robots 34, 3 (2013), 189–206. 3, 10
- Incorporated A. S.: Adobe photoshop user guide, 2016. URL: http://www.photoshop.com/. 9
- Ize T.: Robust bvh ray traversal-revised. Journal of Computer Graphics Techniques (JCGT) 2, 2 (2013), 12–27. 3
- Ju T., Losasso F., Schaefer S., Warren J.: Dual contouring of hermite data. ACM Transactions on Graphics (TOG) 21, 3 (2002), 339–346. 3
- Johanna B., Markus H., Hanspeter P.: State-of-the-art gpu-based large-scale volume visualization. Computer Graphics Forum 34, 8 (2015), 13–37. 3
- Kataoka D.: Art and virtual reality, new tools, new horizons. Silicon Valley VR Expo., 2017. 1
- Kobbelt L. P., Botsch M., Schwanecke U., Seidel H.-P.: Feature sensitive surface extraction from volume data. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques (New York, NY, USA, 2001), SIGGRAPH ‘01, ACM, pp. 57–66. 3
- Kazhdan M., Hoppe H.: Screened poisson surface reconstruction. ACM Transactions on Graphics (TOG) 32, 3 (2013), 29. 3
- Kniss J., Lefohn A., Strzodka R., Sengupta S., Owens J. D.: Octree textures on graphics hardware. In ACM SIGGRAPH 2005 Sketches (2005), SIGGRAPH ‘05, ACM. 3
- Kim B., Tsiotras P., Hong J., Song O.: Interpolation and parallel adjustment of center-sampled trees with new balancing constraints. The Visual Computer 31, 10 (2015), 1351–1363. 4, 10
- Kruger J., Westermann R.: Acceleration techniques for gpu-based volume rendering. In Proceedings of the 14th IEEE Visualization 2003 (VIS'03) (2003), IEEE Computer Society, p. 38. 3
- Lu J., DiVerdi S., Chen W., Barnes C., Finkelstein A.: RealPigment: Paint compositing by example. NPAR 2014, Proceedings of the 12th International Symposium on Non-photorealistic Animation and Rendering (Jun 2014). 10
- Losasso F., Gibou F., Fedkiw R.: Simulating water and smoke with an octree data structure. ACM Transactions on Graphics (TOG) 23, 3 (Aug. 2004), 457–462. 3
- Lefebvre S., Hornus S., Neyret F.: Octree textures on the gpu. In GPU Gems 2, M. Pharr, (Ed.). Addison-Wesley, 2005, pp. 595–613. 3
- Laine S., Karras T.: Efficient sparse voxel octrees. In Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (2010), I3D ‘10, ACM, pp. 55–63. 3, 5
- Lefohn A. E., Sengupta S., Kniss J., Strzodka R., Owens J. D.: Glift: Generic, efficient, random-access gpu data structures. ACM Transactions on Graphics (TOG) 25, 1 (2006), 60–99. 3, 5, 10
- MacDonald J. D., Booth K. S.: Heuristics for ray tracing using space subdivision. The Visual Computer 6, 3 (1990), 153–166. 3, 5
10.1007/BF01911006 Google Scholar
- MOJANG: Official site | minecraft. https://minecraft.net/en-us/?ref=m, 2009–2017. 8
- Museth K.: Vdb: High-resolution sparse volumes with dynamic topology. ACM Transactions on Graphics (TOG) 32, 3 (2013), 27. 3, 10
- Oculus: Quill by story studio. https://storystudio.oculus.com/en-us/, 2016. 2
- Popov S., Günther J., Seidel H.-P., Slusallek P.: Stackless kd-tree traversal for high performance gpu ray tracing. Computer Graphics Forum 26, 3 (2007), 415–424. 3, 5
- Patterson R., Winterbottom M. D., Pierce B. J.: Perceptual issues in the use of head-mounted visual displays. Human factors 48, 3 (2006), 555–573. 5
- Regan C.: An investigation into nausea and other side-effects of head-coupled immersive virtual reality. Virtual Reality 1, 1 (1995), 17–31. 5
10.1007/BF02009710 Google Scholar
- Reichl F., Treib M., Westermann R.: Visualization of big sph simulations via compressed octree grids. In 2013 IEEE International Conference on Big Data (2013), pp. 71–78. 3
- Setaluri R., Aanjaneya M., Bauer S., Sifakis E.: Spgrid: A sparse paged grid structure applied to adaptive smoke simulation. ACM Transactions on Graphics (TOG) 33, 6 (2014), 205. 10
- Samet H.: Implementing ray tracing with octrees and neighbor finding. Computers & Graphics 13, 4 (1989), 445–460. 3, 5
- Zhou K., Gong M., Huang X., Guo B.: Data-parallel octrees for surface reconstruction. IEEE Transactions on Visualization and Computer Graphics 17, 5 (2011), 669–681. 3
