A self-sparse generative adversarial network for autonomous early-stage design of architectural sketches
Wenliang Qian
Key Lab of Smart Prevention and Mitigation of Civil Engineering Disaster, Ministry of Industry and Information Technology, Harbin, China
Key Lab of Structures Dynamic Behavior and Control, Ministry of Education, Harbin, China
School of Civil Engineering, Harbin Institute of Technology, Harbin, China
Search for more papers by this authorYang Xu
Key Lab of Smart Prevention and Mitigation of Civil Engineering Disaster, Ministry of Industry and Information Technology, Harbin, China
Key Lab of Structures Dynamic Behavior and Control, Ministry of Education, Harbin, China
School of Civil Engineering, Harbin Institute of Technology, Harbin, China
Search for more papers by this authorCorresponding Author
Hui Li
Key Lab of Smart Prevention and Mitigation of Civil Engineering Disaster, Ministry of Industry and Information Technology, Harbin, China
Key Lab of Structures Dynamic Behavior and Control, Ministry of Education, Harbin, China
School of Civil Engineering, Harbin Institute of Technology, Harbin, China
Correspondence
Hui Li, School of Civil Engineering, Harbin Institute of Technology, Harbin 150090, China.
Email: [email protected]
Search for more papers by this authorWenliang Qian
Key Lab of Smart Prevention and Mitigation of Civil Engineering Disaster, Ministry of Industry and Information Technology, Harbin, China
Key Lab of Structures Dynamic Behavior and Control, Ministry of Education, Harbin, China
School of Civil Engineering, Harbin Institute of Technology, Harbin, China
Search for more papers by this authorYang Xu
Key Lab of Smart Prevention and Mitigation of Civil Engineering Disaster, Ministry of Industry and Information Technology, Harbin, China
Key Lab of Structures Dynamic Behavior and Control, Ministry of Education, Harbin, China
School of Civil Engineering, Harbin Institute of Technology, Harbin, China
Search for more papers by this authorCorresponding Author
Hui Li
Key Lab of Smart Prevention and Mitigation of Civil Engineering Disaster, Ministry of Industry and Information Technology, Harbin, China
Key Lab of Structures Dynamic Behavior and Control, Ministry of Education, Harbin, China
School of Civil Engineering, Harbin Institute of Technology, Harbin, China
Correspondence
Hui Li, School of Civil Engineering, Harbin Institute of Technology, Harbin 150090, China.
Email: [email protected]
Search for more papers by this authorFunding information:
National Natural Science Foundation of China, Grant Nos.51921006, 51638007, U1711265 and 52008138; National Key R&D Program of China, Grant Nos. 2018YFC0705605 and 2019YFC1511102; China Post-doctoral Science Foundation, Grant Nos. BX20190102 and 2019M661286; Heilongjiang Post-doctoral General Funding, Grant Nos. LBH-TZ2016 and LBH-Z19064
Abstract
This study develops an autonomous design method for architectural shape sketches by a novel self-sparse generative adversarial network (self-sparse GAN), thereby overcoming the problems regarding excessive reliance on sufficient aesthetic knowledge and excessive time consumption in traditional human design. First, a new architectural shape dataset denoted “Sketch” is built by using the eXtended difference-of-Gaussians operator. Second, a self-adaptive sparse transform module (SASTM) is designed following each deconvolution layer of the proposed self-sparse GAN to utilize the sparsity of sketch images by the sparsity decomposition and feature-map recombination. Third, the Frechet inception distance (FID) is adopted to evaluate the quality of the generated sketches by comparing the distribution of the real and generated datasets. Finally, two common image generation approaches, Wasserstein GAN with gradient penalty and self-attention GAN, are compared with the proposed self-sparse GAN, and results show the proposed method achieves the best performance with a relative decrease in the FID score of 11.87%. The proposed autonomous design method can give tens of thousands of sketches for a class of buildings in a few seconds using the trained network, which can help architects to choose the architectural form and/or inspire architects to consider unique schemes in the early stages of design.
REFERENCES
- Arjovsky, M., Chintala, S., & Bottou, L. (2017). Wasserstein GAN. arxiv:1701.07875
- Arjovsky, M., & Bottou, L. (2017). Towards principled methods for training generative adversarial networks. arxiv:1701.04862
- Arvin, S. A., & House, D. H. (2002). Modeling architectural design objectives in physically based space planning. Automation in Construction, 11(2), 213–225.
- Berthelot, D., Schumm, T., & Metz, L. (2017). BEGAN: Boundary equilibrium generative adversarial networks. arxiv:1703.10717
- Chaillou, S. (2020). ArchiGAN: Artificial Intelligence x architecture. In P. F. Yuan, Y. M. M. Xie, N. Leach, J. Yao, & X. Wang (Eds.), Architectural Intelligence (pp. 117–127). Springer.
10.1007/978-981-15-6568-7_8 Google Scholar
- Dai, Z., Almahairi, A., Bachman, P., Hovy, E., & Courville, A. (2017). Calibrating energy-based generative adversarial networks. arxiv:1702.01691
- Eastman, C. M. (1973). Automated space planning. Artificial Intelligence, 4(1), 41–64.
- Foulds, L. R. (1983). Techniques for facilities layout: deciding which pairs of activities should be adjacent. Management Science, 29(12), 1414–1426.
- Fu, J., Liu, J., Tian, H., Li, Y., Bao, Y., Fang, Z., & Lu, H. (2019). Dual attention network for scene segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA (pp. 3146–3154).
- Gao, Y., Kong, B., & Mosalam, K. M. (2019). Deep leaf-bootstrapping generative adversarial network for structural image data augmentation. Computer-Aided Civil and Infrastructure Engineering, 34(9), 755–773.
- Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014). Generative adversarial nets. In Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, & K.Q. Weinberger (Eds.), Advances in neural information processing systems (pp. 2672–2680). Curran Associates Inc.
- Grason, J. (1971). An approach to computerized space planning using graph theory. Proceedings of the 8th Design Automation Workshop, Atlantic City, NJ (pp. 170–178).
- Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. (2017). Improved training of Wasserstein GANs. Neural Information Processing Systems, Long Beach, CA (pp. 5769–5779).
- Hao & Hua. (2016). Irregular architectural layout synthesis with graphical inputs. Automation in Construction, 72, 388–396.
- He, K., Zhang, X., Ren, S., & Sun, J. (2015). Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile (pp. 1026–1034).
- Herbert, D. M. (1993). Architectural study drawings. John Wiley & Sons.
- Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017). GANs trained by a two time-scale update rule converge to a local Nash equilibrium. In Advances in Neural Information Processing Systems, Long Beach, CA (pp. 6626–6637).
- Homayouni, H. (2000). A survey of computational approaches to space layout planning (1965-2000). Department of Architecture and Urban Planning University of Washington.
- Huang, Y., Zhang, H., Li, H., & Wu, S. (2020). Recovering compressed images for automatic crack segmentation using generative models. Mechanical Systems and Signal Processing, 146, 107061.
- Jenni, S., & Favaro, P. (2019). On stabilizing generative adversarial training with noise. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA (pp. 12145–12153).
- Jeong, J. H., Jo, H., & Ditzler, G. (2020). Convolutional neural networks for pavement roughness assessment using calibration-free vehicle dynamics. Computer-Aided Civil and Infrastructure Engineering, 35(11), 1209–1229.
- Jo, J. H., & Gero, J. S. (1998). Space layout planning using an evolutionary approach. Artificial Intelligence in Engineering, 12(3), 149–162.
10.1016/S0954-1810(97)00037-X Google Scholar
- Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arxiv:1412.6980
- Kowaltowski, D, Bianchi, G., & Paiva, V. (2010). Methods that may stimulate creativity and their use in architectural design education. International Journal of Technology & Design Education, 20(4), 453–476
- Maeda, H., Kashiyama, T., Sekimoto, Y., Seto, T., & Omata, H. (2021). Generative adversarial network for road damage detection. Computer-Aided Civil and Infrastructure Engineering, 36(1), 47–60.
- Mao, X., Li, Q., Xie, H., Lau, R. Y. K., Wang, Z., & Smolley, S. P. (2017). Least squares generative adversarial networks. International Conference on Computer Vision, Venice, Italy (pp. 2813–2821).
- Menezes, A., & Lawson, B. (2006). How designers perceive sketches. Design Studies, 27(5), 571–585.
- Mescheder, L., Geiger, A., & Nowozin, S. (2018). Which training methods for GANs do actually converge. International Conference on Machine Learning, Stockholm, Sweden (pp. 3481–3490).
- Michalek, J., & Papalambros, P. (2002). Interactive design optimization of architectural layouts. Engineering Optimization, 34(5), 485–501.
- Miyato, T., Kataoka, T., Koyama, M., & Yoshida, Y. (2018). Spectral normalization for generative adversarial networks. arXiv:1802.05957
- Odena, A., Olah, C., & Shlens, J. (2017). Conditional image synthesis with auxiliary classifier GANs. International Conference on Machine Learning, PMLR, Sydney, Australia (pp. 2642–2651).
- Pfefferkorn, C. E. (1972). The design problem solver: A system for designing equipment or furniture layouts. In C. M. Eastman (Ed.), Spatial synthesis in computer-aided building design. Applied Science Publishers. 98–146.
- Qian, W., Xu, Y., Zuo, W., & Li, H. (2021). Self Sparse Generative Adversarial Networks. arxiv:2101.10556
- Radford, A., Metz, L., & Chintala, S. (2015). Unsupervised representation learning with deep convolutional generative adversarial networks. arxiv:1511.06434
- Rafiei, M. H., & Adeli, H. (2016). A novel machine learning model for estimation of sale prices of real estate units. Journal of Construction Engineering and Management, 142(2), 4015066.
- Rafiei, M. H., Khushefati, W. H., Demirboga, R., & Adeli, H. (2017). Supervised deep restricted Boltzmann machine for estimation of concrete. Aci Materials Journal, 114(2), 237–244.
- Rafiei, M. H., & Adeli, H. (2017a). NEEWS: A novel earthquake early warning model using neural dynamic classification and neural dynamic optimization. Soil Dynamics and Earthquake Engineering, 100, 417–427.
- Rafiei, M. H., & Adeli, H. (2017b). A new neural dynamic classification algorithm. IEEE Transactions on Neural Networks and Learning Systems, 28(12), 3074–3083.
- Rafiei, M. H., & Adeli, H. (2018). Novel machine learning model for construction cost estimation taking into account economic variables and indices. Journal of Construction Engineering and Management, 144(12), 4018106.
- Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016). Improved techniques for training GANs. Advances in Neural Information Processing Systems, Barcelona, Spain (pp. 2234–2242).
- Schon, D. A. (1984). The reflective practitioner: How professionals think in action. Basic Books.
- Schütze, M., Sachse, P., & Römer, A. (2003). Support value of sketching in the design process. Research in Engineering Design, 14(2), 89–97.
- Sharpe, R., & Marksjö, B. S. (1986). Solution of the facilities layout problem by simulated annealing. Computers, Environment and Urban Systems, 11(4), 147–154.
- Shorten, C., & Khoshgoftaar, T. M. (2019). A survey on image data augmentation for deep learning. Journal of Big Data, 6(1), 1–48.
- Simon, H. A. (1973). The structure of ill structured problems. Artificial Intelligence, 4(3-4), 181–201.
- Suwa, M., & Tversky, B. (1996). What architects see in their sketches: Implications for design tools. Conference Companion on Human Factors in Computing Systems, Vancouver, Canada (pp. 191–192).
- Valikhani, A., Jaberi Jahromi, A, Pouyanfar, S, Mantawy, I M., & Azizinamini, A. (2021). Machine learning and image processing approaches for estimating concrete surface roughness using basic cameras. Computer-Aided Civil and Infrastructure Engineering, 36(2), 213–226.
- Winnemöller, H., Kyprianidis, J. E., & Olsen, S. C. (2012). XDoG: An eXtended difference-of-Gaussians compendium including advanced image stylization. Computers & Graphics, 36(6), 740–753.
- Wu, W., Fu, X., Tang, R., Wang, Y., Qi, Y., & Liu, L. (2019). Data-driven interior plan generation for residential buildings. ACM Transactions on Graphics (TOG), 38(6), 1–12.
- Zhang, H., Goodfellow, I., Metaxas, D., & Odena, A. (2019). Self-attention generative adversarial networks. International Conference on Machine Learning, PMLR, Long Beach, CA (pp. 7354–7363).