Algorithmic Design in Virtual Reality

Version	Summary	Created by	Modification	Content Size	Created at	Operation
1		Renata Castelo-Branco	--	2812	2022-04-04 12:00:07	\|
2	Changed the pi character to a symbol instead of a full equation.	Renata Castelo-Branco	Meta information modification	2812	2022-04-04 12:03:56	\| \|
3	update layout and references	Rita Xu	-174 word(s)	2638	2022-04-06 03:53:40	\|

This entry is adapted from the peer-reviewed paper 10.3390/architecture2010003

Virtual reality has been shown to facilitate perception and navigation inside 3D models, while stimulating creativity and enhancing architect/client interaction. In this scenario, in order to better explore paths along the design space that are suggested from this interaction, it is important to support quick updates to the model while still immersed in it. Algorithmic design, an approach to architectural design that uses parametric algorithms to represent a design space, rather than a single design instance, provides such support.

virtual reality algorithmic design live coding

1. Introduction

Current digital representation tools for architectural design employ a visualization strategy that conveys the building’s form through the use of plans, section, elevations, and perspective views. This forces architects and clients to not only combine separate views of the model to form a mental model of the entire scenario ^[1], but also to scale it to real size in their imagination. This process can be considerably improved through Virtual Reality (VR) technology, allowing users to inhabit the building and perceive it at its natural scale ^[2].

Despite the advantages VR brings to architectural design, it has an important limitation: the majority of the ideas suggested during a VR session cannot be implemented and experimented during that session. In fact, most approaches for changing the design, with or without VR, are limited to manual model manipulation and, thus, cannot have extensive effects in the design. Algorithmic Design (AD) addresses this problem through the algorithmic description of the model, allowing parametric changes to the design that preserve its internal logic, thus ensuring the consistency of the modified design. However, AD has not yet been used in combination with VR and, thus, designers cannot update the algorithmic descriptions of their models while immersed in a Virtual Environment (VE).

2. Related Work

In 1965, Ivan Sutherland envisioned the ultimate display ^[3], a reality within which the computer controls the existence of matter itself. Although researchers are still far from achieving this radical vision, VR technology can already provide enough realism and immersiveness to make it an appealing tool for various professions, and architecture is no exception. AD uses algorithms to describe architectural designs, producing complex and parametric results that greatly benefit from immersive visualization. Having this in mind, several approaches have been developed that allow architects to access and change their AD programs from the VE. This section presents the current state of the art on the integration of parametric and algorithmic-design solutions with VR.

2.1. Virtual Reality in Architecture

The architectural industry is rapidly embracing VR since this technology helps users understand building designs, particularly when compared to the traditional alternative, which is based on flat projections that must be combined and scaled to form a mental model of the design ^[1]. Figure 1 illustrates this scenario for the case of the Astana National Library (ANL) project. This building, originally designed by Bjarke Ingels Group in 2008, has the shape of a 3D Moebius strip, a rather complex form for one to understand through projections only. Although architects are known for their 3D visualization capabilities, therefore being more than able to work within these boundaries, their clients, on the other hand, are not. For this reason, early adoption of VR technologies in this industry was mostly motivated by contractors and real estate owners ^[4], but nowadays more and more architectural studios use them to support design processes as well ^[2]^[5]^[6]^[7]^[8].

Figure 1. Rendered ANL model in Rhinoceros 3D shown in the 4 default viewports (top, perspective, front, and right view).

Despite the advantages of VR for facilitating the understanding of a building design, there is one significant limitation: most applications of VR in architecture focus on model visualization only ^[9]^[10]^[11], and the few existing approaches for changing the design while in the VE require virtual manual model manipulation ^[12]^[13] using VR controllers and associated transformation operations. This modeling workflow tends to be slower than the traditional (digital) one, which in turn is already quite slow when systematic changes are needed. There is one approach, however, that overcomes this problem: AD.

2.2. Algorithmic Design

AD defines the creation of architectural designs through algorithmic descriptions, implemented as computer programs ^[14]^[15]. AD allows architects (1) to model complex geometries that would take a considerable amount of time to produce otherwise, (2) automate repetitive and time-consuming tasks, and (3) quickly generate diverse design solutions without having to rework the model for every iteration ^[16]^[17]. Figure 2 presents several variations of the ANL model, obtained by manipulating the radius of the central plaza and the building’s facade torsion.

Figure 2. Variations of the ANL model produced by changing selected parameters in the program: radius of the central plaza and the building’s facade torsion.

Traditionally, AD entails the incremental development of the program that describes a building design and the visualization of the generated model to confirm the correctness of the program. In the case of VR, however, the design process needs to become a real-time and interactive one. Real-time interaction between the program and the model requires the use of a very performative AD tool, so that the designer can get immediate visual feedback on the impact of the changes applied to the program. Grasshopper ^[18], a visual programming environment, and LunaMoth ^[19], a textual programming one, are good examples of tools that possess this liveliness ^[20] aspect. Nevertheless, they were not intended to be used in VR and known integrations with this technology remain within the sphere of static visualization ^[21], failing to take advantage of AD’s interactive potential.

2.3. Live Coding

Live coding is a technique centered upon the writing of interactive programs on the fly ^[20]. In the case of VR, researchers envision the application of live coding in a scenario where the architect can code the algorithmic description of the model alongside the resulting geometry. The idea is similar to the workflow supported by some AD tools such as Dynamo ^[22] and Luna Moth ^[19], where the code (or visual components) is manipulated in the same environment where the model is concurrently being updated.

The concept of using VR for coding is also not a novelty. The Primitive tool ^[23], for instance, has proven the use of VR for collaborative software analysis visualization. Specific coding applications for VR have also been developed, such as NeosVR’s LogiX ^[24]—a VR multiplayer visual programming system; Rumpus ^[25]—a live coding playground for room-scale VR; Fenek ^[26]—a live coding JavaScript framework that allows developers to modify the underlying render engine while immersed in the VE; RiftSketch ^[27] and CodeChisel3D ^[28]—both browser-based live coding environments for VR. However, most of these tools were only tested with simple graphical models, and none were applied to the architectural context.

2.4. Parametric Design Solutions for Virtual Reality

Within the specific context of architectural design, by transporting the programming environment to the VE, researchers can have architects and clients manipulating the models in VR. Given that, in this industry, the production line involves multiple iterations of the solutions discussed by several stakeholders, having these discussions take place synchronously with the project’s development can resolve the existent asymmetric collaboration ^[29]. This, in turn, considerably accelerates the ideation process, thus, saving time and resources.

Solutions that allow for the modification of programs in VR have also been developed. Parametric Modelling Within Immersive Environments ^[30], Shared Immersive Environments for Parametric Model Manipulation ^[31], and Immersive Algorithmic Design ^[32] present solutions to connect AD tools to a VE, where architects are immersed in their models, apply changes to the program, and visualize the corresponding impacts in real time.

All three solutions contemplate a visual programming outline, with the former two ^[30]^[31] only allowing parameter manipulation. This means users do not have access to the entirety of the code in VR, but only to a chosen set of parameters, which they can change via sliders. For this reason, these solutions are framed within the scope of parametric design only, not AD, according to the definitions proposed by ^[33]. The third approach ^[32] goes further, also supporting textual programming and full control over the program. Nevertheless, and despite having presented multiple code manipulation solutions in the entry, the implementation of these solutions within the proposed system is not discussed, nor is the proposal formally evaluated.

3. Algorithmic Design in Virtual Reality

ADVR aims to aid the algorithmic design process by integrating live coding in VR. In this workflow, architects use a Head Mounted Display (HMD) and an AD tool integrated in a VE to code their designs while immersed in them. In the VE, the generated design is concurrently updated in accordance with the changes made to its algorithmic description. Seeing these updates in near real time allows designers to conduct an iterative process of concurrent programming and visualization of the generated model in VR, enhancing the project with each cycle.

Figure 3 presents a conceptual scheme of this loop: architects develop the algorithmic description of the design using an Interactive Development Environment (IDE) or a programming editor to input the coding instructions into the AD tool, which then generates the corresponding model. From the VE designers, then, evaluate the results and, possibly, modify the algorithmic description from there, thus repeating the loop. Figure 4 presents a mock-up of the corresponding VR experience.

Figure 3. Conceptual scheme of the architect/program/model loop happening in VR: architects develop the algorithmic description of the design in VR, generating the corresponding model around them, whose visualization them motivates further changes to the description.

Figure 4. Mock-up of the ADVR experience: the architects live codes the algorithmic description of the design from within the VE.

In order to provide an efficient coding platform in the VE that will enable this workflow, the following components are required: (1) an interactive AD tool that allows for the generation of complex architectural models, along with (2) a VR tool that can be coupled to the AD framework. This tool must be a sufficiently performative visualizer, such as a game engine, to guarantee near real-time feedback; (3) a mechanism that allows designers to code while immersed, i.e., an IDE or a programming editor; (4) text input mechanisms, as well as (5) language and IDE considerations; and, finally, (6) the ability to smoothly update the model even when its complexity hinders performance. The implementations researchers chose to pursuit for each of the numbered items are described below. Figure 5 presents the implementation workflow, featuring the chosen tools.

Figure 5. ADVR implementation: while immersed in the VE, the user accesses the IDE through the headset’s virtual desktop application. The AD tool is responsible for translating the given instructions into operations recognized by the game engine, which is connected to the HMD through the VR plug-in.

3.1. AD Tool

Regarding the AD tool, researchers opted for Khepri ^[34], a portable AD tool that integrates multiple backends for different design purposes, namely Computer-Aided Design (CAD), Building Information Modeling (BIM), game engines, rendering, analysis, and optimization tools. The use of multiple tools along the development process is motivated by their different benefits: while CAD tools outperform the rest in free-form modelling ^[35]; BIM tools are essential for dealing with construction information ^[36]; game engines present a good alternative for fast visualization and navigation ^[37]; rendering tools offer realistic but time-consuming representations of models for presentation; and, finally, analysis and optimization tools inform and guide the design process based on the model’s performance ^[38].

3.2. VR Tool

Regarding the game engine, the choice fell upon Unity ^[39], since it provides good visual quality, including lighting, shadows, and physics; good visualization performance for average-scale architectural projects; platform independence; and availability of assets; as well as VR integration. Through the Steam VR plug-in, Unity communicates with the two tested headsets: Oculus Rift and HTC Vive. It must also be noted that, despite the fast response guaranteed by the game engine, the capacity for real-time feedback will always be conditioned by the model’s complexity.

3.3. IDE Projection

For architects to be able to program from inside the VE, they need to access their preferred IDE while immersed in VR. To this end, researchers use the virtual desktop application provided by most HMDs (including the ones used for this implementation), which mirrors the user’s desktop in VR, allowing the use of any application and, more specifically, of any IDE.

Figure 6 presents the workflow with the ANL model being live coded in VR. In this case, a change in the facade torsion parameter can be observed. Looking at the pictures in the first row, one may also observe that the mirrored desktop represents a partial visual blocker to the scene. However, it should be noted that this two-dimensional representation fails to convey the full 360° experience the user has in the VE. Moreover, the screen can be moved, scaled, or hidden at the designer’s will, as shown in the second row of images in Figure 6.

Figure 6. ADVR of the ANL model: manipulation of the facade torsion parameter (π, 2π, and 4π).

3.4. Text Input

Regarding textual input, there are several solutions currently available for the use of virtual and physical keyboards. Considering the result obtained in previous studies on the matter ^[40]^[41], researchers opted for the latter solution in this implementation. Figure 7 presents a use-case of the assembled workflow, showcasing both the IDE display along with the generated model of ANL in VR, and the interaction mechanisms in action, inside and outside the VE.

Figure 7. ADVR workflow: on the left, the VE where the model, the IDE and the responsive virtual keyboard are visible; on the right, the architect typing in the physical keyboard.

Researchers stress that, despite the performance of the chosen solution in comparison to the remaining ones, it does not yet reach that of typing on a normal keyboard outside the VE, particularly for users who cannot touch type and, thus, heavily rely on both visual and haptic feedback. This question is particularly relevant in the present context, as the majority of programming architects are, in fact, non-experienced typists. Hence, other solutions for the problem must be sought and researchers believe the industry will soon provide them, as some interesting new concepts are already starting to emerge ^[42].

3.5. Language and Editor

In order to guarantee fast typing results, researchers opted for a dynamically-typed programming language, as these tend to be more concise than statically-typed ones. Although the latter offer more performance in run time and can detect static semantic errors, they force the user to provide type information, which is a lot more verbose. The chosen IDE can also help the user in the typing task, particularly, by providing automatic completion for names and for entire syntactical structures, such as function definitions. For this implementation, they used the Julia language running in the Atom editor with the help of the Juno plugin, a combination which considerably augments the user’s typing speed. As shown in Figure 5, the Visual Studio Code editor was also tested, although the lack of a user friendly menu with shortcut buttons forced users to type more in order to run commands.

3.6. Model Update

ADVR takes advantage of game engines’ ability to efficiently process geometry. As a result, researchers can generate large-scale models, as is the case of the ANL, in a matter of seconds. When applying changes to the model in VR, these seconds are, nevertheless, troublesome, since the AD tool deletes and (re)generates the entire model in each iteration, regardless of the number of changes applied. Consequently, as the model grows, a small lag becomes noticeable and the sudden reshaping of the entire VE is disorienting and makes it difficult to understand the effects of the applied changes.

To solve these problems, researchers implemented a multi-buffering approach that keeps the user in an outdated but consistent model, while a new model is invisibly being generated. When finished, the new model replaces the old one, allowing the user to immediately visualize the impact of the changes. It is also possible to have several models available simultaneously, in different buffers, for the user to switch back and forth between them, facilitating comparisons and improving the decision-making process. Figure 6 illustrates this by showing two different views of the variations created.

References

Gobbetti, E.; Scateni, R. Virtual reality: Past, present and future. Stud. Health Technol. Inform. 1998, 58, 3–20.
Portman, M.; Natapov, A.; Fisher-Gewirtzman, D. To go where no man has gone before: Virtual reality in architecture, landscape architecture and environmental planning. Comput. Environ. Urban Syst. 2015, 54, 376–384.
Sutherland, I.E. The Ultimate Display. Proc. Inf. Process. (IFIP) Congr. 1965, 2, 506–508.
Whyte, J. Industrial applications of virtual reality in architecture and construction. J. Inf. Technol. Constr. (ITcon) 2003, 8, 43–50.
Dorta, T.; Pérez, E. Immersive Drafted Virtual Reality: A new approach for ideation within virtual reality. In Proceedings of the 25th Annual Conference of the Association for Computer Aided Design in Architecture (ACADIA), Louisville, Kentucky, USA, 12–15 October 2006; pp. 304–316.
Gu, N.; Kim, M.J.; Maher, M.L. Technological advancements in synchronous collaboration: The effect of 3D virtual worlds and tangible user interfaces on architectural design. Autom. Constr. 2011, 20, 270–278.
Schnabel, M.A. The immersive virtual environment design studio. In Collaborative Design in Virtual Environments; Wang, X., Tsai, J.J.H., Eds.; Intelligent Systems, Control and Automation (ISCA): Science and Engineering; Springer: Dordrecht, The Netherlands, 2011; Volume 48, pp. 177–191.
Wang, X. Mutually augmented virtual environments for architectural design and collaboration. In Proceedings of the 12th International Conference on Computer Aided Architectural Design Futures (CAAD Futures), Sydney, Australia, 11–13 July 2007; Springer: Dordrecht, The Netherlands, 2007; pp. 17–29.
Wang, P.; Wu, P.; Wang, J.; Hung-Lin, C.; Wang, X. A Critical Review of the Use of Virtual Reality in Construction Engineering Education and Training. Int. J. Environ. Res. Public Health 2018, 15, 1204.
Davila Delgado, J.M.; Oyedele, L.; Demian, P.; Beach, T. A research agenda for augmented and virtual reality in architecture, engineering and construction. Adv. Eng. Inform. 2020, 45, 101122.
Alizadehsalehi, S.; Hadavi, A.; Huang, J.C. From BIM to extended reality in AEC industry. Autom. Constr. 2020, 116, 103254.
Anderson, L.; Esser, J.; Interrante, V. A virtual environment for conceptual design in architecture. In Proceedings of the Eurographics Workshop on Virtual Environments (EGVE), Zurich, Switzerland, 22–23 May 2003; Association for Computing Machinery: New York, NY, USA, 2003; pp. 57–63.
de Klerk, R.; Duarte, A.; Medeiros, D.; Pinto Duarte, J.; Jorge, J.; Lopes, D. Usability studies on building early stage architectural models in virtual reality. Autom. Constr. 2019, 103, 104–116.
Burry, M. Scripting Cultures: Architectural Design and Programming; Architectural Design Primer; John Wiley & Sons, Inc.: West Sussex, UK, 2011.
Terzidis, K. Algorithmic Architecture; Architectural Press: New York, NY, USA, 2006.
Lee, J.; Gu, N.; Jupp, J.; Sherratt, S. Evaluating Creativity in Parametric Design Processes and Products: A Pilot Study. In Design Computing and Cognition (DCC’12); Gero, J.S., Ed.; Springer: Dordrecht, The Netherlands, 2014; pp. 165–183.
Woodbury, R. Elements of Parametric Design; Routledge: London, UK, 2010.
Davidson, S. Grasshopper: Algorithmic Modeling for Rhino. 2021. Available online: https://www.grasshopper3d.com (accessed on 27 December 2021).
Alfaiate, P.; Caetano, I.; Leitão, A. Luna Moth: Supporting Creativity in the Cloud. In Proceedings of the 37th Annual Conference of the Association for Computer Aided Design in Architecture (ACADIA), Cambridge, MA, USA, 2–4 November 2017; pp. 72–81.
Rein, P.; Ramson, S.; Lincke, J.; Hirschfeld, R.; Pape, T. Exploratory and Live, Programming and Coding: A Literature Study Comparing Perspectives on Liveness. Program. J. 2018, 3, 1:1–1:33.
Panya, D.S.; Kim, T.; Choo, S. A Methodology of Interactive Motion Facades Design through Parametric Strategies. Appl. Sci. 2020, 10, 1218.
Autodesk, Inc. Dynamo: Open Source Graphical Programming for Design. 2020. Available online: https://dynamobim.org/ (accessed on 27 December 2021).
Prime Software Systems. PRIMITIVE: VR Collaboration for Distributed Teams. 2019. Available online: https://primitive.io/ (accessed on 27 December 2021).
James, P. Watch: ‘LogiX’ is an Impressive Multi-User Visual Programming Interface for VR. 2017. Available online: https://www.roadtovr.com/watch-logix-impressive-multi-user-visual-programming-interface-vr/ (accessed on 27 December 2021).
Iannini, L. Rumpus. 2016. Available online: https://store.steampowered.com/app/458200/Rumpus/ (accessed on 27 December 2021).
Schutz, M.; Wimmer, M. Live Coding of a VR Render Engine in VR. In Proceedings of the 26th Conference on Virtual Reality and 3D User Interfaces, Osaka, Japan, 23–27 March 2019; pp. 1150–1151.
Elliott, A.; Peiris, B.; Parnin, C. Virtual reality in software engineering: Affordances, applications, and challenges. In Proceedings of the 37th International Conference on Software Engineering (ICSE), Florence, Italy, 16–24 May 2015; Volume 2, pp. 547–550.
Krahn, R. CodeChisel3D: Live Programming with three.js and webVR. 2015. Available online: https://robert.kra.hn/past-projects/live-programming-with-three-and-webvr.html (accessed on 27 December 2021).
Clergeaud, D.; Roo, J.S.; Hachet, M.; Guitton, P. Towards Seamless Interaction Between Physical and Virtual Locations for Asymmetric Collaboration. In Proceedings of the 23rd Symposium on Virtual Reality Software and Technology (VRST), Gothenburg, Sweden, 8–10 November 2017; pp. 17:1–17:4.
Coppens, A.; Mens, T.; Gallas, M.A. Parametric Modelling Within Immersive Environments: Building a Bridge Between Existing Tools and Virtual Reality Headsets. In Proceedings of the 36th Education and Research in Computer Aided Architectural Design in Europe (eCAADe) Conference, Lodz, Poland, 17–21 September 2018; pp. 711–716.
Hawton, D.; Cooper-Wooley, B.; Odolphi, J.; Doherty, B.; Fabbri, A.; Gardner, N.; Haeusler, M.H. Shared Immersive Environments for Parametric Model Manipulation—Evaluating a Workflow for Parametric Model Manipulation from Within Immersive Virtual Environments. In Proceedings of the 23rd International Conference on Computer-Aided Architectural Design Research in Asia (CAADRIA), Beijing, China, 17–19 May 2018; pp. 483–492.
Castelo-Branco, R.; Leitão, A.; Santos, G. Immersive Algorithmic Design: Live Coding in Virtual Reality. In Proceedings of the 37th Education and research in Computer Aided Architectural Design in Europe (eCAADe) Conference, Porto, Portugal, 11–13 September 2019; Volume 2, pp. 455–464.
Caetano, I.; Santos, L.; Leitão, A. Computational design in architecture: Defining parametric, generative, and algorithmic design. Front. Archit. Res. 2020, 9, 287–300.
Sammer, M.J.; Leitão, A.; Caetano, I. From Visual Input to Visual Output in Textual Programming. In Proceedings of the 24th International Conference of the Association for Computer-Aided Architectural Design Research in Asia (CAADRIA), Wellington, New Zealand, 15–18 April 2019; Volume 1, pp. 645–654.
Zboinska, M.A. Hybrid CAD/E platform supporting exploratory architectural design. Comput.-Aided Des. 2015, 59, 64–84.
Eastman, C.; Teicholz, P.; Sacks, R.; Liston, K. BIM Handbook: A Guide to Building Information Modeling for Owners, Designers, Engineers, Contractors, and Facility Managers; Wiley: Hoboken, NJ, USA, 2008.
Indraprastha, A.; Shinozaki, M. The Investigation on Using Unity3D Game Engine in Urban Design Study. J. ICT Res. Appl. 2009, 3, 1–18.
Nguyen, A.T.; Reiter, S.; Rigo, P. A review on simulation-based optimization methods applied to building performance analysis. Appl. Energy 2014, 113, 1043–1058.
Hocking, J. Unity in Action: Multiplatform Game Development in C# with Unity 5; Manning Publications: New York, NY, USA, 2015.
Grubert, J.; Witzani, L.; Ofek, E.; Pahud, M.; Kranz, M.; Kristensson, P.O. Text Entry in Immersive Head-Mounted Display-based Virtual Reality using Standard Keyboards. In Proceedings of the 25th Conference on Virtual Reality and 3D User Interfaces, Tuebingen/Reutlingen, Germany, 18–22 March 2018; pp. 159–166.
Walker, J.; Li, B.; Vertanen, K.; Kuhl, S. Efficient Typing on a Visually Occluded Physical Keyboard. In Proceedings of the CHI Conference on Human Factors in Computing Systems, Denver, CO, USA, 6–11 May 2017; Association for Computing Machinery: New York, NY, USA, 2017; pp. 5457–5461.
Vive Team. Introducing the Logitech BRIDGE SDKs. 2017. Available online: https://blog.vive.com/us/2017/11/02/introducing-the-logitech-bridge-sdk/ (accessed on 27 December 2021).

© Text is available under the terms and conditions of the Creative Commons Attribution (CC BY) license; additional terms may apply. By using this site, you agree to the Terms and Conditions and Privacy Policy.

Upload a video for this entry

Information

Subjects: Architecture And Design

Contributors MDPI registered users' name will be linked to their SciProfiles pages. To register with us, please refer to https://encyclopedia.pub/register :

Renata Castelo-Branco

View Times: 801

Update Date: 06 Apr 2022

Table of Contents

Video Upload Options

Confirm