Group Photo: workshop participants and the Data Reef
On November 30th - December 1st 2019, leading scientist-artists Uwe Rieger and Yinan Liu from the Aukland-based arc / sec Lab for Cross Reality Architecture and Interactive Systems were invited to School of Creativity and Art, ShanghaiTech University (SCA) to lead a creative workshop named Data Reef. In this workshop, participants from different backgrounds collaborated to create a Mixed-Reality environment in the form of a digital coral reel that not only connected digital information with physical constructions but also responded to the user.
Photo: Event Recap, video created by art / sec Lab
After thorough communications over email between arc / sec Lab and the host, DIV lab of SCA, and preparation in Shanghai, the workshop was successfully held over the weekend and it engaged more than twenty students from the three Schools of ShanghaiTech University, as well as Shanghai Institute of Visual Arts and Luxun Academy of Fine Arts.
In order to cater to the different interests and skill sets of the students, Uwe and Yinan, together with Prof. YANG Hua of SCA, conducted interviews to identify the level of skills in Unity 3D, 3D modelling, design and graphics. From this information, the instructors formed 7 teams of 4 students to accommodate for this variation of abilities. No students had prior HoloLens experience and only a few with Unity 3D or design experience.
The workshop started with an introduction into the concepts of cross-reality from the arc/sec Lab using recent projects as examples. It explained the technical and design principles of how to create cross-reality environments and outlined potential future applications.
Then the participants were split into 7 groups. Each group designed their own section of the 'digital reef' in form of a physical construction and augmented animation. In the process, all students, despite their previous exposure to technical or design applications, were introduced to the CAD program Rhino 3D, the animation software Unity, marker based tracking app Vuforia and the Microsoft HoloLens.
In the first phase of the workshop, students built a geometrical structure using a pre-prepared 'kit in parts system' out of paper straws and 3D printed joints. A pre-made Rhino 3D template was used to design the structure. The digital model was transferred to the Microsoft HoloLens using Unity 3D and marker based tracking to assist in the physical construction process of the complex geometries.
In the second phase, the students were asked to animate their section of the structure with interactive graphics. Under the theme of a 'coral reef' students used Unity 3D to design interactive elements that populated the physical structure.
The workshop was prepared in steps and with templates so that no pre-existing knowledge was required, but offered the opportunity to go further in depth if they wanted. Accordingly, the workshop had various interesting ideas and outcomes. The students were extremely interested and excited. In the process, they were introduced to a variety of software programs, design strategies, application strategies for XR. Students actively learned how to produce a XR environment and the use of the Microsoft HoloLens. The students were very engaged and learnt very quickly the skills necessary to complete the project. Within 2 days, they designed, built and presented fascinating projects.
The last day ended with a public presentation of the student's work in the foyer of the School of Creativity and Art. Everyone was excited and pleased to how their creations came to life and experienced by others. Using the HoloLens, swimming dolphins, swarms of fish, growing corals, air bubbles and underwater bridges were to be discovered in the impressive 7m long sculpture.
Student Comments:
#1 It is enriching to enter a new field and learn new techniques. The method of construction design after modelling could be widely applied to many areas, which reduce errors and, at the same time, convenient to observe the final effect. I'm looking forward to seeing the commercial use of this technique.
#2 Building an elementary structure and adding complex information on top is a cutting-edge design technique. Through Hololens, it becomes difficult to distinguish between reality and fantasy. Perhaps it is what life is like in 2077.
#3 I agree with the professor that the purpose of virtual reality isn't to encourage people to completely entering the virtual world, but to apply the concept of digital into the real-world, or to realize crazy ideas outside of the screen. I don't know much about architecture or modern arts, but I'm very curious. When I tried to modify geometric graphics and give them meanings, I seem to start to understand them. Different from realism, abstract arts test the degree of implicit between creator and observer.
#4 Wrong: Try to construct wearing VR glasses and look at complicated three-view diagrams are equally silly. Correct: Use Rhino for initial modelling and Unity for neat post effects=contemporary art without glasses, and a fish tank full of jellyfishes with glasses.
#5 I think it has somewhat crossed the boundary of traditional architecture and has fewer restrictions on building materials and techniques. Very creative!
#6 We had a brilliant weekend working together with friends from different academic backgrounds. The moment that impressed me the most was when we designed the geometric structure and faced two choices before assembly:
1. Use the image in our head;
2. Use HoloLens to project the image. In the beginning, we thought we should use both. However, after everyone tried to assemble by HoloLens, we all thought it was amazing!
Accurate manufacture brought by technology enormously increases the efficiency of assembly. However, we might miss the incorrect assemble, as known as creativity. If we can use a technique that could realize in-time capture, calculation and prediction when we voluntarily changed the direction, it may reform a new structure and give future advice. This kind of interactive experience makes me feel like life in the future and hold a lot more expectation for the future.
#7 In the Datareef workshop, we used HoloLens and simple straw connection to experience AR and VR applications in architecture under the guidance of Uwe and Yinan. Being a CV+AR/VR concentration PhD student, my academic vision on AR and VR might possibly oversee the realization and application of this technology. However, this workshop used a rather simple way to apply VR/AR on architecture. It provided a better understanding of AR application, and also gave me more confidence in future academic research.
#8 Real wisdom sparkled, and unexpected results have been created during the collision between art and technology. Being an art student to participate in this workshop from the Datareef, I have really learned a lot and had much fun. Under the guidance of Uwe Rieger and Liu Yinan, we learned to combined the physical structure created by our own with virtual information in our heads and projected the results by HoloLens. It was really amazing and opened a new gate for me.
Even when everyone receives the same original information, we still have different ideas, and the uncertainty brought by creative ideas is always impressive. As a way of art expression, technology can improve the creation of art, and art will fulfill the meaning and concept of technology, which leads to a future with endless possibilities.
#9 As a fine art student, I clearly know that I lack technical understanding. During this opportunity, I want to give it a try on the technology I am interested in. In the beginning, I couldn't focus on the efficient working status, whereas everyone else seems to know more than I do, but then I'm influenced by the atmosphere. It proves that only exploration could lead to breakthroughs, no matter it is model creation, actual assembly or posts interactive. This workshop really makes me feel that the intersection of technology and art will bring more creativity to our works.
About the Guest Lecturers
Uwe Rieger
Uwe Rieger is founder and head of the arc/sec Lab for Cross Reality Architecture and Interactive Systems. He is Associate Professor for Design and Design Technology at the University of Auckland and has worked as architect and researcher in the field of Reactive Architecture for over 20 years. Uwe has exhibited and developed projects for international institutions such as: the Ars Electronica Centre in Austria, the Museum of Modern Art Barcelona (MACBA) in Spain, the Venice Architecture Biennale in Italy, the World EXPO and the International Building Exhibition IBA in Germany, the National Museum of Indonesia, and Te Papa Tongarewa, the National Museum of New Zealand.
Yinan Liu
Yinan Liu received a MArch(Prof)(Hons) degree from the University of Auckland. She is the lead technologist at the arc/sec Lab and coordinator of the Digital Research Hub at the School of Architecture and Planning at the University of Auckland. Yinan is co-founder of arc/sec Solutions ltd., which develops customised applications for cross-reality environments and interactive systems.
About arc/sec Lab
The arc/sec Lab for Cross Reality Architecture and Interactive Systems is based at the University of Auckland. The Lab explores real time Reactive Architecture through the fusion of digital information and physical constructions. Headed by Associate Professor Uwe Rieger with the lead technologist Yinan Liu, the arc/sec team works in different constellations and across diverse disciplines. This includes collaborations with Bioengineering, Medical Science, Computer Science, Music and Dance. Aim of the investigations is to re-connect the intangible computer world to multisensory human perceptions and intuitive forms of engagement. Experiments at the Lab are the starting point for both, the development of practice oriented applications and a speculation on the future of our built environment. Specific to the Lab is the use of large-scale prototypes and performative installations as the driving force for both, the development and the communication of new hybrid design concepts.
About DIV Lab of School of Creativity and Art, ShanghaiTech University
DIV (Design for Interactive Virtualization) Lab of School of Creativity and Art, ShanghaiTech University is dedicated to exploring the possibilities in the creative industry and expanding the boundaries of the creative processes. The creative products would foster a new type of social interactions among citizens, artists and scientists, where everyone could participate in the interactive design in areas of life-style, smart city, smart entertainment, smart social welfare and adolescent education. Adopting a cross-disciplinary approach, the areas of DIV spans the research and practices in design, fine arts, human-computer interaction, participatory design, Design for Behavioral Change, Artificial Intelligence, engineering, computer science, sociology, anthropology, communications, business, cultural studies and other relevant social science areas. Empowered by innovative intelligence technologies, DIV Lab wishes to propose new approaches and concepts in designing for a future-facing and smart living environment.
