Final Project: Extended Reality Development

After finishing the 5 prototypes where I gathered base knowledge of a range of hardwares, softwares and technologies for performing arts. I began work on my final project with my group. We chose to further develop our extended reality prototype and focus on the interactivity aspect of it and mixing the senses of the virtual world and real world.

We gathered extensive feedback from every user who experienced our virtual reality prototype to have a wide range of opinions. The most popular developments we had suggested to us was that users would like to have more moments of interaction with the actresses; to try eating food in the real-world whilst seeing it in the virtual world. Therefore this is what we decided to focus on.

After much experimentation with the Ricoh Theta S camera, we had to refilm each of the three meals we decided the present the viewer with as the initial stage set up ad the lighting too bright and blinded the camera. The Ricoh Theta S uses two separate lenses to film the lighting was picked up different by each resulting in a drastic change in lighting either side of the stitch line. This result was what we ended up with our first attempt of filming using stage lighting and a more fake set up. When testing the camera we discovered that shooting in natural lighting did not pose such a problem, as the light was more evenly distributed across both lenses rather than one. Therefore we refilmed each meal in a more natural homey setting with natural light. I felt that the stage set up we originally used was more effective in making the viewer feel like they are in a performance where as the new home setting allows the viewer to feel familiar and more relaxed in their surroundings. The lighting is really important when filming in 360 because it came make the different between the footage rendering in high quality or low quality.

When it came to edit the footage this was where I found the technology was the weakest. I tried multiple different free 360 specific editing software or general editing software with the capability to input 360 however none of them I found were up to the standard that I expected them to be. In terms of transitions and different effects such as layering different footage on top or adding text the results were often poor and very unprofessional as they were not specifically designed for 360. This meant that we had to try and get the raw footage as perfect as possible with as little editing needed. We only ended up editing the beginning and end of the footage. Due to using one continuous take, unlike normal 2D film you cannot just cut together different takes like a movie with different perspectives we had.

Something that really captivated me with filming in 360 was that everything is constantly in view whether you want the viewer to look at it or not. I tried to control where their looking through audio queues but chances are if someone wants to look behind them their going to do it anyway. I would have liked to use 3D audio as well to further hint at where the audience were supposed to look but we focused on more the interactions with the audience for this project. I would definitely use 3D audio to further develop this project.

“Everything you see in video-based VR is pre-rendered meaning the complex computing work has already been done and all your device has to do is play it back.”

The VR hardware and software that we have used required very little prior knowledge of the technology. Which meant that it is ready to use by all practitioners and very accessible.

To further develop this I would like to implement the use of 3D audio to give audio cues to the user as to where to look. I would also like to refilm the meals using a higher quality 360 camera as the Ricoh Theta S films raw video in high quality but once it is stitched together the quality is considerably poorer.

Please see the videos below for the final product.