Hi, it’s Ying here. It has been quiet a while since my last update, which will be recounted in this post. I will start with the last few week in July after the mid-term presentation first. Then in the following posts I will summarise my work in August.
Spatial Audio Runtime Glitch in Unity
After finishing the spatial audio demo with Google VR SDK in week 6, I found an annoying glitch in Unity. My spatial audio demo had four audio assets – the bell, the chisel, the bike and the ambience. All of them are triggered by the user’s gaze input except the ambience, which is played on awake and always looping in the background. When the clips are playing simultaneously in the runtime, the screen flashes in pink on my macbook. To be honest I don’t know the exact cause of the glitch, probably due to the memory limitation on my mac or that Unity could not handle audio clips at different levels very well. The glitch did not happen after I normalised the individual audio files. I did the normalisation in Audacity, but the task can be done in other DAWs or in Unity as well.
Waveform Analyser Plugin for Audacity
I reported the aforesaid glitch to Enda, and he suggested to bring all the audio clips roughly to the same levels to save the computer from extra calculation. He suggested to me an audio waveform analyser plugin for Audacity < https://gist.github.com/endolith/514077 >, so that I could check the peaks and average levels (RMS) of each soundtracks. This plugin is very useful when I need to know the average decibel of the audio files before putting them into Unity. A solution for the glitch is found. As long as the audios were normalised to the approximate level, the screen would not flash again in Unity 5.4.0b23 in runtime.
Integrating VST in Cubase and Creating UI sounds
I used Cubase for the composition of UI tones. Although my UI sounds are not adopted in the end, I think the process itself worths a bit of documentation. On Tuesday and Wednesday (26-27 July), I was searching for free VSTs and integrating them into Cubase.
Jill preferred to have a percussion called hang for the UI. I could not find a free VST for this wonderful instrument unfortunately. Also, Enda mentioned that he preferred to have a mix of harp and piano for the UI. The notion was inspired by Trinity College Harp, which, according to legend, was once owned by the Irish King Brian Boru. The harp represents the Irish music tradition and aesthetics in many ways. I specifically looked for a harp VST as the HALION Sonic instruments in Cubase are mostly unavailable on the PC I am working with. Also the existing VSTs don’t have a natural sound either for a harp or a piano. By the end of the day I got 10 VSTs installed in Cubase. The names of the VSTs are listed below. I decided to only go with mda Piano and the VS Etherealwinds Harp to make the UI sounds.
mda Piano < http://mda.smartelectronix.com/synths.htm >
VS Etherealwinds Harp < http://vis.versilstudios.net/etherealwinds-harp.html >
Other VSTs: DSKAkoustiK KeyZ, EP-Station, Piano Harp, 4Front EPiano Module, EVM Grand Piano, GlueReeds, Joe’s Jazz Piano I, MrRay 73
I made two UI tones, one for the highlight effect when a UI marker is gazed upon and the other for confirmation when a UI marker is clicked. They are made with the VS Etherealwinds Harp and the mda Piano. I chose a chord for the highlight and two notes for the confirmation. Both are very simple so that they won’t distract the user from the immersion experience.
Research on Irish harp music
This week I also did a little bit of research into the Irish harp music. Turlough O’Carolan (1670-1738) is perhaps the most famous harper and composer in Irish history. He bequeathed generations of Irish musicians with more than 200 pieces for harp performance alone. I selected seven songs from his repertoires in preparation for the video background music in the app.
Captain O’Kane (recorded)
Lady Maxwell (recorded)
Sheeberg & Sheemore
Many songs created by Turlough O’Carolan were not published during his life time, and they were supposed to be learned by ears. When I searched for the sheet music for the above pieces, the scores I found only provided the main theme for the right hand. Performer needs to improvise on the left-hand accompaniment, which has unlimited possibility of variance between performers. Due to constraint of time and the limit of my talent, I only recorded the main theme for Captain O’Kane and Lady Maxwell on Friday, 29 July. I passed the Captain O’Kane piece in MIDI to Enda. He found a better arrangement for that piece in the end.
Finalising Ambience Sound Design Document
I also finalised the ambience sound design document on Thursday, 28 July so that Enda and others could have an idea what kind of sounds would be put into Unity by locations. The design document followed the naming conventions Enda had outlined in Unity. The test recordings I took on 18, 19, 20 July were also taken into account, especially the environment and direction where the sounds are coming from. Having the sound design penned down, I took another two field recordings on Sunday, 31 July and Monday, 1 August.
Audio format considerations
All the ambience was recorded in MS and XY directions. There were stereo audio files with 4 channels in two tracks. I did not recorded the ambience in a 4-channel ambisonic format into one track as I was not sure about the editing workflow in DAW and the support for this format in Unity. I thought the directional 4-channel recording would suffice for the postproduction anyway.
Also I decided not to export the audio as MP3 format. There was 0.04 seconds of silence at the beginning of the sound track, which was unavoidable as long as it was exported in mp3. It would result in a sudden pop, which was bad when the track was looped in Unity. I would rather export the audio file into .wav and let Unity to compress it after the assets were imported.