360 AUDIO AND THE SONY SOUND

"TOMOAKI SATO, Senior Electrical Engineer/Project Leader, Sony Video and Sound"

What got Sony started on the 360 Reality Audio project?

Nageno: Well, technical reasons… Now, Sony has the infrastructure in terms of technology to reproduce audio immersively. That’s one reason. Also, internet speeds are fast enough, and devices have enough capacity now, so we’re ready for 360 audio in the Sony way.

Does that mean you were just waiting for technology to mature?

Nageno: As you know, we previously had headphones that were surround headphones. These were connected to DVD players and Wi-Fi like connections, and produced surround sound, but we had very little sales; very limited customers enjoying cinema sounds in surround. But as you know, immersive means going to all directions.

Up and down as well…

Nageno: Yes, yes. We decided to go to 3D. But we decided to launch the new format with music streaming. Because many people are enjoying music streaming and we want to let people know how immersive audio is attractive. So, we choose the music streaming format. That was the first motivation for us.

And this will work with all speakers and headphones? How is it that it works with all headphones/speakers without any special calibration?

Nageno: We’ve introduced just the technical concept at CES, so no concrete product plan has been established yet. But basically, it has the possibility to reproduce 360 RA with speakers or headphones; any kind of audio reproduction systems. Technically it’s possible.

What’s the hardest part of developing 360-degree audio?

Nageno: We only just started, but one big challenge we foresee is making a full 3D sound, because it’s a challenge to make sound from above and below. Existing audio has only the up side and no down side, so we’re trying to make a canvas just like whole 360-degree vertical.

That’s a challenge. One more thing is getting good quality for streaming data rate.  So under MPEG-H we make something common; an open platform with three immersive quality levels that you can choose based on your connection rate.

With headphones we typically only think of two channels – Left and Right. How many channels can we achieve with 360 Reality Audio?

Nageno: Logically, any number. It’s just software. But typically, we choose 13 virtual speaker locations. 360 audio content format is just object-based, so the sound location can be any place. Each vocalist and each instrument has its own place and soundtrack, so sound location can be any place.

"KOJI NAGENO, Senior Acoustic Architect"

Spatial Audio typically relies on your individual HRTF (Head related transfer function). Is that true for 360 Reality Audio too? How will it work for speakers?

Nageno: HRTF is for headphones reproduction. Each person has a HRTF for each direction so we measure each person’s HRTF and match it to our headphone response in our library. For speakers’ case, the first one is multiple speaker reproduction. Just rendering sound objects to multiple locations. There’s no HRTF technology, just rendering to a sound location.

But there’s other technologies like virtual surround that do have some application of HRTF technology, so example with sound bars, we make calculations based on HRTF how to “place” the virtual speakers.

Will 360 Reality Audio be a new audio format, and will we need new software/players to play it?

Nageno: For now, it’s going to be under MPEG-H; but we have no name for it yet. It should be a new format. We’re talking to many partners to develop the required products and software so that’s under construction. 3D audio seems to be a trend now, with multiple companies pushing out their offerings recently.

Nageno: We believe 3D audio for customers is an impressive thing. With this, we can deliver some new experience for customers with music. One thing is high fidelity of the sound field. When you listen to a live concert, you can feel the ceiling height, the sound of the hall… If you listen through headphones, every sound is coming in just this area *gestures to the sides*.

About 50% is direct sound, and the other 50% is indirect sound. So, for recording engineers, with just two channel recording, they cannot record the full tone. They have to reduce the full tone, because 80% of the sound is overlapped with the echoic sound. This is the first time we’ll be able to record the full tone because everything can be mapped.

Then the second point is about creation. You know Bohemian Rhapsody? It’s something like creation. Galileo comes through the side. If the sound can come from any place, musicians can locate the sound in any place, even if it’s not real. Music can be more attractive with new ideas for sound location. So we deliver a canvas that they can use to create music.

Third, is an interactive experience. Listeners can choose their listening position with 3D audio. For example on some orchestral reproductions, they can choose to listen from the conductor's side, or one of the instruments. Not only with music, but maybe with sports too. It’s just more possibilities for people to enjoy entertainment.

Of course that means capture will become more complex too, because you’ll need to match position for each player on the field.

What would you say is the ultimate audio experience for you?

Sato: I commute by train, so I’m listening everyday by in-ear with Walkman. But my living room has an Atmos set-up that I use to enjoy movies on the weekend.

Nageno: Frankly speaking I listen to any type of music, and professionally I like to check the sound. But personally, I’m not an audio maniac. I enjoy playing instruments like trombone and wind instruments with other people. I generally listen any kind of music, but especially classical music. Not recent recordings, but the older ones; mono-aural recordings. That type of old music satisfles me. Of course, audio quality might be poor, but I can enjoy the music’s essence. Of course I can find new recordings with better quality technically, but sound quality is secondary to the performance.

I’m also listening not as a listener, but a player, so I look out for more subtle expressions. So, I can understand how High Resolution Audio can be a good way to express music.

How does Sony decide what to do next for the Signature series?

Sato: For players, our motivation came from requirements of our customers. We had the portable WM1Z Walkman player, and the larger DMP-Z1 Music player last year. These are two very different types of product.  For DMP-Z1, our requirement was huge power output for headphones, and this actually came from our WM1Z customers especially in the U.S. Now we have two good concepts for the market, so we want to get customers feedback for these products to see what to work on next.

Nageno: Basically, we try any kind of direction. *laughs* We’ll make this product, then if there’s a possibility for another type of product, we’ll try it too. That’s just the primary stage of development, but there’re many projects inside. And then, if we get a new device or big improvement, then we’ll go ahead to the next stage. So we don’t limit ourselves to just one direction.

Mostly finding another direction, not just following one direction is important to creating a new sound. Now, I am 61 years old and half retired, but younger engineers are coming up and maybe they have the next idea.

"Listeners can choose listening position with 3D audio. For example, on orchestral reproduction, they can choose to listen from the coductor's side, or an instrument."