2016 Google Tracker: Everything Google is working on for the new year
arstechnica.com
Android N, a big VR program, Google Glass, and lots more are in store for Alphabet.Google's big push into virtual reality
Enlarge Google is devoting such a large amount of resources to VR, it has to have more than Cardboard planned.
Google
Google sort of has a VR platform today with the ultra-cheap "Google Cardboard" program. Just throw your current smartphone in a specially shaped cardboard box with a pair of plastic lenses, and the experience is enough to get a rough idea of virtual reality. Cardboard isn't really "good," though. It's designed to be cheap above all else, which makes it a sub-par experience that you wouldn't want to use for any significant amount of time. The founder of Oculus, Palmer Luckey, was once asked if the Oculus Rift could compete with the inexpensive Google Cardboard program. Luckey answered, "Yes, because the Rift is actually good. Kind of like how fancy wine competes with muddy water."
The cheapness hasn't stopped Google, though. In April 2015, it launched the "Works with Google Cardboard" program, which allows third parties to make Google Cardboard-compatible viewers like the rebooted Mattel View-Master. There's a VR section for Cardboard in the Play Store, and Google continues to do developer outreach and content partnerships for the program.
Cardboard is just the tip of the iceberg for Google in VR, though. Google seems to be slowly assembling the pieces for a full end-to-end solution that covers hardware and software, but we aren't exactly sure what form this will take. All we know is that there's a ton of resources being devoted to some kind of VR project.
The first is employees. Some of Google's best and brightest, at the highest positions at the company, have left to join the VR team. The lead designer of Google Search, Jon Wiley, stopped working on Google's biggest product to join the VR team. Alex Faaborg, the former lead designer for Firefox, Google Now, and Android Wear, also left some of Google's biggest products to join the VR team. High-ranking people like this aren't taking new positions to work on a mere piece of cardboard.
The Wall Street Journal gave us a small glimpse into what they might be building, saying the company was working on "a version of the Android operating system to power virtual-reality applications." Google is also working on 3D audio of some sort. It recently acquired a company called "Thrive Audio," which is focused on creating an "ultra-realistic soundfield" for VR audio.
Google Processors—A "recommended hardware spec" for smartphone VRTo get enough horsepower to run this mysterious VR project, Google supposedly wants to dive deep into hardware and design its own SoCs. A pair of reports fromThe Information claims Google met with component vendors to discuss co-developing chip designs aimed at virtual reality and augmented reality. The reports quote a source as saying Google was looking to “get into the guts of the chip and make sure chip vendors are provisioning enough horsepower.”
Let's speculate a bit on what this could mean. Any smartphone SoC today is going to be geared primarily for 2D app and OS usage—things like smoothly scrolling through Chrome or a setting page and quickly launching an app. Smartphone SoCs can do some 3D processing for games, but that is a secondary function. The chip design's allocation of power, heat, and die space requirements will prioritize CPU horsepower over GPU horsepower, because when you're primarily browsing the Web and viewing apps, that's what you want.
Smartphone chips are also primarily designed for "burst" power. You load an app or a webpage, and you want everything to power up really fast, load whatever you need, and then go back to sleep to save power. If you run a 3D game for more than a few minutes, you'll quickly overheat the SoC. It will "throttle"—or purposefully slow down—so that it doesn't have a meltdown.For a virtual reality device, you would want to change a lot of this design. For VR, everything has to be rendered in 3D space, and everything has to be rendered twice for your left and right eye. You're basically running a heavy 3D game all the time. You'd want to allocate a lot more of your horsepower budget to the GPU, which a custom chip design could easily do at the expense of a top-end CPU. Since you're doing complex 3D processing all the time, you'd also want to design a chip with a thermal budget that allows it to run at top speed for long periods. The "burst power" design of a regular smartphone SoC wouldn't be ideal.
VR requires a special mix of hardware, and Oculus outlined what it needed when it released the recommended hardware specs for the consumer version of the Oculus Rift. The company said "your CPU needs to be this fast, your GPU needs to be this fast" and so on. Oculus has the luxury of doing this on a PC, though, where the CPU, GPU, and other components are all separate pieces. Customers or PC OEMs can look at this list and easily assemble computers. Google needs to lay out a recommended hardware spec, too, but because it is dealing with a system on a chip where the whole "computer" is a single piece, it must become a chip designer if it wants to specify the CPU, GPU, and other components.
We don't think Google really needs to do anything revolutionary or proprietary when it comes to a VR chip. Off-the-shelf ARM components would be fine for the most part. A VR device just has different system requirements from a smartphone, and current SoC vendors are only building smartphone chips. Once VR takes off, we'd imagine Qualcomm would offer a line of SoCs meant for VR, but since Google is being a trailblazer here, it has to make chips on its own for now.The reports also note that Google wants to make a lot of changes to the camera image processor, which is something that could potentially power an augmented reality device. It's apparently looking for something that can act like a "third eye" and "scan the environment and push images to Google’s cloud-based systems for analysis." Google's servers would then respond and “give you context back.” The reports say Google is also looking to "add support for a wider range of sensors, including one that can measure distance."
Google itself actually seems to back this report up. The company has a public job listing for a " Multimedia Chip Architect" that will "Lead a chip development effort" working in "aspects of multimedia including image processing, video processing, stabilization, etc." The job listing asks for "experience with image/video/display pipeline in SoC," and the eventual goal is to take the chip "to product shipment." That sounds a lot like the rumored image processor mentioned in The Information's report.
A custom camera image processor would be a big help if Google wants to build an augmented reality device. Augmented reality is a lot harder than VR, since it basically combines all the challenges and processing power of virtual reality and then overlays it on top of the environment in front of you in real time. This "overlying" can happen either on top of a video feed or on a transparent display, but either way doing it well means tracking the real world in front of the device in 3D, sending it to the GPU, and combining it with computer-generating elements. It's hard enough to do this once, but all of this has to happen many times a second (video is minimum 24FPS; the Oculus Rift runs at 90 FPS) to make it seem like the virtual object really exists in the 3D world.
A smartphone image processor designed to take pictures and the occasional video just isn't up to the task. Like in our VR example, this will take a smartphone component that is designed to only be used in short bursts and require it to be turned on 24/7. Currently, Google's is working on Project Tango, which is a tablet packed with sensors that allow it to do augmented reality on a smartphone screen. On that device, the computer vision is handled by a co-processor from a company called " Movidius." A Google VR SoC might want to license component designs from Movidius or some other company and integrate it directly on the SoC.
Apple has been building its own chips for the iPhone and iPad for some time now, which gives it a level of hardware and software coordination that puts it at a clear advantage over Android. Apple was the first to release 64-bit devices, and the hardware/software combination is a big part of what made Apple the leader in mobile cameras for a long time. Because Android's hardware and software come from different vendors, any new hardware feature on Android faces a chicken-and-egg problem. There's no software support because there's no hardware, and there's no hardware support because there's no software. By designing its own chips, Google is solving this problem for its VR division.
A software effort is pretty much expected from Google, but the act of wanting to design your own SoC says that Google will probably have a big hand in the hardware design of whatever this is. We still don't know what we're looking at, though. Is this a self-contained headset? Is it a normal-looking smartphone that drops into a headset like Cardboard and Gear VR? Is it an even more specialized smartphone-sized device, like Project Tango, which then drops into a headset?
Cardboard does not feel like the answer to all this. Cardboard is a very poor VR experience, and it seems to be held back on purpose. Google won't even make extremely basic, inexpensive improvements like a head strap. Right now you want to stop using it after five minutes, because you get tired of holding the device to your face. We think this is designed to keep Cardboard as a VR "taste test" and not let it evolve into something more serious. Cardboard feels like a developer kit or funny promotional item more than anything. We think Google's real VR initiative will come later. |
