Tim Cook laid out the playbook for Apple’s next big thing.Apple is the most valuable company in the world, and it makes a lot of profit, but its long-term future depends on it coming up with a new product, a new “one more thing.”
The Apple Watch, which has been selling poorly for Apple, isn’t going to do it. Apple needs a new platform. And Apple CEO Tim Cook knows it. That’s why he’s been dropping hint after hint that Apple’s next major technology initiative is augmented reality — basically, integrating software into the real world.
AR is a close cousin of virtual reality. For now, augmented reality means you can hold up your phone and, for example, identify landmarks, discover a hidden Pokemon, or magically translate signs using your device’s camera. The end game of augmented reality is a pair of computer glasses or perhaps a contact lens that can superimpose computer graphics into the real world seamlessly.
Apple CEO Tim Cook has talked about augmented reality essentially every time he’s been called on to make public remarks recently. “This is something you know it’s coming,” Piper Jaffray managing director Gene Munster told Business Insider. “The only other times they’ve talked like this is before the iPhone came out, they started to indicate they could do something in the phone market. Before the Apple Watch came out they talked about wearables and the wrist being a better option.”
“It’s pretty rare that Apple has been this clear that something is coming,” Munster said. If you parse Cook’s most recent comments on AR, given at an event in Utah in October, he seems to lay out Apple’s strategy for attacking this market, according to analysts and people in the AR industry.
It will be enabled in the operating systems first, said TIM Cook, because it’s a precursor for that to happen for there to be mass adoption of it. I’d look for that to happen in the not-too-distant future. In terms of it becoming a mass adoption [phenomenon], so that, say, everyone in here would have an AR experience, the reality to do that, it has to be something that everyone in here views to be an ‘acceptable thing.’
I do think that a significant portion of the population of developed countries, and eventually all countries, will have AR experiences every day. It’s almost like eating three meals a day, it will become that much a part of you, a lot of us live on our smartphones, the iPhone, I hope, is very important for everyone, so AR will become really big, he concludes.
AR is going to take a while, because there are some really hard technology challenges to overcome. It’s already permeating many platforms today, and the business cases are also mushrooming. Adding key AR building blocks, like the ability for a camera to identify objects, into iPhone software and Apple apps is part of Apples strategy for the near future. This includes adding AR-specific hardware to the iPhone, like a 3D sensor as part of Primesense, a company Apple bought in 2013.
Opening up the “platform to developers”, so that programmers can make augmented reality apps without dealing with the complicated computer vision and physics algorithms currently being developed by people at Apple and other companies with is a step in the direction of Apples strategy to do the heavy lifting and allow end-users and mid-users to innovate around use cases.
One of the insights I see is the Apple may eventually be releasing an AR-specific product, perhaps a pair of glasses, when it’s Technology is ready. Such an embedded capability in iOS and MacOS will make taking advantage of AR apps in iPhone exceedingly easier with seamless experienece.
“AR is related to all Apple’s current businesses; the key is that AR is an innovative human-machine interface that could be used in various devices & applications,” KGI Securities analyst Ming-Chi Kuo wrote in a note distributed to clients last week seen by Business Insider. “All of Apple’s past successes were related to human-machine interfaces.”
‘Enabled in operating system first’
“We believe Apple could embed early AR features in iOS 11,” the version of iOS expected to be unveiled next June, UBS analyst Steven Milunovich wrote.There are already signs that Apple has experimented with integrating some headset-related features into its iOS. In 2015, it purchased Metaio, a company that had developed a mobile AR browser as well as a whole suite of AR tools.
Last month, developer Steve Troughton-Smith uncovered some interesting features currently included in Apple’s iOS. These features seem to have been abandoned for the time being, Troughton-Smith told Business Insider.
“I think the kind of VR you build for something dumb and simplistic like Google Cardboard is very different from what real VR needs. I imagine they got to the point where they realized they were going down the wrong path if they want to do this seriously…” Troughton-Smith said.
It isn’t the first time that Apple has built an augmented reality application that’s yet to see the light of day. 9to5Mac reported last year that the company had been experimenting with a feature in Apple Maps, based out of the Points of Interest system, that would show an augmented reality view for local listings.
“The augmented reality feature allows a user to hold up her iPhone in the Maps application, and point her camera toward a particular business or an area. Pointed towards a cafe, for example, the screen could show a virtual view of menu items or daily specials,” 9to5Mac reported.
Apple is also reportedly working on a Snapchat-style messaging app that could use augmented reality technology to apply filters or manipulate faces in real-time.
“We expect Apple to generate preliminary results for AR in the next 1-2 years at the earliest and working with iPhone may be the first step,” Kuo wrote.
living on our smartphones
“There are two phases to this: the phone is becoming a better AR device and that’s probably what Apple’s going to do over the next five years,” Munster said. “They’re going to start to make it more clear that the iPhone is an AR platform, then it’s up to developers to create experiences.”
“Our best guess is next year, AR is going to be one of the selling points,” Munster said. The next step would be to add depth-sensing capabilities to the iPhone. Depth-sensing is critical for AR — if the computer can tell how far away something is, it’s much easier to manipulate on a screen.
Apple’s PrimeSense 3D-sensor has yet to be fully integrated into the iPhone. But Apple did hint at AR-specific hardware with the iPhone 7 Plus. The iPhone 7 Plus has a dual-lens camera. Basically, the two lenses can act like a pair of eyes, and with some math and triangulation, can give some rough distance estimations. But there are issues with the lenses currently on the iPhone 7 Plus.
Experts say that the lenses are too close to each other to do much useful depth sensing. Imagine if there was no distance between your eyes, for example. Fast Company talked to a developer who thinks the max distance that it could accurately identify distance is perhaps 50 centimeters — or slightly farther than you typically hold your smartphone away from your face.
Developers are already eager to make AR apps for the iPhone — as soon as Apple releases an AR platform, say, ARKit, there will be a slew of new apps and experiments. Lenovo launched a phone this week that includes a 3D sensor using Google’s technology.
The challenges to overcome
The real step forward will be having the augmentation of hardware, say glasses that could provide for the experienece required. “The future of AR is only possible with a mixture of single person viewers and other wearables. As technology rapidly increases by way of electronics miniaturization and manufacturing, its possible that by 2020 we will see some very interesting and natural designs by way of wearables etc.
Some glasses are already on the market — Microsoft sells a version of its Hololens AR headset to developers already, and it will go on sale to consumers in the next two years. Magic Leap, a Google-backed startup with $1.39 billion in venture funding, is also gearing up to launch a product in the next year or two.
Apple has been designing headsets in its prototype labs, according to the Financial Times. Aince Apple is rarely an early adopter, there are several issues with headsets that exist, including size, weight and portability. These headsets are difficult to integrate with batteries without getting them chunky, and the tethering available today is mostly hinged on chords. There is also the other important part of resolution. Currently available AR optics have issues with high resolution, color, and field of view.
It’s hard to see why Apple would want to bet the future of its company on a new kind of technology that might not ever catch on én mass. Bank of America Merrill Lynch estimates that AR could have 75% of a $90-$117 billion market by 2020, with that sounding very bullish.
It’s easier to see what Apple’s competitors see in the field. Microsoft launched a new computer last month, but instead of focusing on Windows, it spent a lot of time talking about 3D modeling and its Hololens product.
Microsoft missed the boat on mobile. Apple took over 100% of the profit in the smartphone sector this past quarter. So if it wants to become a major platform player again, it needs to find the next one. Smart people think it’s going to be AR, even though the market is only nascent.
“Today, Microsoft is focusing on business applications with Hololens, and that is not what Apple focuses on,” Blau said. “They really only ever focus on consumer technology for everyone.”
“If Apple does anything, it is going to be an every-person’s technology,” Blau said.
What makes Apple a special company in Silicon Valley is it can design both the unique hardware and software for the project, and it remains one of the best companies at user interface and design. If it were to launch consumer AR based on that, it would be ahead of nearly every other major tech company in this space.
If Apple nails this transition, it could continue to be the world’s most valuable company for years to come. If it doesn’t, Apple becomes a less exciting company, Cook suddenly starts to look a lot like former Microsoft CEO Steve Ballmer — a caretaker who missed the boat on the next big computer platform.