The iPhone 12 Pro and 12 Pro Max carry Light Detection and Ranging LiDAR scanners. What is this, and why should it matter to your enterprise?
Both the iPhone 12 Pro and 12 Pro Max possess Light Detection and Ranging (LiDAR) scanners. Why should it matter to your enterprise?
What is LiDAR?
LiDAR systems are capable of measuring how long it takes for light to reflect back from a surface in order to measure the distance travelled and create 3D depth maps of what they see. The technology helps support realistic AR experiences, and also provides a huge boost to photography in low-light.
LiDAR is not a new technology. It relies on tiny lasers that bounce light pulses against objects around them to figure out spatial information based on the journey it takes to send and return that light. Most people first became aware of it when it was used to help Apollo 15 map the moon’s surface. (NASA is now developing the technology for Mars missions.)
LiDAR has seen multiple uses since – it’s being used to inform collision detection systems in autonomous and semi-autonomous vehicles, and it’s expected to be deployed in all new cars sold (smart or otherwise) from 2030. It is also used in some smart vacuum cleaners (and security researchers [PDF], who have figured out how to use these systems to eavesdrop in homes).
Apple and LiDAR
Vehicles and robot cleaners use LiDAR to sense objects around them and to prevent collisions, but the technology has other uses on Apple’s smartphones.
It’s important to note that Apple’s LiDAR implementation relies on a system that sends out multiple beams of light, while cheaper systems use just one.
With support from the A14 Bionic chip, Apple’s implementation is faster, more accurate, and with longer range. It also means the devices can successfully figure out how to place virtual objects in complex scenes using object occlusion.
This is why if you project a landscape onto your room the grass will grow only on the floor for more realism.
It uses Single Photon Avalanche Diodes (SPADS) and Vertical Cavity Surface Emitting Lasers (VCSELs), which means it can accurately measure objects at a distance of around 20-feet. ArsTechnica points out that the one more thing to this is that the LiDAR tech now used in iPhones once cost upwards of $70,000, so Apple has bought this to the mass market.
At present the primary uses of LiDAR on iPhones are:
Augmented Reality (AR): These iPhones can detect and identify surfaces to generate a 3D map of what’s around you. You can use this to scan rooms, measure people and items, apply imaging effects and place AR objects.
Low light and night time photography: On iPhones that support it LiDAR provides a huge improvement in autofocus in low light conditions, which means you can take the photo faster and more accurately. It also works with the camera and the machine intelligence locked inside Apple’s Neural Engine to capture extraordinarily good night shots.
Measurements: The Measure app inside your iOS device uses LiDAR. When the app detects a person in the viewfinder it automatically measures their height. Tap the shutter button to take an image of that person with their height measurement.
Of course, all three of these uses support developer-focused APIs, which means innovative new apps are emerging regularly at this point.
Why Apple is using LiDAR
We all believe that Apple is developing an AR headset, a system expected to deploy all Apple’s relevant technologies (such as AR, Maps, Siri, A-series processors, AI and vision intelligence). LiDAR quite clearly has an important role to play in these devices, which it’s reasonable to predict will enable users to merge virtual and actual space without inadvertently wandering into the path of a speeding train.
Apple appears to agree. The company is researching implementations such as a recently revealed patent for real-time LiDAR surface tracking. What’s lesser known is that the company has been researching in this space for around a decade, and that its patents include one which could make use of LiDAR to inform a headset’s lens system to improve the eyesight of the wearer. We’ll also see better at night.
The company’s extensive research in autonomous vehicles is also a matter of record, and it has multiple patents in this space, including at least one that uses LiDAR to power vehicle collision detection and prediction systems.
We can’t expect self-driving vehicles for some time longer yet, but LiDAR guidance and collision systems are being deployed by the big automobile firms today – though not (yet) from Apple. There are also opportunities in road traffic management, automated traffic toll systems, and other forms of transport infrastructure.
Apple is currently nurturing innovation in the AR and LiDAR space with a view to supporting future platforms and to maintain its mission to bring AR to the mass and enterprise markets.
Get to know LiDAR today
If you’re one of the many enterprise customers who has already or intends to raid the IT budget in order to purchase an iPhone 12 Pro or Pro Max “for testing”, then you should explore this handful of apps which show different ways in which LiDAR can be applied and how it may be useful to your business.
- IKEA Place: An app that makes use of LiDAR to place to scale representations of items sold by the giant retailer. Another take on this is available in the Wayfair app.
- Hot Lava: This Apple Arcade game transforms your room into a gaming environment. It is also an excellent illustration to show how LiDAR and AR can be used together to augment space – think warehousing, field maintenance, or even emergency despatch.
- Canvas: This app scans and measures your room to create accurate 3D models of your space which can be converted to CAD, useful for interior design and home renovations. This is the kind of information likely used by Apple’s teams in the construction of a new London HQ.
- Primer: A similar app with a slightly different focus, Primer uses LiDAR and AR to help you measure your space and virtually place curated paint, tiles and wallpapers before purchasing them for your DIY projects.
It may also be useful to explore some of the ways in which enterprises are using these technologies are making use of the tech in non-mass market scenarios.
How enterprises are already exploring LiDAR
We’ve mentioned autonomous vehicles, autonomous and semi-autonomous indoor and outdoor robots and space travel, but LiDAR has useful applications across many industries. These include the automotive, consumer electronics, consumer health, industrial and security markets, according to two former Apple engineers who recently launched 4D LiDAR chip company, Aeva. Backed by Porsche, Aeva’s chip brings the capacity to measure velocity, which makes it effective for tracking moving objects.
Here are some other usage cases:
Exploration: Researchers based at Tulane University recently made use of LiDAR to reveal an astonishing 61,000 undiscovered Mayan structures in Guatemala’s Petén region. That’s an eye-catching use of a technology which is also seeing use in geographic and natural resource exploration.
Architecture, design and project management: Network Rail made use of high-end 3D LiDAR scanning technology to build a photo-realistic scan of an historically important 300-year old railway bridge. This has helped guide the project.
Virtual experiences: Pokemon Go showed the future. This is now a well-travelled path with local information, maps and LiDaR distance measurement combined to create virtualized experiences. Think how such augmentation might be used in retail to help guide the customer journey, for example.
Security systems: In conjunction with machine vision intelligence, LiDAR is seeing use in security systems as its time-of-flight sensors and accuracy helps reduce false positives, even in low lighting situations.
Agriculture: Agricultural management systems based on highly accurate LiDAR-supplemented soil analysis (PDF) is already being explored. The Netherlands is currently testing autonomous LiDAR-guided boats and agriculturalists make use of LiDAR vision in drones to help manage large farms and distributed livestock. The Department of Engineering at Aarhus University used LiDAR to map wheat crops.
There’s much more research into real world applications of this technology across multiple industries, from biology and conservation to climate change, pollution modelling to forestry management, hydrographics to sewer and manhole maintenance, and much more.
But, while the majority of the industrial systems currently in use in many of these scenarios depend on highly expensive LiDAR equipment, what Apple has done is take this potential to a wider audience, enabling enterprises and developers to visualize and realize multiple mass market applications of this powerful technology.
