How Herschel uses AR on product pages with headless commerce (and so can you)

Augmented and virtual reality is gaining traction with brands and retailers looking to enhance buying experiences. And it’s getting easier to adopt and inject into product pages thanks to headless ecommerce APIs and progressive web applications (PWAs).

Today’s late-model mobile devices are shipping with enhanced native AR support, with toolkits like Apple’s ARKit and Google’s ARCore offering developers a way to connect AR to the storefront. And unlike early iterations of AR for ecommerce that required native app downloads, today’s experiences can be launched through mobile browsers.

While web-based AR still can only be used by shoppers with compatible hardware and operating systems, within one more upgrade cycle (~2 years) we can expect AR accessibility to hit critical mass.

For those who want a head start, adding 3D models and AR to product pages may be simpler than you think.

How Herschel Supply uses AR on product pages

The first step to AR-izing products is to leverage a 3D scanning application such as Qlone, ThreeKit or Herschel’s choice, Vertebrae. Many scanning apps allow you to create models in-house with just a smartphone (All3DP has a great review of popular options).

Because 3D scanning can be a time consuming process, consider starting with your top-selling evergreen products, and choose one master SKU variant for each (e.g. color).

The beauty of capturing 3D images is any visitor can view them when added to your image gallery without AR -- so it’s time well invested! For users with AR-compatible devices, you can display an icon or toggle to switch to AR mode.

You can use device targeting to show AR options only on compatible devices, or display it for everyone. Herschel displays the option for all Web visitors through any browser. Clicking View 3D and AR launches a QR code with instructions on device and OS requirements.

Users can then “place” the selected item, which appears true to scale, and drag, rotate or view the item from any angle.

Check out this experience yourself on Herschel.com

Using ARKit and ARCore (and more)

Developers can use Apple’s ARKit to build experiences for newer iPhones (8 and higher), tapping into their native neural processors and AR features. (Technically, iPhone 7 also supports AR, but with slower, more battery-sucking performance).

Serving AR to iPhones through the Safari browser requires ARKit’s Quick Look extension (iOS 12), which enables 3D models to be viewed in the real-world context through a user’s camera.

How Quick Look works

To play nicely with Quick Look, your 3D-scanned images must be saved with the USDZ file format and hosted through your CDN. AR tags will only display on compatible devices and when a corresponding USDZ file exists and is detected through your CDN (and called to the storefront via page view). This ensures empty tags don’t litter your catalog.

When a user visits a product page with a corresponding USDZ file and activates AR mode, the file will be loaded into the Safari browser and rendered in a full-screen Apple AR viewer app.

For a more technical description of how to use ARKit for ecommerce, our own Senior Software Engineer Shaun Maharaj documents how our Product team built AR into product pages using Quick Look in our React.js demo store/reference application.

(Elastic Path customers: check out our documentation for ARKit Quick Look integration)

Google’s Scene Viewer

For Android, Google’s Scene Viewer is ARCore’s answer to Quick Look. The twist is you can leverage Google’s AR wrapper with both Apple’s USDZ and Android’s GLB 3D file formats with a single line of code. Users of both mobile OS can launch 3D products in their respective viewers (Quick Look or Android’s Model Viewer), or with Magic Leap headsets.

However ARCore’s framerate and accuracy is reportedly not as good as ARKit’s at this time. And while Android today has a larger install base than iOS, it’s more fragmented, and not all users are on the latest compatible OS version (Q).

Both Quick Look and Scene Viewer support sharing of AR models through SMS, email, messenger apps and chatbots.

Third-party AR viewers

Third-party tools like Vectary are also available for merchants who want to serve WebAR to both iOS and Android users though a universal viewer. Vectary’s AR features are opened in its own viewer via an iFrame you place on your product pages (enabling desktop users to play with 3D images as well).

Vectary also aims to make it easier for designers to add AR experiences without code. Its “CMS for AR” approach pushes changes from its 3D editor directly to your site.

What about WebXR?

WebAR/VR (now WebXR) APIs were conceived in the spirit of democratizing development and accessibility beyond iOS and Android, and are supported by Microsoft (including Hololens devices), Google and Firefox (desktop and mobile), Samsung Internet, Servo (desktop) and Microsoft Edge and Hololens devices. (Phew!)

The biggest challenge with WebXR is Apple’s on the fence whether they’ll support it. For that reason, we advise ecommerce developers and designers to focus efforts on ARKit, ARCore or 3P tools that play nicely with iOS.

AR viewer limitations

Quick Look, Scene Viewer and their 3P cousins work best when you want to render a single, fixed model in AR versus create custom UI environments or display multiple images (products) within the same view.

In other words, they’re:

• Great for virtual try on of a single item, more difficult building an outfit.
• Great for overlaying an image through a customer’s camera feature, less good for arranging multiple products in a virtual space.
• Great for seeing a product in context, not so much for swapping colors, finishes or patterns with a single tap.

More advanced AR requires not just 3D imagery, but scaled and textured modeling, and plane and spacial recognition to display multiple products or to display them in a truly virtual environment. This is important for ecommerce, as less-than-realistic AR and VR can lead to disappointing unboxing experiences -- especially when patterns, finishes and textures are key attributes of a product.

Reality Composer and AFrame

To handle more advanced AR (and even VR), enter ARKit’s Reality Composer, a prototyping tool that allows you to build AR experiences within “virtual space.”

You may have guessed it, but yes -- Apple’s Reality Composer requires USDZ files (and .reality files) and doesn’t extend to Android, though you can build in Reality Composer and serve through a 3P viewer like Vectary to serve both platforms.

Reality Composer is also friendly to non-techies. Amir from Augmentop shares how they pulled price, customer reviews and a cart button into an AR experience in about an hour, with no coding skills.

Alternatively, virtual reality can be built with AFrame (leveraging WebXR), an open source, HTML-based framework. For example, the Elastic Path reference store chose this option to work natively through the Web and PWAs. (Test our sample product in a VR world here - click the “viewer” icon in the top left of the image).

Build AR/VR without coding

For those that want to explore AR and VR without technical skills, SaaS platforms and toolkits are available.

Why headless commerce for AR/VR

Accessibility and ease-of-adoption

The first step out of the walled garden of native apps is to choose a headless commerce platform to support a PWA front end. This enables you to leverage web-friendly AR without an app download.

Apps also don’t allow easy navigation between ecommerce pages and features like cart, checkout and browsing history, nor can app AR be accessed from any merchandising zone or product page in your catalog. While possible, it’s complicated, and the user may not have the appetite to keep switching between the app and the Web.

Choosing to deliver AR/VR through mobile browsers helps ROI. It’s critical that your new AR and VR features are available and easily accessible to as many customers as possible. Reducing cross-platform development and maintenance is another benefit.

Direct access through APIs

One of the biggest advantages of using headless commerce, and microservices (or Packaged Business Capabilities) in particular, is direct access through APIs. This helps you inject product data, prices, attributes, promotions, customer reviews and more into your AR or VR experience.

Because your commerce APIs have all the technology you need and can connect with device and toolkit APIs, there’s not much that needs to be recreated.

In-AR transactions

Today’s mobile AR viewers don’t natively support checkout and payments -- users must exit the viewer and return to the product page. But with headless commerce and APIs (and some clever development), you can support frictionless AR and VR shopping, keeping your catalog, cart and even payments within one experience.

Support “future commerce”

As toolkits evolve, expect to see more shoppable AR in-store. For example, walking through aisles and seeing prices, promotions, loyalty rewards and endless-aisle capabilities appear through your camera. Or play gamified “Pokemon Go” shopping games anywhere.

Or personalize with cross-sells and upsells appearing as customers browse a physical shop, engage with signage and outdoor advertising, or explore a virtual shopping experience through a headset or other wearable. Connected to CRM, personalization engines, location awareness and even browser history, there are many ways AR and VR will become part of the new “digital-in-store.”