How To Create An AR app

Our last article was about choosing a VR tech stack, today’s article is focused on choosing an AR tech stack. While there are overlaps, building for AR is a slightly different ball game. 

‍

Just like with building for VR, you have to consider who your audience is and the platform or device you want to have the experience on. But it’s also a bit more complicated. We’ve listed all the things to consider below:

1. The first consideration is whether we’re building a WebAR app or a native mobile app. Android? iOS? WebAR? 
2. Who is your audience and what device/devices will they experience your AR app with? What platforms are they on? Which mobile operating system do they use? What types of network connections are they on? You need as much data as you can gather on your customers and the type of devices they use. Have a baseline device that serves as a minimum requirement.
3. How will you trigger the app's AR functions? Is it with a marker like a QR code or a special type of graphic, thus a marker-based app? Or will it work with the device’s sensors to kickstart in real-time, thus being markerless? Or will it work with GPS and trigger only in specific locations thus being a GPS-based app? Those are the three major ways to trigger your app's AR functions to kick off at the right time. Which one works best for you?
4. The next thing to consider would be what SDK you intend to use for the app and this depends on the previous two points, your audience, and their devices, and how the AR functions in the app will be triggered. You’d need an SDK that supports the platform of your choice and supports a game engine, e.g., Unity and you’d also need to consider the triggering system the SDK supports (i.e., marker-based, markerless, or location-based.)
5. Finally, you’d need to think about what features you’d want the application to have e.g., the type of interactive elements, whether it’s a solo or a multi-user app, whether it’s 3D AR, projection-based AR, Outline AR etc.

‍

Let’s go a bit more in-depth on each of these considerations.

‍

1. Audience and Platform

There are many ways to share and feature AR experiences. The most popular way is via mobile phones, courtesy of constant upgrades in processing power, camera quality, the addition of sensors like gyroscopes and accelerometers, and GPS functionality to name a few features. Other methods include Smart Glasses,  headsets with Mixed Reality capabilities (e.g., Meta Quest 3 & Apple Vision Pro), or holograms.

The platform you build for, depends on the audience you’re trying to reach. Many developers choose to build cross-platform apps, however, this has its drawbacks, as there’s a limit to how well cross-platform apps can take advantage of device features compared to apps designed specifically for a platform or device. For example, an app designed to take advantage of iOS will provide a better AR experience than one that was built with cross-platform support in mind. This is because the iOS app is entirely focused and specialized for iOS devices as opposed to the cross-platform app, which must function with a range of devices.

Choosing your audience paves the way for deciding how you want the app to work, and much more as we’ll cover in the coming sub headings. 

‍

2. Features

This basically covers how you want to integrate AR features into your app. 

Do you want your AR experience to be a 3D projection i.e., it projects a 3D object like a chair or equipment into space or when it scans the marker? Or a superimposition i.e., it adds features to an already existing object like an Instagram filter adds features to your face? Is it a passive experience or an interactive experience? Can it be shared on social media? The crux here is deciding what you want your app to do and how you want it to happen. This is where you outline the experience.

‍

3. AR Triggers

As mentioned earlier, there are three main ways to kickstart the AR features on a device. 

You can either use a physical marker like a QR code or an image, or you can use a markerless app, i.e. it doesn’t require any physical markers and works with the device’s sensors to detect lighting and plane surfaces to generate the AR experience. The final option is to use the device’s GPS capabilities to detect the audience’s location and thus kickstart the experience when the user has the app open in the right location. 

Discussing the differences between these methods requires its own article, but we found this nifty table from Kevuru Games, a game development company that gives a quick summary of the pros and cons of each method. 

‍

Determining what type of trigger you want will help you determine the next step, which is choosing an SDK

‍

4. SDK

We defined SDKs in our previous article on VR tech stacks and as mentioned they’re toolkits that come with libraries, guides, elements, and blueprints that developers can use to create XR experiences for devices. Your choice of SDK depends heavily on the platform and audience you’re building for, among others. Below is a list of factors to consider when choosing an SDK:

• Free or paid: Many SDKs are free because most manufacturers provide SDKs for developers to use; Meta has Meta Open XR, Microsoft has Mixed Reality Toolkit, Apple supports ARKit and Vision OS, the list is long. Things are different with web-based XR, however, many of the popular and reliable development tools, e.g., 8th Wall, Spark AR, etc. all require payment.
• Platform: Are you developing for iOS, Android, or the web? 
• Device: Is it for PCs, mobile phones, Mixed Reality headsets, or Smart Glasses?
• Cloud support: Does the SDK offer cloud integration? Cloud storage reduces the memory and performance demands running an AR app places on your device. SDKs like ARCore and ARit offer cloud support via content delivery networks (CDNs) like AWS and Google Cloud platform.
• Game engine support: Not all SDKs work with all game engines, you want to ensure you pick an SDK that’s compatible with the game engine of your choice. 
• GeoLocation support: Necessary if you want to build a location-based AR app
• SLAM (Simultaneous Localization and Mapping): SLAM technology is what allows self-driving cars and automated robots to map an environment and detect objects in real-time. Your SDK should support this technology if you’re working with a markerless AR app
• 3D tracking: Does the SDK have functionality to track 3D objects?

The above list is a summary of all the features one must consider when choosing an SDK. Some popular SDKs are Google AR Core which is specifically for Android devices, Apple AR Kit which is for iOS devices, Vuforia which is cross-platform, 8th Wall which is browser-based, Meta’s Spark AR, Needle Tools, MyWebAR, Tik Tok’s Effect House, A-Frame, and Unity’s ARFoundation. However, there are other options out there.

‍

‍

Conclusion

There’s a lot more detail to be covered when it comes to creating an AR app, but this is just an overview. It should also be noted that this list is not exactly chronological. You should have a clear audience and platform in mind and you should know what features you want your app to have before development starts, but like with every process you may need to review decisions and choices. 

‍

Our next article will touch on building a Mixed Reality app and the tech stack involved. Till then, stay building.

‍