We only have half of the Magic Leap SDK

Recently, Magic Leap released its SDK to developers so they can start playing with what is possible on the Magic Leap platform. With these tools you can develop a full Magic Leap application to run on the device when it is eventually released. But the documentation and toolkit is incomplete. There is a huge section that is still yet to be released to the public. 

That is, the Lumin Runtime. And it could completely change how you think about apps.

Making an App

When you go to make an app on Android or iOS you have to choose the tools you want to use to make that application. There are 3 main choices: A Web App, using the official toolkit, or going native.

These days, web apps are so powerful that in most cases this is a good choice for developers. But if they need a bit more, they can use the official toolkits provided by the platform. This is using Java or Kotlin in Android and Objective-C or Swift on iOS. Most Apps are developed this way. But some apps, mostly games, need direct access to the hardware and need to more carefully manage compute resources. Platforms expose a specific SDK for this. On Android this is called the NDK or native development kit. 

What Magic Leap has released is the equivalent of Androids NDK. It has yet to release its application toolkit. This is important as there is a good chance that most applications made for Magic Leap will be written using this toolkit and not the 3D engines already released.

Lumin Runtime vs. 3D Engines


The Lumin Runtime sits beside the 3D engines in the Lumin OS stack. The SDK for these 3D engines are released already but we only have one short section on what the lumin runtime will be in the docs. It seems this runtime is geared towards developers, such as web developers, who might not be as comfortable working with game engines or starting from a blank canvas. It is for simple applications. For example, making an email client from scratch in unity or unreal is quite a daunting proposition for most developers. The Lumin Runtime helps developers by giving them a framework to use. It will provide a common design language for developers and help them navigate the challenging task of developing applications in an unknown 3D space. 

Throw out the App Model

One other detail about the Lumin runtime is that it is built for multiple applications to run at the same time stating: "It is capable of running and rendering multiple native applications simultaneously." This is not true for the 3D engines which appear to take over the whole device. 

Traditionally, on mobile hardware, applications are installed and to use them you have to open them up and run them. One at a time. On android you have some more flexibility with this but generally most applications stick to this model. I believe (watch me wildly speculate again!) that Magic Leap will be a bit different. You won't click a grid of icons to open an app but an app will live in a physical place somewhere. Let's say you have a word processing app. It will live at your desk. To use it you can just walk up to your desk and it is there. A television app will live on the wall. You install it like a real TV. You might stick a mapping application to your front door so you can grab it on the way out. Perhaps have a recipe app beside your stove. The point is that you won't need to scroll through your app drawer and find an icon to click on, the app will just be where you left it in physical space. This is all speculation on my part but I think it makes sense. 

The documentation says that the runtime will "facilitate application sharing out of the box, allowing application developers to collaborate with multiple users." So if I install a TV app on the wall, everyone will see it. It is persistent and as part of the world as a real television. I could "hang" digital art on the wall and everyone who comes to my house would see that art through their own set of ML glasses. 

This is a radically new way to think about applications and computing in general. It is fundamentally tied to physical space and multi users. It truly is augmenting reality and I think one of the more exciting aspects of what Magic Leap is doing.