Today is November 6th, 2022, the day of the SAO Incident. Thousands of VRMMORPG gamers were trapped by a mad scientist inside a death game that could only be escaped through completion. If their hit points dropped to zero, their brain would be bombarded by extraordinarily powerful microwaves, supposedly killing the user. The same would happen if anyone in the real world tampered with their NerveGear, the virtual reality head-mounted-display that transported their minds and souls to Aincrad, the primary setting of Sword Art Online.
This type of scenario has been a staple of science-fiction for decades, but Sword Art Online exploded in popularity at exactly the right time to have a massive impact on the real world. The already-popular web novel was adapted into an anime series that aired just as the Oculus Rift went from internet forum curiosity for turbonerds to siren of the games industry, endorsed by giants like John Carmack, Gabe Newell, and Cliff Bleszinski – the 4th episode of SAO, “The Black Swordsman” (黒の剣士) aired in Japan just as the Kickstarter for the Oculus Rift DK1 launched!
The popularity of SAO led to massive otaku enthusiasm for Oculus, especially in Japan, which quickly became our 2nd largest market. In turn, the existence of the Rift made SAO itself seem far more plausible and grounded – a story that had been written in a world where VR was a dead technology was now straight out of the gamer hype headlines. This synergy had meaningful impact on our dev kit sales and adoption – literally thousands of people reached out to me asking variations of “Have you seen Sword Art Online? When will you make the NerveGear real?!”. The Oculus SDK was the closest thing to SAO’s World Seed, a software tool that enabled anyone to create incredible VR worlds.
There are dozens of stories I could tell, many of them worth a post of their own. How we ended up with a Kirito mural on the wall of the first Oculus office, how it was eventually moved to Facebook HQ, why it was removed shortly after I was fired. Personally showing Oculus DK2 to Reki Kawahara at Anime Expo, my wife meeting him while cosplaying as Sinon at the next Anime Expo, getting into minutiae of SAO canon at the Hollywood premiere of SAO: Ordinal Scale, the audience screaming as Yuuki came back from the dead to perform a two-user Mother’s Rosario attack combo with Asuna against the 100th floor boss of Aincrad. Arguing with people who only watched the anime about the supposed plot and tech holes, explaining that the anime was only ever meant to act as a companion piece for fans of the novel series that so frequently predicts the future of AR, VR, AI, and more. The beautiful wedding gift from Kawahara-san that shows he is a double-threat writer/artist combo!
But that isn’t what you are here for. You want NerveGear, the incredible device that perfectly recreates reality using a direct neural interface that is also capable of killing the user. The idea of tying your real life to your virtual avatar has always fascinated me – you instantly raise the stakes to the maximum level and force people to fundamentally rethink how they interact with the virtual world and the players inside it. Pumped up graphics might make a game look more real, but only the threat of serious consequences can make a game feel real to you and every other person in the game. This is an area of videogame mechanics that has never been explored, despite the long history of real-world sports revolving around similar stakes.
The good news is that we are halfway to making a true NerveGear The bad news is that so far, I have only figured out the half that kills you. The perfect-VR half of the equation is still many years out.
In SAO, the NerveGear contained a microwave emitter that could be overdriven to lethal levels, something the creator of SAO and the NerveGear itself (Akihiko Kayaba) was able to hide from his employees, regulators, and contract manufacturing partners. I am a pretty smart guy, but I couldn’t come up with any way to make anything like this work, not without attaching the headset to gigantic pieces of equipment. In lieu of this, I used three of the explosive charge modules I usually use for a different project, tying them to a narrow-band photosensor that can detect when the screen flashes red at a specific frequency, making game-over integration on the part of the developer very easy. When an appropriate game-over screen is displayed, the charges fire, instantly destroying the brain of the user.
This isn’t a perfect system, of course. I have plans for an anti-tamper mechanism that, like the NerveGear, will make it impossible to remove or destroy the headset. Even so, there are a huge variety of failures that could occur and kill the user at the wrong time. This is why I have not worked up the balls to actually use it myself, and also why I am convinced that, like in SAO, the final triggering should really be tied to a high-intelligence agent that can readily determine if conditions for termination are actually correct.
At this point, it is just a piece of office art, a thought-provoking reminder of unexplored avenues in game design. It is also, as far as I know, the first non-fiction example of a VR device that can actually kill the user. It won’t be the last.
Today marks ten years since the start of Oculus. I decided to drop out of school when I was 19 to make a go at turning my extremely niche hobby of building high-performance head-mounted-displays into a business that could get everyone interested in VR and AR. It took a couple hours to come up with the name, register the website, and spread the news on all the internet forums I frequented. It would take almost two months to get back final paperwork for Oculus LLC from the California Franchise Tax Board, of course!
Just a few days short of five years later, I was fired from my own company.
I learned a lot of lessons and made a lot of money along the way, and my new company Anduril wouldn’t be where it is if it wasn’t for Oculus. Over the next year, I am going to try to distill some of those lessons into vignettes others might find useful, posting more or less in time with the decennial of various Oculus milestones (crowdfunding, fundraising, shipping, acquisition, termination, etc). I have quite a bit to share, especially regarding portions of Oculus history that have been subject to extensive revisionism in public accounts. This year is also the right time to finally unveil some VR technologies I haven’t been able to talk about for a variety of reasons. For now, though, I just want to share the first official communication from Oculus VR, the “About” page text from oculusvr.com posted on April 15th, 2012:
Let me get this out of the way: I am a huge HMD nut. As far as I know, I have the largest private collection of unique HMDs in the world, totaling 43 units, not including the ones I have built myself. Part of this HMD obsession, as with any other obsession, is trying to convince other people to join in. And what a seductive dream, that technology can transport us into worlds we cannot hope to experience in real life, or augment our reality to shape it closer to our desires!
Unfortunately, virtual reality has risen and fallen many times, with a lot more emphasis on the latter portion. The tech has never gotten far enough to be truly convincing, and great VR hardware has been far out of reach for the average person… Until now.
Oculus is my tilt at trying to change that. The tech has improved, and we can build hardware and software that is better, stronger, and faster than the old guard, companies that create niche, wildly expensive products. Don’t get me wrong, these companies are important, and they have to solve some very tough engineering challenges to satisfy their customers. But the reality is that as gamers and dreamers, we have a different set of challenges to meet. Massive field of view to engulf your visual senses, low latency tracking to maximize presence, light weight and comfortable for long term use, and perhaps most importantly, prices measured in the hundreds of dollars, not tens of thousands. I have worked long and hard with a lot of brilliant people to try and meet those challenges, and now it is time to put it in your hands.
Why the name “Oculus? Because it is the Latin word for “eye”, and someone used the word in a meeting several months ago. I thought it was a nifty word, and was better than the alternative, “StepN2theGAME”.
This guy has eyeballs that allow him to love his Rift S. His friends wish they had that luxury.
Rift S is very cool! It takes concepts that have been around for years and puts them into a fully functional product for the first time. Sure, sure, I see people complaining about how Rift S is worse than CV1 concerning audio quality, display characteristics, and ergonomics – some of the tradeoffs are real, some are imaginary, and people should really wait for it to come out before passing final judgement. All in all, it is going to be a great HMD.
For about 70% of the population.
My IPD (interpupillary distance, the distance between my eyes) is a hair under 70mm and slightly skewed to the right side of my face. One of my best friends has an IPD of 59mm. I don’t know what your IPD is, but both of us were perfectly served by the IPD adjustment mechanism on Rift CV1, a mechanism that was an important part of our goal to be compatible with male and female users from 5th to 95th percentile. Anyone within the supported range (about 58mm to 72mm) got a perfect optical experience – field curvature on the focal plane was matched, geometric distortion was properly corrected, world scale was at the right size, and pupil swim was more or less even. Sharp imagery from edge to edge of your field of view was the norm. The small handful of people with an IPD outside that range would not get a perfect experience, but at least they would be in the right ballpark. IPD skews in different directions by gender, race, and age, but we managed to cover almost everyone, and we were proud of that.
This is not the case with Rift S. Like Oculus Go, it uses two lenses that are set about 64mm apart, perfect for a perfectly average person. Everyone who fits Cinderella’s shoe will get a perfect experience, anyone close will deal with minor eyestrain problems that impact their perception of VR at a mostly subconscious level. Everyone else is screwed, including me. Imagery is hard to fuse, details are blurry, distortion is wrong, mismatched pupil swim screws up VOR, and everything is at the wrong scale. “Software IPD adjustment” can solve that last bit, but not much else – it adjusts a single variable that happens to be related to IPD, but is not comparable in any way to an actual IPD adjustment mechanism . This is the main reason I cannot use my Oculus Go, even after heavy modification on other fronts.
Does that mean Rift S should have had mechanical IPD adjustment like CV1? No. That is the easiest way to look at things, but it misses the point by focusing on a particular solution to a broad problem. HMD design is all about tradeoffs, and finding the right balance is very difficult. More important than IPD adjustment is IPD Tolerance, an all-encompassing set of factors that determines your ability to use a particular VR headset both at rest and during active motion that slips the headset around on your head a little bit. There are a lot of ways to approach IPD Tolerance, here are the four most common:
1. Mechanical IPD Adjustment An oldie, but a goodie. This is the way most VR headsets have tackled IPD tolerance over the decades, driven largely by microdisplay-based architectures with narrow eyeboxes that required perfect positioning to work at all. It allows the HMD designer to focus on other qualities like FOV and field curvature. Tying physical adjustment to the distance between virtual cameras is a great way to ensure the scale of the virtual world is correct, and a single HMD can be adjusted to fit a wide variety of users. It does add a small amount of weight and a moderate amount of fragility. This was the approach used in Rift CV1, though we tied the adjustments together in a way that slightly impacts people like myself with asymmetrical faces. Good enough!
2. Custom Sizing This is my personal favorite, and will dominate the VR industry in the long run. Every adjustment on an HMD adds weight, bulk, complexity, cost, and fragility. If the headset is fitted to match the end user from the start, you can minimize and sometimes eliminate the need for adjustments! This is a common principle behind almost all wearable goods, from shoes to clothes to watches to sunglasses – it is the reason your wardrobe is not dominated by one-size-fits-all unisex jumpsuits.
Given the small size of the VR market, it is hard to justify fully custom sizing for every single HMD sold. Luckily, you can get huge gains out of even a handful of options! I am not a huge fan of Magic Leap, but one thing they did right was achieving IPD Tolerance by making two different versions of the ML1, one for wide IPD and one for narrow IPD. Focals by North is taking a similar approach with their smart glasses, but have dozens of different sizing options.
I won’t mince words. Rift S should have done this. The logistical overhead of managing a handful of different SKUs with slightly different plastic pieces holding the lenses at slightly different distances would have allowed Rift S to keep costs low and expand the addressable market for VR without cutting out new and old customers alike.
3. Perfect Collimation Nothing is perfect, but . It is possible to design optical systems with a very wide sweet spot/eyebox and little need for physical adjustment, but not without tradeoffs. Playstation VR is a good modern example, using solid and heavy lenses with a fairly long focal length to create an wide enough eyebox for most users. This comes with a penalty to weight, cost, size and panel utilization (and thus resolution), but it is hard to argue with the results, especially for a headset that regularly sells for $199. The Rift DK2 also had a large-ish eyebox that allowed it to be a useful development tool, but there is a reason some users created aftermarket modifications for lens spacing – we knew it would not be responsible to launch DK2 as a consumer product, and it took nearly two years from the launch of DK2 to make something we felt comfortable pushing as the One For All solution.
4. Everything is Fine One of the most common approaches to IPD Tolerance over the decades, and one of the many reasons VR has always failed. You can’t solve engineering decisions with marketing, but that won’t stop PR professionals from trying. Doing nothing and claiming Everything is Fine does nothing to keep customers from churning out and failing to engage as a result of blurry images, headaches, and eyestrain. Some people will only see these effects once they start actively using their headset – more people will be within an acceptable range when standing still than during active play, because the same factors that allow for a wide range of IPDs also allow for more active playstyles.
The most obvious sign of this strategy is when any company refuses to admit what their supported IPD range is, especially companies that have proudly marketed it in the past – providing a simple number that would allow prospective customers to make smart decisions would also give customers and press the ability to objectively calculate the number of people being cut off (or, alternatively, just how far the acceptable quality bar has dropped). Much easier to treat it as some kind of insignificant, unfortunate, unavoidable minuscule problem with no demographic-based distinctions whatsoever.
Why are you so bitter about Rift S in particular? What about the other bad HMDs? Thanks for asking. Yes, there are lots of other headsets that don’t tolerate a wide range of different IPDs. Many early Windows MR headsets, in particular, were prominent members of the Everything is Fine club. There are a lot of headsets in the SteamVR ecosystem that totally blow, as well. I have been quiet about this for two reasons: First, I don’t really use those platforms. Yes, I own almost every HMD ever made, and I keep up with developments in the VR marketplace, but I am completely tied to the Oculus platform. I buy my games from the Oculus Store, I launch my games through the Oculus UI, and I develop VR applications for the military using the Oculus SDK. I have a lot more interest in Oculus because I want to continue doing all of these things.
Secondly and more importantly, Rift S is the only way to use the Oculus PC ecosystem. It is the singular option, a full replacement for the now-discontinued CV1. Anyone who can’t use Rift S is going to be effectively locked out of the ecosystem, including people who have invested thousands of dollars into their content library. For Windows MR and Steam VR, on the other hand, bare-bones headsets are just part of a broader ecosystem that also includes headsets that do support a wide range of IPDs. I have no problem with the existence of low-cost headsets that only work for 70% or 80% of the population as long as they are flanked by options for the rest of us. Oculus Quest is great, but it is not a PC headset – it is an option for people who want mobile VR and can’t use Go, and that is all.
I spent much of my later tenure at Oculus working on supporting headsets from other vendors, in part to avoid this type of situation. As things stand, I find myself shunned by an ecosystem I spent most of my adult life helping to create. To be fair, Oculus did not show up to the GDC announcement of Rift S without a plan to address the concerns of users who find themselves with no future in the Oculus ecosystem. If anyone falls outside of the carefully unspecified IPD range of Rift S, they might want to stock up on the original Oculus Rift while they still can!
Oh
I love most of you Oculus guys. Keep trying to cut a few more tools for Rift S IPD+ and Rift S IPD-. Fingers crossed for a future version spaced a hair under 70mm, skewed to the right.
The RR1 repair kit took less than three hours to design. Testing it on all the broken Rifts sent to me by the community took a bit longer.
TL;DR: The RR1 repair kit can repair the busted audio on your Oculus Rift. I will send one to you for free. Vive la VR revolution!
The Oculus Rift CV1 is not perfect. Some issues are the result of carefully considered design tradeoffs, but others are design flaws that did not become apparent as such until well after launch. Eventual failure of the complex electromechanical assembly that gets audio from the Rift to your ears is one of the latter.
This issue and the root causes are well documented across the internet, so I will limit this post to information that is actually helpful. In short, long-term use of the Oculus Rift CV1 often leads to failure of important electrical paths in a specific ribbon cable that winds through the strap. For most, the failure is limited to the ground (GND) trace for the right headphone audio module. This sometimes develops into a full audio cutout for both ears, though that is less common. The RR1 repair kit can address both types of failures, and has been designed so the average PC gamer will have no problems with installation.
As of this post, Oculus seems generally willing to replace headsets with this failure as long as they are under warranty and in a supported country. Unfortunately, the longtime Rift users who are most likely to suffer from this problem are usually well outside the warranty period. Even so, the first step for any user with this problem should be to contact Oculus Customer Support and see what they can do. In some cases, you might just need to clean some contacts, tighten the connection between your headphones and straps, or fix your PC’s sound settings. If you are unsuccessful in resolving your problem, please forward your ticket and a mailing address to RiftRepairOne@gmail.com, and I will send you an RR1 repair kit free of charge so you can get back in the Metaverse as quickly as possible. Oculus Customer Support, feel free to direct customers who are out of warranty to my kit.
The RR1 comes with more detailed instructions, but here is the gist: The kit works as an external wiring harness for the Oculus Rift CV1 that can be configured in a variety of ways to address a variety of different failures while running in parallel to the existing electrical system. Most people only need a common ground between their right and left headphone audio modules – just slap the wiring harness on, slip the bypass discs between the pogo pins on the headphones and the strap receptacle, you will be back in business! Users with full audio failure will also benefit from the ability to drive entirely from external audio sources, both wired or wireless. The kit weighs about 32 grams, but because the weight is rear of the facial interface and close to the head, you won’t be able to notice the difference. Please note that the RR1 cannot fix failures in the ribbon cable that affect the rear Constellation tracking LEDs.
The kit can also function as a third-party headphone adapter on fully-functional Rifts, but I don’t have any plans to distribute the kit for that purpose – I am doing this because I feel bad for people who bought a Rift from me and can’t use it properly anymore, not to make using third party headphones slightly more convenient! Maximizing the number of people in the VR ecosystem is also important to me, and the people who have been using their headsets for years on end tend to be among the most engaged, most valuable users who dump tons of money into the content ecosystem. Properly functioning, high quality audio that does not rely on fumbling with third party headphones and cables is an important part of getting users to regularly engage, and the market research bears it out – there is a reason everyone is building audio into their headsets, even companies that initially doubted the importance.
Big thanks to everyone who responded to my call on Reddit and Twitter for broken Rifts, this would not have been possible without your help. (Before anyone gives me shit for ripping people off, please know that I paid everyone enough to buy a new Rift+Touch for themselves in their native country)
Oculus Go, like all products, is the result of a series of compromises around cost, schedule, quality, and competence. It is clearly the best standalone VR platform on the market, but anyone with a bit of technical skill can make it even better.
I have been modding my Go over the past few months, and the resulting device is a huge step up in terms of overall experience. Once you GoBlack, you never go back! Here is a list of upgrades, along with a bit of explanation on the thinking behind each of them.
Color
The shell of a stock Oculus Go is made of ABS plastic, and the facial interface and head straps are a blend of various synthetics. The coloring is a nice mainstream-friendly gray that does a good job of hiding minor defects in the injection molding process. I am a strong believer in black VR headsets, so going dark was at the top of the upgrade list. This is especially important for all the surfaces on the interior of the facial interface, given the critical role they play in controlling contrast and other aberrations. Any light from the displays, nose opening, or other gaps is reflected off those surfaces and back into the lenses/optical cavity – if you don’t like god rays, learn to embrace matte black surface treatments!
Painting the outer shelI would have increased weight and still not solved the facial interface problem, so I decided to dye everything with polyester dye. WARNING: Be careful if you try this yourself. Dying plastics requires high temperature near-boiling liquid solutions, and if you go even a few degrees too hot, you will warp your plastics almost instantly. If you go too cool, the dye won’t permeate properly. Also, you can burn yourself.
Don’t drink this! Make sure you thoroughly rinse all soft goods with cold water, you don’t want concentrated polyester dye hanging out on your skin!Left is an unmodified Oculus Go facial interface, right is the dyed GoBlack facial interface. Guess which one reflects less light back into your optics? I also replaced the retainer clips around the lenses with something a little lower profile. When designing HMDs, always minimize even mildly reflective surfaces. That is one of the reasons the Rift has matte black fabric wrapped all the way to the lenses and minimal exposed plastic.
Weight
Oculus Go is pretty heavy. The overall weight pencils out well on paper against the Rift, but most of that is due to the much lighter fabric straps, which don’t really contribute much to perceived weight or skin shear during head movement. The actual face-borne unit weighs just over 400 grams.
With a few mods, I was able to get that down to just 280 grams, and most of the gains are out at the front edge of the headset where it matters most. Totally subjectively, it feels like a 50% weight reduction, especially during head movement. Here is how I got there:
Battery Swap – I removed the battery from the front of the headset and relocated it to the top of the head, where it attaches with velcro to the over-head strap. It was originally a pretty low-end 2600mAh 18650 cell that provides about two hours of battery life, but I decided to spend a few dollars on a nice 3500mAh that pushes over three hours of juice. It connects through a new magnetic pogo-pin connector I installed, which allows me to easily swap to a new battery pack whenever I want to keep playing. Much nicer than charging while playing, which can overheat the headset. It would be easy to create larger battery modules that allow for all-day use, but I don’t think I am likely to bother.
Thermal Management/Heatsink: Oculus Go was designed with a passive cooling system that uses a heat pipe to spread heat from the Qualcomm Snapdragon 821 SOC across the aluminum faceplate. Unfortunately, the system is both heavy and located way out at the front of the headset, maximizing impact on comfort. I am brutal on headset weight – HMDs, like rockets, get huge gains out of seemingly small decreases in weight.
To solve this, I installed an entirely new cooling system. It uses a much shorter and lighter heat pipe to move heat from the SOC into a nearby array of small aluminum fins that are cooled by the flow of a small micro-blower. Power for the system is pulled off the mainboard, and the aluminum faceplate was replaced with lightweight tinted/polarized plastic. I think it looks pretty sweet.
FCC Stuff: I pulled the lids and shielding off everything. This only saved a couple grams, but I liked the aesthetics of being able to see all the support circuitry for each chip.
Plastic Reduction: I could have been more extreme on this, but I wanted to maintain durability, rigidity, and a somewhat stock appearance. I was able to trim a lot of the plastic that was used to mount and support the thermal management system and front faceplate, but my friend GOROman really took this to the extreme in his modded Go!
Misc Other Stuff
IPD Adjustment: I gave up on this for a variety of reasons. Oculus Go is a fixed-IPD headset, so the higher resolution is not useful for people with particularly wide or narrow IPD. I have a pretty wide interpupillary distance, around 70mm, and Oculus Go is somewhat unusable for me – mismatched geometric distortion, scale, and field curvature causes eyestrain and blurriness. I suspect most of the people using Go on a regular basis have a more average IPD, which is unfortunate – we designed the Rift to accommodate male and female users from 5th to 95th percentile, and we were really proud of that. As displays become higher resolution, proper optical adjustment (or custom sizing) will become even more important. The VR revolution should be accessible for everyone, no matter the distance between their eyes. I will be writing more on this in the future.
Keyboard for Mobile Computing: One of the most exciting things about Oculus Go, in my opinion, is Virtual Desktop support! It allows me to have a full Windows setup almost anywhere I go, complete with giant monitor. I built myself a custom PC inside one of my favorite mechanical keyboards, a dampened Happy Hacking Keyboard Professional Type 2 S with blank PFU keycaps (no need for labels you can’t see in VR anyway!) I might do a writeup on this at some point, having a full size keyboard+Windows+GoBlack is so much better than any laptop could ever be!
6DOF, Oculus Quest, NOLO, etc: I am not using GoBlack very much at the moment, mostly because of the IPD issues mentioned earlier. I am very glad Oculus Quest has IPD adjustment, and really look forward to it! For anyone who cares, wait just a little bit longer for the vestibular fix I have been talking about.
I am pretty impressed with the latest version of NOLO VR, a third party tracking system for Oculus Go that has gotten a lot better over the last few months via firmware updates. If you can’t get your hands on a Quest Dev kit, this is a reasonable facsimile that allows you to keep building on top of the Oculus SDK and engine integrations.
What if I told you of a new entrant to the VR market that allows you to truly experience anything, anywhere? What if I told you they provide a full-dive virtual reality experience that is absolutely indistinguishable from the most thrilling lucid dream or the most mundane real-world experience, all in a lightweight visor that costs only $99 USD?
Quite a few people would jump to purchase as soon as they heard, and the rest of humanity would follow as soon as they have reason to believe – as The Matrix told us in 1999, no one can be told what the matrix is. You have to see it for yourself.
More importantly, though, they would use it. Oh, would they use it! Network effects kick in, users buy and subscribe to deep libraries of content to keep themselves occupied, content developers make piles of money, and they use that money to develop better and broader content ad infinitum. In the end, hardware sales are a meaningless metric for the success of VR. They matter only as a means to an end, a foundation to enable the one thing that truly matters: Engagement. Engagement is all that matters. Engagement is Everything!
This is just as true in the present day. Hardware sales get a lot of attention and speculation from analysts and consumers alike, but the real name of the game revolves around the number of people logging in and spending money each week, the life force that makes everything actually go. Recent market experiments with cheap VR hardware have shown that there are millions of people willing to buy said hardware, but very few among them continue to use the hardware or invest in the software ecosystem for very long. This is true even when people get the hardware for free – the millions of cardboard boxes fulfilling their ultimate destiny on the back shelf of a closet don’t do much for the VR industry. Why the lack of use? Quality of experience. If the free hardware was as good as the visor described in the first paragraph and paired with good content, a mass-market VR revolution would occur practically overnight.
And what if that visor cost $999 instead of $99? Price is certainly a relevant factor in the rate of VR adoption, but not a dominant one – as someone who has had to eat my hat multiple times in pursuit of keeping costs low, I feel like I intimately understand what it must have felt like to deal with the response to the E3 2006 Playstation 3 price announcement. Five hundred and ninety nine US dollars?! The hypothetical visor provides quite a bit more for your money, though – it may not sell billions of units, but it would certainly sell by the hundreds of millions. Lower pricing for existing VR technology can help expand the size of the active and engaged userbase, but not to nearly the degree many people would expect. I want to take this a step further and make a bold claim: No existing or imminent VR hardware is good enough to go truly mainstream, even at a price of $0.00. You could give a Rift+PC to every single person in the developed world for free, and the vast majority would cease to use it in a matter of weeks or months. I know this from seeing the results of large scale real-world market testing, not just my own imagination – hardcore gamers and technology enthusiasts are entranced by the VR of today, as am I, but stickiness drops off steeply outside of that core demographic. Free is still not cheap enough for most people, because cost is not what holds them back actively or passively. I hope and pray I am wrong, but most people are not like you or me. If I had to make a concrete bet, I would put a hypothetical ultimate ceiling for VR in the next two years at perhaps 50 million active users, and that could only happen with an unreasonable amount of investment that would be better spent on other parts of the problem. That is okay! That is fine! That is great, even! That is more than enough for a healthy VR ecosystem, especially given the high spending potential for engaged VR users, but well short of the ultimate potential.
Virtual Reality is reasonable candidate for most important technology of the century. It is hard to prove with current technology and adoption, of course, but the argument is trivially easy if you assume even moderate technological advancement compounded over decades. The real debate should be over the timeline for adoption and how many boom/bust cycles VR will see between inception in the 1960s and eventual, inevitable dominance as the final platform (yes, AR is also cool).
We can minimize and perhaps eliminate those cycles with better hardware, broader content, and a deep understanding of how to best interface with a human perceptual system that varies significantly across age, gender, and race. Every dollar that goes into making those things better now will pay huge dividends down the road, especially when compared with forced marketing to segments of the world that are not yet ready to embrace VR. This is an opinion shared by most True Believers I know, especially the ones who see themselves personally working on VR across the rest of their lives – they know we have come a long way, and know we have a long way to go. The number of True Believers continues to grow in the wake of the 2012 re-vitalization, despite dilution by people who see VR as a fashionable stepping-stone to their next adventure. True Believers are the reason we have the VR of today, and they are the best chance we have for the VR of tomorrow.
The title of this review was carefully chosen, not glibly. I want what is best for VR and all other technologies on the Reality–Virtuality Continuum, Magic Leap included. Unfortunately, their current offering is a tragedy in the classical sense, even more so when you consider how their massive funding and carefully crafted hype sucked all the air out of the room in the AR space. It is less of a functional developer kit and more of a flashy hype vehicle that almost nobody can actually use in a meaningful way, and many of their design decisions seem to be driven by that reality. It does not deliver on almost any of the promises that allowed them to monopolize funding in the AR investment community.
There are lots of general overviews of ML1 at this point, so I will be focusing on a few specific points that have not been as widely covered. If you want to check out a broad overview, this video from Tested is a great place to start. If you want to see the guts and how they work, I got to help iFixit to tear down my ML1.
The Controller
Tracking is bad. There is no other way to put it. The controller is slow to respond, drifts all over the place, and becomes essentially unusable near large steel objects – fine if you want to use it in a house made of sticks, bad if you want to work in any kind of industrial environment. Magnetic tracking is hard to pull off in the best of cases, but this is probably the worst implementation I have seen released to the public. For VR enthusiasts who are familiar with systems from Polhemus (the OG magnetic tracker people), the Razer Hydra, or the ever-elusive Sixense STEM, you know where the bar is. From the Magic Leap developer manual: “6DoF tracking is stable in slow to moderate motion. It also recovers and relocates quickly in case of fast or sudden motion (e.g. boxing or fishing – like movements).”
I understand that Magic Leap wanted a controller that did not require line-of-sight to the headset or a bulbous protrusion to report position, but that was a terrible tradeoff to make, especially for developers who need a controller that just works – there are good reasons no other company decided to walk this path. Parlor tricks like holding the controller behind your back are fun, but ML1 could and should have used basically any other type of tracking system. Several other companies have managed to pull off inside-out optical tracking without Magic Leap’s billions of dollars in funding, and if they can’t pull that off, they could definitely use a system with an external reference to get things going. As things stand, a lot of their software and UI limitations seem to be driven by the bad controller.
In another bizarre departure from competing devices, the trackpad is not clickable. The Steam Controller, HTC Vive wands, Oculus Go, Lenovo Mirage Solo, etc all have a clickable trackpad, and designers have heavily relied on that feature. Even the Playstation 4 controller has one! What this means in practice is that selection via the touchpad requires either lifting and tapping (terrible for precision) or clicking the trigger while holding (also terrible for precision). It also means that they cannot use the trackpad to emulate buttons or other selection schema. Everyone else in the industry is using components from ALPS (great company BTW), they should have just called them up and told them they needed a custom trackpad with fun RGB LEDs.
Final note on controllers: Unlike most magnetic tracking systems, the transmitter side is in the controller. This means there is a giant ferrous core wound with copper wire hanging out just above the trigger. To balance things out, Magic Leap had to install metal counterbalance weights in the bottom of the controller. That does make the controller feel “premium” at first heft, but really sucks for long term ergonomics.
The Computer
They call it the “Lightpack”. It is basically the guts of a tablet computer in an oversized hockey puck that you wear on your belt. This is the best part of the device by far, A+! I would have expected Magic Leap to do the fashionable thing and throw all their render hardware and battery power on the headset itself for looks, but some group of sane people appear to have recognized that putting your heaviest components on the most weight sensitive part of your body is a bad idea if you want people to actually wear your product for any period of time – this is a longer topic for another day, but the data shows that you need to be BRUTAL when it comes to reducing HMD weight. This approach also allows them to use much more powerful chips than they could feasibly cram into a head worn device.
The cables are robust, and the weight tugging at the back of your head actually helps counterbalance things a little. They should have made the battery replaceable, but nobody is going to use their ML1 long enough for that to matter to anyone but collectors with an aim to preserve the history of AR and VR.
The Headset
They call it the “Lightwear”. This is the part that has gotten the most hype over the years, with endless talk of “Photonic Lightfield Chips”, “Fiber Scanning Laser Displays”, “projecting a digital light field into the user’s eye”, and the holy-grail promise of solving vergence-accommodation conflict, an issue that has plagued HMDs for decades – in other words, ensuring that the focus of your eyes always matches their convergence, something that Magic Leap has touted as critical in avoiding “permanent neurologic deficits” and brain damage. It is even more important for AR than VR, since you have to blend digital elements with real-world elements that are consistently correct.
TL;DR: The supposed “Photonic Lightfield Chips” are just waveguides paired with reflective sequential-color LCOS displays and LED illumination, the same technology everyone else has been using for years, including Microsoft in their last-gen HoloLens. The ML1 is a not a “lightfield projector” or display by any broadly accepted definition, and as a Bi-Focal Display, only solves vergence-accommodation conflict in contrived demos that put all UI and environmental elements at one of two focus planes. Mismatch occurs at all other depths. In much the same way, a broken clock displays the correct time twice a day.
In more detail: The ML1 uses six waveguides all stacked atop one another, three for each RGB color channel at two different focal planes. You can think of it as a Bi-Focal Display – a display system that can move the focus of the display between two different values based on eye-tracking with no variability in between, unlike continuously vari-focal displays such as Oculus Half-Dome or Nvidia’s true lightfield displays. I don’t have exact measurements yet, but it seems like the near-plane is focused at about 0.75 meters, and the far-plane is focused at about 5 meters. If they stick with this technology (and I have not seen any indication that they can do anything else, especially not their much-hyped fiber display), each additional focus plane will require even more stacks of waveguides and infeasibly high framerates (each plane consumes a minimum of 60hz from the temporal budget). I don’t think that will happen within reasonable weight, image quality, and cost constraints.
More than one plane is good, don’t get me wrong! It allows developers to avoid extreme mismatch with very close or very far objects. That said, spinning hype and monopolizing investment with promises that cannot be met is bad for the entire XR industry, not just Magic Leap. Hardware manufacturers have a responsibility to clearly communicate the capabilities of their hardware to developers, even when the capabilities fall short of what they would prefer.
On to the other parts of the headset: The tracking is good compared to most other players in the AR/VR industry, but worse than most of the big guys, including Hololens. Expect jitter in ideal environments. If you want a comparison, think halfway between PSVR and Rift. The meshing system is good, but not nearly as fast as Hololens. It is pretty similar to what you see from companies with a few orders of magnitude less funding, like Stereolabs.
Bi-focal abilities aside, the image quality is acceptable. Have you seen Hololens? Think that, but with slightly larger FOV. The rainbow artifacts are a bit worse owing to the large number of stacked waveguides and the black levels are a bit better, but Magic Leap is playing in the ballpark as everyone else. Despite drawing enough power to keep the headset nice and toasty (seriously, it is hard to touch the magnesium shell if you are in a warm room), the display is far too dim to use outdoors. That is a shame, since the transparency is about the same as a pair of dark sunglasses – not exactly indoor material. How does the eye-tracking work? Impossible to say, because nothing uses it. That is not a great indicator.
A true leap would have been a FOV that is wide enough to be useful, something that Magic Leap could have done if they prioritized user experience over meeting device size expectations. For a good example, check out Dreamworld at 90 degrees – the tracking is not at all comparable, but the experience is pretty exciting.
The Operating System
Magic Leap says they have “built a whole new operating system” called LuminOS to take advantage of their “spatial computing system“. It is actually just Android with custom stuff on top, the same approach most people take when they want to claim they have built a whole operating system.
I will keep this part short. I hope Magic Leap does cool stuff in the future, but the current UI is basically an Android Wear watch menu that floats in front of you. The menus are made of flat panels that can only be interacted with through the previously discussed non-clickable trackpack. Eye tracking and rotation/position of the controller are ignored, as is headlook. You can toss Windows 8 style application windows all over the place, floating in space or even attached to walls! That is nifty, mostly useless, and also exactly what Microsoft started showing off about three years ago. It is some of the worst parts of phone UI slammed into some of the most gimmicky parts of VR UI, and I hope developers create better stuff in the near future.
Sales Figures and Developer Adoption
The Magic Leap order system was really easy to figure out for the first few days after launch. I gathered some order numbers from friends and compared their order times, and I am pretty confident about predicting first-week sales. Unfortunately, they changed the system shortly after I tweeted about it. Based on what I do know, it looks like they sold about 2,000 units in the first week, with a very heavy bias towards the first 48 hours. If I had to guess, I would put total sales at well under 3,000 units at this point. This is unfortunate for obvious reasons – I know over a hundred people with an ML1, and almost none of them are AR developers. Most are tech executives, “influencers”, or early adopters who work in the industry but have no plans to actually build AR apps. This was a big problem in the early VR industry, and that was with many tens of thousands of developers among hundreds of thousands of development kits sold! Multiplying the problem by a couple orders of magnitude is going to be rough for ML.
Conclusion
Magic Leap needed to really blow people away to justify the last few years. The product they put out is reasonably solid, but is nowhere close to what they had hyped up, and has several flaws that prevent it from becoming a broadly useful tool for development of AR applications. That is not good for the XR industry. It is slightly better than Hololens in some ways, slightly worse in others, and generally a small step past what was state of the art three years ago – this is more Hololens 1.1 than Consumer AR 1.0. Consumer AR can’t happen without advancement, and it seems those advancements will be coming from other companies. There is, of course, a chance that Magic Leap is sandbagging us; maybe the real deal is just behind the next curtain! Past experience suggests otherwise…
Above is a telling picture from a piece Magic Leap did with Wired magazine a couple years ago, back when they were still hyping up scanning fiber displays. See the fancy-looking, high-tech light up strands? They don’t do anything. It is just electro-luminescent wire. It looks great to casual observers, but does not hold up to any kind of scrutiny from people who are in the know. If you want to try using it to dress up your own costume, gaming PC, or multi-billion dollar hype machine, you can buy a nice assortment for $20 here. ———————————————————————————————————–
Update: About 45 minutes after I posted this, the CEO of Magic Leap decided to share his love of the television series Avatar: The Last Airbender. He also points out that the photographic stereoscope was invented in 1838, very interesting!
Silicon Valley needs to get smart on national security. In many cases, it will mean picking sides. Which side will you choose?
You can read more here: https://www.washingtonpost.com/opinions/silicon-valley-should-stop-ostracizing-the-military/2018/08/08/7a7e0658-974f-11e8-80e1-00e80e1fdf43_story.html
Below are some pictures of what makes up an Oculus Go. There is only so much I can say, but I plan on replacing some of the components with different components – stay tuned,
