⌨️Why Typing in XR Still Sucks?
Olivier Plante is the CEO of Fleksy, a white-label keyboard SDK that powers the fastest typing experiences in the world. With 12+ years of experience, he has worked on making text input seamless across mobile and XR.
In this conversation, we dive deep into:
- Why typing in XR is still so frustrating
- Why voice won’t replace typing anytime soon
- How AI is transforming the way we type
Before getting into the interview, I wanted to quickly introduce you to today’s sponsor Gracia AI. Gracia AI is the only app that allows you to experience Gaussian Splatting volumetric videos on a standalone headset, either in VR or MR.
It is a truly impressive experience and I recommend you to try it out right now on your Quest or PC-powered headset.
Interview with Olivier Plante
Why does typing in XR still suck?
Olivier Plante: It’s a great question, and it comes down to three big challenges. First, no tactile feedback. With physical keyboards, you feel the keys. Even on touchscreens, there’s still a surface to interact with. But in XR, you’re typing in the air—so the system has to guess if you've actually pressed a key or not. Second, input precision is much harder. On mobile, you touch a screen, and we track every millisecond of your movement. In XR, the system must interpret where in 3D space your fingers are, which leads to more errors. Third, no universal standard. Apple’s Vision Pro keyboard is different from Meta’s approach, and none of them feel truly optimized for XR yet.
Besides the lack of tactile feedback, what else makes XR keyboards frustrating?
Olivier Plante: The biggest issue is the engine behind the keyboard. It has to understand: “Did the user mean to type Q, W, or E?” In XR, you don’t have a fixed point of contact like on a touchscreen. So the software has to be extremely forgiving while also ensuring the user doesn’t get completely wrong words. Companies like Meta have put a ton of research into this, but it’s still a tough challenge. Our expertise at Fleksy is handling sloppy typing and our engine can correct input even when the user is being imprecise. That’s what’s needed for XR typing to actually work.
How do you measure how good a virtual keyboard is?
Olivier Plante: There are several benchmarks we use. One is F-score, which evaluates autocorrection accuracy, next-word prediction, and swipe input precision. Then, there’s typing speed, which we measure in words per minute (WPM). A good touchscreen keyboard today reaches 80-100 WPM. But in XR? Nowhere near that yet! The quality of the language model is also crucial. How well does it handle different languages, slang, and technical terms? All these factors define whether a keyboard actually helps or frustrates users.
So what makes Fleksy a perfect fit for XR?
Olivier Plante: Unlike Apple or Google’s keyboards, Fleksy doesn’t need retraining when the keyboard position changes. In XR, you might want the keyboard on your wrist, floating in space, or projected onto a desk. With traditional models, every change in position means retraining the AI, which is slow and expensive. Fleksy works dynamically: you can resize it from a tiny smartwatch-sized keyboard to a full desktop layout, and it still works just as accurately.
Can you share a real-world example of Fleksy in action?
Olivier Plante: Sure! One of my favorite examples is healthcare. Some of our clients use Fleksy’s typing data to detect early signs of Parkinson’s, Alzheimer’s, and dementia by analyzing how users type over time. Another example is AI-powered keyboards that let users rewrite messages with one tap using ChatGPT-like suggestions inside WhatsApp.
How is Fleksy working with developers to build better typing experiences?
Olivier Plante: We provide a keyboard SDK that lets developers build a custom, high-performance keyboard without reinventing the wheel. Whether it’s for XR, AI-powered typing, or secure banking applications, our SDK helps developers build faster, more accurate keyboards in less time. We already have 60+ clients across industries like healthcare, AI, and automotive.
But won’t users make more mistakes on a smaller keyboard?
Olivier Plante: Surprisingly, no. Because our AI compensates for sloppy typing, we can make the keyboard much smaller without sacrificing accuracy. Imagine you’re on a bus, and you invoke a tiny wrist-mounted keyboard to reply to a message with swipe input. Or you're in VR and want a floating keyboard on the side of your field of view. With Fleksy, that’s all possible because our technology adapts to different sizes and positions automatically.
You mentioned swipe input, how do you see that evolving in XR?
Olivier Plante: Swipe input is going to be the dominant method for XR typing. If you’re using hand tracking, swiping across a virtual keyboard is way easier than trying to tap individual keys in midair. But swiping doesn’t have to mean dragging your finger across a keyboard. With neural interfaces like Meta’s wristband, you could swipe using micro-gestures, just subtle finger movements. That’s the direction I see XR input evolving.
Why don’t we have these advanced XR keyboards yet?
Olivier Plante: Honestly? Big companies are too focused on reinventing everything from scratch instead of looking at what already works. There’s a hunger for "radical innovation", but sometimes, the best solution is a smarter evolution of what we already have. Right now, XR keyboards are either floating QWERTY layouts or experimental, hard-to-use systems. The best path forward is keeping familiar layouts while introducing AI-powered improvements—like dynamic resizing, better word prediction, and adaptive input methods.
So why not just use voice input instead?
Olivier Plante: We hear this all the time. “Why do we even need keyboards when we have voice recognition?” The answer is simple: voice is public. If I’m in a quiet office, a bus, or a meeting, I don’t want to speak my messages out loud. Plus, voice input is imprecise: it struggles with technical terms, names, and accents. Voice and keyboards will coexist, but typing isn’t going away.
How does AI impact keyboard technology?
Olivier Plante: AI is transforming keyboards in two big ways. First, context-aware suggestions. Instead of just predicting the next word, future keyboards will predict entire sentences based on your conversation. Second, dynamic layouts. Imagine a keyboard that rearranges itself in real-time based on what you’re typing. If you type “I am going to,” the keyboard might surface words like "the store," "bed," or "work" as big buttons for faster selection. That’s the future.
That’s it for today, and don’t forget to subscribe to the newsletter if you find this interesting
Check out the full Video interview here
See you next week