Pezen is a restaurant point-of-sale I am reviving into a business. Its standout feature is an AI waiter that runs on the iPad itself.

Pezen is a restaurant point-of-sale app, and it is the one project here I want to turn into a business. I first built it in 2019 and kept improving it through 2022, and then it sat dormant for years. I am reviving it now with that goal in mind. A point-of-sale on its own is a crowded market, so the thing that makes Pezen worth building is what I added to it recently, an AI waiter that lets a diner order out loud, with no human waiter needed. That AI came out of Challenge 1 at the Apple Developer Institute for AIML, whose theme was making something useful out of audio data. There were three of us, Eko, Fitri, and me, and to get the most out of it we each wrote our own inference pipeline. I took on the shared foundation. I redesigned the architecture and directed an AI coding assistant to reimplement it in the stack I wanted, then built the Table App that lets an operator point inference at either an on-device model or a cloud server. From there I focused on the on-device pipeline while Eko and Fitri built cloud ones. This case study is about both halves, the product and the AI I built into it.

Before the AI, Pezen is a working point-of-sale. A diner or a server builds an order, adjusts quantities and per-item notes, and checks out, and the cart and the kitchen stay in sync throughout. The table terminal runs on an iPad, backed by a Rust server that owns the menu, the cart, and the orders. Almost nothing about this layer is exotic, and that is deliberate. A point-of-sale has to be dependable and unsurprising in the places that matter, so that the one genuinely ambitious part has room to take a risk. For Pezen, that part is the waiter.

A spoken order is nothing like a typed one. People stumble, they add fillers like 'ehm' and 'anu', and they correct themselves halfway through ('bakso tiga, eh dua'). On top of that a restaurant is loud, and the person at the next table is talking too. The heart of the problem, though, is a tradeoff. To run on-device with no server and no per-order cost, the model has to be small, around three billion parameters. Small models are good at language and bad at bookkeeping. Ask one to also copy a 24-character menu ID or scan a menu for the closest match, and it will cheerfully invent an item or grab the wrong one. So the real question was how to get a small model's language skill without trusting it with anything it is bad at.

The answer shapes the whole system. The language model is only allowed to do language. It cleans up the messy speech, then extracts what dishes the customer named, as plain text, and what they want to do. It never picks a menu item and never touches an ID. Everything factual is handled by deterministic code. Plain Swift matches each spoken dish name to the real menu, decides whether it is a confident match, ambiguous, or not on the menu at all, resolves the real IDs, does the cart math, and even writes the confirmation sentence so the reply can never disagree with what was actually added. The language runs as two small steps, one to repair the disfluent Describing the natural stumbles of speech, like fillers and self-correctionsspeech into a single clean sentence and one to pull the intent out of it, and then the deterministic matcher takes over. A small model asked to match 'bakso' to a menu will happily return 'Mie Ayam Bakso' because they share a word. Matching in code, anchored on the first word, simply cannot make that mistake. That split is the thing I am proudest of.

Before any of that, the audio has to become clean text. A neural voice detector decides exactly when an utterance starts and ends, so each recording holds one spoken phrase and no dead air. Restaurant audio comes in very quiet, so the first real step is normalizing its loudness, which on its own was the single biggest boost to transcription accuracy. Then the audio is split by speaker, so two people ordering into one recording do not get transcribed as one garbled turn, and finally each piece is transcribed by an on-device Whisper OpenAI's family of speech-to-text models, here running on-devicemodel pinned to Indonesian. There is one more trick that makes it feel responsive. The customer never has to wait for the system to finish before speaking again. A new utterance cancels the in-flight run and is resent together with everything said since the last result, so a correction a second later resolves against the whole batch at once, and the earlier answer is quietly thrown away.

Every time we tested the waiter it worked beautifully, and it took me a while to notice why. We were staying silent while it was processing. A real restaurant is nothing like that. It is full of voices, people ordering at other tables, staff calling out, someone in the next seat ranting about something that has nothing to do with food. And knowing that someone is speaking is not the same as knowing they are speaking to you. Every stray voice trips the detector, gets captured, and, worst of all, cancels the diner's own in-progress order. It is the same crosstalk that sank early drive-thru voice systems. I needed the app to listen only when it actually should. The fix came from thinking about how I order in real life. I do not just start talking, I look at the waiter first. So I added a small piece of computer vision that watches the front camera and opens the mic only when a person is there and facing the iPad. Look at it to talk to it, look away and it stops listening. For rooms or mounts where a camera does not fit, a tap-to-talk button and a wake phrase work as alternatives, and behind all of them a near-field Sound coming from right next to the device, as opposed to across the roomguard measures each capture against the room's noise floor and drops anything that is not clearly someone leaning in to talk. The diner clears it easily. The next table does not.

Because the three of us were each building a pipeline, I made the boundary between the app and the inference a clean contract. The Table App is a dumb terminal. It captures audio and applies cart actions, and it sends every turn to whatever pipeline the operator has configured, the on-device model or a teammate's cloud server, as long as that pipeline answers the same request and response shape. To make all of it debuggable, I built a logger. Every stage of a turn posts itself to the backend, and a web viewer replays the whole thing as a timeline, the raw request, the transcript, the prompt, the model's output, and the final response, with the audio at each stage playable right in the browser. Because it is just an open HTTP contract, Eko and Fitri's cloud pipelines log into the exact same viewer, so we could line all three builds up side by side.

The AI waiter began as a challenge project, and I want to be clear that it is not in a real restaurant yet. But it works end to end on a single iPad, from a messy spoken sentence in a noisy room to the right items in the cart, with nothing leaving the device, and that is enough to convince me it is worth pursuing. Pezen is the one thing here I would like to grow into a real business, and the waiter is what would set it apart from every other point-of-sale on the market. The lesson underneath it is the one that shaped everything. Let a model do only what it is genuinely good at, and let plain code handle the rest. It is the same instinct behind my computer-vision work, and I keep finding it is the line between a demo that impresses once and a system a restaurant could actually trust with a live order. Crossing that line is the whole point.