Smart audio mixing is one of my favorite, lesser-known smartphone AI features

As you’ve probably noticed, the hype around generative AI tends to be focused on a few key areas when it comes to smartphones. If it’s not about answering knowledge questions, it’s about image editing and generation, or rewriting messages to friends or coworkers. You’d be forgiven for thinking that OpenAI is worth billions because people really, really need help planning vacations or crafting basic work emails.

Something that gets lost in the shuffle is smart audio mixing. It’s not that it gets zero attention — it’s a marketing bulletpoint for phones like the iPhone 16, Galaxy S25, and Pixel 9 — but I feel like it’s underrated next to its utility. So much so that I really hope companies like Apple and Google dramatically increase the attention it gets in the next generation of devices.

What makes smart audio mixing so great?

Literally, finding the signal in the noise

Apple

While modern smartphones can produce excellent video — even comparable to professional cameras under ideal conditions — audio remains a sore point. Their tiny microphones just aren’t that great beyond mouth range, so it’s easy to end up with faint voices muffled by a soup of ambient noise, if those voices aren’t drowned out completely. Good luck recording clean vocals when watching a stadium event or riding a PEV — there’s a reason your favorite YouTubers use dedicated audio gear.

AI will never completely replace separate microphones, but it can do plenty to clean up the audio you do have. One example is Google’s Audio Magic Eraser tool, which lets Pixel owners reduce wind and other ambient noises, or amplify vocals. Something I appreciate is that while there’s an Auto button, there are also manual sliders for each category — if you want to preserve some ambient sound, like city traffic, you can leave that in. When everything comes together, it really does feel like magic.

iPhone owners don’t get as much control, but if you’ve got an iPhone 16 or 16 Pro, you can take advantage of something called Audio Mix. This offers four options — Standard, In-Frame, Studio, and Cinematic — that not only suppress ambient noise, but determine where vocals “sit” in the mix using Apple’s Spatial Audio technology. It can take time to figure out which option is best for a particular scene, but the good news is that, like Audio Magic Eraser, everything is edited in post. If the Studio or In-Frame modes sound strange, you can always revert to Standard or Cinematic.

Samsung has an even more nuanced Audio Eraser tool of its own for the Galaxy S25. This identifies a range of different sound types, such as crowds and car alarms, and lets you dial them in independently. Used intelligently, and under the right circumstances, the tool can deliver even better results than Apple or Google.

All of this matters because audio has a greater impact on videos than we’re consciously aware of. One of the things that made David Lynch’s movies classic, for example, was his emphasis on sound design — he knew how to unsettle people with an ambient soundscape, or let us listen in on the smallest secrets.

Related

Will anyone actually want Apple’s AR glasses when they’re ready?

Apple seems to be struggling to understand AR and VR.

The work that needs to be done

Glaring but not insurmountable flaws

Perhaps the most serious issue with AI audio mixing, currently, is that it’s too easy to make things sound artificial, and not in the sense of being pristine. Processed voices can occasionally sound flat, distorted, or clipped. I’ve heard samples from a Galaxy S25 Ultra that sound outright robotic, which is the opposite of what anyone wants.

You can potentially avoid these issues by learning when they’re likely to be triggered, but, of course, that somewhat defeats the purpose of AI mixing. Any video will sound great under perfect conditions. Work needs to be done on adapting AI to every scenario, although the best results might require better built-in mics, making it easier to separate and filter out sources.

Other welcome additions would be automatic clean-up and EQ balancing. The ideal for most smartphone videos is clean, punchy, cinema-quality sound, and while there’s only so much you can do with tiny mics, I’m almost surprised better processing isn’t de facto already. There’s the risk of making things too polished, as I hinted a moment ago, but better that than it being patently obvious you were using a smartphone.

Related

The Pixel 10 might end up being the smartphone surprise of the year

An annual Pixel refresh is about as reliable as death and taxes.

Why wait for the next generation of devices for improvements?

A question of technology and willpower

Supcase

One issue, I suspect, is processing power. Devices like the iPhone 16 are already handling 4K resolution, spatial mixing, and Dolby VIsion, so expecting the mobile equivalent of Logic or Pro Tools might be a bit much. Phone processors are continuing to evolve with AI in mind, however, so any technical barriers should eventually disappear, particularly if microphones improve as well.

The greater obstacle may be development priorities. Smartphone videos often sound good enough for casual use, and if I’m honest, there are aspects of AI that do deserve a higher priority. It’s more important that my phone interprets me properly when I try to control apps or smart home devices, and some people legitimately depend on AI assistants for research or how-to advice. I can deal with muddy memories of my birthday if it means people aren’t getting hallucinated facts about science or history.

It’s hard to judge how much Apple, Google, and Samsung care about smart audio mixing. Given that they’ve made it a part of their marketing campaigns, though, it seems reasonable to expect work to continue. Who knows — in a few years, it might be legitimately difficult to tell some phone videos apart from a TV show.

Related

Is iPhone better than Android for sound quality?

The answer isn’t as clear-cut as it once might’ve been.