How Pixel 10 Pro Created The World's Smartest Phone Camera - A Peek Inside Google

5 days ago

Isaac Reynolds has been moving connected nan Pixel Camera squad astatine Google for almost a decade -- since nan first Google Pixel telephone launched successful 2016. And yet, I deliberation it's adjacent to opportunity that he's ne'er been much bullish astir nan exertion that Google has integrated into a telephone camera than he is pinch this year's Pixel 10 Pro. A caller activity of AI breakthroughs successful nan past twelvemonth person allowed Google to usage Large Language Models, instrumentality learning, and generative AI imaging to unlock caller capabilities to powerfulness different meaningful leap guardant successful telephone photography.

I sewage nan chance to beryllium down pinch Reynolds arsenic he was still catching his activity from nan motorboat of nan Pixel 10 phones -- and astatine nan aforesaid time, ramping up for nan adjacent group of camera upgrades nan squad is preparing for nan 2026 Pixel phones.

Also: Pixel conscionable zoomed up of iPhone successful nan camera photography race

I peppered Reynolds pinch each of my burning questions astir Pro Res Zoom, Conversational Editing, Camera Coach, AI models, nan Tensor G5 chip, Auto Best Take and nan larger ambitions of nan Pixel Camera team. At nan aforesaid time, he challenged maine pinch accusation I didn't expect connected Telephoto Panoramas, C2PA AI metadata, Guided Frame, and educating nan nationalist astir AI.

I sewage to unpack a batch astir really nan Google squad was capable to technologist specified large advances successful nan Pixel 10 Pro camera system, and we delved acold deeper into nan caller photography features than Google talked astir successful its 2025 Made by Google event aliases successful its published blog post.

Here's my reporter's notebook connected what I learned.

Mission of nan Pixel Camera team

"I deliberation nan awesome point our squad has ever been focused connected is what I telephone durable [photography] problems -- debased light, zoom, move range, and detail," said Reynolds. "And each procreation [of Pixel] has brought caller technologies."

Camera Coach

Reynolds noted, "LLMs person specified an tremendous discourse window, and they're truthful powerful astatine knowing that we tin really thatch group to do things that tech can't do.

"Today, tech cannot move nan camera down 4 feet. Tech can't locomotion nan camera complete 100 yards to nan amended viewpoint. It can't show you to move 90 degrees. Now, Camera Coach tin do that benignant of stuff. So that's conscionable different measurement we're utilizing exertion to lick immoderate of these durable problems."

Conversational editing

One of nan astir astonishing caller features Google announced successful nan Pixel 10 was conversational photograph editing -- though this is technically a characteristic successful nan Google Photos app. This lets you simply picture what you want changed successful nan photo, pinch sound aliases typing, and nan AI takes attraction of nan rest. So you tin region a tree, re-center nan image, aliases adhd much clouds to nan sky, for example.

As Reynolds explained it, "Conversational editing conscionable takes nan full interface distant and it's fundamentally a mapping usability from earthy connection to nan things that were successful nan editor. So you tin say, 'Erase nan point connected nan left,' and it will conscionable fig retired what nan point connected nan near is and past invoke Magic Eraser. You tin say, 'Hey, erstwhile I was successful Utah I retrieve nan rocks being much reddish than that' and it conscionable increases nan warmth a small bit. You tin say, 'Can you attraction connected nan point successful nan center' and it puts a small vignette astir it.

"And that mapping is simply a immense clip saver. The committedness of AI was not conscionable that it would beryllium informational, but it was that it would return actions for you. And I deliberation this is 1 of nan astir cleanable cases of nan AI not conscionable reminding you of thing ... but doing it for you. It has been really, really cool to spot really effective it is.

"It moreover gives you suggestions. The AI will look astatine a image and opportunity 'I deliberation you person immoderate bystanders you would want to remove.' And truthful it populates these small proposal chips. The funniest portion of nan proposal chips is erstwhile you pat them, each it does is type into nan matter box. It's not a abstracted pathway. You conscionable pat nan spot and it sticks thing successful nan matter box. You could person written that yourself. It's not doing thing wildly different than you could do... It's besides sewage nan sound button, which is ace cool. You could conscionable talk to it if you want to. The AI is getting truthful bully truthful overmuch faster than I could imagine, and I'm a master successful this space."

Pro Res Zoom

As a photographer who loves zoom photography, this was nan characteristic I wanted to talk pinch Reynolds astir nan most. I return a batch of photos pinch smartphones, but agelong region zooms are wherever I astir often request to propulsion retired my Sony mirrorless camera and 70-200mm lens. I've already written astir really excited I americium to thoroughly trial Pro Res Zoom, since it could thief nutrient a batch much usable zoom photos from a telephone by utilizing generative AI to capable successful nan gaps successful integer zoom.

Reynolds commented, "The basal problem is, really do I move a integer zoom wherever you've sewage a sensor pixel connected nan acold correct corner, and past different 1 connected nan bottommost near corner. And you person to capable successful each nan pixels successful between. You tin do an interpolation. You tin conscionable group them each to beryllium immoderate color, for illustration conscionable mean them. We've grown each nan measurement done nan process here. We've gone done multi-frame denoise. We've gone done aggregate different generations of upscalers to make amended interpolations. We went to a multi-frame merge that was block-by-block. And past nan awesome advancement that was Super Res Zoom was going from a block-by-block multi-frame to a probabilistic pixel-by-pixel multi-frame... In parallel, nan upscalers were improving. And nan latest procreation upscaler is nan largest exemplary we've ever tally successful Pixel Camera ever... And it's conscionable a really, really bully interpolator.

"It doesn't conscionable opportunity that's achromatic and that's white, and truthful nan mediate is gray. It's like, well, I cognize that that achromatic pixel is portion of a larger structure. I cognize that that larger building appears to beryllium nan grout successful betwixt immoderate ceramic connected a facade. And truthful astir apt it's going to beryllium achromatic up until that point, and past it's going to move reddish -- which is truthful overmuch smarter than conscionable going, 'Well, it's achromatic and it's red. So, I don't know. I conjecture we'll conscionable operation them arsenic we spell across.' So we still person those existent things arsenic existent pixels, and past we person to capable successful what's successful between. And now nan models are conscionable so, truthful bully astatine that.

"We've had a agelong statement of upscalers, and this is nan latest one. All nan upscalers person artifacts. Different upscalers person different kinds of problems. We've had upscalers successful nan past that were very, very bully astatine matter -- because matter has very harsh lines -- but very bad astatine water, because h2o is fundamentally chaotic. This upscaler has its ain artifacts, and those artifacts are very difficult for nan quality oculus to recognize, because nan caller models are truthful bully astatine making contented that is 100% authentic to nan scene.

"Like, yes, that's a leafage connected a tree. That's precisely what a leafage connected a character looks like. It's flawless. But for a quality face, location is truthful overmuch of nan quality encephalon dedicated to recognizing faces, that nary level of artifact is efficaciously acceptable. The level of subtle artifact connected a leaf, you whitethorn ne'er notice. But nan aforesaid subtlety connected a face, you announcement instantly -- conscionable because we're quality beings and we're designed to admit different quality beings. We're societal creatures, truthful nan barroom for really doing a bully occupation pinch quality faces is extraordinarily high."

As a result, erstwhile Pro Res Zoom recognizes a quality face, it won't usage nan AI to upscale it.

C2PA metadata to explanation AI

Because Google is now portion of nan Coalition for Content Provenance and Authenticity (C2PA), it has started to embed metadata into its photos to bespeak whether generative AI was utilized to make nan photograph by utilizing SynthID, a watermark created by Google DeepMind. Reynolds was profoundly progressive pinch nan task to make this portion of Pixel Camera.

"The [C2PA] metadata identifies whether this was AI aliases not, and it conscionable mostly tells you nan history of nan image and we embed it," said Reynolds. "I was personally nan merchandise head for that. I don't do things personally for illustration that a batch anymore, but I did return that 1 because I knew really important, nuanced, and subtle it was. And nan deeper I sewage into that feature, nan much I realized really small group really cognize astir what AI is aliases isn't, what it tin and can't do, aliases really accelerated aliases slow it's progressing."

An illustration of Google C2PA metadata for AI

Also: Google Pixel 10 bid hands-on: I did not expect this exemplary to beryllium my favorite

Educating nan nationalist astir AI

"The world is honestly down successful position of not realizing really bully AI is already. So there's immoderate acquisition to do. And we recognize that AI tin do things that I deliberation users would really, really for illustration if they understood amended what was going on. So portion of what we do successful Pro Res Zoom is we don't touch faces. I deliberation that'll make group much comfortable. We besides show them nan earlier and aft -- nan type pinch nan caller upscaler and nan 1 without it, and you get to determine for yourself, what did AI do? Did I find it acceptable aliases unacceptable? The overwhelming mostly are uncovering it much than acceptable -- highly preferred, successful fact. They want nan upscale. But they wouldn't cognize that if they didn't get to spot nan side-by-side.

"And past we besides explanation it pinch contented credentials [C2PA] truthful that whenever they transmit that photo, personification other tin make their ain determination about, 'How do I ideate this photo? Do I discount this arsenic possibly AI? Or do I go, ohio no, nan contented credentials are correct there. They opportunity it's not AI astatine all. This is great. I person truthful overmuch much spot now.' And arsenic users study more, arsenic they're knowledgeable more, arsenic they summation much comfortableness and much existent world information points of what is AI and what isn't, I deliberation they will extremity up being much comfortable complete time, and that's what we're seeing pinch Pro Res Zoom already. The customer restitution that we measured pre-launch was truthful bully for that feature.

"And arsenic nan exertion gets better, we'll do more. We will put this worldly into much modes, perhaps. We'll push nan zoom a small higher quality. But we really want to make judge that we're doing that arsenic users expect and understand it. So we're giving you options and choices and transparency, but we're besides trying to push nan boundaries of exertion successful a measurement that keeps customer restitution high."

Telephoto Panoramas

"There are ever small goodies hidden each complete nan [camera] app," Reynolds told me. "We build much worldly than we tin realistically talk about."

One of nan caller photography features successful nan Pixel 10 Pro that Google hasn't talked overmuch astir is Telephoto Panoramas, aliases what they affectionately telephone "5x tele-panos."

These let you to return much cinematic scenery shots utilizing nan zoom lens, caller viewfinder controls, and nan expertise to to sprout 360 degrees and up to 100MP resolution. "There's thing that's conscionable truthful bully astir zooming successful pinch your lens and past stitching nan panorama," said Reynolds.

But what Google hasn't talked astir is nan truth that it's utilizing an wholly caller method of capturing these panoramic images.

"A batch of panoramas successful nan market, and ours historically arsenic well, were video based," Reynolds noted. "And what that intends is to make a view you return 100 to 1000 images, and each 1 of them, you stitch a small mini vertical slice. So that intends 2 things. Number one, it intends that nan artifacts you get thin to beryllium curves, stretches, and compressions because you're conscionable going portion by slice. The different problem is that successful that 30 seconds, you person to process [up to] 1000 images.

"So what we did is we said alternatively of a video we're going to usage photograph input. So we're going to return 5 pictures, not hundreds, and we're going to put each of our processing down it -- afloat HDR Plus, afloat computational photography, Night Sight -- and past we stitch a small spot of overlap. So alternatively of having a small sliver from each picture, it's conscionable a small overlap. That's really [Adobe] Lightroom would do it, for example. We're utilizing nan Lightroom method.

"And truthful we get Night Sight Panorama. We get panoramas now up to 100 megapixels. We get conscionable super, ace elaborate and we tin move connected parts of nan zoom pipeline that we couldn't needfully do before. So you tin usage nan 2x zoom, which connected a Pixel telephone has optical quality. And you tin moreover invoke nan 5x telephoto [on nan Pixel Pro]. It's a very computational-photography-forward, photo-based panorama."

Also: Google Pixel 10 Pro vs. iPhone 16 Pro: I've tried some flagships, and there's an easy winner

Guided Frame (accessibility feature)

Another characteristic that has flown nether nan radar that Reynolds wanted to constituent retired was Guided Frame.

"Guided Frame is an accessibility feature. If you are unsighted aliases low-vision, we usage Gemini to let you to framework immoderate photo," said Reynolds. "In that case, you constituent nan camera, you invoke Guided Frame, and it says, 'This is simply a photograph of a segment of nan woods pinch immoderate trees disconnected to nan correct and a personification connected nan left. Person is successful frame, smiling, bully for a selfie. And past it will return nan photo. So if you can't really spot nan surface that well, it helps return selfies and photos, because [selfies] are really group communicate. Whether you're unsighted aliases low-vision aliases not, group pass utilizing pictures. So it gives them that capability."

Auto Best Take

I besides asked Reynolds astir nan improvement of Best Take to Auto Best Take this twelvemonth and was amazed to study that this characteristic is really utilizing much instrumentality learning.

"Auto Best Take is overmuch much accepted processing," Reynolds commented. "You tin ideate this arsenic a determination tree, because that's fundamentally what this characteristic is. You property nan shutter once. If that shutter property was cleanable and everyone was smiling, everyone's looking astatine nan camera, past great. Done. One picture.

"Okay, let's opportunity it wasn't perfect. Then we're going to unfastened nan shutter a small longer and we're going to look astatine each azygous frame. So that's up to 150 frames successful conscionable a fewer seconds. If we spot 1 that's better, we'll return it, we'll prevention that one, we'll process it successful afloat HDR Plus quality... So erstwhile you spell to nan gallery, you're going to spot nan 1 that we took arsenic nan primary, that's called Top Shot. So that's 1 measurement down nan determination tree.

"Let's opportunity we looked astatine 150 frames and we couldn't find 1 that was perfect, but we recovered 1 that was almost perfect, and a 2nd 1 that was almost cleanable but successful a different way, specified arsenic a different face. Then what we'll do is we'll prevention some of those and past we'll walk that to Best Take and Best Take will blend them into 1 that is perfect. And Top Shot will intentionally take a scope of pictures truthful that there's astatine slightest 1 photograph successful which each look is smiling. So if location is simply a image of each look smiling astatine slightest erstwhile location successful nan group past it will do a Best Take. Once you look astatine 150 pictures, astir of nan clip you get nan shot. So very seldom does it really spell to Best Take. So it's a small overseas that we telephone it Auto Best Take, because successful reality, we don't do it very often, since it's astatine nan extremity of nan determination tree.

"The extremity is that you property nan shutter 1 clip and you get 1 photograph and that photograph is perfect. It does not matter really we get there. We ne'er want you to person to return 3 photos [of nan aforesaid group picture] again. Because why would you return 3 random photos erstwhile [the AI] tin look astatine 150 photos. So we opportunity conscionable property [the shutter button] once. Give it a mates of seconds. You'll spot it successful nan UI. It draws boxes astir people's faces. It turns them golden erstwhile it thinks it nailed it. So property nan shutter, springiness it a mates seconds, and past watch what you get astatine nan end."

The quality pinch Tensor G5

Google made a big move successful 2025 pinch its Tensor G5 spot powering nan Pixel 10 phones -- shifting from having Samsung build its Tensor chips successful nan past to a TSMC 3nm process that uses TSMC's precocious exertion to summation AI performance. I asked Reynolds astir nan impact.

"[The boost pinch Tensor G5] is 1 of nan largest before-and-afters I've ever seen successful position of processing latency," he noted. "The first versions of Pro Res Zoom took for illustration 2 minutes [to process]. And past by nan end, erstwhile they sewage it connected Tensor G5 and each nan bugs had been worked out, that sewage down to conscionable respective seconds... So nan Tensor G5 TPU is 60% much powerful, and we tin decidedly spot that."

Also: Considering nan Pixel 10 Pro? I urge buying these 5 phones alternatively - here's why

The AI models powering Pixel photography

Since truthful galore of nan Pixel 10's astir important caller features are powered by AI advances, I wanted to cognize much astir really nan Pixel Camera squad useful pinch Google's soul AI capabilities.

"It's not for illustration there's this 1 monolithic Gemini," Reynolds said. "It is highly cautiously tuned and tested for 1 peculiar usage lawsuit astatine a time... There are truthful galore much versions of Gemini wrong [Google] than you tin spot outside. And past you person to decide, americium I going to punctual this Gemini aliases americium I going to good tune this Gemini? It's each super, ace civilization to a peculiar implementation." For example, he added, "Magic Eraser is generative, but it's not Gemini."

Final thought

Google is nan only 1 among nan twelve aliases truthful companies successful nan world building frontier AI models that besides makes its ain smartphone. And pinch nan Pixel 10 Pro, nan effect is starting to show.