
The case for local inference is made in three clauses representing different kinds of constraint: cost, where a local model removes an API call; availability, where the application works on a flight; and data, where requirements prevent information leaving the device. Only the third changes what is buildable rather than what is affordable. What makes this newly practical is unglamorous — instruction set extensions integrated into the runtime rather than a modelling breakthrough — with around thirty per cent improvement reported in image editing functions. The guidance on fine-tuning inverts the usual advice: adaptation matters most for the smallest models, because their capability budget is already spent and getting them to perform on your problem means spending some of it there. The two examples do the real work, since neither is a cheaper version of a cloud application.






