Edge AI & model optimization
Many models are trained on large servers, then need to run somewhere much smaller: a camera, a vehicle, a machine on the factory floor or a single local server.
We shrink and speed up models so they fit the hardware you have, and we measure how much accuracy that costs.
What model optimization does
When a model runs close to where data is produced, answers come faster, nothing depends on a network connection and sensitive data does not have to travel.
Devices, however, have limited memory, computing power and energy, and a model that runs comfortably on a server may be too large or too slow for them. Optimization makes it smaller and faster and adapts it to the chip it will run on.
The same work also lowers running costs on ordinary servers, which is often reason enough on its own.
What we build
Smaller versions of large models
A compact model is trained to imitate a larger one. It keeps most of the useful behavior at a fraction of the size.
Lower-precision models
Storing and computing a model's numbers at lower precision cuts memory use. The model also runs faster, especially on chips built for low-precision arithmetic.
Models tuned for a specific chip
We prepare the model for the exact processor, accelerator or phone chip it will run on, using the tools that hardware supports.
On-device vision and sensor models
Detection, inspection and monitoring models that run inside cameras, vehicles or machines without sending data elsewhere.
Local language models
Language models small enough to run on a single local server or workstation for tasks with a clear, limited scope.
Faster models on existing servers
On the servers you already have, a faster model costs less to run and the same hardware can handle more requests.
When it's a good fit
Worth the effort
The model must respond faster than a round trip to a server allows, the device has a weak connection or none, data should not leave the site, or the current model costs too much to run at your volume.
A server may be simpler
If none of these applies, running the full model on an ordinary server is usually the easier choice. In that case we will recommend it.
How a project runs
-
Agree on the target
Together we set the target hardware, how fast the model has to respond, how much memory and power it may use, and how much accuracy you can give up in return.
-
Measure the starting point
We test the current model on your data and time it on the target hardware. Every later version is compared against that baseline.
-
Try several ways to shrink it
We combine and compare different techniques and measure each version's size, speed and accuracy on the target device itself, as well as on a development machine.
-
Package it for the device
We package the chosen version with the code to run it on the target hardware, ready to install alongside your existing software.
Knowing what you give up
Making a model smaller almost always costs some accuracy, and it matters where that loss falls. A drop that looks small on average can be concentrated in the cases you care about most, such as rare defects or poor lighting.
So we break down the comparison with the original model by the situations that matter to you. Before choosing a version, you can see where accuracy drops.
What you receive
- An optimized model packaged for your target hardware
- A side-by-side comparison with the original model on speed, size and accuracy
- The scripts used to repeat the optimization when the model is retrained
- Code and documentation, handed over to your team
Frequently asked questions
Can an optimized model run on the hardware we already have?
Yes. We start from the devices you have and suggest different hardware only if yours can't meet the requirements.
How much accuracy does a model lose when it is made smaller?
There is no general figure. It depends on the model, the task and how far the model has to shrink, so we measure it on your own test cases.
What do you need from us to optimize a model?
The current model or a clear description of the task, some test data that reflects real use, and access to the target device or an identical one.
Does our data have to leave our site for model optimization?
No. We can do the optimization work in your environment, and the finished model runs on your devices, where the data is produced.
What happens when the model is retrained later?
The retrained model can go through the same optimization with the scripts we hand over. We can also help with those updates if you prefer.
Work with us
Tell us in a few lines what you're working on. We'll get back to you.
From our blog
All articles
Self-hosted LLM or API? Rent the generalist, own the specialist
Self-host an LLM or use an API? Use an API for open-ended work. For narrow, repeated tasks, a small model you run can win on cost, privacy, and control.
Read article
How to choose an AI model architecture
AI model architecture shapes what a system can learn and where performance plateaus. Prompting or more data cannot fix an architectural mismatch.
Read article
When should you use synthetic data?
Synthetic data can help when real data is scarce, sensitive, or unavailable. When is it worth using, and how can you tell whether it improves results?
Read article