Lesson 07 · 15 minutes · Preparation concept
Fast enough means
measured on target.
Your next win: distinguish a requested accelerator from the device and performance actually observed.
One model, several execution questions
| Question | Evidence needed |
|---|---|
| Does exported inference match the baseline? | Paired task results on the same frozen scenes. |
| Which device actually ran? | Runtime-reported device and precision. |
| Is it fast enough for the application? | Cold start, warm p50/p95, throughput, memory, and end-to-end timing. |
OpenVINO is the planned Intel inference runtime. The correct path is CPU correctness first, then measured iGPU acceleration if available. An NPU is optional and component-specific. A request for a device is not proof that the runtime selected it; fallbacks must stay visible in the benchmark record.
Latency is not one number
Cold-start latency includes initialization. Warm p50 describes a typical repeated request, while p95 shows a slower but still common tail. Throughput measures completed work over time. End-to-end timing includes more than model inference: image preparation, planning, action validation, simulation, and record writing can all matter.
Practice an honest result
Write a two-line report for this hypothetical: “iGPU requested, CPU used, 10 paired seeds retained, success unchanged.” State the fallback and the remaining limitation. Ask the agent to review it.
Primary reading: OpenVINO’s supported-model documentation and the project’s Intel deployment plan.