Ilya Savin
Engineering Manager
Engineering Manager with 10+ years in mobile and product engineering. I started as an Android engineer, still care deeply about the craft, and now spend a lot of my time helping teams use AI in practical ways: speeding up delivery, cutting busywork, and making room for better product thinking. I like building things that are useful, polished, and actually make it into customers’ hands.
Measuring AI Without Fooling Ourselves
Most AI coding tools show usage numbers: active users, tokens, and acceptance rates. These help track rollout, but rarely tell leadership whether review, delivery, and quality actually improved.
The harder question is what changed in the workflow: how much AI-assisted code reaches merged PRs, what happens to cycle and review time, whether defects stay under control, and how to measure it without handing vendors broad repo access. That shifts focus from tool usage to the coding harness around the model: permissions, repo access, hooks, logs, and PR flow. It matters more when teams switch between Cursor, Claude Code, Codex, or Copilot.
You will leave knowing:
- how to estimate AI-assisted code in merged PRs without broad vendor access
- which signals beat provider dashboards: cycle time, review time, defects
- where measurement gets risky: privacy, access rights, vendor-only metrics
- how to keep metrics useful when the tool or its pricing change