Back

About

Artem Mukhin

Software Engineer at Microsoft with 10+ years across Microsoft and Yandex. Most recently built the quality, debugging and routing-reliability stack for a production LLM agent that executes write actions for business customers: trace replay, stage-isolated testing, end-to-end state verification, production failure detection, and retrieval fixes for skill routing. Background in analytics engineering, frontend and developer tooling. Author of a book on Claude Code for developer experience (Taylor & Francis / CRC Press, forthcoming).

Applied AI: building agents

  • Traced a family of multi-turn routing failures to one root cause: embedding-based skill routing reacted to the surface form of entity values (names, emails, license titles), so the same intent landed on different skills depending on the person's name. Disproved the naive normalization fix and documented it as a cross-cutting architecture issue
  • Prototyped typed-slot masking before retrieval (validated PoC): a deterministic + NER cascade replaces entity values with typed slots for routing only, ~76 ms p50. Name-dependent misroutes dropped from 6/6 to 2/6 intents; cross-language consistency rose from 74% to 90%
  • Built a skill-space lab that reproduces the agent's two-stage retrieval over hundreds of skills and visualizes skill collisions in 2D/3D - routing debates became measurable geometry instead of prompt folklore
  • Measured planner over-refusal (~4% of calls, independent of the query) and shipped a retry fix
  • Designed deterministic forms for common write actions (validated PoC): the LLM only suggests intent and slots, code owns resolution, risk, auth and execution. Defined the risk boundary and mapped the full skill catalog to seven UI patterns

Applied AI: testing agents

  • Built a stage-isolated testing toolkit (retrieval, chooser, planner A/B, then end-to-end) that cut test cycles 10-100x - a ~1-second targeted check instead of a ~20-minute end-to-end eval
  • Wrote an end-to-end API suite of 30+ single- and multi-turn scenarios that verifies the requested change actually happened, not that the answer sounded right
  • The suite caught a P1 failure class that offline evals structurally could not see, ahead of the private preview - including a fix that passed offline evals but did not fix the behavior
  • Redefined how agent success is measured: dashboards counted valid clarifications as failures and silent fallbacks as successes. Designed a deterministic outcome classifier over existing telemetry, with zero product changes
  • Turned "tests for one agent" into a phased E2E testing pattern shared with other agent teams

Applied AI: seeing agents in production

  • Built a trace-replay debugger - like browser DevTools, but for an AI agent - that joins production, eval and E2E logs into one conversation timeline (10-20x faster root-cause). Became the team's primary investigation entry point
  • Detected production failures without user text: defined nine structural failure signatures from execution metadata alone, surfacing real bugs that looked green in error-count dashboards
  • Presented the team's eval methodology at an org-level AI review and wrote the full reliability handoff: executive brief, pitch deck and a developer book on tracing, testing, routing and observability

Engineering & tools

  • Conceived and built MediaPlanner at Yandex through three iterations (Excel to Access to React+TypeScript) - cut media-plan creation ~6x for 100-150 Enterprise sales managers with near-zero errors; core calculations later shipped in Yandex.Direct
  • Shipped internal tools solo as developer, PM, and UX designer; led the Storybook migration behind Cloud UI / Tornado UI (now open source)
  • Built self-service OLAP dashboards (SSRS + MDX) for 50+ sales managers and analysts - instant data instead of multi-day waits
  • Automated analytics and reporting early on (VBA) - reduced a month of recurring work to 2-4 days
  • Founded an 800-member internal analytics community at Yandex

Developer experience

  • Created a dedicated DX role at Microsoft Skype - the first of its kind in the team
  • Cut mobile Fast Refresh from 60+ seconds to 2-5 seconds (up to 30x)
  • Served as the last engineer with full UI context during Skype's sunset - led the final ship migrating 22.8M users to Teams
  • Author of Bottom-Up DevEx methodology (peer-reviewed paper)
  • My article on The New Stack hit #1 Trending in Software Development (featured in WeAreDevelopers DevDigest, 200K+ subscribers)
  • Built DX-Ray - DX knowledge base, diagnostics, and resource library

Communities

6,250+ members across communities I've built and maintained:

  • legalgd - 4,500 members, 8+ years. Largest Russian-speaking community at the intersection of game law and development.
  • Game Analytics - 900 members, 9 years. Participants from Wargaming, Playrix, devtodev.
  • DX Level Up - YouTube channel on Developer Experience.

Recognition

  • Raptors Fellow - peer-reviewed, invite-only membership
  • The Hangar (dx.community) - selective DX professional community
  • Hackathon judge and creator (DX-Ray Hackathon, Vanilla Web, Code Olympics)
  • Mentor at RS School (JavaScript), 2 students over 20 weeks
Full career history on LinkedIn