Daniel Korat

Principal Engineer, AI Optimization · Microsoft

I make language models, and the agents built on them, cheaper to run.

I'm a Principal Software Engineer in Microsoft's AI Frameworks group, optimizing AI models for Microsoft's AI accelerators. Before that I spent eleven years at Intel on efficient NLP, LLM inference and AI agents: speculative decoding, few-shot learning and test-time compute, from first prototype to code merged into Hugging Face Transformers and vLLM.

Portrait of Daniel Korat
2.8×lossless LLM speedup from speculative decoding
2M+SetFit downloads a year
11papers at top-tier AI/ML venues
24%lower coding-agent operating cost at equal latency

01Selected work

2023–2025
Inference

Speculative decoding with any assistant model

Co-developed three methods, Dynamic Speculation Lookahead, Universal Assisted Generation and Token-Level Intersection, merged into Hugging Face Transformers; Token-Level Intersection is also in vLLM, where it is the only cross-tokenizer drafting path. Up to 2.8× faster generation with no change in output. First author on Universal Assisted Generation, which lets a small model draft for a large one even when their tokenizers differ.

ICML 2025 paperHF blog: UniversalHF blog: DynamicHF blog: TLI

2025–2026
Agents

Cost-efficient AI agents

Built and shipped a Claude Code plugin in which a frontier model hands work to a local specialist model served on commodity CPU servers: up to 24% lower operating cost than cloud-only, at equal latency. Built a deep-research agent that runs fully on-premises across CPU and GPU; profiling each stage led to a redesign that cut about 51% of the tokens per query.

hybrid-deep-research

2022
Few-shot

SetFit: few-shot text classification without prompts

Co-created with Hugging Face: train a strong classifier from a handful of labelled examples. 2M+ downloads a year; a later 8-bit version runs 7.8× faster on CPU.

GitHubPaperHF blog: 7.8× on CPU

2025
Test-time compute

Portfolio Beam Search

Spends a fixed test-time compute budget across several search strategies. At 16 beams it matches standard beam search at 32.

HF blog

02Selected papers

All 12 papers on Google Scholar →

03Background

Experience

MicrosoftPrincipal AI Optimization Software Engineer · 2026–present

IntelAI Research Engineer / Scientist · 2025–2026Deep Learning Research Engineer · 2017–2025Research Assistant · 2015–2017

Co-founder of NLP Architect, Intel's open-source NLP library. One granted U.S. patent.

Education

M.Sc. Computer Science, Reichman UniversityMagna Cum Laude · 2018–2022

B.Sc. Computer Science and Linguistics, Tel Aviv UniversitySumma Cum Laude in Linguistics · 2013–2016

04Contact

The best way to reach me is a message on LinkedIn.