inference/
Faster, with less
I work on vLLM and FP8 serving, EAGLE-3 speculative decoding, and MXFP4/GPTQ quantization, connecting model optimization to efficiency in GPU operations.
$ whoami
Taeyoung Lee.
a.k.a. Full-stack Latte · sh2orc
AI platform & inference engineer
Building financial payment systems and data platforms taught me how to design and operate services that keep running. Today I apply that experience to designing and developing LLM inference and agent platforms.
I improve inference efficiency through vLLM serving, quantization and speculative decoding, then connect it to environments where agents do real work. I also adapt diverse LLMs to Korean language and contexts and create the datasets they need.
In my personal workshop, I build the IYAGI terminal, window-management tools and apps for my family. I share what I can as open source, and write and speak about what I learn along the way.
profile.jsonjson
{
"handle": "sh2orc",
"pen_name": "Full-stack Latte",
"role": "AI platform & inference engineer",
"building": ["inference platforms","agent tools"],
"research": ["Korean LLMs","datasets"],
"personal": ["iyagi","ieum"],
"focus": ["vLLM","MXFP4","EAGLE-3"],
"stack": ["python","rust","typescript"],
"fuel": "latte"
} always a work in progress
$ ls ~/interests
What I’m exploring now.
agents/
A workspace for agents
I develop computer-use and multi-agent orchestration environments where AI performs real tasks. With IYAGI, I explore ways to keep multiple coding-agent sessions running.
data-platform/
Making data usable by AI
From Hadoop to Databricks-based NL2SQL and MongoDB vector/keyword hybrid search, I connect payment data to AI services.
operations/
AI through to operations
I work on H200 cluster operations, B300 capacity planning, and OpenShift AI/llm-d serving. Risk classification, approvals, monitoring and audit trails are part of the platform too.
$ git log --oneline --reverse
One step at a time.
- 2001–04
start: A first startup in educational software
I began building products through mathematics education software and solutions for businesses and public institutions.
- 2004–06
serve: Helping people on the emergency front line
During firefighting service, I responded to more than 2,300 dispatches, contributed to 12 rescues and received a commendation.
- 2010
build: A Hadoop platform for financial data
I built a foundation for analyzing payment data at scale and continued developing financial payment systems and data platforms.
- 2018
study: Researching abnormal payment detection
I presented research using scalable architecture and Redis Cluster at IEEE PlatCon and completed a master’s in Electronic and Computer Engineering.
- 2020
ship: A zero-downtime core payment migration
Starting in 2017, I built Linux x86 high-availability architecture and applied ML fault detection with recovery within 0.3 seconds.
- 2021–23
ship: Launching cross-border QR payments and art commerce
I led an in-house startup team to launch QR payments connecting five Asian countries and digital art commerce for emerging artists and artists with disabilities.
- 2023
build: Beginning in-house LLM development
I set an enterprise AI/ML roadmap and began development with a 27B LLM. We later expanded the lineup for coding and agentic workloads.
- 2024.06
build: Designing cloud infrastructure for AI services
I led cloud-native AI service design, LLM research and adoption, establishing a foundation for using models in real services.
- 2024.12
build: Building inference, RAG and agent platforms together
I lead a team using XP agile practices to design and develop the AI services we need.
- 2025
share: Sharing AI experience through talks and policy advice
I presented our AI platform development journey and advised on financial-sector AI R&D environments and public-sector digital and AI adoption.
- 2026.05
speak: Sharing inference optimization internationally
I presented the SLM Swarm platform built with my team, including up to 70% lower GPU usage and more than 3× faster inference.
- 2026.08
release: Training and releasing a model with in-house MXFP4 quantization
moai-gemma4-31b-mxfp4 ranked 10th overall and 7th on K-MMLU in the K-Leaderboard, as of August 2026.
- 2026.09
scale: Expanding an agent development environment to 200 developers
We evaluated MoAI MATE in a 39-developer POC, observing a 56.4% productivity gain and 64% fewer bugs, before expanding it to 200 developers.
- 2026
study: Studying data science alongside work
I balanced work and graduate study from 2024 and completed a master’s in Data Science.
- 2026
release: Open-sourcing tools I use myself
I released IYAGI Terminal for multiple AI coding agents and IYAGI Layout, a macOS window manager built with Rust and Tauri 2.
- HEADWriting the next commit_
$ cat ~/research/README.md
Learning and research through practice.
From computer science to data science
B.S. in Computer Science, University of Seoul (2010); M.S. in Electronic and Computer Engineering, Korea University (2018); M.S. in Data Science, Illinois Institute of Technology (2026).
Real-time payment analysis and anomaly detection
Payment analysis using scalable network architecture and in-memory computing (2017), and abnormal payment detection using scalable architecture and Redis Cluster (IEEE PlatCon, 2018).
Six granted Korean patents · quantization patent pending
Patents cover payments, card and merchant recommendations, and device interfaces. A multilingual Hessian regularization method for MXFP4 quantization is patent pending.
$ ls ~/talks-and-writing
Sharing what I learn by building.
- 2026.09
KT Group Connect Week
In-house AI transformation with BCGPT, MoAI and SLM Swarm
- 2026.09
MongoDB.local Seoul
BC Card’s AX journey: turning payment data into AI-ready assets
- 2026.05
Red Hat Summit · Atlanta
An agentic AI operations platform built on SLM Swarm and Red Hat
- 2025.10
Red Hat Summit Connect Seoul
From operations to innovation: building an AI platform on Red Hat OpenShift