About Kog
Kog builds the fastest LLM inference engine on standard datacenter GPUs. Our Kog Inference Engine generates 3,000 output tokens per second per request on a single 8× AMD MI300X node and 2,100 on an 8× NVIDIA H200 node (FP16, batch size 1, no speculative decoding).
We co-design the model architecture and the execution engine together. Our Laneformer model uses Delayed Tensor Parallelism (DTP), a novel architecture that restructures the Transformer dependency graph so inter-GPU communication overlaps with computation rather than blocking it.
We pre-trained a 2B-parameter DTP model on 6T tokens on 256 H100 GPUs.
We are a team of 11 people, including 10 engineers and 5 PhDs.
Test it at playground.kog.ai http://playground.kog.ai. Read the technical details on the Kog Labs blog https://blog.kog.ai.
What you will work on
You will imagine, design, and run experiments to understand how architectural decisions propagate through inference behavior, morph existing open-weight models into architecture variants optimized for speed, and turn findings into measurable gains in generation speed and model quality.
- Design new model architecture variants, including routing strategies, attention mechanisms, and MoE structure, with execution constraints as a first-order design input.
- Extend the Laneformer thesis by exploring inference-aware architectural variants such as DTP, Ladder Residual, and PT-Transformer, and finding what compounds at scale.
- Own the post-training pipeline across fine-tuning, evaluation methodology, and adaptation of existing open-weight models toward architecture variants optimized for inference speed.
- Scale the stack to large MoE models such as DeepSeek v4 and Qwen 3, working through routing, expert parallelism, and communication patterns at inference time.
- Write up findings as research papers, submit them to top venues, and present them at conferences.
- Contribute to building AI agents that will perform architecture research and training experiments autonomously, starting from the research foundations we are building now.
What we look for
- You have designed or changed model architecture, where the structure itself was the object of the work. Showing that work, a paper, a repository, or a thesis, is a requirement to move forward.
- You reason about model design and hardware together, tracing how communication structure and layer dependencies shape inference behavior, with fluency in Transformers and MoE deep enough to weigh trade-offs.
- Stronger signals include inference-aware architectural variants such as DTP, Ladder Residual, or PT-Transformer, and post-training methods such as fine-tuning, preference optimization, or quantization, including at research scale.
- A top engineering school or a PhD with concrete architecture work counts, even without industry experience.
What we offer
- Direct access to AMD and NVIDIA datacenter GPUs from day one
- A team where creativity and technical judgment carry weight and where the people closest to the problem shape the key decisions
- Problems that sit on the critical path of model execution speed and that directly influence what the system can become
- A remote-friendly working model, with one mandatory week per month in our Paris office. Travel and accommodation covered by the company.
- Compensation aligned with top AI research profiles, including equity