12-08, 13:00–13:30 (UTC), LLM Track
This talk examines using open-source LLMs for real-world purposes. It compares the benefits and drawbacks of open-source LLMs to proprietary options like OpenAI. The discussion covers the economics of hosting open-source LLMs, highlights serving frameworks, explores cloud GPU availability, and gives an overview of key open-source LLMs.
The talk will cover:
1. Pros and cons of open-source LLMs vs OpenAI and alike
2. Basic economics of hosting open-source LLMs
3. Open-source serving frameworks
4. State of cloud GPU
5. An overview of main open-source LLMs
No previous knowledge expected
Andrey is the core contributor to dstack, an open-source framework for running LLM workloads across any clouds. Before dstack, Andrey worked at JetBrains as a PM for PyCharm and other dev tools.