12-08, 12:00–12:30 (UTC), LLM Track
The landscape of Large Language Models (LLMs) has expanded rapidly, offering users a diverse range of options for text generation and analysis. However, the cost associated with utilizing these LLMs can turn out to be very expensive. During this presentation, I will delve into practical strategies aimed at achieving a delicate balance: reducing inference costs while simultaneously elevating model performance, enhancing quality, and optimizing latency. Additionally, I will discuss essential architectural principles for constructing LLM-based systems and products, alongside pragmatic methodologies to fine-tune open-source LLM models, enhancing their performance in specific use-cases. I will also explore some practical evaluation methods for benchmarking models against baseline standards, delve into embedding techniques for precise query classification, and unravel the intricacies of shot-prompting strategies to bolster adaptability to unfamiliar data.
The talk will be delivered in two parts:
- Architecting LLM-Based Systems: I will delve into the underlying patterns and principles essential for constructing robust LLM-driven systems and products. Understanding these architectural foundations can significantly enhance the efficacy of LLM integration. (15 minutes)
- Key Strategies for optimizing latency, quality and cost - Strategies to efficiently reduce costs and latency by directing requests to the most suitable models based on user preferences and feedback, while also improving response quality through the evaluation of user interactions using the most appropriate LLMs. (15 minutes)
No previous knowledge expected
Shashank is Data Sciences leader with diverse experience across verticals including Telecom, CPG, Retail, Hitech and E-commerce domains. He is the founder of Gen AI focused startup AIOrdinate and is building a state of the art industry first LLM Ops platform which will enable secured deployment of LLM applications and products at a fraction of cost of individual LLMs with very low latency and high accuracy. In the past, he has worked in VMware, Amazon, Flipkart, Subex and Target and has been involved in solving various complex business problems using Machine Learning and Deep Learning. He has been part of the program committee of several international conferences like ICDM and MLDM and was selected as a mentor in Global Datathon 2018 organized by Data Sciences Society. He has multiple patents and publications in the field of artificial intelligence, machine learning, deep learning and image recognition in several international journals of repute to his credit. He has spoken at many summits and conferences like PyData Global, APAC Data Innovation Summit, Big Data Lake Summit, PlugIn etc. He has also published three open-source libraries on Python and is an active contributor to the global Python community.