4th Annual MLOps World Conference on Machine Learning in Production 2023

MLOps World

A global initiative to help put more models into production, effectively, responsibly and efficiently. Incl. access to Gen AI World!

Renaissance Austin Hotel

<h3>📌 A 2-FOR-1 ticket, meaning if you're attending MLOps, you'll have full access to <a href="https://generative-ai-summit.com/#" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">Gen AI Summit</a>, and vice-versa.</h3><h3>Thank you for joining us here. </h3><h3>The goal for MLOps World is to help companies put more models into production environments, effectively, responsibly, and efficiently. Whether you're working towards a live production environment, or currently working in production, this is geared towards you on your journey.</h3><h3>We hope to see you soon. </h3><h3>- MLOps World Team</h3>

<p><strong><em>Each ticket includes;  </em></strong></p><ul><li>Access to 30+ virtual workshops to help build and deploy  (Kubernetes, etc.) - <strong>Oct 24th</strong></li><li>Access to 60+ in-person talks - <strong>Oct 25th &amp; 26th</strong></li><li>Access 90+ hours of recordings</li><li>Access to app-brain dates and networking</li><li>Network and connect through our event app </li><li>Q+A with speakers</li><li>Channels to share your work with the community</li><li>Run your chat groups and virtual gatherings!</li></ul><h2><a href="https://mlopsworld.com/" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">https://mlopsworld.com/</a></h2><p>To few companies have effective AI leaders and an effective AI strategy. </p><p>Taken from the real-life experiences of our community, the Steering Committee has selected the top applications, achievements and knowledge areas to highlight across this dynamic event.</p><p><strong>S﻿peakers and topics to be announced</strong> </p><h2>Talk Tracks include: </h2><h3>- Real World Case Studies</h3><h3>- Business &amp; Strategy</h3><h3>- Technical &amp; Research (levels 1-7)</h3><h3>- Workshops (levels 1-7)</h3><h3>- In-person coding sessions </h3><ul><p></p></ul><h2>Top Industries Served:</h2><ul><li>Technology &amp; Service</li><li>Computer Software</li><li>Banking &amp; Financial Services</li><li>Insurance</li><li>Hospital &amp; Health Care</li><li>Automotive</li><li>Telecommunications</li><li>Environmental Services</li><li>Food &amp; Beverages</li><li>Marketing &amp; Advertising</li></ul><p>We believe these events should be as accessible as possible and set our ticket passes accordingly </p><p><em>MLOps World is an international community group of practitioners trying to better understand the science of deploying ML models into live production environments, and everything both technical and non-technical that goes with it! </em></p><p><em>Created initially by the Toronto Machine Learning Society (TMLS) this initiative is intended to unite and support the wider AI Ecosystem, companies, practitioners, and academics and contributors to open-source communities operating within it.</em></p><p><em>With an explorative approach, our initiatives address the unique needs of our community of over 16,000+ ML researchers, professionals, entrepreneurs and engineers. Intended to empower its members and propel productionized ML. Our community gatherings and events attempt to re-imagining what it means to have a connected community; offering support, growth, inclusion for all participants.</em></p><p></p><h2>FAQs</h2><p><strong>Q: This a virtual or an in-person conference </strong></p><p>A portion is virtual and a portion in person (The full conference will not be completely hybrid)</p><p>- October 24th, Bonus workshops are held virtually (for ticket holders)</p><p>- October 25th &amp; 26th, Conference talks, expo, workshops, coding sessions (in person)</p><p>**If you'd like to attend virtually only, you can request a special pass by emailing info@mlopsworld.com with ONLINE ONLY PASS in the Subject Header.**</p><p><strong>Q: Which sessions are going to be recorded? When will the recordings be available and do I have access to them?</strong></p><p>Most sessions will be recorded during the event (provided speaker permissions) and will be made available to attendees approximately 2-4 weeks after the event and be available for 12 months after release.</p><p><strong>Q: Are there ID or minimum age requirements to enter the event? </strong>There is not. Everyone is welcome.</p><p><strong>Q: How can I contact the organizer with any questions? </strong>Please email info@mlopsworld.com</p><p><strong>Q: What's the refund policy? </strong>Tickets are refundable up to 30 days before the event.</p><p><strong>Q: Why should I attend ?  </strong>From over 300+ submissions the committee has selected the top sessions to help your learning. From hands-on coding workshops to case-studies, you won't find a conference gathering that packs as much information, and at such a low-cost of entry. </p><p>Come join our community and celebrate the major triumps of the year, as well as the main learning lessons. As well, aside from the sessions there will also be brain-dates via the app for networking  and evening socials that provide opportunities to meet peers and build your network. </p><p><strong>Q: Who will attend? </strong>Please see our <a href="https://mlopsworld.com/who-attends/" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">Who Attends</a> Section for a full breakdown. Participants range from data scientists, to engineers, business exectutives, and students.  We'll have multiple tracks and in app brain-dates to accommodate various vantage points and maturity levels. <a href="https://mlopsworld.com/#who-attends" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer"></a></p><p><strong>Q: Can I speak at the event?</strong></p><p>You can <a href="https://mlopsworld.com/call-for-speakers/" target="_blank" data-msys-clicktrack="0" rel="nofollow noopener noreferrer">submit an abstract here</a>. Submissions are reviewed by our committee. </p><p>*Content is non-commercial and speaking spots cannot be purchased. </p><p><strong>Q: Will you give out the attendee list? </strong>No, we do our best to ensure attendees are not inundated with messages, We allow attendees to stay in contact through our slack channel and follow-up monthly socials.</p><p><strong>Q: Can my company have a display</strong>? Yes, there will be spaces for company displays. You can inquire at faraz@mlopsworld.com</p><p>If you're unemployed, or in a difficult financial situation, but would stand to benefit from this event, please email info@mlopsworld.com and we can help you out !</p>

<h3>Getting unstuck - How not to get locked into one LLM and benefit from all best in class models!</h3><h3>3:25 PM - :25 PM</h3><p>Liran Hason, CEO &amp; Co-Founder, Aporia </p><p><strong>Business Strategy</strong></p><p><strong>Abstract: </strong>Generative AI represents an exciting new frontier, but deploying successful generative AI products comes with challenges. In this talk, we&#x27;ll examine common roadblocks organizations face and provide a framework to overcome them. By having the right approach we can accelerate time-to-market and drive impact.</p><p><strong>Speakers Bio:</strong>  Liran Hason is the Co-Founder and CEO of Aporia, the AI Performance Platform. Aporia is trusted by Fortune 500 companies and data science teams in every industry to enable responsible AI and monitor, improve, and scale AI products. Prior to founding Aporia, Liran was an ML Architect at Adallom (acquired by Microsoft), and later an investor at Vertex Ventures. Liran founded Aporia after seeing first-hand the risks of AI without guardrails. In 2022, Forbes named Aporia as the “Next Billion-Dollar Company</p><p><strong>How technical is this talk?: 3</strong>/7</p><p><strong>What You&#x27;ll Learn: </strong> The audience will learn how to navigate organizational challenges in starting a AI/ML ecosystem at a large company with deeply rooted traditions. This will include pitfalls to avoid, building for your customer, and navigating team topology that works best for your ecosystem.</p><h3></h3><h3>Generative AI: the open source way</h3><h3>1:55 PM-2:00 PM</h3><p>Andreea Munteanu, MLOps Product Manager, <strong>Canonical </strong></p><p><strong>Abstract: </strong></p><p>Generative AI is probably the topic of the year. Leaders across the world feel the pressure of missed opportunities related to the latest technology. Professionals are also left worried about the impact that latest technology will have on their roles. Data scientists and machine learning engineers are challenge and need to quickly upskill. With such a stretched picture in mind, will generative AI deliver up to the great promises that it has right now? </p><p>This lighting talk will depict the opportunities that open source gives to generative AI to accelerate innovation.  Between anxiety and enthusiasms, the latest technologies bring a new angle to the market. Let’s learn together about genAI with open source: from models to tooling to applications</p><p><strong>Speaker Bio: </strong> Andreea is a Product Manager at Canonical, leading the MLOps area. With a background in Data Science in various industries, such as retail or telecommunications, Andreea used AI techniques to enable enterprises to benefit from their initiatives and make data-driven decisions. Andreea is looking to help enterprises get started with their AI projects and then deploy them to production, using open-source, secure, stable solutions. </p><h3></h3><h3>Building an end-to-end MLOps Pipeline</h3><h3>2:00 PM-2:05 PM</h3><p>Aurimas Griciūnas, Head of Product, <strong>Neptune.ai</strong></p><p><strong>Abstract: </strong></p><p>The talk will be about MLOps and the lifecycle of ML projects. I will go through stages involved in the ML project lifecycle and some key highlights from each of them. I will also explain how CI/CD is different in Machine Learning project when compared to regular software and highlight how it evolves with the maturity of MLOps processes in the organisation. I will also ground the explanations with real life examples of building out MLOps capabilities, successes and failures.</p><p><strong>Speaker Bio: </strong> Aurimas has over a decade of work experience in various data-related fields: Data Analytics, Data Science, Machine Learning, Data Engineering, and Cloud Engineering. For a few years he also led teams working with Data and Infrastructure. Today, Aurimas is Head of Product at neptune.ai.</p><p></p><h3>Your AI applications need Guardrails: Here&#x27;s how to build them</h3><h3>2:45 PM-3:15 PM</h3><p>Shreya Rajpal, Founder, <strong>Guardrails AI</strong></p><p><strong>Which talk track does this best fit into?: </strong>Advanced/Technical Research</p><p>Abstract: Large Language Models (LLMs) such as ChatGPT have revolutionized AI applications, offering unprecedented potential for complex real-world scenarios. However, fully harnessing this potential comes with unique challenges such as model brittleness and the need for consistent, accurate outputs. These hurdles become more pronounced when developing production-grade applications that utilize LLMs as a software abstraction layer.In this talk, we will tackle these challenges head-on. We introduce Guardrails AI, an open-source platform designed to mitigate risks and enhance the safety and efficiency of LLMs. We will delve into specific techniques and advanced control mechanisms that enable developers to optimize model performance effectively. Furthermore, we will explore how implementing these safeguards can significantly improve the development process of LLMs, ultimately leading to safer, more reliable, and robust real-world AI applications. </p><p><strong>About the speaker: </strong>Shreya Rajpal is the creator and maintainer of Guardrails AI, an open source platform developed to ensure increased safety, reliability, and robustness of large language models in real-world applications. Her expertise spans a decade in the field of machine learning and AI. Most recently, she was the founding engineer at Predibase, where she led the ML infrastructure team. In earlier roles, she was part of the cross-functional ML team within Apple&#x27;s Special Projects Group and developed computer vision models for autonomous driving perception systems at Drive.ai. </p><p><strong>How technical is this talk?:</strong> 4/7</p><p><strong>What You&#x27;ll Learn:</strong> The audience will learn about the challenges and risks associated with Large Language Models (LLMs) and how the open-source platform, Guardrails AI, addresses these issues by providing specific techniques and advanced control mechanisms to optimize model performance, leading to safer, more reliable, and robust real-world AI applications.</p><p><strong>Why is this takeaway(s) important?:</strong> It demonstrates how guardrails can provide developers with tools to optimize model performance, resulting in safer and more reliable real-world AI applications.</p><p><strong>What is unique about this talk: </strong>This talk will cover implementing end-to-end guardrails and safety interventions for Large Language Models (LLMs). These measures will promote a comprehensive approach to evaluating safety-first LLM development.</p><p></p><h3>MLOps for Graph-Based Recommender Systems: Orchestrating Intelligent Connections</h3><h3>12:05 PM-12:35 PM</h3><p>Aishwarya Reganti, Applied Scientist<strong>, Amazon</strong></p><p><strong>Advanced Technical/Research</strong></p><p><strong>Abstract: </strong>As graph-based recommender systems continue to gain prominence in large tech companies, the need for efficient management of machine learning operations (MLOps) becomes paramount. This talk highlights the specific challenges faced in operationalizing graph-based recommender systems and how MLOps addresses these complexities. We delve into techniques for efficient data ingestion, feature engineering, and model training within the graph context. We emphasize the significance of versioning and managing graph models, ensuring reproducibility and scalability in the MLOps pipeline.</p><p><strong>Speakers Bio: </strong>Aishwarya is an Applied Scientist in the Amazon Search Science and AI Org. She works on developing large scale graph-based ML techniques that improve Amazon Search Quality, Trust and Recommendations. She obtained her Master&#x27;s degree in Computer Science (MCDS) from Carnegie Mellon&#x27;s Language Technology Institute, Pittsburgh. She has over 6+ years of hands-on Machine Learning experience and 20+ publications in top-tier conferences like AAAI, ACL, CVPR, NeurIPS, EACL e.t.c. She has worked on a wide spectrum of problems that involve Large Scale Graph Neural Networks, Machine Translation, Multimodal Summarization, Social Media and Social Networks, Human Centric ML, Artificial Social Intelligence, Code-Mixing e.t.c. She has mentored several Masters and PhD students in the aforementioned areas. She has also served as a reviewer in various NLP and Graph ML conferences like ACL, EMNLP, AAAI, LoG e.t.c. She has worked with some of the best minds in both academia and industry through collaborations and internships in Microsoft Research, University of Michigan, NTU Singapore, IIIT-Delhi, NTNU-Norway, University of South Carolina e.t.c.</p><p><strong>Have you given this talk before? </strong>No</p><p><strong>What are some of the infrastructure you plan to discuss?: </strong>Distributed Graph Training using AWS Clusters, CPU-GPU training using optimal AWS machines</p><p><strong>What are some of the languages you plan to discuss? </strong>PyTorch, DistDGL</p><p><strong>What kind of Dev ops tools you plan to discuss? Open source? </strong>None</p><p><strong>How technical is this talk? </strong>4</p><p><strong>What You&#x27;ll Learn: </strong>The audience will learn about the unique advantages and challenges of implementing graph-based MLOps in recommender systems. They will gain insights into how graph structures enable more accurate recommendations. The talk will cover techniques for data preprocessing, feature engineering, and model training specific to graph data using the DGL library. Overall, attendees will gain a comprehensive understanding of the transformative potential of graph-based MLOps in building and managing recommender systems.</p><p><strong>Why is this takeaway(s) important?  </strong>Graph-based recommender systems present unique operational challenges that differ from traditional Euclidian data. Most available MLOPs tools do not account for graph structure making it extremely cumbersome for scientists and engineers to build graph models, this talk will go over graph specific challenges and provide modular and scalable solutions for the same.</p><p><strong>What is unique about this talk: </strong>Graph-based recommender systems represent a novel product and research domain that I have been immersed in for approximately three years while working at Amazon. It is worth noting that many companies are yet to establish formal methodologies for effectively managing this area, given its relative newness. As a result, my experience, primarily gained at Amazon, offers unique insights that are not readily available in public resources or online platforms. </p><h3></h3><h3></h3><h3></h3><h3>The Evolution of ML Monitoring in Production: From ML 1.0 to LLMs</h3><h3>1:35 PM - 1:40 PM</h3><p>Gon Rappaport, MLOps Solutions Architect, Aporia</p><p><strong>Abstract:</strong>  In this talk, we’ll explore the landscape of ML Monitoring in production, highlighting best practices for tracking real-world AI products like Recommender Systems and Fraud Detection models (“ML 1.0”) and LLM-based applications such as Chatbots. We’ll further explore the intricacies of LLMs, focusing on monitoring applications that use Retrieval Augmented Generation (enriching LLMs with external insights), LLMs serving as controllers capable of activating external APIs or other ML models, as well as LLM Guardrails.</p><p><strong>Speakers Bio: </strong>Gon Rappaport is an MLOps Solutions Architect at Aporia, the AI Performance Platform, providing Observability and Guardrails to drive responsible, high-performing AI products. A software engineer at heart, Gon works closely with ML teams in every industry, seamlessly blending his knack for meme-based executive reports with his commitment to overcoming the visible and hidden challenges of machine learning models in production.</p><h3>LLMs from hallucinations to relevant responses</h3><h3>1:40 PM - 1:45 PM</h3><p>Eddie Mattia, Data Scientist, Outerbounds</p><p><strong>Abstract:</strong>  Present a taxonomy of methods for controlling LLMs. Listeners will learn the broad strokes of how apps based on retrieval-augmented generation (RAG) and instruction tuning work and where they fit into the big picture of generative AI. We focus on how these techniques can be used to make generated responses more relevant.</p><p><strong>Speakers Bio: </strong>Eddie Mattia is a data scientist working on Metaflow and foundation models at Outerbounds. He began using Python to teach applied math in grad school. Since then, Eddie has worked at startups and at Intel building machine learning software.</p><h3>Supporting Community Competitions to Develop LLMs</h3><h3>1:50 PM - 1:55 PM</h3><p>Rick Izzo, Tech Lead, Lighting AI</p><p><strong>Abstract:</strong>  Present a taxonomy of methods for controlling LLMs. Listeners will learn the broad strokes of how apps based on retrieval-augmented generation (RAG) and instruction tuning work and where they fit into the big picture of generative AI. We focus on how these techniques can be used to make generated responses more relevant.</p><p><strong>Speakers Bio: </strong>Eddie Mattia is a data scientist working on Metaflow and foundation models at Outerbounds. He began using Python to teach applied math in grad school. Since then, Eddie has worked at startups and at Intel building machine learning software.</p><h3>LLMs, Big Data, and Audio: Breaching an Untapped Gold Mine</h3><h3>2:00 PM - 2:05 PM</h3><p>Jose Nicholas Francisco, ML Developer Advocate, Deepgram</p><p><strong>Abstract:</strong>  Present a taxonomy of methods for controlling LLMs. Listeners will learn the broad strokes of how apps based on retrieval-augmented generation (RAG) and instruction tuning work and where they fit into the big picture of generative AI. We focus on how these techniques can be used to make generated responses more relevant.</p><p><strong>Speakers Bio: </strong>Eddie Mattia is a data scientist working on Metaflow and foundation models at Outerbounds. He began using Python to teach applied math in grad school. Since then, Eddie has worked at startups and at Intel building machine learning software.</p><p></p><p></p><h3>Your AI applications need Guardrails: Here&#x27;s how to build them </h3><h3>2:45 PM-3:15 PM</h3><p>Shreya Rajpal, Founder, Guardrails AI</p><p><strong>Advanced/Technical Research</strong></p><p><strong>Abstract</strong>: Large Language Models (LLMs) such as ChatGPT have revolutionized AI applications, offering unprecedented potential for complex real-world scenarios. However, fully harnessing this potential comes with unique challenges such as model brittleness and the need for consistent, accurate outputs. These hurdles become more pronounced when developing production-grade applications that utilize LLMs as a software abstraction layer.In this talk, we will tackle these challenges head-on. We introduce Guardrails AI, an open-source platform designed to mitigate risks and enhance the safety and efficiency of LLMs. We will delve into specific techniques and advanced control mechanisms that enable developers to optimize model performance effectively. Furthermore, we will explore how implementing these safeguards can significantly improve the development process of LLMs, ultimately leading to safer, more reliable, and robust real-world AI applications. </p><p><strong>Speakers Bio: </strong> Shreya Rajpal is the creator and maintainer of Guardrails AI, an open source platform developed to ensure increased safety, reliability, and robustness of large language models in real-world applications. Her expertise spans a decade in the field of machine learning and AI. Most recently, she was the founding engineer at Predibase, where she led the ML infrastructure team. In earlier roles, she was part of the cross-functional ML team within Apple&#x27;s Special Projects Group and developed computer vision models for autonomous driving perception systems at Drive.ai.</p><p><strong>How technical is this talk?</strong> 4/7</p><p><strong>What You&#x27;ll Learn: </strong> The audience will learn about the challenges and risks associated with Large Language Models (LLMs) and how the open-source platform, Guardrails AI, addresses these issues by providing specific techniques and advanced control mechanisms to optimize model performance, leading to safer, more reliable, and robust real-world AI applications.</p><p><strong>Why is this takeaway(s) important?</strong>  It demonstrates how guardrails can provide developers with tools to optimize model performance, resulting in safer and more reliable real-world AI applications.</p><p><strong>What is unique about this talk: </strong> This talk will cover implementing end-to-end guardrails and safety interventions for Large Language Models (LLMs). These measures will promote a comprehensive approach to evaluating safety-first LLM development.</p><h3>Build your own ChatGPT with open source tooling</h3><h3>3:45 PM-4:45 PM</h3><p>Andreea Munteanu, MLOps product manager, Canonical</p><p>Workshop</p><p><strong>Abstract</strong>:  LLMs are gaining huge popularity with projects such ChatGPT, LLaMA or PaLM being open to everyone. Yet, enterprises feel overwhelmed by the large number of applications that requires, at first sight, a complete reshuffle of the existing infrastructure, in order to accommodate the needs for powerful cloud computing.Depending on the industry, there are various use cases, as well as legacy architectures in place. Generative AI, however, can be deployed on any environment, whether it ia a public or private cloud. From the very beginning, when doing estimations of the cluster size, to upper layers in the stack where inference infrastructure is considered, there are a bunch of key factors such as GPU types, MLOps tooling or artefact types that influence the infrastructure of the project.This talk will cover how you can build your infrastructure for a generative AI project, with a focus on building your own conversational assistant. It will go through the entire stack, including hardware and software applications, that cover the entire machine learning lifecycle, focusing on open source tooling and models. The session will feature a case study by a finserv company that build their own chatbot, using their data. After this presentation, you will understand how you can build and automate your infrastructure for a genAI project, using open source tooling.</p><p><strong>Speakers Bio: </strong>  I am a Product Manager at Canonical, leading the MLOps area. With a background in Data Science in various industries, such as retail or telecommunications, I used AI techniques to enable enterprises to benefit from their initiatives and make data-driven decisions. I am looking to help enterprises get started with their AI projects and then deploy them to production, using open-source, secure, stable solutions. I am a driven professional, passionate about machine learning and open source. I always look for opportunities to improve, both myself and people within the teams that I am part of. I enjoy sharing my knowledge, mentoring young professionals and having an educational impact in the industry. </p><p><strong>How technical is this talk?</strong> 5/7</p><p><strong>What You&#x27;ll Learn: </strong>Fine tuning LLMs using open source tooling</p>

Oct 24

With the emergence of IaC (infrastructure as code) tools, we have seen GitOps become an increasingly popular DevOps pattern that facilitates automation, reproducibility, and security. While hugely beneficial, applying the same principles in MLOps is not straightforward due to the specific aspects of the field such as the need to work with large amounts of data and the experimental nature of ML development. In this talk, we will see how we can bridge these gaps by using tools such as DVC. Step by step, we will create an end-to-end MLOps pipeline which is centered around the git repository as its single source of truth.

Applying GitOps principles at every step of an E2E MLOps project

In this workshop, attendees will learn about methods for working with LLMs. Our stories will be guided by examples you can run on your laptop or in a (free) hosted cloud environment provided to attendees. Developers will expand their awareness of how researchers and product designers are working with LLMs, with emphasis on connecting high-level concepts such as fine-tuning and vector databases to the fundamental math and APIs data scientists should understand. Business-minded executives can either get hands - on or follow the higher-level stories to deepen their sense of what is possible with LLMs, the technicalities behind risks they introduce, and how they fit into the arc of ML. The primary value of this workshop will be as a guide to help teams set reasonable goals in the complex and fast-moving world of LLMs, and understand what you need to successfully support your team’s next LLM projects.

LLMs in practice: a guide to recent techniques and trends

Serving a machine learning model is not particularly easy, especially if we add two or three models in parallel to the mix, in which case, a single model deployment recipe might start to crumble. To tackle the challenges around serving individual or multiple models in production, we have handy tools like MLServer and Seldon Core. The former is a python library that allows us create machine learning microservices with one or multiple models in the same service, and the latter allows us to build simple-to-complex inference graphs that can help us handle A/B testing, shadow and canary deployment, feature transformations, and model monitoring. If you want to learn how to use open-source tools to build microservices based on your different use cases and model recipes, come and join this hands-on workshop and get started with several of the key steps in the machine learning workflow as we walk through fun examples from the broader music industry.

Introduction to Building ML Microservices

Today, foundation LLMs can only be trained by a handful of organizations possessing the compute resources required to pre-train models with more than a hundred billion parameters over internet-scale data. These foundation models are then fine-tuned by the wider ML community for specific applications. Even though fine-tuning can be more compute and memory efficient than full parameter tuning, a significant challenge to fine-tuning is provisioning the appropriate infrastructure. In this session, Niels will demonstrate how to use Flyte, a Linux Foundation open-source orchestration platform to fine-tune a LLM on the Flyte codebase itself ?. Flyte allows for the declarative specification of the infrastructure needed for a broad range of ML workloads, including fine-tuning LLMs with limited resources by leveraging multi-node, multi-gpu distributed training.

Learn Your Codebase: Fine-tuning CodeLlama with Flyte… to Learn Flyte

Generative AI models are all the hype nowadays, but how do you actually deploy them in a scalable way? In this talk we will discuss best practices when moving models to production, as well as show an interactive example of how to deploy one using UbiOps. UbiOps is a serverless and cloud agnostic platform for AI & ML models, built to help data science teams run and scale models in production. We will pay special attention to typical hurdles encountered in deploying (generative) AI models at scale. Python knowledge is all you need for following along!

Deploying generative AI models: best practices and an interactive example

While there's a lot of work done on defining the risks, goals, and policies for Responsible AI, less is known about what you can apply today to build safe, fair, and reliable models. This session introduces open-source tools and examples of using them in real-world projects - to address three common challenges.The first is robustness - testing and improving a model's ability to handle accidental or intentional minor changes in input that can uncover model fragility and failure points. The third is bias - testing that a model performs equally across gender, age, race, ethnicity, or other population groups. The third is data leakage, in combination with leakage caused by using personally identifiable information in training data. The open-source LangTest library is used to demonstrate how to generate tests, run tests, augment data, and integrate these evaluations into MLOps workflows.This session is intended for data science practitioners and leaders who need to know what they can do tod

Applying Responsible AI with the Open-Source LangTest Library

Oct 25

MLOps, as an engineering discipline, has been widely adopted by enterprises to productionalize ML applications. Generative AI and LLMs can bring transformative values to businesses, but is there an emerging stack that can effectively support and operationalize LLM applications? Can MLOps take on those additional responsibilities? This presentation first argues that the current form of MLOps is insufficient in supporting LLM applications by highlighting the unique set of challenges they bring. These challenges include: The ambiguity inherent in interacting with LLMs in natural languages The diverse needs of different types of LLM applications The new risks associated with incorporating LLM into customer-facing applications. The presentation will then conclude by providing an overview of an emerging stack called LLMOps.

LLMOps: An emerging stack to productionalize LLM Applications

Model Inference at Capital One as across the Fintech sector is a key aspect of the Model Development Lifecycle. In order to reap the benefits of machine learning models trained by Data Scientists, we are required to deploy said models in a production environment. This enables critical business applications, such as credit decisions or fraud detection. Increasingly, we are faced with demanding non-functional requirements for high resilience and low latency service response leading us to invest into service oriented architectures for model inference. Seldon has emerged over the last years a key solution to address these challenges. In this presentation we will take a close look at the recently released Seldon V2 and our findings for its application for financial services at enterprise scale, comparing it to its V1 specification. We will summarize our findings as they relate to the benefits of novel improvements as well as challenges we see in establishing much needed controls to establis

Low-latency Model Inference in Finance 

MLOps is used in various organizations that operate on very sensitive datasets. For instance, pharmaceutical and life science companies handle human DNA samples, healthcare institutions training models on patient data, or highly regulated environments like telecom and financial companies. End users, machine learning engineers or data scientists can be concerned about cloud-native workloads would expose them more to software vulnerabilities, data leaks, or any lack of data protection measures. In reality, it's just the opposite. This presentation will cover how you can improve compliance and security with features like Kubernetes strict confinement, blockchain-based tokenization and privacy-enhancing technologies like confidential computing. The talk will feature a case study by a life sciences company that created customized treatments using these technology building blocks. After the talk, you will understand how you can apply them yourself on cloud envorinments MLOps using Kubeflow.

MLOps on Highly Sensitive Data - Strict Confinement, Confidential Computing 

In a world where Generative AI is shaping enterprise innovation, this fireside chat explores the integration of reasoning, logic, and cutting-edge techniques like Language Models (LLM) and mixture of experts tactics. Led by Reuven Cohen, CTO of Generative AI at EY Americas, the discussion uncovers how LLM fuels new chain-of-thought processes, while the mixture of experts approach optimizes decision-making within businesses. Join us for a thought-provoking conversation that delves into the real-world applications of these technologies, offering valuable insights into their transformative impact on modern enterprises. This talk is a must for anyone interested in the intersection of human intellect, artificial intelligence, and the future of business innovation.""" Speakers Bio: "Reuven Cohen is an influential technology leader with over three decades of experience, particularly recognized as the CTO, Generative AI at EY Americas. In this role, he is spearheading the Generative AI strateg

Unlocking Potential: A Fireside Chat on Generative AI, Reasoning, and Logic 

Stitch Fix utilizes a sophisticated multi-tiered recommender system stack, encompassing feature generation, scoring, ranking, and business policy decision-making. This presentation delves into the training architecture of the scoring model, a deep learning model that predicts the likelihood of a user purchasing an item. I will walk through our journey, detailing the transition from training on a single GPU to leveraging multiple GPUs through pytorch's Distributed Data Parallel (DDP) strategy. Additionally, I will share empirical results highlighting the efficiency of GPU utilization as we scale up with DDP across multiple GPUs.

Supercharging Recommender Systems: Unleashing the Power of Distributed Model T

As graph-based recommender systems continue to gain prominence in large tech companies, the need for efficient management of machine learning operations (MLOps) becomes paramount. This talk highlights the specific challenges faced in operationalizing graph-based recommender systems and how MLOps addresses these complexities. We delve into techniques for efficient data ingestion, feature engineering, and model training within the graph context. We emphasize the significance of versioning and managing graph models, ensuring reproducibility and scalability in the MLOps pipeline.

MLOps for Graph-Based Recommender Systems: Orchestrating Intelligent Connectio

Large language models (LLMs) represent an exciting trend in AI, with many new commercial and open-source models released recently. However, selecting the right LLM for your needs has become increasingly complex. This tutorial provides data scientists and machine learning engineers with practical tools and best practices for evaluating and choosing LLMs.The tutorial will cover the existing research on the capabilities of LLMs versus small traditional ML models. If an LLM is the best solution, the tutorial covers several techniques, including evaluation suites like the EleutherAI Harness, head-to-head competition approaches, and using LLMs for evaluating other LLMs. The tutorial will also touch on subtle factors that affect evaluation, including role of prompts, tokenization, and requirements for factual accuracy. Finally, a discussion of model bias and ethics will be integrated into the working examples.Attendees will gain an in-depth understanding of LLM evaluation tradeoffs and method

Evaluation Techniques for Large Language Models

In applied ML projects, experienced data scientists know that improving data brings higher ROI than tinkering with models. However the process of finding and fixing problems in a dataset is highly manual (ad hoc ideas explored in Jupyter notebooks). Cleanlab develops open-source software to help make this process more: efficient (via novel algorithms that automatically detect certain issues in data) and systematic (with better coverage to detect different types of issues). This talk will describe how high-level ideas from data-centric AI can be operationalized across a wide variety of datasets (image, text, tabular, etc). I will introduce novel algorithmic strategies to automatically identify various issues in data that we have researched and published papers on with extensive benchmarks. These include detection of label errors, bad data annotators, out-of-distribution examples, and other dataset problems that once identified can be easily addressed to significantly improve trained mo

Operationalizing data-centric AI: Practical algorithms + software 

In this talk, we’ll explore the landscape of ML Monitoring in production, highlighting best practices for tracking real-world AI products like Recommender Systems and Fraud Detection models (“ML 1.0”) and LLM-based applications such as Chatbots. We’ll further explore the intricacies of LLMs, focusing on monitoring applications that use Retrieval Augmented Generation (enriching LLMs with external insights), LLMs serving as controllers capable of activating external APIs or other ML models, as well as LLM Guardrails.

The Evolution of ML Monitoring in Production: From ML 1.0 to LLMs

Present a taxonomy of methods for controlling LLMs. Listeners will learn the broad strokes of how apps based on retrieval-augmented generation (RAG) and instruction tuning work and where they fit into the big picture of generative AI. We focus on how these techniques can be used to make generated responses more relevant.

LLMs from hallucinations to relevant responses

Lightning AI Supported the NEURIPS LLM Challenge by providing the base LIT-GPT model upon which contestants were asked to improve training of & finetune. This talk describes what it's like to support a supporting a community competition focused on llm efficiency; From setting up the leaderboards, to creating a model challenge that allows participants to demonstrate that you can meaningfully finetune a model on a single GPU (using quantization) in one day, and get to the best evaluation possible.

Supporting Community Competitions to Develop LLMs

Large Language Models (LLMs) offer a new machine learning interaction paradigm: in-context learning. This approach is clearly much better than approaches that rely on explicit labelled data for a wide variety of generative tasks (e.g. summarisation, question answering, paraphrasing). In-context learning can also be applied to predictive tasks such as text categorization and entity recognition, with few or no labelled exemplars. But how does in-context learning actually compare to supervised approaches on those tasks? The key advantage is you need less data, but how many labelled examples do you need on different problems before a BERT-sized model can beat GPT4 in accuracy? The answer might surprise you: models with fewer than 1b parameters are actually very good at classic predictive NLP, while in-context learning struggles on many problem shapes --- especially tasks with many labels or that require structured prediction. Methods of improving in-context learning accuracy

How many Labelled Examples do you need for a BERT-sized Model to Beat GPT4 on 

Almost a year ago, with the introduction of ChatGPT and, subsequently GPT-4, the sphere of machine learning transformed completely. These advancements and LLMs unlocked the capability to address previously unsolvable problems. In this talk, Hannes explains how Digits’ machine learning team has adapted to the new world of LLMs, how their MLOps processes have changed, and the team’s learning around fine-tuning and deploying LLMs as part of a small, highly focused ML team. He will also discuss the key questions you must ask to determine if you should fine-tune open-source LLMs or integrate with a 3rd party API, the challenges and ethical concerns of using advanced language models via APIs, and how these risks in your projects can be mitigated through engineering.

Fine-tune LLMs or Integrate 3rd party APIs? A financial Case-study

Large Language Models (LLMs) such as ChatGPT have revolutionized AI applications, offering unprecedented potential for complex real-world scenarios. However, fully harnessing this potential comes with unique challenges such as model brittleness and the need for consistent, accurate outputs. These hurdles become more pronounced when developing production-grade applications that utilize LLMs as a software abstraction layer.In this talk, we will tackle these challenges head-on. We introduce Guardrails AI, an open-source platform designed to mitigate risks and enhance the safety and efficiency of LLMs. We will delve into specific techniques and advanced control mechanisms that enable developers to optimize model performance effectively. Furthermore, we will explore how implementing these safeguards can significantly improve the development process of LLMs, ultimately leading to safer, more reliable, and robust real-world AI applications.

Your AI applications need Guardrails: Here's how to build them

"As AI rapidly evolves, organizations must keep pace by iterating on ML models quickly. In the MLOps field, we face several challenges that make it hard to prototype and deploy quickly. In this presentation, we share how we leverage a mix of open-source tools and our own solutions at Dropbox to achieve faster iteration speeds, reducing prototyping and deployment times from weeks to under an hour. The session will explore how to establish an end-to-end system to achieve rapid prototyping and deployment while maintaining security best practices. Attendees will gain insights into the integration of open source frameworks, such as TorchServe, Triton, KServe, and Kubernetes, along with internal tools to accelerate the process. The discussion will also delve into the challenges of iterating on ML models and their deployments and explore strategies for optimizing resource allocation, managing dependencies, and automating deployment processes. "

 From Prototype to Product: Rapid iteration and ML model deployment at Dropbox

Model Drift monitoring is a critical activity in the MLOps life cycle. Models in production may drift and decay due to several reasons. Identifying drift quickly and taking corrective action is critical for continued success of models. A severe limitation in drift monitoring in a number of cases, is the lack of true labels for production data. How do we monitor drift when true labels are not available? We built a technique called Scouter models, to identify concept drift, when true labels are not available. In this talk, we will talk about what a scouter model is, how to identify the right scouter feature and best practices in building and managing such models. This would be of great interest to data scientists and MLOps engineers.

Using Scouter Models to monitor Model Drift - A novel approach

Large language models like those in the GPT and Llama series are primarily trained on massive amounts of *text* data. However, the vast majority of language and communication doesn't take place over text, but rather through voice. Cues in vocal tone carry information that the plaintext cannot convey—think about the last time you've witnessed or experienced a miscommunication over text/email/Slack. Thus, in this talk, I argue that training language models on audio data is the next step to improving them. Then, I'll propose a way of integrating audio data with text data in a larger dataset that can then be used for training various LLMs.

LLMs, Big Data, and Audio: Breaching an Untapped Gold Mine

Oct 26

In the rapidly evolving landscape of GenAI, large US enterprises face unique challenges when considering its implementation. Beyond the well-acknowledged concerns of data privacy, security, bias, and regulatory compliance, our journey in executing GenAI within mission-critical applications has revealed additional complexities. In this session we will walk through a number of real examples of failed implementations and the lessons learned from them.

Lessons Learned from Implementing GenAI at Large Enterprises

We'll start by breaking down the machine learning development lifecycle into experimentation (design + develop) and production (deploy + iterate). We'll walk through the best practices for developing and executing ML workloads and iteratively build a production workflow around it (manual to CI/CD to continual learning). We'll also take a look at the biggest obstacles in the way of taking machine learning in production (standardization, integrations, scaling workloads in Python, dev to prod transition, reliability, etc.) While our specific use case will be fine-tuning an LLM for a supervised NLP use case, all the content will easily extend to any algorithm (regression to LLMs), application (NLP, CV, tabular, etc.), tooling stacks and scales.

End-to-end Production Machine Learning Workflows

Tenstorrent is an innovative hardware architecture designed to enhance the efficiency and scalability of artificial intelligence (AI) workloads. Addressing the compute-intensive nature of modern AI applications, Tenstorrent leverages a unique grid-based architecture to enable efficient execution of both sparse and dense computations. Its dynamic code generation and execution capabilities allow flexibility across various AI models and algorithms. Furthermore, Tenstorrent emphasizes scalability, with the potential to deploy individual units in massive, interconnected networks for larger AI tasks. The system is designed to optimize power efficiency and performance, making it an ideal solution for advanced machine-learning jobs. Architectures like Tenstorrent will be crucial in harnessing its full potential as AI drives technological progress.

Building Computers for AI

Discover how Instacart's search journey has been revolutionized through the implementation of Language Models (LLMs). By leveraging the power of LLMs, we have achieved significant enhancements, transforming the search experience for Instacart users. Join us to discover real-world use cases, gain insights into our seamless integration strategies, and witness how LLMs have empowered us to overcome challenges, deliver personalized recommendations, and elevate the overall search experience at Instacart. [An example - https://techcrunch.com/2023/05/31/instacart-in-app-ai-search-tool-powered-by-chatgpt/]

Supercharging Search with LLMs: The Instacart Journey

As artificial intelligence (AI) continues to reshape industries and drive transformative advancements, the need for accelerated AI innovation has become paramount. Organizations across the globe are seeking novel ways to leverage AI technologies to gain a competitive edge, optimize processes, and deliver enhanced customer experiences. However, as AI applications grow in complexity and scale, so do the challenges related to scalability and security. This talk aims to address the critical aspects of accelerating Enterprise AI innovation and will delve into the following key topics: - Harnessing the Power of PyTorch and PyTorch Lightning: We will explore the benefits of modern AI frameworks, such as PyTorch, PyTorch Lightning, and Fabric, and the techniques that facilitate rapid development and deployment of AI solutions. - Scalability: The exponential growth of data and increasing demands for AI applications necessitate scalable architectures. We will discuss strategies for designing

Gen AI and Lightning: Accelerating AI Innovation while Ensuring Scalability an

Ryan Turner will be presenting on the various pitfalls of productionizing ML models. Common issues include dependency management, correct testing processes, and over reliance on Python notebooks. There will be no shortage of satire and metaphor. The talk will draw upon the collective experience of several ML engineers at DVC.

How NOT to get ML Models into production

We built a Large Language Model to generate memes. We show you how by constructing your own Large Language Model using an open-source model as the foundation enables you to retain complete control over your data and AI application.

MemeGPT: Creating a Large Language Model to Generate Memes

Feature Store is a misnomer. Their objective is not to simply store feature values but rather to facilitate the organization, collaboration, versioning, discovery, and serving of feature definitions. In this Ignite talk, we'll break down the actual goal of feature stores, their value, and where they fit into the MLOps stack.

Feature Store are NOT about Storing Features

"Generative AI is probably the topic of the year. Leaders across the world feel the pressure of missed opportunities related to the latest technology. Professionals are also left worried about the impact that latest technology will have on their roles. Data scientists and machine learning engineers are challenge and need to quickly upskill. With such a stretched picture in mind, will generative AI deliver up to the great promises that it has right now? This lighting talk will depict the opportunities that open source gives to generative AI to accelerate innovation. Between anxiety and enthusiasms, the latest technologies bring a new angle to the market. Let’s learn together about genAI with open source: from models to tooling to applications"

Generative AI: the open source way

The talk will be about MLOps and the lifecycle of ML projects. I will go through stages involved in the ML project lifecycle and some key highlights from each of them. I will also explain how CI/CD is different in Machine Learning project when compared to regular software and highlight how it evolves with the maturity of MLOps processes in the organisation. I will also ground the explanations with real life examples of building out MLOps capabilities, successes and failures.

Building an end-to-end MLOps Pipeline

Training large AI language models is a challenging task that requires a deep understanding of natural language processing, machine learning, and distributed computing. In this talk, we will go over lessons learned from training models with billions of parameters across hundreds of GPUs. We will discuss the challenges of handling massive amounts of data, designing effective model architectures, optimizing training procedures, and managing computational resources. This talk is suitable for ML researchers, practitioners, and anyone curious about the "sausage making" behind training large language models.

Training large language models: lessons from the trenches

Come get a first hand account of how NASA is leveraging Generative Design to reduce the cost and increase the performance of spaceflight missions. Also, how the concept of AI Prompt Engineering can be practically applied to diverse fields, such as structures development.

4th Annual MLOps World Conference on Machine Learning in Production 2023

More events from MLOps World

Discover more events from MLOps World, from Science & Tech to other experiences you might love.

Still looking for the right event?

Explore all events in Austin and filter by date, category, and more to find the perfect fit.

4th Annual MLOps World Conference on Machine Learning in Production 2023

More events from MLOps World

Discover more events from MLOps World, from Science & Tech to other experiences you might love.

You might also like...

Browse more events with different dates, prices, and formats to find your next great experience.

Still looking for the right event?

Explore all events in Austin and filter by date, category, and more to find the perfect fit.