Cloud Native PDX May: Change-driven Architecture and Serving Inference at Scale

This May is all about scaling; scaling your infrastructure management, or scaling your LLM inference serving. Join us to find out about some open source tools to make managing large modern stacks easier.

Date: Thursday, May 14 Time: 5:30–7:30 PM Location: Reperio Health, 4784 SE 17th Ave Suite 120, Portland, OR

Recording: Talks are typically recorded (opt-in by speakers)

A big thank you to Microsoft for sponsoring food & beverage, and to Reperio Health, our venue host.

Drasi, a new take on Change Driven Architectures: Aman Singh, Microsoft

Modern cloud-native systems constantly generate data changes, and applications often need to react to them. Building change-driven solutions that respond to specific changes in distributed data is challenging. This talk introduces Drasi, a CNCF Sandbox project that simplifies the design and implementation of change-driven architectures using Graph Queries and pluggable components. For example, with Drasi you can declaratively write automation to detect and respond to running containers with newly identified vulnerabilities across pods and deployments in a Kubernetes cluster. Join us for a walkthrough of real-world use cases that show how Drasi’s approach brings structure and responsiveness to complex distributed environments - without writing custom code.

Dynamo: Large Scale Distributed Inference David Zeir, Director, DL System Software, Nvidia Neelay Shah, Distinguished Engineer, Nvidia

This talk introduces Dynamo, NVIDIA's open-source Kubernetes-native distributed inference platform. We'll cover the problem space, walk through Dynamo's architecture — disaggregated prefill/decode, KV-cache-aware routing, and a transport layer that moves KV blocks directly between GPUs — and dig into the Kubernetes integration for scheduling, autoscaling, and graceful failure handling. We'll close with a demo of Dynamo serving a real workload.

Hello, language model enthusiasts and technology integration developers!

Ready to dive into app development with Large Language Models (LLMs) and other machine learning models? Eager to navigate cutting-edge technologies and learn from others' experiences?

You're in luck! Join us at our ultimate developer-focused, LLM app-building inaugural event!

• 📅 DATE: May 11th
• ⌚ TIME: 5pm - 7pm
• 🌍 LOCATION: GRAYBOX, 107 SE Washington St #700, Portland, OR

Experience an evening of LLM exploration and app development perfect for those creating or interested in LLM-based apps. Master key development concepts and integration code through knowledge-sharing and collaboration. Don't miss out!

🍕🍺 FREE Pizza and Beer: Because coding on an empty stomach is no fun!

🎤🖥️ FANTASTIC Presentations:

1. "AI for Beginners: Crafting a Chatbot to Transform Customer Service" with Chris Schuermyer (beginner-level)
2. “Summarize Long Documents with LangChain: MapReduce and Refine” with Jon Page (intermediate-level)
3. TBD: Stay tuned for our third presenter

👩‍💻👨‍💻 ONE Hour of Hands-On Experience:

Develop your LLM app or start with an API, surrounded by skilled and like-minded developers. Bring your app ideas to life!

🤔✅ JOIN US!

Don't wait! Mark your calendar, grab your laptop, and experience an unforgettable night of coding, networking, and app development!

• Your organizers, Brady & Marty

How to run a Large Lanuage Model on your PC or Laptop

Large language models are essentially large files (think GB in size!) that contain information somone can draw upon with a prompt. You've undoubtedly heard of (or used) OpenAI, which uses the ChatGPT LLM.

All OpenAI is is someone else's computer with a large language model installed. So why not do the same thing on your home computer?

For this event I'll walk through how to install an interface to chat with an AI model, download a model that works for you, and how to interface it.

This is intended to be more of a workshop, but since it takes a long time to install and download the models, this is more of a class format (it will be recorded and shared later with attendees).

After the tutorial attendees will be able to contact me with any questions about the process.

What will you need for the tutorial?

A computer. And, of course, an Internet connection. If you have a computer with a graphic card, opt for that, but you can use a local LLM on your laptop all the same.

To Make the Most of the Class

This isn't required, but if you have the technical know-how, I'd highly recommend doing the following:

• downloading (and installing) https://github.com/oobabooga/text-generation-webui for your platform.
• Once it's installed, use the "Download Models" tab to download TheBloke/dolphin-2.6-mistral-7B-GGUF (model info here: https://huggingface.co/TheBloke/dolphin-2.6-mistral-7B-GGUF)

Again, if these instructions don't make sense or you find them confusing, don't worry! I'm not expecting anyone to have done this.

Still have questions? Feel free to reach out to me on XMPP: xmpp://[email protected]

Join AI Portland for a relaxed evening of conversation and connection at our March Happy Hour! This is the perfect opportunity to unwind and mingle with fellow AI enthusiasts and professionals in a laid-back atmosphere. Share your experiences, discuss the latest AI trends, or simply enjoy the company of like-minded individuals. Food and drinks can be purchased at your leisure, so you can snack and sip while engaging in stimulating discussions about all things AI.