Karanveer Benipal

Nvidia recently dropped an awesome blog post about shrinking LLMs using some cool new tricks. Instead of the usual precision quantization stuff, they're playing around with pruning and knowledge distillation to cut the model size in half. Beginner friendly definition: Pruning: Removing unnecessary parameters and layers that contribute less to model performance. Distillation: Using a larger, more capable model to 'teach' a smaller model, transferring complex knowledge in a simplified manner ( spoon feeding) By combining these techniques, NVIDIA created a 4B parameter model (Llama-3.1-Minitron 4B) from the original 8B model. Why is this important ? - It provides a rare, detailed optimization cookbook for LLM reduction. ( I have never seen one before, have you?) Now it opens up amazing opportunities for people to shrink models even further. We might soon see 100M parameter models created for specific tasks by anyone who wants to give it a shot. These tiny models could run on pretty much any device you can think of - kinda like how we've got a phone app for everything these days!! I don't want to go into more details here but the blog contains more information on: How they decided which parts of the model to remove The methods used to train the smaller model Comparisons of performance with other models. If you're curious about the nitty-gritty, it's definitely worth a read! Head over to https://lnkd.in/gZWQfKyi Thanks Saurav Muralidharan NVIDIA for the contribution :) HF main page with model card and paper : https://lnkd.in/gGevxbav Feel free to explore and let me know whats your take on these smaller models and of course shoot some questions if you need better explanation on any of the terms used! #llama3 #nvidia

33

5 Comments

🌟 Building Retrieval-Augmented Generation (RAG) Systems with LLMs and LangChain 🌟 I've been diving into how Large Language Models (LLMs) like GPT, Cohere, and Ollama, along with vector databases such as Pinecone and Qdrant, can create highly accurate and context-aware AI systems using LangChain. This innovative approach is known as Retrieval-Augmented Generation (RAG). 🔑 What is RAG? RAG is an AI architecture that enhances text generation by integrating information retrieval. It retrieves relevant data from external sources through a vector database, enriching the text generation process to ensure more informed and precise responses from LLMs. ⚙️ Steps to Build a RAG System: Create Embeddings: Use LLMs like GPT or Cohere to generate vector embeddings from unstructured data (e.g., text). Store Embeddings: Save these embeddings in a vector database like Pinecone or Qdrant for fast, similarity-based retrieval. Query Processing: Convert user queries into embeddings and retrieve similar documents from the vector database. Augment Generation: Feed the retrieved documents into an LLM via LangChain, enhancing the LLM's response generation with real-time data. LangChain Integration: Use LangChain to connect LLMs with vector databases, creating seamless workflows for retrieving and generating context-aware text. 🔍 Why Use RAG? RAG significantly boosts accuracy and contextual understanding by grounding LLM outputs in external, up-to-date information. This approach reduces hallucinations—instances where models generate incorrect facts—and provides contextually relevant responses. It’s particularly useful for applications like knowledge assistants, content generation, and recommendation systems. 🧠 Technologies Involved: LLMs: GPT, Cohere , Ollama are used for embedding generation and response generation. Vector Databases: Pinecone, Qdrant, and ChromaDB are utilized for storing and retrieving embeddings based on semantic similarity. LangChain: This framework connects LLMs with vector databases, enabling smooth RAG pipelines for enhanced AI responses. 🚀 Applications: Knowledge Retrieval: Systems that pull context-aware, real-time data for addressing complex queries. Personalized Content Generation: Generate accurate and fact-based content by retrieving the latest information. AI Assistants: Use retrieval-augmented LLMs to build intelligent assistants that provide relevant and timely information to users. Combining LLMs with vector databases and LangChain enables the creation of intelligent, scalable, and contextually aware systems for AI-driven applications. Exploring these emerging technologies has been an exciting part of my internship journey! Resources to dive deeper: LangChain Docs: https://docs.langchain.com Pinecone + LangChain Integration: https://lnkd.in/dZNbEZD9 OpenAI Embeddings Guide: https://lnkd.in/dxzCfzfV #RAG #LLMs #LangChain #VectorDatabases #AI #KnowledgeRetrieval #GPT #Cohere #Ollama #Qdrant

9

Want to learn the best programming combo? Pick your free time and dive deep into pybind11, Python, and C++. Try to create all possible bindings you can get from the standard library, and compose with a procedural/functional style. It's absolutely amazing how fast and easy you can make your application run. You can have a custom, strongly typed Python minimizing side effects—absolutely amazing.

39

6 Comments

🖥️ Frontend Developer Meets Startup Bootcamp: My Summer at USC Viterbi Just wrapped up an exhilarating 8-week journey at the USC Viterbi Startup Garage Summer Smasher Program! As a frontend developer, this experience was a game-changer. Here's what I accomplished: 👨‍💻 Tech Triumph: Spearheaded frontend development for Kintsugi Global, Inc., an innovative anime-focused startup. Transformed complex designs into responsive, user-friendly interfaces. 🛠️ Skills Sharpened: Mastered new frameworks and libraries Optimized code for performance and scalability Collaborated with UX/UI designers to bring concepts to life 🚀 Startup Savvy: Dove deep into the startup ecosystem, learning about: Pitch deck creation and presentation Market analysis and product-market fit Agile development in a fast-paced environment 🙏 Grateful for the mentorship from a diverse group of experts Ron Scovil, Arcelia Navarro, Dr. David Gerber, Ellis Meng, and Dave Eastman. Their insights were invaluable in bridging the gap between coding and entrepreneurship. 💡 Key Takeaway: The fusion of technical expertise and business acumen is powerful. I'm now equipped to not just code, but to contribute strategically for company's growth. #Kintsugi Global, Inc. #FrontendDevelopment #TechStartups #USC Viterbi School of Engineering #USC Viterbi Startup Garage

43

4 Comments

What matters the most to me in Generative AI is any aspect of control. It often feels like a slot machine as we're are betting to get a generation close to what we want it to look like. How could we add more control? Here is "Tora: Trajectory-oriented Diffusion Transformer for Video Generation" the new method from Alibaba Group. This paper introduces Tora, the first trajectory-oriented DiT framework that integrates textual, visual, and trajectory conditions concurrently for video generation. Specifically, Tora consists of a Trajectory Extractor~(TE), a Spatial-Temporal DiT, and a Motion-guidance Fuser~(MGF). The TE encodes arbitrary trajectories into hierarchical spacetime motion patches with a 3D video compression network. The MGF integrates the motion patches into the DiT blocks to generate consistent videos following trajectories. Their design aligns seamlessly with DiT's scalability, allowing precise control of video content's dynamics with diverse durations, aspect ratios, and resolutions. Extensive experiments demonstrate Tora's excellence in achieving high motion fidelity, while also meticulously simulating the movement of the physical world. Check the links in the comments! #generativeai #videogeneration #control #motionguidance #machinelearning #deeplearning #futurism

23

1 Comment