From the course: A Start to Using Generative AI in .NET

Unlock the full course today

Join today to access over 23,400 courses taught by industry experts.

Implementing streaming completion

Implementing streaming completion - .NET Tutorial

From the course: A Start to Using Generative AI in .NET

Start my 1-month free trial Buy for my team

Implementing streaming completion

“

- [Instructor] Let's implement streaming chat completion. I will use the result of the error handling video as a starting point. Before we actually start, let's make sure that the MakTokens line is removed, if you still have that in your code. I will use GPT-4 for this example because it is slower and we will be able to see the streaming better. The key to streaming is to replace the GetChatCompletions method with the GetChatCompletionsStreamingAsync. This is the most complex chat completion variant, which not only streams the response but also uses it as an asynchronous enumerable. I'll also change the name of the variable to streamingResult. Now we process the result in an await foreach loop, so await foreach car update in streamingResult., and then here we have Result, .EnumerateValues. This is a new feature in C# eight and it's perfect for streaming. Don't worry if you haven't seen this before. Think of it as a foreach loop that gives up control to the caller while waiting for the…

Contents