Loading...
applications, namely prompt templates, models, and parsers. And you'll learn a bit about how to combine them together to create chains with LangChain Expression Language. Let's dive in. Before we get too far in, let's tackle the elephant in the room. Why LangChain.js? Well, JavaScript is the biggest ecosystem of developers in the world, and many of them prefer, surprisingly, to use JavaScript. You also might choose it for some of the easy deployment tools and powerful scaling features of frameworks like Next.js, cloud offerings like Vercel's Edge functions or Cloudflare Workers, and generally the advanced tool chains that come with them. You also might want to build for multiple platforms or different platforms. If you're building directly in the browser for Chrome extensions or mobile apps with React Native, desktop apps with Electron, environments like that. Another housekeeping note, this notebook uses the Deno Jupyter kernel, which is a JavaScript runtime with slight differences from node and web environments. For the most part, you'll be able to copy paste code directly into different environments. And when there are differences, we'll call them out. So a bit about LangChain Expression Language. LangChain uses this to compose chains of components together, and components that implement and are usable with this language are called runnables. They define some core methods and an allowed set of input types, and output types. And allow you to use methods like invoke stream and batch right out of the box, which are all common methods used when building and using LLM applications, there's also means of modifying parameters at runtime with the bind method. So a concrete example of this is the prompt LLM output parser trio that we'll be going over in this lesson. And some other benefits of the runnable protocol are that you get fallbacks and parallelism right out of the box with our batch method, as well as logging and tracing built in via LangSmith, our tracing and observability tool. Throughout the course, we'll link to a few explorable traces in LangSmith that illustrate how different chains work. Let's start with one of the most fundamental pieces of LangChain, the language model. And Langchain supports two different types of language model. In fact, there are text LLMs, which take a string as input and return a string as output. So string to string. And then there are chat models, which take a list of messages as input and return a single message output. And examples of text models are things like popular apps like ChatGPT and GPT-4. So because they look like strings, text LLM inputs and outputs are easy to visualize. So let's look at what calling a chat model directly looks like. So we'll import some environment variables. We'll be using OpenAI's GPT 3.5 Turbo for the majority of the course, which is a chat model. And we'll import the LangChain wrapper for it, as import chat OpenAI from LangChain chat models OpenAI. Then we'll import a human message class that we'll use to wrap and create our chat model inputs. And then we'll initialize our model like so. And again, be using GPT-Turbo. And we'll use the latest version of time of filming, which was from November 6th. And then let's try querying it, or invoking it with a list of messages, which in this case will just be a single human message corresponding to our input. Tell me a joke. We'll also be testing ChatGPT's humor here or rather opening eyes humor. Cool. Let's give it a try why don't skeletons fight each other? They don't have the guts. Hilarious. So one thing you'll notice about the message output from the chat model is that it contains a content field containing the text value of the message, and an associated role that corresponds to the entity sending the message. In this case, we started with the human sending the original message, which is us, and the AI responded with an AI message. And again, while we're using GPT 3.5 Turbo, LangChain supports models from many different providers, and you can try swapping the provided class in any of the code examples. So here's a good place to pause and try editing the prompt yourself, or the message yourself rather, to tell a specific joke, perhaps about parrots, or even change the input message to ask about something else entirely. Give it a shot. OK, let's move on to another building block, the prompt template. So while calling models in isolation can be useful, as you saw above, and you can do some pretty advanced things with that, it's often more convenient to factor out the logic behind model inputs into reusable parameterized components, rather than typing out the full query each time. And for this, LangChain includes prompt templates, which are responsible for formatting user input for later model calls. Let's see what that looks like. We'll import a chat prompt template class here and initialize it like so. What are three good names for a company that makes product? One thing to note here, we've denoted an input variable using curly braces, in this case product, and anything we pass into the prompt will be injected and formatted into this part. Prompt templates are also useful for smoothing over some of the differences in model input types. Here we construct a prompt template directly from a string, but we can use this prompt template to generate both string input for an LLM using the format method like this, and this is how coincidentally we pass input variables. So we'll say colorful socks for a product and that'll get injected to our prompt right here. Let's have a question mark so it's not. So there we go. And if you run this, we get a string output with a human prefix because we've created a chat prompt template. We get what are three good names for a company that makes colorful socks. And we get a string output, which is also our input variable here. So this is output we could pass directly to an LLM, but we can also format our prompt to output a message array useful for calling chat models. So await prompt format messages this time, and we'll give it the same product because who doesn't love colorful socks? Colorful socks. And this time we get a single human message within an array that we could pass directly to our chat model ,and you'll notice that if we use this from template method with a string as input for convenience, we turn that into a human message here, but we can also create a prompt template for messages directly for finer grading control over what types of messages are passed to the prompt. For example many models and model providers rely on a system message to define certain behavior, and this is a great way to format your input. So here's an example. We'll import some other methods here, and then we'll create our prompt slightly differently using these new message prompt templates. So prompt for messages will be a chat prompt template from these messages. And now this time, instead of constructing our prompt template from a string, we're going to construct it from a system message prompt template and a human message prompt template such that our output and our formatted output rather, our output will be multiple formatted messages instead of just a single one. So let's try it with a product shiny objects. And this time we get a system message, which remains the same because we didn't have any template variables here. And a human message, what are three good names for a company that makes shiny objects? Awesome. And some nice shorthand for this, by the way, so you don't have to import those message prompt template classes. Let's just use a tuple with the role of the template. So for example, we can initialize it like this. The same exact output as before, but instead of using those message prompt templates, we use tuple with the role and then the template string. And if we run format, we get the exact same output. So this will be useful later for passing around history because you can inject history messages directly into the prompt. Though we can pass these formatted values directly into a model, there's actually a more elegant way to use prompts and models together that we'll go over next. And we've already mentioned it. That's right, it's expression language, or LCEL for short. LCEL is a composable syntax for chaining LangChain models together. And again, objects that are compatible with LCEL are called runnables. We can construct a simple chain from the prompt and model we declared above like this. We'll use this pipe method. And what this is going to do is it's going to create a chain where the input is the same as the first step in the sequence, our prompt here, which in this case will be an object with a single property called product. The prompt template is invoked with this input and passes the properly formatted result as input into the next step of the chain, the chat model here. Here's what it looks like in action. There we go, await chain.invoke, Products colorful socks. If you recall, the prompt asked the LLM to describe three fun names for a company that makes this product. And if we run it, that's exactly what we'll see a single chat message response with three fun names for a company that makes colorful socks chroma sock bright step color vibe sock comb. The final consideration we'll go over in this lesson is formatting our output. For example, it's often easier to work with the raw string value of a chat models's output rather than an AI message. And the LangChain abstract for this is called an output parser. So we'll import it like this. This is one that will take our chat model output, a single message, and coerce it into a string. So let's redeclare our chain like this. Awesome. Now, if we run this chain again, perhaps with a more fun name, or a different product rather, let's see if we can generate some good names for a company that makes fancy cookies. And this time you'll see we get three names like before, but this time instead of a chat message output we get a string. Delicate crumbs, Gourmet Cookie Creations, and Elegant Sweets Co. These three pieces form the core of many complicated chains, so it's important to get a good handle on them now. And if you're more of a visual person, you can check out this Langsmith trace illustrating what's going on internally. And I'll paste it here and let's go look at it. So we see that we have a runnable sequence here, which corresponds to our chain. And that wraps a few steps here. First, our chat prompt template, where it takes in our input, fancy cookies, that's a product, and injects it into the prompt. It outputs a single human message wrapped in an array, and that's the format that our chat model expects here. So our input, what are three good names for a company that makes fancy cookies? And the output, an AI message with three good names for a company that makes fancy cookies. And then finally, a string output parser converts that from an AI message into a string. And by the way, while these pipe operations are convenient for small numbers of outputs, they can get a little unwieldy for more complicated chains. For this, you can use the runnableSequence.from method to construct a chain from an array of runnables. Here's how it works. We'll import it here. And we'll reconstruct our chain above using that method, and it's as simple as prompt model output parser. And that's equivalent to the pipe operation above. So let's invoke it and see. And yep, we get three names for a company that makes fancy cookies. And let's go over some of the other methods that we get for free with expression language here. One is streaming. That's a very important one for many of you in the webdev community, and we'll take great advantage of that later. But all these runnables and sequences of runnables themselves are also runnables, by the way, get a dot stream method which returns output from the chain in an iterable stream. And because LLM responses often take a long time to finish, this is useful in situations where showing feedback quickly is important. And here's an example with the chain we just composed. You'll see here that we call this dot stream method instead of dot invoke with a different product name this time, really cool robots. And we can use this asynciterator syntax for await, const chunk of stream. To loop over the return and stream chunks from our stream above. And when we run it, you'll see we get three names for companies that make really cool robots, but this time, instead of a single chunk back, we get individual string chunks containing part of our output. Robotech Innovations, FutureBot Solutions, and MechanoWorks Unlimited. And we could show these in a front end application much more quickly than waiting for the entire response. You'll also note that the output parser transforms those chunks from the model as they are generated, resulting in string output chunks rather than models. And finally, let's look at batch. This is useful for performing concurrent operations and multiple generations simultaneously. To show what this looks like, we'll define a set of inputs as an array, and each one of these should match the syntax that our prompt template takes. So product will be some new product, let's say large calculators, a little whimsical, and product alpaca-wool-sweaters. And this time, instead of using streamer invoke, we'll use the dot batch method. And what we expect to see are two string outputs corresponding to names for two different companies. The first that would make large calculators. So we have BigCalco, GiantDigits nc, and JumboMath Solutions. And a second that would make Alpaca Wool Sweaters, which you can see here are not too bad. So now you've seen some of the fundamental modules behind LangChain and LangChain Expression Language, the glue that holds them all together. Feel free to pause before going on to the next lesson and try modifying some of these prompts and templates and even expression language chains for yourself to get a better sense of how they fit together. We'll see you in the next lesson where we'll go over the basics of retrieval augmented generation and talk about loading and preparing our data.