Loading...
lesson and add on to it to create a conversational agent. This will combine tool usage with chat memory and let us create something that ends up looking like ChatGPT. Let's chat about conversational agents. In this lesson, we'll be building a conversational agent. First, let's go over the basics of agents. So what are agents? They're a combination of language models and code. So the language model reasons about what steps to take and what the inputs to those actions should be, and then there's an agent loop which basically uses that agent to choose the tool, and then it calls that tool, observes the output, What are these stopping criterias? They could be several things. The first and most popular one is just using the language model to determine when to stop. So if you remember the previous lesson, we talked about the idea of an agent finish. This is when the language model just responds. This is one such stopping criteria. But you can have others like hard-coded rules, such as the maximum number of iterations or something like that. In this lab, we're going to use the tools that we built in the previous lesson. We're going to take that concept of routing and choosing and then calling the tools, and we're going to put that into our own agent loop, We're then going to use the agent executor class in LangChain We're then going to go ahead and create the logic that takes in the input and determines what tool to call. So we did this in the previous lesson. So we're importing a model, the prompt template, the formatting of tools to open AI functions, and then this output parser. We're then going to create functions by calling format tool to open AI function on our list of tools. We're going to create our model, setting temperature equals equals to zero and binding it with functions equals functions. We're then going to create our prompt again pretty simple with just a single system message you are helpful but sassy assistant, And so if we call this on some input we can get back a tool that it recommends using, and the tool input. And again, that's because the input here is specifically aimed at calling one of the tools. So what we want to do is create this loop that determines what tool to use, then calls that tool, and passes it back in and repeats until some stopping criteria is met. If you look at the prompt, what that means is we need a place in the prompt to pass back in this history of tools that are called and the corresponding outputs. So we're going to have to change the prompt a little bit to add that in. Specifically, we want to add in a place where we can pass in a list of messages. And that's because we're going to convert this tool selection to tool observation into a list of messages and pass it back in. So we're going to use messages placeholder to be a placeholder for this list of messages. And then in our prompt template, we're going to have the same first two elements, but then we're going to add in this agent scratchpad messages placeholder. We create that prompt, we create a chain now with this prompt and let's call it again on the same input. And this time we also need to pass in agent scratchpad, because this is the variable here that we need to pass in. We're going to pass in this being an empty list, because we haven't taken any actions yet. We'll call that, and we'll get result 1. Then what we can do is we can see what function result 1 wants us to call. We can then call that function and get back an observation. If we look at this observation, we can see that it is a string. And of course, we have result one, which is this agent action message log. So now how are we going to take these two things and convert them back into a list of messages that we can pass into agent scratchpad? In order to do that, we're gonna use this function, format to open AI functions, Behind the scenes, what's going on in format to open AI functions? What's happening is that we're gonna take result 1, So if we look at it, this contains the chat message that has this additional quarks with this function call saying get current temperature and then the arguments with the JSON string, the exact response from OpenAI. And so we're keeping that around in this message log because now when it comes time to construct this agent scratch pad we can just put it right back in there, So we're going to call this function and we're going to call it on a list of tuples corresponding to the agent action and the observation. And the reason that this is a list of tuples is that if we add in more steps for the agent to take we can just keep on passing in a list of tuples and it will keep on generating this list of messages. So here we have an AI message and this corresponds to what was outputted by the language model to make it take this agent action and then we have this function message with the content being the observation and the name being get current temperature. If we then take that and call the chain again with the same input and now with agent scratchpad set equal to this format to open AI functions parameter, if we look at what we get back from result 2, So let's bundle all of that up into a function. So what we're going to do is we're going to create this run agent function. This is going to take in the user input. It's going to initialize an empty list. This is the intermediate steps. And then it's going to enter into a while loop. And while this loop is running, it's going to call the chain with this user input and with this agent scratchpad. And each time we're calling format to open AI functions on this list We're going to make one minor modification to this chain before continuing to make it a true agent chain. And basically what we're going to do is we're going to put this format to open AI functions logic in the chain itself. That's going to make it a little bit more portable so it just has to take in the user input and the intermediate steps. We're going to create this agent chain by using a runnable pass-through. What a runnable pass-through does is it basically takes the initial input and passes it through. What we're going to do is we're going to add this assign variable here. This assign method here is creating a new argument to the dictionary that's getting passed through. This argument has the name agent scratchpad and this is where we call format to open AI functions on intermediate steps. We're then going to pipe the result of that which is a dictionary into the chain and we're going to get a full end-to-end chain that now takes in the input in intermediate steps, does the necessary pre-processing of the intermediate steps to create the agent scratchpad, then passes it to the prompt, then to the model, and then to the agent output parser. With this change we can now create a slightly more simple loop here where if you notice we're just calling the agent chain on the inputs directly, Let's now run this agent. So let's call it on what is the weather in SF? And we get back the current temperature in San Francisco is 22.9 Degrees Celsius. If we run the agent now, we can see that we get back a response. And let's also call it on a simple input. Let's call it on just hi. Hello, how can I assist you today? That seems right. So we've created this run agent function, which is this loop that loops over the LLM, decides what to do, then takes the next step. We have a class for this in LangChain called agent executor, It adds in error handling. So if the language model outputs something that isn't valid JSON, which remember it can still do, we handle that nicely. And it adds in also error handling for tools. So if you call a tool and the tool errors, we then handle that nicely. We catch that error. We pass it back into the language model and ask it to correct it. We can use this pretty simply by just importing agent executor from langchain.agents. We can then initialize this with agent equals to the agent chain. We pass in the tools because it needs to have the tools to be able to call it, and then we pass in verbose equals true to get some nice logging. If we now call this on what is LangChain, This agent executor in this loop under the hood is very similar to a lot of what's powering chat GPT. So if you've interacted with the code interpreter there or the plug-in system there, you'll notice that the language model often thinks about what to do, then it calls a tool, then it gets back the response, then it maybe calls another one, then it gets back the response from that, and then it returns to the user. So this is a really powerful loop and technique that can be used to create something like ChatGPT. Of course, what ChatGPT can also do is it can also have a conversation with you. So can we do that with our current agent executor? Let's try it out. So let's try it out with something simple like my name is Bob. So it responds. "Hello Bob, how can I assist you today?" Let's then ask it a follow-up question that any good chatbot should be able to answer. So let's ask it, what is my name? And I've just told it that my name is Bob, but it doesn't seem to remember that. So what's going on? What's happening is that we haven't added any mechanism to actually pass in previous messages. So it actually doesn't remember the my name is Bob message. Let's add that in and see what happens after that. In order to add that in, we're going to add another messages placeholder, this time for chat history. This is going to be before the user input, because we want to put messages that have appeared before and so we're going to put it in between the system message and the user question. We'll then create the same agent chain as before with the updated prompt, What this is going to do is it's just going to keep a list of messages in memory. We're going to return these messages in message format. And basically, If this was set to false, it would return it as a string. And so if we wanted to, we could get a string representation of all these messages. This would be useful if we wanted to put that in a normal string template, but here we don't. We want to pass it in as a list of messages, so we're setting return messages equals true. We're then also setting memory key equals chat history. This is just lining up the messages that are returned with this variable here so it's keeping those in sync. And then all we have to do when creating our agent executor is pass in the updated chain, pass in the same tools as before, we're going to set verbose equals to true as before, and then we're also going to pass in memory equals to the memory object above. Let's now run this on the input as before. So we're going to say, "Hi, If we now ask what my name is, it realizes my name is Bob. Good. And of course we can use it as before and ask it about the current weather and it will know how to use a tool, it will call that tool and then we'll get back the response. So this has the same functionality as before but we've added in this memory component that just remembers previous interactions. Now is a good time to pause and try it out on some questions of your own, try adding in some new tools, try adding in a different system prompt. There's a lot of different things you can play around with here. For our final bit here, we're going to put it all together and create a nice looking chatbot that we can interact with. Let's add in another tool. You guys should feel free to replace this with whatever you want. Remember to update this description here when you're done. For us here we're just going to have a really simple reverse operator that's taking in the query and just reversing the string. We're going to update our list of tools, Using a tool called panel, we're going to create a nice little UI for this chatbot. And so taking a look at this very briefly, we can see here that we have our functions, which is calling the same function as before, format tool to open AI functions. We've got our model, which is a chat open AI model, and we're binding functions to it. We've got memory, which is same as before. We've got our prompt, which is same as before. So it's got system message, We're then creating the agent chain, same as before. And then we're creating the agent executor, same as before. With that in place, we can now run panel and create a nice little dashboard with a Q&A bot. And so we can start having a conversation with it. So we can again do, hi, my name is Bob. Press Enter and we can see it pop up. What's my name? Your name is Bob. We can ask it questions about the weather. What's the weather in SF. See it takes a little bit longer because now it's calling a tool but it manages to get the current temperature okay. We can ask it some fun questions like what tools do you have available. So we can see that it lists out the tools that it has available, get current temperature, search Wikipedia, create your own. Let's now force it to call the create your own. So we're going to tell it to call the create your own tool with the input I love LangChain and return the result. So if we look at what happens, the result of the create your own tool with the input I love LangChain is something that's unpronounceable, but it looks right. This is the end of the lesson. So now's a great time to add in more tools, try it out with different inputs. You've got a chatbot that's similar to ChatGPT. It can have a conversation, it can take actions, you can see the results of that, try making it do multiple hops. There's a lot of amazing things that you can try out and build and I really look forward to seeing what you guys build with this technology.