Loading...
In this lesson you'll get to implement function calling with Mistral. Function calling allows Mistral Models to connect to external tools, making it easy for you to build applications catering to specific use cases and practical problems. Let's check it out. At a high level, there are four steps with function calling. Step one is for users to define tools and use a query. A tool can be a user defined function or an external API. For example, users can write two functions to extract the payment status and payment date information, and then when user asks a question what's the status of my payment? This question is directly related to the payment status tool, and we should use this tool to address the question. Step two: is for our model to generate function arguments. when applicable, based on the tools and the user query, our model is able to determine that the function we should use is payment status and the function argument is transaction ID equals T1001. Step three: is for users to execute the function to obtain two results. We simply passing the function arguments directly in the function, and we get the result from the function which is paid in this example. Step four: is for a model to generate a final answer according to the information available, your transaction id t1001 has been paid. Is there anything else I can assist you with? Okay, now hopefully you have a good understanding of how function calling works with Mistral. Let's take a look at that code. Let's import pandas as PD. And assume we have the following data. We have this data frame of transaction ID and the payment information. And we would like to ask questions about this data frame. So, how can we ask questions about this data without function calling. We could simply passing this data in the prompt and then ask the question. Given the following data what is the payment status for the transaction ID, which is T1001? Again, let's load our Mixture function and run the Mistral model. Okay, we can see the answer. The payment status for the transaction with the ID T1001 is paid, and the result tells us how it was able to get the answer from the data. As you can see, Mistral Model is pretty good, if you give it the right prompts. But a normal use cases, you may have a pretty large database of transactions. If you use your entire database and feed into a large language model, this may exceed the context window. A more efficient, affordable, and reliable way to do this is to use function calling to run code. To perform this kind of search. You just need to prompt the large language model to know when to call each function. Let's take a look at how we do function calling step by step. Users first need to define all the necessary tools for their use cases. For example, here we have a function retrieve payment status. If we define the function argument, transaction id is T1001. We get the status paid from this function. Let's try another function. Retrieve payment date. And we can retrieve the payment date information based on a transaction ID. And then we get the date. So how do Mistral Models understand these functions. In order for Mistral models to understand the functions you can align the function specs with a json schema. For example here is the json schema for the function retrieve payment status. Let's specify the tool type which has a function in this case, the function name. The function description. This will tells our model. What does this function do. The parameters of the function, which includes the arguments of the function. The type of the argument and the description of the argument. Let's also specify the required function argument, which is the transaction id here. Please keep in mind the name of the function should be exactly the name of the function we just defined. Also, please make sure you give a good enough description of your function and a good description of the argument, so that our language model can understand which function to use and the needed argument to use. Similarly, let's define the json specs for the retrieve payment function. Let's combine these two json specs into a list called tools. Now we organized the two functions into a dictionary, where the keys represent the function name and the values are the functions with the data frame defined. This allows us to call each function based on its function name. To give you an example, we can call the retrieve payment status function with the argument transaction id equals T1001. Using this names to function dictionary. Okay. Next we need to define a user query. Suppose a user asks the following question. What's the status of my transaction? A stand alone large language model will not be able to answer this question. As it doesn't have the business logic backend to access the necessary data. But what if we have the exact tool we can use to answer this question? We could potentially provide an answer. So let's see how function calling works in the next step. Step two is for our model to generate function arguments. So how do Mistral models know about these functions and know which function to use? We provide both the user query and the tool specs. To Mistral models. You can have the language model to automatically choose if we should use a tool or not. With the two choice equals auto. Or you can define tool choice as any to force to use. Or if you don't want to use any tool, you can set tool choice equals none. To prevent tool, use. Okay, let's run this code. Let's see the content of the message. To check the status of your transaction I need the transaction id. Could you please provide it? As you can see, our model was able to identify if there is any essential information missing for the function. And it will ask for this essential information. Now let's save the history of the conversation we have in the chat history and add a user message. My transaction ID is T1001. Now in the chat message we have the original user question. What's the status of my transaction ID? We have the assistant message asking us to provide a transaction ID, and then we have a user message telling the model my transaction ID is T1001. Let's run our model again with this chat history. Let's take a look at the response. Now the content of the assistant message is actually empty. But it was able to return the information in tool calls telling us the function name is a retrieve payment status and the arguments we should use for this function. Let's again append the model response to the chat history. Step three is for user to execute the function to obtain tool results. Currently is the user's responsibility to execute these functions and the function execution lies on the user side. Let's extract some of the function information from the model response. We get the function name and the function arguments. Now we can get the function results based on the function name and the function arguments. We'll get the results status as paid. Let's define this message with the role tool name as the function name and the content as the function result. Let's save this tool message using the message method with the role as tool. Name is the function name. The content is the function result and we append it to the chat history. And let's take a look at the chat history right now. Here, as you can see we have the assistant message telling us the result of function tool calls with the function information. And we have the tool results returning the results of the function. Now step four is for model to generate a final answer. Let's just give our model the entire chat history and it will return a personalized answer. The status of your transaction T1001 is paid. Is there anything else you need help with? So, that's the four steps for a function calling. Mistral model also provides parallel function calling. Feel free to pause the video and change the prompt. See how can you call both functions at the same time? For example, in your user message, if you ask for both the status and the date of my transaction ID T1001, let's see what would the model respond? Let's run this cell. As you can see here, our model actually responded with two function calls. One is the retrieve payment status. Another one is the retrieve payment date. Now you can execute both functions to get the desired output. That's it for function calling. In the next lesson, we'll learn about RAG and how you can use RAG as a tool for function calling. See you in the next lesson.
Getting Started with Mistral