Calling Functions Through Dialogue with OpenAI and Langchain

1. LLMs and Langchain

Many people may never get the chance to train, or even fine-tune, a large model, but using large models is the wave of the future. So how should we embrace this change? The answer is Langchain.

A large model provides a broad, general-purpose foundation. So far I have seen two main ways of putting it to work:

AIGC based on generative capability: scripts, code, QR codes, short videos, molecular structures, all emerging one after another
AutoGPT based on comprehension capability: combined with an execution engine, it directly changes machine state and performs automated control

In our current work scenarios, large models are often still not enough to directly replace human work at scale. There are two reasons:

A lot of private data has not been given to large models for training. Only after fine-tuning and combining with a knowledge base can good results be achieved. This is determined by the fundamental positioning of large models
Large models have not been around long enough. ToB efficiency-tool services operate on a ten-year cycle, and the market has not yet formed

The adoption of large models is inevitable. Today the large model is our Copilot; in the future we may be the large model’s Copilot. Combining large models with real business scenarios to build Copilot tools and improve efficiency is what I have been thinking about recently.

Building applications means frameworks. Langchain positions itself as the framework for building large-model applications, solving some of the common problems in getting large models into production. For example, connecting to multiple large models, Prompt management, context, external document loading, vector store integration, Chains task pipelines, and so on.

Langchain has already gone from a personal project to a commercially operated company, and raised several funding rounds in 2023. You can see from this that the venture capital industry is very bullish on application development built on large models. Since the investors have already made the judgment for us, all we need to do is learn and use Langchain more.

2. Calling Functions Directly Through Dialogue

In June 2023, OpenAI and Langchain successively released versions supporting direct function calling. This means a large model is not only good for chatting — it can also be used to trigger business logic.

2.1 Let’s See the Result First

Run the program

1
python function_bot.py

Interactive test

1
2
3
4
manual_input:获取 default 这个命名空间的全部 pod
function_bot: ["pod1", "pod2", "pod3"]
manual_input:获取 c1 这个集群的全部节点
function_bot: ["node1", "node2", "node3"]

You type natural language, the function runs automatically, and the result comes back. Two examples are shown here: one gets all Pods in the default namespace, the other gets all nodes in the c1 cluster.

2.2 Code Implementation

Set the environment variables

1
2
export OPENAI_API_BASE="https://api.openai.com/v1"
export OPENAI_API_KEY="xxx"

When using the OpenAI API, the API KEY set in the environment variables is read automatically.

Full code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
# -*- coding: utf-8 -*-
import json
from typing import Type
from pydantic import BaseModel, Field, create_model
from typing import Optional
from langchain.tools import BaseTool
from langchain.callbacks.manager import (
    AsyncCallbackManagerForToolRun,
    CallbackManagerForToolRun,
)
from langchain.tools import format_tool_to_openai_function
import openai


class GetClusterNodes(BaseTool):
    name: str = "get_cluster_nodes"
    description: str = "get all nodes in kubernetes cluster"

    args_schema: Type[BaseModel] = create_model(
        "GetClusterNodesArgs",
        cluster=(str, Field(
            description="the cluster of you want to query", type="string")),
    )

    def _run(
        self, query: str,
        run_manager: Optional[CallbackManagerForToolRun] = None
    ) -> str:
        return json.dumps(["node1", "node2", "node3"])

    async def _arun(
        self, query: str,
        run_manager: Optional[AsyncCallbackManagerForToolRun] = None
    ) -> str:
        return json.dumps(["node1", "node2", "node3"])


class GetClusterPodsByNamespaces(BaseTool):
    name: str = "get_cluster_pods_by_namespace"
    description: str = "get special pods in kubernetes special namespace"

    args_schema: Type[BaseModel] = create_model(
        "GetClusterPodsByNamespacesArgs",
        namespace=(str, Field(
            description="the namespace of you want to query", type="string")),
    )

    def _run(
        self, query: str,
        run_manager: Optional[CallbackManagerForToolRun] = None
    ) -> str:
        return json.dumps(["pod1", "pod2", "pod3"])

    async def _arun(
        self, query: str,
        run_manager: Optional[AsyncCallbackManagerForToolRun] = None
    ) -> str:
        return json.dumps(["pod1", "pod2", "pod3"])


functions_list: list = [GetClusterNodes, GetClusterPodsByNamespaces]
functions_map: dict = {fun().name: fun for fun in functions_list}


def run(msg: str):
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": msg}],
        functions=[
            format_tool_to_openai_function(t()) for t in functions_list],
        function_call="auto",
    )
    message = response["choices"][0]["message"]
    if message.get("function_call"):
        function_name = message["function_call"]["name"]
        function_response = functions_map[function_name]().run(
            message["function_call"]["arguments"])
        return function_response

if __name__ == "__main__":
    while True:
        user_input = input("manual_input:")

        if user_input == "exit":
            break

        print("function_bot:", run(user_input))

To keep the implementation simple, each _run here just returns directly without actually calling a function. In real production we need to call the function based on the incoming query and return the result.

2.3 Step-by-Step Walkthrough

Core code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
def run(msg: str):
    response = openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[{"role": "user", "content": msg}],
        functions=[
            format_tool_to_openai_function(t()) for t in functions_list],
        function_call="auto",
    )
    message = response["choices"][0]["message"]
    if message.get("function_call"):
        function_name = message["function_call"]["name"]
        function_response = functions_map[function_name]().run(
            message["function_call"]["arguments"])
        return function_response

Two parameters are set in openai.ChatCompletion.create:

function_call is set to auto, which is also the default; the model decides on its own whether to call a function. This is not an actual call — it returns some function metadata.

functions is a list object. Based on the function descriptions passed in plus the user’s input msg, OpenAI makes a judgment and returns the function name and the parameters it extracted.

Custom functions

There are two ways to define them: concatenating list objects directly, or subclassing BaseTool.

Here is the list-object concatenation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
[
  {
    "name": "get_cluster_nodes",
    "description": "get all nodes in kubernetes cluster"
  },
  {
    "name": "get_cluster_pods_by_namespace",
    "description": "get special pods in kubernetes special namespace",
    "parameters": {
      "type": "object",
      "properties": {
        "namespace": {
          "type": "string",
          "description": "filter pods in namespace"
        }
      },
      "required": ["namespace"]
    }
  }
]

The full example code above uses the BaseTool subclass approach:

1
2
class GetClusterNodes(BaseTool):
class GetClusterPodsByNamespaces(BaseTool):

The BaseTool approach still ultimately needs format_tool_to_openai_function to extract the information from the custom functions and generate a list, but managing functions with BaseTool is a cleaner way to do it.

Defining function parameters matters a lot

If you do not describe the parameters in detail, the parameter format OpenAI recognizes is very likely to look like this:

1
2
3
4
"function_call": {
    "name": "get_cluster_nodes",
    "arguments": "{\n\"__arg1\": \"c1\"\n}"
}

But once properties or args_schema is set, the function arguments OpenAI returns match expectations very well.

1
2
3
4
"function_call": {
    "name": "get_cluster_nodes",
    "arguments": "{\n\"cluster\": \"c1\"\n}"
}

Where the actual business logic goes

If you use the direct list-concatenation form, just write it directly in the function. If you subclass BaseTool, you need to implement its synchronous _run function and asynchronous _arun function.

In the full example code above:

1
2
function_response = functions_map[function_name]().run(
            message["function_call"]["arguments"])

The returned arguments are passed straight to the function being called — here that is _run. Inside _run, json.loads(query) gives you the arguments matching the args_schema definition.

[Optional] Have OpenAI clean up the message response a second time

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
def format(msg: str, function_response: str):
    return openai.ChatCompletion.create(
        model="gpt-3.5-turbo",
        messages=[
            {"role": "user", "content": msg},
            {
                "role": "function",
                "name": "get_cluster_nodes",
                "content": function_response,
            },
        ],
    )["choices"][0]["message"]["content"]

The code is as above. After getting the function response, if you still want to clean up the format and content of the response a second time, you can set a message with the function role, attach the function response, and send it to OpenAI together with the user’s input. OpenAI will then give a more complete response.

But this step is not required. If the function response already matches expectations, you can simply return it.

3. Summary

This article used OpenAI and Langchain to build an example of calling functions directly with natural language.

A large model is not only good for chatting — it can also be used to trigger business logic. When we build Copilots, we often need this kind of glue function, binding the large model to the business logic.

Building out the large-model ecosystem has two parts: cognition and execution. Cognition depends on the model’s parameter scale, network structure, and training data; execution mainly depends on the external connections it has.

I think that even if you cannot take part in training large models, you can still try to organize an industry knowledge base, and you still have a chance to take part in the execution side. Around execution, we can open up product APIs and add more touchpoints for the large model to connect to systems; we can also build SDKs and toolkits that help developers integrate large models quickly — for example, putting together a BaseTool class library that wraps all kinds of APIs and script functions. And of course, we can redesign business processes and execution logic around the way large models think.