So far, you've seen how to create a server and a client.
The client has been able to call the server explicitly to list its tools, resources, and prompts.
However, this is not a very practical approach.
Your users live in the agentic era and expect to use prompts and communicate with an LLM instead.
They do not care whether you use MCP to store your capabilities; they simply expect to interact using natural language.
So how do we solve this?
The solution is to add an LLM to the client.
In this lesson we focus on adding an LLM to do your client and show how this provides a much better experience for your user.
By the end of this lesson, you will be able to:
Let's try to understand the approach we need to take. Adding an LLM sounds simple, but will we actually do this?
Here's how the client will interact with the server:
1. Establish connection with server.
1. List capabilities, prompts, resources and tools, and save down their schema.
1. Add an LLM and pass the saved capabilities and their schema in a format the LLM understands.
1. Handle a user prompt by passing it to the LLM together with the tools listed by the client.
Great, now we understand how we can do this at high level, let's try this out in below exercise.
In this exercise, we will learn to add an LLM to our client.
Creating a GitHub token is a straightforward process. Here’s how you can do it:
Let's create our client first:
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
import { Transport } from "@modelcontextprotocol/sdk/shared/transport.js";
import OpenAI from "openai";
import { z } from "zod"; // Import zod for schema validation
class MCPClient {
private openai: OpenAI;
private client: Client;
constructor(){
this.openai = new OpenAI({
baseURL: "https://models.inference.ai.azure.com",
apiKey: process.env.GITHUB_TOKEN,
});
this.client = new Client(
{
name: "example-client",
version: "1.0.0"
},
{
capabilities: {
prompts: {},
resources: {},
tools: {}
}
}
);
}
}
In the preceding code we've:
client and openai that will help us manage a client and interact with an LLM respectively.baseUrl to point to the inference API.
from mcp import ClientSession, StdioServerParameters, types
from mcp.client.stdio import stdio_client
# Create server parameters for stdio connection
server_params = StdioServerParameters(
command="mcp", # Executable
args=["run", "server.py"], # Optional command line arguments
env=None, # Optional environment variables
)
async def run():
async with stdio_client(server_params) as (read, write):
async with ClientSession(
read, write
) as session:
# Initialize the connection
await session.initialize()
if __name__ == "__main__":
import asyncio
asyncio.run(run())
In the preceding code we've:
using Azure;
using Azure.AI.Inference;
using Azure.Identity;
using System.Text.Json;
using ModelContextProtocol.Client;
using System.Text.Json;
var clientTransport = new StdioClientTransport(new()
{
Name = "Demo Server",
Command = "/workspaces/mcp-for-beginners/03-GettingStarted/02-client/solution/server/bin/Debug/net8.0/server",
Arguments = [],
});
await using var mcpClient = await McpClient.CreateAsync(clientTransport);
First, you'll need to add the LangChain4j dependencies to your pom.xml file.
Add these dependencies to enable MCP integration and GitHub Models support:
<properties>
<langchain4j.version>1.0.0-beta3</langchain4j.version>
</properties>
<dependencies>
<!-- LangChain4j MCP Integration -->
<dependency>
<groupId>dev.langchain4j</groupId>
<artifactId>langchain4j-mcp</artifactId>
<version>${langchain4j.version}</version>
</dependency>
<!-- OpenAI Official API Client -->
<dependency>
<groupId>dev.langchain4j</groupId>
<artifactId>langchain4j-open-ai-official</artifactId>
<version>${langchain4j.version}</version>
</dependency>
<!-- GitHub Models Support -->
<dependency>
<groupId>dev.langchain4j</groupId>
<artifactId>langchain4j-github-models</artifactId>
<version>${langchain4j.version}</version>
</dependency>
<!-- Spring Boot Starter (optional, for production apps) -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
</dependencies>
Then create your Java client class:
import dev.langchain4j.mcp.McpToolProvider;
import dev.langchain4j.mcp.client.DefaultMcpClient;
import dev.langchain4j.mcp.client.McpClient;
import dev.langchain4j.mcp.client.transport.McpTransport;
import dev.langchain4j.mcp.client.transport.http.HttpMcpTransport;
import dev.langchain4j.model.chat.ChatLanguageModel;
import dev.langchain4j.model.openaiofficial.OpenAiOfficialChatModel;
import dev.langchain4j.service.AiServices;
import dev.langchain4j.service.tool.ToolProvider;
import java.time.Duration;
import java.util.List;
public class LangChain4jClient {
public static void main(String[] args) throws Exception { // Configure the LLM to use GitHub Models
ChatLanguageModel model = OpenAiOfficialChatModel.builder()
.isGitHubModels(true)
.apiKey(System.getenv("GITHUB_TOKEN"))
.timeout(Duration.ofSeconds(60))
.modelName("gpt-4.1-nano")
.build();
// Create MCP transport for connecting to server
McpTransport transport = new HttpMcpTransport.Builder()
.sseUrl("http://localhost:8080/sse")
.timeout(Duration.ofSeconds(60))
.logRequests(true)
.logResponses(true)
.build();
// Create MCP client
McpClient mcpClient = new DefaultMcpClient.Builder()
.transport(transport)
.build();
}
}
In the preceding code we've:
ChatLanguageModel: Configured to use GitHub Models with your GitHub tokenThis example assumes you have a Rust based MCP server running. If you don't have one, refer back to the 01-first-server lesson to create the server.
Once you have your Rust MCP server, open a terminal and navigate to the same directory as the server. Then run the following command to create a new LLM client project:
mkdir calculator-llmclient
cd calculator-llmclient
cargo init
Add the following dependencies to your Cargo.toml file:
[dependencies]
async-openai = { version = "0.29.0", features = ["byot"] }
rmcp = { version = "0.5.0", features = ["client", "transport-child-process"] }
serde_json = "1.0.141"
tokio = { version = "1.46.1", features = ["rt-multi-thread"] }
> [!NOTE]
> There isn't an official Rust library for OpenAI, however, the async-openai crate is a community maintained library that is commonly used.
Open the src/main.rs file and replace its content with the following code:
use async_openai::{Client, config::OpenAIConfig};
use rmcp::{
RmcpError,
model::{CallToolRequestParam, ListToolsResult},
service::{RoleClient, RunningService, ServiceExt},
transport::{ConfigureCommandExt, TokioChildProcess},
};
use serde_json::{Value, json};
use std::error::Error;
use tokio::process::Command;
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
// Initial message
let mut messages = vec![json!({"role": "user", "content": "What is the sum of 3 and 2?"})];
// Setup OpenAI client
let api_key = std::env::var("OPENAI_API_KEY")?;
let openai_client = Client::with_config(
OpenAIConfig::new()
.with_api_base("https://models.github.ai/inference/chat")
.with_api_key(api_key),
);
// Setup MCP client
let server_dir = std::path::Path::new(env!("CARGO_MANIFEST_DIR"))
.parent()
.unwrap()
.join("calculator-server");
let mcp_client = ()
.serve(
TokioChildProcess::new(Command::new("cargo").configure(|cmd| {
cmd.arg("run").current_dir(server_dir);
}))
.map_err(RmcpError::transport_creation::<TokioChildProcess>)?,
)
.await?;
// TODO: Get MCP tool listing
// TODO: LLM conversation with tool calls
Ok(())
}
This code sets up a basic Rust application that will connect to an MCP server and GitHub Models for LLM interactions.
> [!IMPORTANT]
> Make sure to set the OPENAI_API_KEY environment variable with your GitHub token before running the application.
Great, for our next step, let's list the capabilities on the server.
Now we will connect to the server and ask for its capabilities:
In the same class, add the following methods:
async connectToServer(transport: Transport) {
await this.client.connect(transport);
this.run();
console.error("MCPClient started on stdin/stdout");
}
async run() {
console.log("Asking server for available tools");
// listing tools
const toolsResult = await this.client.listTools();
}
In the preceding code we've:
connectToServer.run method responsible for handling our app flow. So far it only lists the tools but we will add more to it shortly.
# List available resources
resources = await session.list_resources()
print("LISTING RESOURCES")
for resource in resources:
print("Resource: ", resource)
# List available tools
tools = await session.list_tools()
print("LISTING TOOLS")
for tool in tools.tools:
print("Tool: ", tool.name)
print("Tool", tool.inputSchema["properties"])
Here's what we added:
inputSchema which we use later.
async Task<List<ChatCompletionsToolDefinition>> GetMcpTools()
{
Console.WriteLine("Listing tools");
var tools = await mcpClient.ListToolsAsync();
List<ChatCompletionsToolDefinition> toolDefinitions = new List<ChatCompletionsToolDefinition>();
foreach (var tool in tools)
{
Console.WriteLine($"Connected to server with tools: {tool.Name}");
Console.WriteLine($"Tool description: {tool.Description}");
Console.WriteLine($"Tool parameters: {tool.JsonSchema}");
// TODO: convert tool definition from MCP tool to LLm tool
}
return toolDefinitions;
}
In the preceding code we've:
// Create a tool provider that automatically discovers MCP tools
ToolProvider toolProvider = McpToolProvider.builder()
.mcpClients(List.of(mcpClient))
.build();
// The MCP tool provider automatically handles:
// - Listing available tools from the MCP server
// - Converting MCP tool schemas to LangChain4j format
// - Managing tool execution and responses
In the preceding code we've:
McpToolProvider that automatically discovers and registers all tools from the MCP serverRetrieving tools from the MCP server is done using the list_tools method.
In your main function, after setting up the MCP client, add the following code:
// Get MCP tool listing
let tools = mcp_client.list_tools(Default::default()).await?;
Next step after listing server capabilities is to convert them into a format that the LLM understands. Once we do that, we can provide these capabilities as tools to our LLM.
1. Add the following code to convert response from MCP Server to a tool format the LLM can use:
```typescript
openAiToolAdapter(tool: {
name: string;
description?: string;
input_schema: any;
}) {
// Create a zod schema based on the input_schema
const schema = z.object(tool.input_schema);
return {
type: "function" as const, // Explicitly set type to "function"
function: {
name: tool.name,
description: tool.description,
parameters: {
type: "object",
properties: tool.input_schema.properties,
required: tool.input_schema.required,
},
},
};
}
```
The above code takes a response from the MCP Server and converts that to a tool definition format the LLM can understand.
2. Let's update the run method next to list server capabilities:
```typescript
async run() {
console.log("Asking server for available tools");
const toolsResult = await this.client.listTools();
const tools = toolsResult.tools.map((tool) => {
return this.openAiToolAdapter({
name: tool.name,
description: tool.description,
input_schema: tool.inputSchema,
});
});
}
```
In the preceding code, we've update the run method to map through the result and for each entry call openAiToolAdapter.
1. First, let's create the following converter function
```python
def convert_to_llm_tool(tool):
tool_schema = {
"type": "function",
"function": {
"name": tool.name,
"description": tool.description,
"type": "function",
"parameters": {
"type": "object",
"properties": tool.inputSchema["properties"]
}
}
}
return tool_schema
```
In the function above convert_to_llm_tools we take an MCP tool response and convert it to a format that the LLM can understand.
2. Next, let's update our client code to leverage this function like so:
```python
functions = []
for tool in tools.tools:
print("Tool: ", tool.name)
print("Tool", tool.inputSchema["properties"])
functions.append(convert_to_llm_tool(tool))
```
Here, we're adding a call to convert_to_llm_tool to convert the MCP tool response to something we can feed the LLM later.
1. Let's add code to convert the MCP tool response to something the LLM can understand
ChatCompletionsToolDefinition ConvertFrom(string name, string description, JsonElement jsonElement)
{
// convert the tool to a function definition
FunctionDefinition functionDefinition = new FunctionDefinition(name)
{
Description = description,
Parameters = BinaryData.FromObjectAsJson(new
{
Type = "object",
Properties = jsonElement
},
new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase })
};
// create a tool definition
ChatCompletionsToolDefinition toolDefinition = new ChatCompletionsToolDefinition(functionDefinition);
return toolDefinition;
}
In the preceding code we've:
ConvertFrom that takes, name, description and input schema.2. Let's see how we can update some existing code to take advantage of this function above:
```csharp
async Task
{
Console.WriteLine("Listing tools");
var tools = await mcpClient.ListToolsAsync();
List
foreach (var tool in tools)
{
Console.WriteLine($"Connected to server with tools: {tool.Name}");
Console.WriteLine($"Tool description: {tool.Description}");
Console.WriteLine($"Tool parameters: {tool.JsonSchema}");
JsonElement propertiesElement;
tool.JsonSchema.TryGetProperty("properties", out propertiesElement);
var def = ConvertFrom(tool.Name, tool.Description, propertiesElement);
Console.WriteLine($"Tool definition: {def}");
toolDefinitions.Add(def);
Console.WriteLine($"Properties: {propertiesElement}");
}
return toolDefinitions;
}
``` In the preceding code, we've:
- Update the function to convert the MCP tool response to an LLm tool. Let's highlight the code we added:
```csharp
JsonElement propertiesElement;
tool.JsonSchema.TryGetProperty("properties", out propertiesElement);
var def = ConvertFrom(tool.Name, tool.Description, propertiesElement);
Console.WriteLine($"Tool definition: {def}");
toolDefinitions.Add(def);
```
The input schema is part of the tool response but on the "properties" attribute, so we need to extract.
Furthermore, we now call ConvertFrom with the tool details.
Now we've done the heavy lifting, let's see how it call comes together as we handle a user prompt next.
// Create a Bot interface for natural language interaction
public interface Bot {
String chat(String prompt);
}
// Configure the AI service with LLM and MCP tools
Bot bot = AiServices.builder(Bot.class)
.chatLanguageModel(model)
.toolProvider(toolProvider)
.build();
In the preceding code we've:
Bot interface for natural language interactionsAiServices to automatically bind the LLM with the MCP tool providerTo convert the MCP tool response to a format that the LLM can understand, we will add a helper function that formats the tools listing.
Add the following code to your main.rs file below the main function.
This will be called when making requests to the LLM:
async fn format_tools(tools: &ListToolsResult) -> Result<Vec<Value>, Box<dyn Error>> {
let tools_json = serde_json::to_value(tools)?;
let Some(tools_array) = tools_json.get("tools").and_then(|t| t.as_array()) else {
return Ok(vec![]);
};
let formatted_tools = tools_array
.iter()
.filter_map(|tool| {
let name = tool.get("name")?.as_str()?;
let description = tool.get("description")?.as_str()?;
let schema = tool.get("inputSchema")?;
Some(json!({
"type": "function",
"function": {
"name": name,
"description": description,
"parameters": {
"type": "object",
"properties": schema.get("properties").unwrap_or(&json!({})),
"required": schema.get("required").unwrap_or(&json!([]))
}
}
}))
})
.collect();
Ok(formatted_tools)
}
Great, we're not set up to handle any user requests, so let's tackle that next.
In this part of the code, we will handle user requests.
1. Add a method that will be used to call our LLM:
```typescript
async callTools(
tool_calls: OpenAI.Chat.Completions.ChatCompletionMessageToolCall[],
toolResults: any[]
) {
for (const tool_call of tool_calls) {
const toolName = tool_call.function.name;
const args = tool_call.function.arguments;
console.log(Calling tool ${toolName} with args ${JSON.stringify(args)});
// 2. Call the server's tool
const toolResult = await this.client.callTool({
name: toolName,
arguments: JSON.parse(args),
});
console.log("Tool result: ", toolResult);
// 3. Do something with the result
// TODO
}
}
```
In the preceding code we:
- Added a method callTools.
- The method takes an LLM response and checks to see what tools have been called, if any:
```typescript
for (const tool_call of tool_calls) {
const toolName = tool_call.function.name;
const args = tool_call.function.arguments;
console.log(Calling tool ${toolName} with args ${JSON.stringify(args)});
// call tool
}
```
- Calls a tool, if LLM indicates it should be called:
```typescript
// 2. Call the server's tool
const toolResult = await this.client.callTool({
name: toolName,
arguments: JSON.parse(args),
});
console.log("Tool result: ", toolResult);
// 3. Do something with the result
// TODO
```
2.
Update the run method to include calls to the LLM and calling callTools:
```typescript
// 1. Create messages that's input for the LLM
const prompt = "What is the sum of 2 and 3?"
const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
{
role: "user",
content: prompt,
},
];
console.log("Querying LLM: ", messages[0].content);
// 2. Calling the LLM
let response = this.openai.chat.completions.create({
model: "gpt-4.1-mini",
max_tokens: 1000,
messages,
tools: tools,
});
let results: any[] = [];
// 3. Go through the LLM response,for each choice, check if it has tool calls
(await response).choices.map(async (choice: { message: any; }) => {
const message = choice.message;
if (message.tool_calls) {
console.log("Making tool call")
await this.callTools(message.tool_calls, results);
}
});
```
Great, let's list the code in full:
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
import { Transport } from "@modelcontextprotocol/sdk/shared/transport.js";
import OpenAI from "openai";
import { z } from "zod"; // Import zod for schema validation
class MyClient {
private openai: OpenAI;
private client: Client;
constructor(){
this.openai = new OpenAI({
baseURL: "https://models.inference.ai.azure.com", // might need to change to this url in the future: https://models.github.ai/inference
apiKey: process.env.GITHUB_TOKEN,
});
this.client = new Client(
{
name: "example-client",
version: "1.0.0"
},
{
capabilities: {
prompts: {},
resources: {},
tools: {}
}
}
);
}
async connectToServer(transport: Transport) {
await this.client.connect(transport);
this.run();
console.error("MCPClient started on stdin/stdout");
}
openAiToolAdapter(tool: {
name: string;
description?: string;
input_schema: any;
}) {
// Create a zod schema based on the input_schema
const schema = z.object(tool.input_schema);
return {
type: "function" as const, // Explicitly set type to "function"
function: {
name: tool.name,
description: tool.description,
parameters: {
type: "object",
properties: tool.input_schema.properties,
required: tool.input_schema.required,
},
},
};
}
async callTools(
tool_calls: OpenAI.Chat.Completions.ChatCompletionMessageToolCall[],
toolResults: any[]
) {
for (const tool_call of tool_calls) {
const toolName = tool_call.function.name;
const args = tool_call.function.arguments;
console.log(`Calling tool ${toolName} with args ${JSON.stringify(args)}`);
// 2. Call the server's tool
const toolResult = await this.client.callTool({
name: toolName,
arguments: JSON.parse(args),
});
console.log("Tool result: ", toolResult);
// 3. Do something with the result
// TODO
}
}
async run() {
console.log("Asking server for available tools");
const toolsResult = await this.client.listTools();
const tools = toolsResult.tools.map((tool) => {
return this.openAiToolAdapter({
name: tool.name,
description: tool.description,
input_schema: tool.inputSchema,
});
});
const prompt = "What is the sum of 2 and 3?";
const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
{
role: "user",
content: prompt,
},
];
console.log("Querying LLM: ", messages[0].content);
let response = this.openai.chat.completions.create({
model: "gpt-4.1-mini",
max_tokens: 1000,
messages,
tools: tools,
});
let results: any[] = [];
// 3. Go through the LLM response, for each choice, check if it has tool calls
(await response).choices.map(async (choice: { message: any; }) => {
const message = choice.message;
if (message.tool_calls) {
console.log("Making tool call")
await this.callTools(message.tool_calls, results);
}
});
}
}
let client = new MyClient();
const transport = new StdioClientTransport({
command: "node",
args: ["./build/index.js"]
});
client.connectToServer(transport);
1. Let's add some imports needed to call an LLM
```python
# llm
import os
from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.models import SystemMessage, UserMessage
from azure.core.credentials import AzureKeyCredential
import json
```
2. Next, let's add the function that will call the LLM:
```python
# llm
def call_llm(prompt, functions):
token = os.environ["GITHUB_TOKEN"]
endpoint = "https://models.inference.ai.azure.com"
model_name = "gpt-4o"
client = ChatCompletionsClient(
endpoint=endpoint,
credential=AzureKeyCredential(token),
)
print("CALLING LLM")
response = client.complete(
messages=[
{
"role": "system",
"content": "You are a helpful assistant.",
},
{
"role": "user",
"content": prompt,
},
],
model=model_name,
tools = functions,
# Optional parameters
temperature=1.,
max_tokens=1000,
top_p=1.
)
response_message = response.choices[0].message
functions_to_call = []
if response_message.tool_calls:
for tool_call in response_message.tool_calls:
print("TOOL: ", tool_call)
name = tool_call.function.name
args = json.loads(tool_call.function.arguments)
functions_to_call.append({ "name": name, "args": args })
return functions_to_call
```
In the preceding code we've:
- Passed our functions, that we found on the MCP server and converted, to the LLM.
- Then we called the LLM with said functions.
- Then, we're inspecting the result to see what functions we should call, if any.
- Finally, we pass an array of functions to call.
3. Final step, let's update our main code:
```python
prompt = "Add 2 to 20"
# ask LLM what tools to all, if any
functions_to_call = call_llm(prompt, functions)
# call suggested functions
for f in functions_to_call:
result = await session.call_tool(f["name"], arguments=f["args"])
print("TOOLS result: ", result.content)
```
There, that was the final step, in the code above we're:
- Calling an MCP tool via call_tool using a function that the LLM thought we should call based on our prompt.
- Printing the result of the tool call to the MCP Server.
1. Let's show some code for doing an LLM prompt request:
```csharp
var tools = await GetMcpTools();
for (int i = 0; i < tools.Count; i++)
{
var tool = tools[i];
Console.WriteLine($"MCP Tools def: {i}: {tool}");
}
// 0. Define the chat history and the user message
var userMessage = "add 2 and 4";
chatHistory.Add(new ChatRequestUserMessage(userMessage));
// 1. Define tools
ChatCompletionsToolDefinition def = CreateToolDefinition();
// 2. Define options, including the tools
var options = new ChatCompletionsOptions(chatHistory)
{
Model = "gpt-4.1-mini",
Tools = { tools[0] }
};
// 3. Call the model
ChatCompletions? response = await client.CompleteAsync(options);
var content = response.Content;
```
In the preceding code we've:
- Fetched tools from the MCP server, var tools = await GetMcpTools().
- Defined a user prompt userMessage.
- Constructor an options object specifying model and tools.
- Made a request towards the LLM.
2. One last step, let's see if the LLM thinks we should call a function:
```csharp
// 4. Check if the response contains a function call
ChatCompletionsToolCall? calls = response.ToolCalls.FirstOrDefault();
for (int i = 0; i < response.ToolCalls.Count; i++)
{
var call = response.ToolCalls[i];
Console.WriteLine($"Tool call {i}: {call.Name} with arguments {call.Arguments}");
//Tool call 0: add with arguments {"a":2,"b":4}
var dict = JsonSerializer.Deserialize
var result = await mcpClient.CallToolAsync(
call.Name,
dict!,
cancellationToken: CancellationToken.None
);
Console.WriteLine(result.Content.First(c => c.Type == "text").Text);
}
```
In the preceding code we've:
- Looped through a list of function calls.
- For each tool call, parse out name and arguments and call the tool on the MCP server using the MCP client. Finally we print the results.
Here's the code in full:
using Azure;
using Azure.AI.Inference;
using Azure.Identity;
using System.Text.Json;
using ModelContextProtocol.Client;
using ModelContextProtocol.Protocol;
var endpoint = "https://models.inference.ai.azure.com";
var token = Environment.GetEnvironmentVariable("GITHUB_TOKEN"); // Your GitHub Access Token
var client = new ChatCompletionsClient(new Uri(endpoint), new AzureKeyCredential(token));
var chatHistory = new List<ChatRequestMessage>
{
new ChatRequestSystemMessage("You are a helpful assistant that knows about AI")
};
var clientTransport = new StdioClientTransport(new()
{
Name = "Demo Server",
Command = "/workspaces/mcp-for-beginners/03-GettingStarted/02-client/solution/server/bin/Debug/net8.0/server",
Arguments = [],
});
Console.WriteLine("Setting up stdio transport");
await using var mcpClient = await McpClient.CreateAsync(clientTransport);
ChatCompletionsToolDefinition ConvertFrom(string name, string description, JsonElement jsonElement)
{
// convert the tool to a function definition
FunctionDefinition functionDefinition = new FunctionDefinition(name)
{
Description = description,
Parameters = BinaryData.FromObjectAsJson(new
{
Type = "object",
Properties = jsonElement
},
new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase })
};
// create a tool definition
ChatCompletionsToolDefinition toolDefinition = new ChatCompletionsToolDefinition(functionDefinition);
return toolDefinition;
}
async Task<List<ChatCompletionsToolDefinition>> GetMcpTools()
{
Console.WriteLine("Listing tools");
var tools = await mcpClient.ListToolsAsync();
List<ChatCompletionsToolDefinition> toolDefinitions = new List<ChatCompletionsToolDefinition>();
foreach (var tool in tools)
{
Console.WriteLine($"Connected to server with tools: {tool.Name}");
Console.WriteLine($"Tool description: {tool.Description}");
Console.WriteLine($"Tool parameters: {tool.JsonSchema}");
JsonElement propertiesElement;
tool.JsonSchema.TryGetProperty("properties", out propertiesElement);
var def = ConvertFrom(tool.Name, tool.Description, propertiesElement);
Console.WriteLine($"Tool definition: {def}");
toolDefinitions.Add(def);
Console.WriteLine($"Properties: {propertiesElement}");
}
return toolDefinitions;
}
// 1. List tools on mcp server
var tools = await GetMcpTools();
for (int i = 0; i < tools.Count; i++)
{
var tool = tools[i];
Console.WriteLine($"MCP Tools def: {i}: {tool}");
}
// 2. Define the chat history and the user message
var userMessage = "add 2 and 4";
chatHistory.Add(new ChatRequestUserMessage(userMessage));
// 3. Define options, including the tools
var options = new ChatCompletionsOptions(chatHistory)
{
Model = "gpt-4.1-mini",
Tools = { tools[0] }
};
// 4. Call the model
ChatCompletions? response = await client.CompleteAsync(options);
var content = response.Content;
// 5. Check if the response contains a function call
ChatCompletionsToolCall? calls = response.ToolCalls.FirstOrDefault();
for (int i = 0; i < response.ToolCalls.Count; i++)
{
var call = response.ToolCalls[i];
Console.WriteLine($"Tool call {i}: {call.Name} with arguments {call.Arguments}");
//Tool call 0: add with arguments {"a":2,"b":4}
var dict = JsonSerializer.Deserialize<Dictionary<string, object>>(call.Arguments);
var result = await mcpClient.CallToolAsync(
call.Name,
dict!,
cancellationToken: CancellationToken.None
);
Console.WriteLine(result.Content.OfType<TextContentBlock>().First().Text);
}
// 6. Print the generic response
Console.WriteLine($"Assistant response: {content}");
try {
// Execute natural language requests that automatically use MCP tools
String response = bot.chat("Calculate the sum of 24.5 and 17.3 using the calculator service");
System.out.println(response);
response = bot.chat("What's the square root of 144?");
System.out.println(response);
response = bot.chat("Show me the help for the calculator service");
System.out.println(response);
} finally {
mcpClient.close();
}
In the preceding code we've:
- Converting user prompts to tool calls when needed
- Calling the appropriate MCP tools based on the LLM's decision
- Managing the conversation flow between the LLM and MCP server
bot.chat() method returns natural language responses that may include results from MCP tool executionsComplete code example:
public class LangChain4jClient {
public static void main(String[] args) throws Exception { ChatLanguageModel model = OpenAiOfficialChatModel.builder()
.isGitHubModels(true)
.apiKey(System.getenv("GITHUB_TOKEN"))
.timeout(Duration.ofSeconds(60))
.modelName("gpt-4.1-nano")
.timeout(Duration.ofSeconds(60))
.build();
McpTransport transport = new HttpMcpTransport.Builder()
.sseUrl("http://localhost:8080/sse")
.timeout(Duration.ofSeconds(60))
.logRequests(true)
.logResponses(true)
.build();
McpClient mcpClient = new DefaultMcpClient.Builder()
.transport(transport)
.build();
ToolProvider toolProvider = McpToolProvider.builder()
.mcpClients(List.of(mcpClient))
.build();
Bot bot = AiServices.builder(Bot.class)
.chatLanguageModel(model)
.toolProvider(toolProvider)
.build();
try {
String response = bot.chat("Calculate the sum of 24.5 and 17.3 using the calculator service");
System.out.println(response);
response = bot.chat("What's the square root of 144?");
System.out.println(response);
response = bot.chat("Show me the help for the calculator service");
System.out.println(response);
} finally {
mcpClient.close();
}
}
}
Here is where the majority of the work happens.
We will call the LLM with the initial user prompt, then process the response to see if any tools need to be called.
If so, we will call those tools and continue the conversation with the LLM until no more tool calls are needed and we have a final response.
We will be making multiple calls to the LLM, so let's define a function that will handle the LLM call.
Add the following function to your main.rs file:
async fn call_llm(
client: &Client<OpenAIConfig>,
messages: &[Value],
tools: &ListToolsResult,
) -> Result<Value, Box<dyn Error>> {
let response = client
.completions()
.create_byot(json!({
"messages": messages,
"model": "openai/gpt-4.1",
"tools": format_tools(tools).await?,
}))
.await?;
Ok(response)
}
This function takes the LLM client, a list of messages (including the user prompt), tools from the MCP server, and sends a request to the LLM, returning the response.
The response from the LLM will contain an array of choices.
We will need to process the result to see if any tool_calls are present.
This let's us know the LLM is requesting a specific tool should be called with arguments.
Add the following code to the bottom of your main.rs file to define a function to handle the LLM response:
async fn process_llm_response(
llm_response: &Value,
mcp_client: &RunningService<RoleClient, ()>,
openai_client: &Client<OpenAIConfig>,
mcp_tools: &ListToolsResult,
messages: &mut Vec<Value>,
) -> Result<(), Box<dyn Error>> {
let Some(message) = llm_response
.get("choices")
.and_then(|c| c.as_array())
.and_then(|choices| choices.first())
.and_then(|choice| choice.get("message"))
else {
return Ok(());
};
// Print content if available
if let Some(content) = message.get("content").and_then(|c| c.as_str()) {
println!("🤖 {}", content);
}
// Handle tool calls
if let Some(tool_calls) = message.get("tool_calls").and_then(|tc| tc.as_array()) {
messages.push(message.clone()); // Add assistant message
// Execute each tool call
for tool_call in tool_calls {
let (tool_id, name, args) = extract_tool_call_info(tool_call)?;
println!("⚡ Calling tool: {}", name);
let result = mcp_client
.call_tool(CallToolRequestParam {
name: name.into(),
arguments: serde_json::from_str::<Value>(&args)?.as_object().cloned(),
})
.await?;
// Add tool result to messages
messages.push(json!({
"role": "tool",
"tool_call_id": tool_id,
"content": serde_json::to_string_pretty(&result)?
}));
}
// Continue conversation with tool results
let response = call_llm(openai_client, messages, mcp_tools).await?;
Box::pin(process_llm_response(
&response,
mcp_client,
openai_client,
mcp_tools,
messages,
))
.await?;
}
Ok(())
}
If tool_calls are present, it extracts the tool information, calls the MCP server with the tool request, and adds the results to the conversation messages.
It then continues the conversation with the LLM and the messages are updated with the assistant's response and tool call results.
To extract tool call information that the LLM returns for MCP calls, we will add another helper function to extract everything needed to make the call.
Add the following code to the bottom of your main.rs file:
fn extract_tool_call_info(tool_call: &Value) -> Result<(String, String, String), Box<dyn Error>> {
let tool_id = tool_call
.get("id")
.and_then(|id| id.as_str())
.unwrap_or("")
.to_string();
let function = tool_call.get("function").ok_or("Missing function")?;
let name = function
.get("name")
.and_then(|n| n.as_str())
.unwrap_or("")
.to_string();
let args = function
.get("arguments")
.and_then(|a| a.as_str())
.unwrap_or("{}")
.to_string();
Ok((tool_id, name, args))
}
With all the pieces in place, we can now handle the initial user prompt and call the LLM. Update your main function to include the following code:
// LLM conversation with tool calls
let response = call_llm(&openai_client, &messages, &tools).await?;
process_llm_response(
&response,
&mcp_client,
&openai_client,
&tools,
&mut messages,
)
.await?;
This will query the LLM with the initial user prompt asking for the sum of two numbers, and it will process the response to dynamically handle tool calls.
Great, you did it!
Take the code from the exercise and build out the server with some more tools.
Then create a client with an LLM, like in the exercise, and test it out with different prompts to make sure all your server tools gets called dynamically.
This way of building a client means the end user will have a great user experience as they're able to use prompts, instead of exact client commands, and be oblivious to any MCP server being called.
지금까지 서버와 클라이언트를 만드는 방법을 보았습니다.
클라이언트는 서버를 명시적으로 호출하여 도구, 리소스, 프롬프트를 나열할 수 있었습니다.
하지만 이는 그리 실용적인 방법이 아닙니다.
사용자는 에이전트 시대에 살고 있으며 프롬프트를 사용하고 LLM과 소통하기를 기대합니다.
사용자가 MCP를 사용하여 기능을 저장하는지 여부는 신경 쓰지 않고 단순히 자연어를 사용하여 상호작용하기를 기대합니다.
그렇다면 이것을 어떻게 해결할 수 있을까요?
해결책은 클라이언트에 LLM을 추가하는 것입니다.
이번 수업에서는 클라이언트에 LLM을 추가하는 데 중점을 두며, 이를 통해 사용자에게 훨씬 더 나은 경험을 제공하는 방법을 보여줍니다.
이 수업이 끝나면 다음을 할 수 있습니다:
우리가 취해야 할 접근 방식을 이해해 봅시다. LLM을 추가하는 것은 간단해 보이지만 실제로 이렇게 할 수 있을까요?
클라이언트가 서버와 상호작용하는 방식은 다음과 같습니다:
1. 서버와 연결을 설정합니다.
1. 기능, 프롬프트, 리소스 및 도구를 나열하고 해당 스키마를 저장합니다.
1. LLM을 추가하고, 저장된 기능과 그들의 스키마를 LLM이 이해하는 형식으로 전달합니다.
1. 사용자 프롬프트를 처리하여 클라이언트가 나열한 도구와 함께 LLM에 전달합니다.
좋습니다. 이제 어떻게 고수준에서 이 작업을 수행할 수 있는지 이해했으니, 아래 연습 문제에서 직접 시도해 봅시다.
이 연습 문제에서는 클라이언트에 LLM을 추가하는 방법을 배웁니다.
GitHub 토큰 만드는 과정은 간단합니다. 방법은 다음과 같습니다:
먼저 클라이언트를 만들어 봅시다:
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
import { Transport } from "@modelcontextprotocol/sdk/shared/transport.js";
import OpenAI from "openai";
import { z } from "zod"; // 스키마 검증을 위해 zod를 가져옵니다
class MCPClient {
private openai: OpenAI;
private client: Client;
constructor(){
this.openai = new OpenAI({
baseURL: "https://models.inference.ai.azure.com",
apiKey: process.env.GITHUB_TOKEN,
});
this.client = new Client(
{
name: "example-client",
version: "1.0.0"
},
{
capabilities: {
prompts: {},
resources: {},
tools: {}
}
}
);
}
}
위 코드에서는:
client와 openai가 있는 클래스를 생성했습니다.baseUrl을 추론 API로 설정하여 GitHub 모델을 사용하도록 LLM 인스턴스를 구성했습니다.
from mcp import ClientSession, StdioServerParameters, types
from mcp.client.stdio import stdio_client
# stdio 연결을 위한 서버 매개변수 생성
server_params = StdioServerParameters(
command="mcp", # 실행 파일
args=["run", "server.py"], # 선택적 명령줄 인수
env=None, # 선택적 환경 변수
)
async def run():
async with stdio_client(server_params) as (read, write):
async with ClientSession(
read, write
) as session:
# 연결 초기화
await session.initialize()
if __name__ == "__main__":
import asyncio
asyncio.run(run())
위 코드에서는:
using Azure;
using Azure.AI.Inference;
using Azure.Identity;
using System.Text.Json;
using ModelContextProtocol.Client;
using System.Text.Json;
var clientTransport = new StdioClientTransport(new()
{
Name = "Demo Server",
Command = "/workspaces/mcp-for-beginners/03-GettingStarted/02-client/solution/server/bin/Debug/net8.0/server",
Arguments = [],
});
await using var mcpClient = await McpClient.CreateAsync(clientTransport);
먼저 pom.xml 파일에 LangChain4j 의존성을 추가해야 합니다. MCP 통합 및 GitHub 모델 지원을 가능하게 하는 의존성을 추가하세요:
<properties>
<langchain4j.version>1.0.0-beta3</langchain4j.version>
</properties>
<dependencies>
<!-- LangChain4j MCP Integration -->
<dependency>
<groupId>dev.langchain4j</groupId>
<artifactId>langchain4j-mcp</artifactId>
<version>${langchain4j.version}</version>
</dependency>
<!-- OpenAI Official API Client -->
<dependency>
<groupId>dev.langchain4j</groupId>
<artifactId>langchain4j-open-ai-official</artifactId>
<version>${langchain4j.version}</version>
</dependency>
<!-- GitHub Models Support -->
<dependency>
<groupId>dev.langchain4j</groupId>
<artifactId>langchain4j-github-models</artifactId>
<version>${langchain4j.version}</version>
</dependency>
<!-- Spring Boot Starter (optional, for production apps) -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
</dependencies>
그런 다음 Java 클라이언트 클래스를 만듭니다:
import dev.langchain4j.mcp.McpToolProvider;
import dev.langchain4j.mcp.client.DefaultMcpClient;
import dev.langchain4j.mcp.client.McpClient;
import dev.langchain4j.mcp.client.transport.McpTransport;
import dev.langchain4j.mcp.client.transport.http.HttpMcpTransport;
import dev.langchain4j.model.chat.ChatLanguageModel;
import dev.langchain4j.model.openaiofficial.OpenAiOfficialChatModel;
import dev.langchain4j.service.AiServices;
import dev.langchain4j.service.tool.ToolProvider;
import java.time.Duration;
import java.util.List;
public class LangChain4jClient {
public static void main(String[] args) throws Exception { // LLM이 GitHub 모델을 사용하도록 구성하기
ChatLanguageModel model = OpenAiOfficialChatModel.builder()
.isGitHubModels(true)
.apiKey(System.getenv("GITHUB_TOKEN"))
.timeout(Duration.ofSeconds(60))
.modelName("gpt-4.1-nano")
.build();
// 서버에 연결하기 위한 MCP 전송 생성하기
McpTransport transport = new HttpMcpTransport.Builder()
.sseUrl("http://localhost:8080/sse")
.timeout(Duration.ofSeconds(60))
.logRequests(true)
.logResponses(true)
.build();
// MCP 클라이언트 생성하기
McpClient mcpClient = new DefaultMcpClient.Builder()
.transport(transport)
.build();
}
}
위 코드에서는:
ChatLanguageModel 생성: GitHub 토큰으로 GitHub 모델을 사용하도록 구성함이 예제는 Rust 기반 MCP 서버가 실행 중임을 가정합니다. 서버가 없다면 01-first-server 수업으로 돌아가서 서버를 만드세요.
Rust MCP 서버가 준비되면 터미널을 열고 서버와 같은 디렉터리로 이동한 후 다음 명령어로 새 LLM 클라이언트 프로젝트를 생성합니다:
mkdir calculator-llmclient
cd calculator-llmclient
cargo init
Cargo.toml 파일에 다음 의존성을 추가하세요:
[dependencies]
async-openai = { version = "0.29.0", features = ["byot"] }
rmcp = { version = "0.5.0", features = ["client", "transport-child-process"] }
serde_json = "1.0.141"
tokio = { version = "1.46.1", features = ["rt-multi-thread"] }
> [!NOTE]
> OpenAI 공식 Rust 라이브러리는 없지만, async-openai 크레이트는 많이 사용되는 커뮤니티 유지 라이브러리입니다.
src/main.rs 파일을 열고 내용을 다음 코드로 교체하세요:
use async_openai::{Client, config::OpenAIConfig};
use rmcp::{
RmcpError,
model::{CallToolRequestParam, ListToolsResult},
service::{RoleClient, RunningService, ServiceExt},
transport::{ConfigureCommandExt, TokioChildProcess},
};
use serde_json::{Value, json};
use std::error::Error;
use tokio::process::Command;
#[tokio::main]
async fn main() -> Result<(), Box<dyn Error>> {
// 초기 메시지
let mut messages = vec![json!({"role": "user", "content": "What is the sum of 3 and 2?"})];
// OpenAI 클라이언트 설정
let api_key = std::env::var("OPENAI_API_KEY")?;
let openai_client = Client::with_config(
OpenAIConfig::new()
.with_api_base("https://models.github.ai/inference/chat")
.with_api_key(api_key),
);
// MCP 클라이언트 설정
let server_dir = std::path::Path::new(env!("CARGO_MANIFEST_DIR"))
.parent()
.unwrap()
.join("calculator-server");
let mcp_client = ()
.serve(
TokioChildProcess::new(Command::new("cargo").configure(|cmd| {
cmd.arg("run").current_dir(server_dir);
}))
.map_err(RmcpError::transport_creation::<TokioChildProcess>)?,
)
.await?;
// TODO: MCP 도구 목록 가져오기
// TODO: 도구 호출과 함께 LLM 대화
Ok(())
}
이 코드는 MCP 서버와 GitHub 모델에 연결할 기본 Rust 애플리케이션을 설정합니다.
> [!IMPORTANT]
> 애플리케이션을 실행하기 전에 반드시 OPENAI_API_KEY 환경 변수에 GitHub 토큰을 설정하세요.
좋습니다, 다음 단계로 서버에서 기능 목록을 불러옵시다.
이제 서버에 연결하여 기능을 요청해 봅시다:
같은 클래스에 다음 메서드를 추가하세요:
async connectToServer(transport: Transport) {
await this.client.connect(transport);
this.run();
console.error("MCPClient started on stdin/stdout");
}
async run() {
console.log("Asking server for available tools");
// 도구 나열하기
const toolsResult = await this.client.listTools();
}
위 코드에서:
connectToServer 코드를 추가했습니다.run 메서드를 만들었습니다. 현재는 도구만 나열하지만 곧 더 추가할 예정입니다.
# 사용 가능한 리소스 목록
resources = await session.list_resources()
print("LISTING RESOURCES")
for resource in resources:
print("Resource: ", resource)
# 사용 가능한 도구 목록
tools = await session.list_tools()
print("LISTING TOOLS")
for tool in tools.tools:
print("Tool: ", tool.name)
print("Tool", tool.inputSchema["properties"])
다음과 같이 추가했습니다:
inputSchema도 나열했습니다.
async Task<List<ChatCompletionsToolDefinition>> GetMcpTools()
{
Console.WriteLine("Listing tools");
var tools = await mcpClient.ListToolsAsync();
List<ChatCompletionsToolDefinition> toolDefinitions = new List<ChatCompletionsToolDefinition>();
foreach (var tool in tools)
{
Console.WriteLine($"Connected to server with tools: {tool.Name}");
Console.WriteLine($"Tool description: {tool.Description}");
Console.WriteLine($"Tool parameters: {tool.JsonSchema}");
// TODO: convert tool definition from MCP tool to LLm tool
}
return toolDefinitions;
}
위 코드에서:
// MCP 도구를 자동으로 검색하는 도구 제공자 생성
ToolProvider toolProvider = McpToolProvider.builder()
.mcpClients(List.of(mcpClient))
.build();
// MCP 도구 제공자는 자동으로 다음을 처리합니다:
// - MCP 서버에서 사용 가능한 도구 목록 작성
// - MCP 도구 스키마를 LangChain4j 형식으로 변환
// - 도구 실행 및 응답 관리
위 코드에서:
McpToolProvider를 만들었습니다.MCP 서버에서 도구를 가져오는 것은 list_tools 메서드를 사용합니다. main 함수에서 MCP 클라이언트를 설정한 후 다음 코드를 추가하세요:
// MCP 도구 목록 가져오기
let tools = mcp_client.list_tools(Default::default()).await?;
서버 기능 목록을 불러온 다음, LLM이 이해할 수 있는 형식으로 변환해야 합니다. 변환하면 이를 LLM의 도구로 제공할 수 있습니다.
1. 다음 코드를 추가하여 MCP 서버 응답을 LLM이 사용할 도구 형식으로 변환하세요:
```typescript
openAiToolAdapter(tool: {
name: string;
description?: string;
input_schema: any;
}) {
// input_schema를 기반으로 zod 스키마 생성
const schema = z.object(tool.input_schema);
return {
type: "function" as const, // 타입을 명시적으로 "function"으로 설정
function: {
name: tool.name,
description: tool.description,
parameters: {
type: "object",
properties: tool.input_schema.properties,
required: tool.input_schema.required,
},
},
};
}
```
위 코드는 MCP 서버 응답을 받아 LLM이 이해할 수 있는 도구 정의 형식으로 변환합니다.
2. 다음으로 run 메서드를 업데이트하여 서버 기능을 나열해 봅시다:
```typescript
async run() {
console.log("Asking server for available tools");
const toolsResult = await this.client.listTools();
const tools = toolsResult.tools.map((tool) => {
return this.openAiToolAdapter({
name: tool.name,
description: tool.description,
input_schema: tool.inputSchema,
});
});
}
```
앞 코드에서 run 메서드를 업데이트하여 결과에서 각 항목에 대해 openAiToolAdapter를 호출하도록 했습니다.
1. 먼저 다음 변환 함수를 만듭니다:
```python
def convert_to_llm_tool(tool):
tool_schema = {
"type": "function",
"function": {
"name": tool.name,
"description": tool.description,
"type": "function",
"parameters": {
"type": "object",
"properties": tool.inputSchema["properties"]
}
}
}
return tool_schema
```
위 함수 convert_to_llm_tools는 MCP 도구 응답을 받아 LLM이 이해할 수 있는 형식으로 변환합니다.
2. 다음으로 이 함수를 활용하도록 클라이언트 코드를 업데이트합니다:
```python
functions = []
for tool in tools.tools:
print("Tool: ", tool.name)
print("Tool", tool.inputSchema["properties"])
functions.append(convert_to_llm_tool(tool))
```
여기서는 MCP 도구 응답을 LLM에 공급할 수 있게 변환하는 convert_to_llm_tool 호출을 추가했습니다.
1. MCP 도구 응답을 LLM이 이해할 수 있는 형태로 변환하는 코드를 추가합니다:
ChatCompletionsToolDefinition ConvertFrom(string name, string description, JsonElement jsonElement)
{
// convert the tool to a function definition
FunctionDefinition functionDefinition = new FunctionDefinition(name)
{
Description = description,
Parameters = BinaryData.FromObjectAsJson(new
{
Type = "object",
Properties = jsonElement
},
new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase })
};
// create a tool definition
ChatCompletionsToolDefinition toolDefinition = new ChatCompletionsToolDefinition(functionDefinition);
return toolDefinition;
}
위 코드에서는:
ConvertFrom 함수를 만들었습니다.FunctionDefinition을 생성하며, 이는 ChatCompletionsDefinition에 전달됩니다. 후자는 LLM이 이해하는 형식입니다.2. 위 함수를 활용하도록 기존 코드를 업데이트해 봅시다:
```csharp
async Task
{
Console.WriteLine("Listing tools");
var tools = await mcpClient.ListToolsAsync();
List
foreach (var tool in tools)
{
Console.WriteLine($"Connected to server with tools: {tool.Name}");
Console.WriteLine($"Tool description: {tool.Description}");
Console.WriteLine($"Tool parameters: {tool.JsonSchema}");
JsonElement propertiesElement;
tool.JsonSchema.TryGetProperty("properties", out propertiesElement);
var def = ConvertFrom(tool.Name, tool.Description, propertiesElement);
Console.WriteLine($"Tool definition: {def}");
toolDefinitions.Add(def);
Console.WriteLine($"Properties: {propertiesElement}");
}
return toolDefinitions;
}
``` In the preceding code, we've:
- Update the function to convert the MCP tool response to an LLm tool. Let's highlight the code we added:
```csharp
JsonElement propertiesElement;
tool.JsonSchema.TryGetProperty("properties", out propertiesElement);
var def = ConvertFrom(tool.Name, tool.Description, propertiesElement);
Console.WriteLine($"Tool definition: {def}");
toolDefinitions.Add(def);
```
The input schema is part of the tool response but on the "properties" attribute, so we need to extract.
Furthermore, we now call ConvertFrom with the tool details.
Now we've done the heavy lifting, let's see how it call comes together as we handle a user prompt next.
// 자연어 상호작용을 위한 Bot 인터페이스 생성
public interface Bot {
String chat(String prompt);
}
// LLM 및 MCP 도구로 AI 서비스 구성
Bot bot = AiServices.builder(Bot.class)
.chatLanguageModel(model)
.toolProvider(toolProvider)
.build();
위 코드에서는:
Bot 인터페이스를 정의했습니다.AiServices를 사용해 LLM과 MCP 도구 제공자를 자동 바인딩했습니다.MCP 도구 응답을 LLM이 이해할 수 있는 형식으로 변환하려면 도구 목록을 형식화하는 헬퍼 함수를 추가해야 합니다. main 함수 아래에 다음 코드를 추가하세요. LLM에 요청할 때 호출됩니다:
async fn format_tools(tools: &ListToolsResult) -> Result<Vec<Value>, Box<dyn Error>> {
let tools_json = serde_json::to_value(tools)?;
let Some(tools_array) = tools_json.get("tools").and_then(|t| t.as_array()) else {
return Ok(vec![]);
};
let formatted_tools = tools_array
.iter()
.filter_map(|tool| {
let name = tool.get("name")?.as_str()?;
let description = tool.get("description")?.as_str()?;
let schema = tool.get("inputSchema")?;
Some(json!({
"type": "function",
"function": {
"name": name,
"description": description,
"parameters": {
"type": "object",
"properties": schema.get("properties").unwrap_or(&json!({})),
"required": schema.get("required").unwrap_or(&json!([]))
}
}
}))
})
.collect();
Ok(formatted_tools)
}
좋습니다, 이제 사용자 요청을 처리할 준비가 되었으니 다음 단계로 넘어갑시다.
이 코드 부분에서는 사용자 요청을 처리합니다.
1. LLM을 호출하는데 사용할 메서드를 추가하세요:
```typescript
async callTools(
tool_calls: OpenAI.Chat.Completions.ChatCompletionMessageToolCall[],
toolResults: any[]
) {
for (const tool_call of tool_calls) {
const toolName = tool_call.function.name;
const args = tool_call.function.arguments;
console.log(Calling tool ${toolName} with args ${JSON.stringify(args)});
// 2. 서버의 도구를 호출합니다
const toolResult = await this.client.callTool({
name: toolName,
arguments: JSON.parse(args),
});
console.log("Tool result: ", toolResult);
// 3. 결과를 가지고 무언가를 합니다
// 할 일
}
}
```
위 코드에서:
- callTools 메서드를 추가했습니다.
- 메서드는 LLM 응답을 받고 어떤 도구가 호출되었는지 검사합니다:
```typescript
for (const tool_call of tool_calls) {
const toolName = tool_call.function.name;
const args = tool_call.function.arguments;
console.log(Calling tool ${toolName} with args ${JSON.stringify(args)});
// 도구 호출
}
```
- LLM이 호출할 것을 지시하면 도구를 호출합니다:
```typescript
// 2. 서버의 도구를 호출합니다
const toolResult = await this.client.callTool({
name: toolName,
arguments: JSON.parse(args),
});
console.log("Tool result: ", toolResult);
// 3. 결과로 무언가를 합니다
// 할 일
```
2. run 메서드를 업데이트하여 LLM 호출과 callTools 호출을 포함시킵니다:
```typescript
// 1. LLM의 입력으로 사용할 메시지를 생성합니다
const prompt = "What is the sum of 2 and 3?"
const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
{
role: "user",
content: prompt,
},
];
console.log("Querying LLM: ", messages[0].content);
// 2. LLM 호출
let response = this.openai.chat.completions.create({
model: "gpt-4.1-mini",
max_tokens: 1000,
messages,
tools: tools,
});
let results: any[] = [];
// 3. LLM 응답을 확인하며 각 선택지에 도구 호출이 있는지 점검합니다
(await response).choices.map(async (choice: { message: any; }) => {
const message = choice.message;
if (message.tool_calls) {
console.log("Making tool call")
await this.callTools(message.tool_calls, results);
}
});
```
좋습니다, 전체 코드를 나열해 봅시다:
import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StdioClientTransport } from "@modelcontextprotocol/sdk/client/stdio.js";
import { Transport } from "@modelcontextprotocol/sdk/shared/transport.js";
import OpenAI from "openai";
import { z } from "zod"; // 스키마 유효성 검사를 위해 zod를 가져옵니다
class MyClient {
private openai: OpenAI;
private client: Client;
constructor(){
this.openai = new OpenAI({
baseURL: "https://models.inference.ai.azure.com", // 미래에 이 URL로 변경해야 할 수도 있습니다: https://models.github.ai/inference
apiKey: process.env.GITHUB_TOKEN,
});
this.client = new Client(
{
name: "example-client",
version: "1.0.0"
},
{
capabilities: {
prompts: {},
resources: {},
tools: {}
}
}
);
}
async connectToServer(transport: Transport) {
await this.client.connect(transport);
this.run();
console.error("MCPClient started on stdin/stdout");
}
openAiToolAdapter(tool: {
name: string;
description?: string;
input_schema: any;
}) {
// input_schema를 기반으로 zod 스키마를 생성합니다
const schema = z.object(tool.input_schema);
return {
type: "function" as const, // 타입을 명시적으로 "function"으로 설정합니다
function: {
name: tool.name,
description: tool.description,
parameters: {
type: "object",
properties: tool.input_schema.properties,
required: tool.input_schema.required,
},
},
};
}
async callTools(
tool_calls: OpenAI.Chat.Completions.ChatCompletionMessageToolCall[],
toolResults: any[]
) {
for (const tool_call of tool_calls) {
const toolName = tool_call.function.name;
const args = tool_call.function.arguments;
console.log(`Calling tool ${toolName} with args ${JSON.stringify(args)}`);
// 2. 서버의 도구를 호출합니다
const toolResult = await this.client.callTool({
name: toolName,
arguments: JSON.parse(args),
});
console.log("Tool result: ", toolResult);
// 3. 결과로 무언가를 수행합니다
// 해야 할 일
}
}
async run() {
console.log("Asking server for available tools");
const toolsResult = await this.client.listTools();
const tools = toolsResult.tools.map((tool) => {
return this.openAiToolAdapter({
name: tool.name,
description: tool.description,
input_schema: tool.inputSchema,
});
});
const prompt = "What is the sum of 2 and 3?";
const messages: OpenAI.Chat.Completions.ChatCompletionMessageParam[] = [
{
role: "user",
content: prompt,
},
];
console.log("Querying LLM: ", messages[0].content);
let response = this.openai.chat.completions.create({
model: "gpt-4.1-mini",
max_tokens: 1000,
messages,
tools: tools,
});
let results: any[] = [];
// 3. LLM 응답을 확인하고, 각 선택지에 도구 호출이 있는지 검사합니다
(await response).choices.map(async (choice: { message: any; }) => {
const message = choice.message;
if (message.tool_calls) {
console.log("Making tool call")
await this.callTools(message.tool_calls, results);
}
});
}
}
let client = new MyClient();
const transport = new StdioClientTransport({
command: "node",
args: ["./build/index.js"]
});
client.connectToServer(transport);
1. LLM 호출에 필요한 일부 import를 추가합시다:
```python
# 대형 언어 모델
import os
from azure.ai.inference import ChatCompletionsClient
from azure.ai.inference.models import SystemMessage, UserMessage
from azure.core.credentials import AzureKeyCredential
import json
```
2. LLM을 호출하는 함수를 추가합시다:
```python
# llm
def call_llm(prompt, functions):
token = os.environ["GITHUB_TOKEN"]
endpoint = "https://models.inference.ai.azure.com"
model_name = "gpt-4o"
client = ChatCompletionsClient(
endpoint=endpoint,
credential=AzureKeyCredential(token),
)
print("CALLING LLM")
response = client.complete(
messages=[
{
"role": "system",
"content": "You are a helpful assistant.",
},
{
"role": "user",
"content": prompt,
},
],
model=model_name,
tools = functions,
# 선택적 매개변수
temperature=1.,
max_tokens=1000,
top_p=1.
)
response_message = response.choices[0].message
functions_to_call = []
if response_message.tool_calls:
for tool_call in response_message.tool_calls:
print("TOOL: ", tool_call)
name = tool_call.function.name
args = json.loads(tool_call.function.arguments)
functions_to_call.append({ "name": name, "args": args })
return functions_to_call
```
위 코드에서:
- MCP 서버에서 찾고 변환한 함수들을 LLM에 전달했습니다.
- 해당 함수들과 함께 LLM을 호출했습니다.
- 결과를 검사하여 호출해야 할 함수가 있으면 호출합니다.
- 호출할 함수 목록 배열을 전달합니다.
3. 마지막으로 메인 코드를 업데이트합시다:
```python
prompt = "Add 2 to 20"
# 어떤 도구가 모두 필요한지 LLM에 물어보세요, 만약 있다면
functions_to_call = call_llm(prompt, functions)
# 제안된 함수를 호출하세요
for f in functions_to_call:
result = await session.call_tool(f["name"], arguments=f["args"])
print("TOOLS result: ", result.content)
```
위 코드가 마지막 단계입니다. 여기서 우리는:
- LLM이 프롬프트에 따라 호출해야 한다고 판단한 함수를 사용해 call_tool로 MCP 도구를 호출했습니다.
- 도구 호출 결과를 출력했습니다.
1. LLM 프롬프트 요청을 수행하는 코드를 봅시다:
```csharp
var tools = await GetMcpTools();
for (int i = 0; i < tools.Count; i++)
{
var tool = tools[i];
Console.WriteLine($"MCP Tools def: {i}: {tool}");
}
// 0. Define the chat history and the user message
var userMessage = "add 2 and 4";
chatHistory.Add(new ChatRequestUserMessage(userMessage));
// 1. Define tools
ChatCompletionsToolDefinition def = CreateToolDefinition();
// 2. Define options, including the tools
var options = new ChatCompletionsOptions(chatHistory)
{
Model = "gpt-4.1-mini",
Tools = { tools[0] }
};
// 3. Call the model
ChatCompletions? response = await client.CompleteAsync(options);
var content = response.Content;
```
위 코드에서는:
- MCP 서버에서 도구를 가져왔습니다: var tools = await GetMcpTools().
- 사용자 프롬프트를 정의했습니다: userMessage.
- 모델과 도구를 지정하는 옵션 객체를 만들었습니다.
- LLM에 요청을 보냈습니다.
2. 마지막 단계로 LLM이 함수 호출이 필요하다고 판단하는지 확인합니다:
```csharp
// 4. Check if the response contains a function call
ChatCompletionsToolCall? calls = response.ToolCalls.FirstOrDefault();
for (int i = 0; i < response.ToolCalls.Count; i++)
{
var call = response.ToolCalls[i];
Console.WriteLine($"Tool call {i}: {call.Name} with arguments {call.Arguments}");
//Tool call 0: add with arguments {"a":2,"b":4}
var dict = JsonSerializer.Deserialize
var result = await mcpClient.CallToolAsync(
call.Name,
dict!,
cancellationToken: CancellationToken.None
);
Console.WriteLine(result.Content.First(c => c.Type == "text").Text);
}
```
위 코드에서는:
- 함수 호출 목록을 반복했습니다.
- 각 도구 호출에 대해 이름과 인수를 파싱하고, MCP 클라이언트를 통해 MCP 서버에서 도구를 호출합니다. 결과를 출력합니다.
전체 코드는 다음과 같습니다:
using Azure;
using Azure.AI.Inference;
using Azure.Identity;
using System.Text.Json;
using ModelContextProtocol.Client;
using ModelContextProtocol.Protocol;
var endpoint = "https://models.inference.ai.azure.com";
var token = Environment.GetEnvironmentVariable("GITHUB_TOKEN"); // Your GitHub Access Token
var client = new ChatCompletionsClient(new Uri(endpoint), new AzureKeyCredential(token));
var chatHistory = new List<ChatRequestMessage>
{
new ChatRequestSystemMessage("You are a helpful assistant that knows about AI")
};
var clientTransport = new StdioClientTransport(new()
{
Name = "Demo Server",
Command = "/workspaces/mcp-for-beginners/03-GettingStarted/02-client/solution/server/bin/Debug/net8.0/server",
Arguments = [],
});
Console.WriteLine("Setting up stdio transport");
await using var mcpClient = await McpClient.CreateAsync(clientTransport);
ChatCompletionsToolDefinition ConvertFrom(string name, string description, JsonElement jsonElement)
{
// convert the tool to a function definition
FunctionDefinition functionDefinition = new FunctionDefinition(name)
{
Description = description,
Parameters = BinaryData.FromObjectAsJson(new
{
Type = "object",
Properties = jsonElement
},
new JsonSerializerOptions() { PropertyNamingPolicy = JsonNamingPolicy.CamelCase })
};
// create a tool definition
ChatCompletionsToolDefinition toolDefinition = new ChatCompletionsToolDefinition(functionDefinition);
return toolDefinition;
}
async Task<List<ChatCompletionsToolDefinition>> GetMcpTools()
{
Console.WriteLine("Listing tools");
var tools = await mcpClient.ListToolsAsync();
List<ChatCompletionsToolDefinition> toolDefinitions = new List<ChatCompletionsToolDefinition>();
foreach (var tool in tools)
{
Console.WriteLine($"Connected to server with tools: {tool.Name}");
Console.WriteLine($"Tool description: {tool.Description}");
Console.WriteLine($"Tool parameters: {tool.JsonSchema}");
JsonElement propertiesElement;
tool.JsonSchema.TryGetProperty("properties", out propertiesElement);
var def = ConvertFrom(tool.Name, tool.Description, propertiesElement);
Console.WriteLine($"Tool definition: {def}");
toolDefinitions.Add(def);
Console.WriteLine($"Properties: {propertiesElement}");
}
return toolDefinitions;
}
// 1. List tools on mcp server
var tools = await GetMcpTools();
for (int i = 0; i < tools.Count; i++)
{
var tool = tools[i];
Console.WriteLine($"MCP Tools def: {i}: {tool}");
}
// 2. Define the chat history and the user message
var userMessage = "add 2 and 4";
chatHistory.Add(new ChatRequestUserMessage(userMessage));
// 3. Define options, including the tools
var options = new ChatCompletionsOptions(chatHistory)
{
Model = "gpt-4.1-mini",
Tools = { tools[0] }
};
// 4. Call the model
ChatCompletions? response = await client.CompleteAsync(options);
var content = response.Content;
// 5. Check if the response contains a function call
ChatCompletionsToolCall? calls = response.ToolCalls.FirstOrDefault();
for (int i = 0; i < response.ToolCalls.Count; i++)
{
var call = response.ToolCalls[i];
Console.WriteLine($"Tool call {i}: {call.Name} with arguments {call.Arguments}");
//Tool call 0: add with arguments {"a":2,"b":4}
var dict = JsonSerializer.Deserialize<Dictionary<string, object>>(call.Arguments);
var result = await mcpClient.CallToolAsync(
call.Name,
dict!,
cancellationToken: CancellationToken.None
);
Console.WriteLine(result.Content.OfType<TextContentBlock>().First().Text);
}
// 6. Print the generic response
Console.WriteLine($"Assistant response: {content}");
try {
// MCP 도구를 자동으로 사용하는 자연어 요청 실행
String response = bot.chat("Calculate the sum of 24.5 and 17.3 using the calculator service");
System.out.println(response);
response = bot.chat("What's the square root of 144?");
System.out.println(response);
response = bot.chat("Show me the help for the calculator service");
System.out.println(response);
} finally {
mcpClient.close();
}
위 코드에서는:
- 필요 시 사용자 프롬프트를 도구 호출로 변환
- LLM 판단에 따라 적절한 MCP 도구 호출
- LLM과 MCP 서버 간 대화 흐름 관리
bot.chat() 메서드는 MCP 도구 실행 결과를 포함할 수 있는 자연어 응답을 반환합니다.전체 코드 예제:
public class LangChain4jClient {
public static void main(String[] args) throws Exception { ChatLanguageModel model = OpenAiOfficialChatModel.builder()
.isGitHubModels(true)
.apiKey(System.getenv("GITHUB_TOKEN"))
.timeout(Duration.ofSeconds(60))
.modelName("gpt-4.1-nano")
.timeout(Duration.ofSeconds(60))
.build();
McpTransport transport = new HttpMcpTransport.Builder()
.sseUrl("http://localhost:8080/sse")
.timeout(Duration.ofSeconds(60))
.logRequests(true)
.logResponses(true)
.build();
McpClient mcpClient = new DefaultMcpClient.Builder()
.transport(transport)
.build();
ToolProvider toolProvider = McpToolProvider.builder()
.mcpClients(List.of(mcpClient))
.build();
Bot bot = AiServices.builder(Bot.class)
.chatLanguageModel(model)
.toolProvider(toolProvider)
.build();
try {
String response = bot.chat("Calculate the sum of 24.5 and 17.3 using the calculator service");
System.out.println(response);
response = bot.chat("What's the square root of 144?");
System.out.println(response);
response = bot.chat("Show me the help for the calculator service");
System.out.println(response);
} finally {
mcpClient.close();
}
}
}
여기서 대부분 작업이 일어납니다. 초기 사용자 프롬프트로 LLM을 호출한 후 응답을 처리하여 호출해야 할 도구가 있는지 확인합니다. 있다면 해당 도구들을 호출하고, 더 이상 도구 호출이 필요 없고 최종 응답이 나올 때까지 LLM과 대화를 계속합니다.
LLM을 여러 번 호출해야 하므로, LLM 호출을 처리하는 함수를 정의합시다. main.rs 파일에 다음 함수를 추가하세요:
async fn call_llm(
client: &Client<OpenAIConfig>,
messages: &[Value],
tools: &ListToolsResult,
) -> Result<Value, Box<dyn Error>> {
let response = client
.completions()
.create_byot(json!({
"messages": messages,
"model": "openai/gpt-4.1",
"tools": format_tools(tools).await?,
}))
.await?;
Ok(response)
}
이 함수는 LLM 클라이언트, 메시지 목록(사용자 프롬프트 포함), MCP 서버 도구를 받아 LLM에 요청을 보내고 응답을 반환합니다.
LLM 응답에는 choices 배열이 포함됩니다.
결과를 처리하여 tool_calls가 있는지 확인해야 합니다.
이를 통해 LLM이 인수와 함께 특정 도구를 호출하도록 요청하는지를 알 수 있습니다.
다음 코드를 main.rs 파일 하단에 추가하여 LLM 응답을 처리하는 함수를 정의하세요:
async fn process_llm_response(
llm_response: &Value,
mcp_client: &RunningService<RoleClient, ()>,
openai_client: &Client<OpenAIConfig>,
mcp_tools: &ListToolsResult,
messages: &mut Vec<Value>,
) -> Result<(), Box<dyn Error>> {
let Some(message) = llm_response
.get("choices")
.and_then(|c| c.as_array())
.and_then(|choices| choices.first())
.and_then(|choice| choice.get("message"))
else {
return Ok(());
};
// 사용 가능한 경우 내용을 출력합니다
if let Some(content) = message.get("content").and_then(|c| c.as_str()) {
println!("🤖 {}", content);
}
// 도구 호출 처리
if let Some(tool_calls) = message.get("tool_calls").and_then(|tc| tc.as_array()) {
messages.push(message.clone()); // 어시스턴트 메시지 추가
// 각 도구 호출 실행
for tool_call in tool_calls {
let (tool_id, name, args) = extract_tool_call_info(tool_call)?;
println!("⚡ Calling tool: {}", name);
let result = mcp_client
.call_tool(CallToolRequestParam {
name: name.into(),
arguments: serde_json::from_str::<Value>(&args)?.as_object().cloned(),
})
.await?;
// 도구 결과를 메시지에 추가
messages.push(json!({
"role": "tool",
"tool_call_id": tool_id,
"content": serde_json::to_string_pretty(&result)?
}));
}
// 도구 결과로 대화 계속하기
let response = call_llm(openai_client, messages, mcp_tools).await?;
Box::pin(process_llm_response(
&response,
mcp_client,
openai_client,
mcp_tools,
messages,
))
.await?;
}
Ok(())
}
tool_calls가 있으면 도구 정보를 추출하고, MCP 서버에 도구 요청을 호출한 다음 결과를 대화 메시지에 추가합니다. 그런 다음 LLM과 계속 대화를 나누며, 메시지는 어시스턴트 응답과 도구 호출 결과로 업데이트됩니다.
LLM이 MCP 호출을 위해 반환하는 도구 호출 정보를 추출하려면, 호출에 필요한 모든 정보를 추출하는 또 다른 도우미 함수를 추가합니다. 다음 코드를 main.rs 파일 하단에 추가하세요:
fn extract_tool_call_info(tool_call: &Value) -> Result<(String, String, String), Box<dyn Error>> {
let tool_id = tool_call
.get("id")
.and_then(|id| id.as_str())
.unwrap_or("")
.to_string();
let function = tool_call.get("function").ok_or("Missing function")?;
let name = function
.get("name")
.and_then(|n| n.as_str())
.unwrap_or("")
.to_string();
let args = function
.get("arguments")
.and_then(|a| a.as_str())
.unwrap_or("{}")
.to_string();
Ok((tool_id, name, args))
}
모든 부분이 준비되었으니 초기 사용자 프롬프트를 처리하고 LLM을 호출할 수 있습니다. main 함수를 다음 코드로 업데이트하세요:
// 도구 호출이 포함된 LLM 대화
let response = call_llm(&openai_client, &messages, &tools).await?;
process_llm_response(
&response,
&mcp_client,
&openai_client,
&tools,
&mut messages,
)
.await?;
이 코드는 두 숫자의 합을 요청하는 초기 사용자 프롬프트로 LLM에 질의하며, 응답을 처리하여 도구 호출을 동적으로 처리합니다.
잘하셨습니다!
연습에서 작성한 코드를 바탕으로 더 많은 도구를 갖춘 서버를 구축하세요. 그런 다음 연습과 같이 LLM 클라이언트를 만들고, 다양한 프롬프트로 테스트하여 모든 서버 도구가 동적으로 호출되는지 확인하세요. 이렇게 클라이언트를 구축하면 최종 사용자가 정확한 클라이언트 명령 대신 프롬프트를 사용할 수 있어 훌륭한 사용자 경험을 제공합니다. MCP 서버 호출이 있는지도 인지하지 못합니다.
---
면책 조항:
이 문서는 AI 번역 서비스 Co-op Translator를 사용하여 번역되었습니다.
정확성을 위해 노력하고 있으나, 자동 번역에는 오류나 부정확한 부분이 있을 수 있음을 양지하시기 바랍니다.
원문 문서가 권위 있는 출처로 간주되어야 합니다.
중요한 정보의 경우 전문적인 인간 번역을 권장합니다.
이 번역 사용으로 인해 발생하는 오해나 오해석에 대해 당사는 책임을 지지 않습니다.