A Graph Database

In the era of big data, the way we store and manage information has evolved significantly. Traditional relational databases, while effective for many applications, often struggle with complex data relationships. Enter Neo4j, a leading graph database that allows users to model and query data in a way that reflects real-world relationships. This guide will walk you through the essentials of using Neo4j, from installation to practical applications, ensuring you have a solid foundation to leverage this powerful tool.

Understanding Neo4j

What is a Graph Database?

At its core, a graph database is designed to represent and store data in the form of nodes and relationships. In Neo4j:

• Nodes represent entities (e.g., people, products, locations).
• Relationships define how these entities are connected (e.g., friendships, purchases, and geographical proximity).

This structure makes graph databases particularly well-suited for applications that require complex querying of interconnected data, such as social networks, recommendation systems, and fraud detection.

The Cypher Query Language

Neo4j utilizes Cypher, a declarative query language specifically designed for graph data. Cypher allows users to express what data they want to retrieve without needing to specify how to get it. This makes it intuitive and powerful for querying complex relationships.

Getting Started with Neo4j

Installation

To begin using Neo4j, you need to install it on your machine. There are two primary methods for installation:

1. Download the Neo4j Community Edition:

• Visit the official Neo4j website and download the Community Edition, which is free and open-source.

1. Using Docker:

• If you prefer containerization, you can easily run Neo4j using Docker. Open your terminal and execute the following commands:
  bash docker pull neo4j docker run -p7474:7474 -p7687:7687 neo4j
• This command pulls the latest Neo4j image and runs it, exposing the necessary ports for web access and Bolt protocol.

Using Neo4j Sandbox

For those who are new to Neo4j or want to experiment without installation, the Neo4j Sandbox is an excellent resource. It provides a cloud-based environment with sample datasets and guided tutorials, allowing you to explore Neo4j’s capabilities without any setup.

Basic Cypher Commands

Once you have Neo4j up and running, you can start interacting with it using Cypher. Here are some fundamental commands to get you started:

Creating Nodes

To create a new node, you can use the following command:

CREATE (n:Person {name: 'Alice', age: 30})

This command creates a node labeled Person with properties name and age.

Creating Relationships

To establish a relationship between two nodes, you can use the MATCH and CREATE commands:

MATCH (a:Person {name: 'Alice'})
CREATE (a)-[:FRIENDS_WITH]->(b:Person {name: 'Bob'})

This command finds the node representing Alice and creates a FRIENDS_WITH relationship to a new node representing Bob.

Querying Data

To retrieve data from your graph, you can use the MATCH command:

MATCH (n:Person) RETURN n

This command returns all nodes labeled Person, allowing you to see the data you’ve created.

Real-World Use Cases

Neo4j’s graph structure lends itself to various applications across different industries. Here are some notable use cases:

1. Social Networks

Graph databases excel at modeling relationships, making them ideal for social networking applications. You can easily represent users, their connections, and interactions, enabling features like friend suggestions and relationship analysis.

2. Recommendation Systems

By analyzing user behavior and preferences, Neo4j can power recommendation engines. For instance, you can suggest products based on users’ past purchases or their friends’ activities, enhancing user engagement and satisfaction.

3. Fraud Detection

In financial services, Neo4j can help identify fraudulent activities by analyzing connections between transactions. By visualizing relationships, you can uncover suspicious patterns that may indicate fraud, allowing for timely intervention.

Resources for Learning Neo4j

To deepen your understanding of Neo4j and its capabilities, consider exploring the following resources:

Tutorials

• TutorialsPoint Neo4j Tutorial: A comprehensive guide that covers everything from the basics to advanced topics in Neo4j.
• DataCamp Neo4j Tutorial: This tutorial focuses on using Neo4j with Python, including data ingestion and querying techniques.

Video Tutorials

• Introduction to Neo4j: A beginner-friendly video that covers installation and basic usage, perfect for visual learners.

Conclusion

Neo4j is a powerful graph database that offers a unique approach to managing and querying complex data relationships. As you saw in the previous blog post, it is the framework that Agent Memory for .NET leverages.

Get it? An elephant never forgets. Get it? Get it??

Overview of Agent Memory for .NET

Agent Memory for .NET is a sophisticated solution that allows AI agents to store and recall information across sessions. By utilizing a graph database structure, that is, one that organizes data as nodes, edges and properties, it enables agents to create a rich knowledge graph that captures entities, relationships, and interactions over time.

Key Features and Innovations

Types of Memory:

• Short-term Memory: This component captures the immediate context of conversations, allowing agents to respond appropriately to ongoing dialogues.
• Long-term Memory: This aspect stores a comprehensive knowledge graph that includes entities and relationships, enabling agents to recall past interactions and provide personalized responses.
• Reasoning Memory: By recording the agent’s actions and decisions, this memory type enhances the agent’s ability to make informed decisions in future interactions.

Time-aware Memory: One of the standout features of Agent Memory for .NET is its support for bitemporal recall. Bitemporal recall is the ability of a data system to track and query information across two distinct timelines — in this case valid time (when the fact was true in the real world) and transaction time (when the fact was recorded in the Database). This allows agents to answer questions based on both past beliefs and current knowledge, providing a more nuanced understanding of user queries.

Integration: The framework is designed to be compatible with the Microsoft Agent Framework and other .NET applications. This seamless integration makes it easy for developers to incorporate Agent Memory into existing systems without significant overhead.

Graph-Native Structure: By leveraging Neo4j’s graph database capabilities, Agent Memory for .NET can store and query memory efficiently. This structure allows for complex relationships and interactions to be represented in a way that is both intuitive and powerful.

Implementation: Getting Started with Agent Memory for .NET

To illustrate how to set up Agent Memory for .NET, let’s walk through a simple code example. This demonstrates how to initialize the memory store, store a memory, and retrieve it.

Prerequisites

Before you begin, ensure you have the following:

• .NET SDK installed on your machine.
• A running instance of Neo4j. You can download and install Neo4j from the official website.

Code Example

Here’s a straightforward example of how to set up Agent Memory for .NET using Neo4j:

using Neo4j.Driver;
using AgentMemory;

class Program
{
    static async Task Main(string[] args)
    {
        // Initialize Neo4j Driver
        var driver = GraphDatabase.Driver("bolt://localhost:7687", AuthTokens.Basic("neo4j", "password"));

        // Create a new memory store
        var memoryStore = new MemoryStore(driver);

        // Store a memory
        await memoryStore.StoreMemory("user123", "What is the capital of France?", "Paris");

        // Retrieve a memory
        var response = await memoryStore.RetrieveMemory("user123", "What is the capital of France?");
        Console.WriteLine(response); // Outputs: Paris
    }
}

Explanation of the Code

1. Initialize Neo4j Driver: The first step is to create a connection to your Neo4j database using the GraphDatabase.Driver method. Replace the connection string and authentication details with your own.
2. Create a Memory Store: An instance of MemoryStore is created, which will handle the storage and retrieval of memories.
3. Store a Memory: The StoreMemory method is called to save a memory associated with a specific user. In this case, we store the question “What is the capital of France?” along with the answer “Paris”.
4. Retrieve a Memory: Finally, we retrieve the stored memory using the RetrieveMemory method and print the response to the console.

By leveraging the power of Neo4j, Agent Memory for .NET provides a solution for enhancing applications with persistent memory.

For more information and resources, see the Agent Memory for .NET GitHub Repository and the Neo4j Blog on Agent Memory.

What is Microsoft Agent Framework?

To quickly review, Microsoft Agent Framework is a development framework designed specifically for creating AI agents. These agents are capable of interacting with various services and data sources, making them versatile tools for developers. The framework provides a rich set of tools and libraries that enable developers to build intelligent applications that can automate tasks, respond to user queries, and integrate with other systems.

Key Features of Microsoft Agent Framework

1. Development Tools: The framework includes a variety of libraries and APIs that simplify the process of building AI agents. Developers can leverage these tools to create agents that can understand natural language, process data, and perform complex tasks.
2. Integration Capabilities: The framework is designed to work with various data sources and services, allowing developers to create agents that can pull information from multiple platforms and provide comprehensive responses to user queries.
3. Flexibility: Developers can use the Microsoft Agent Framework alongside other frameworks, such as the OpenAI Agents SDK, to create a wide range of applications, from simple chatbots to complex AI-driven solutions.

For much more on Microsoft Agent Framework and agentics in general see the blog posts beginning here.

What is Microsoft Foundry?

Microsoft Foundry is a managed platform that provides a comprehensive environment for building, deploying, and scaling AI applications. It offers a suite of tools and services that enable developers to utilize various AI models and frameworks, making it easier to create sophisticated applications.

Key Features of Microsoft Foundry

1. Managed Environment: Foundry provides a fully managed environment, which means developers can focus on building their applications without worrying about the underlying infrastructure. This allows for faster development cycles and easier scaling.
2. AI Model Integration: Foundry supports a wide range of AI models and tools, enabling developers to leverage the latest advancements in AI technology. This integration allows for the creation of more intelligent and capable agents.
3. Governance and Observability: Foundry includes features that help organizations maintain compliance with regulations, making it particularly beneficial for industries that require strict governance, such as finance and healthcare.

The Relationship Between Microsoft Agent Framework and Microsoft Foundry

The relationship between the Microsoft Agent Framework and Microsoft Foundry is one of synergy and integration. Together, they provide a robust environment for developing AI agents that can handle complex tasks and workflows. Here are some key aspects of their relationship:

1. Integration of Services

The Foundry Agent Service acts as a bridge between the Microsoft Agent Framework and various data sources and other agents. This integration enables seamless communication and data exchange, which is crucial for developing multi-agent workflows that can manage complex business processes.

Additionally, the Responses API serves as a single entry point for accessing Foundry models and tools. This allows developers to build agents using the Agent Framework while leveraging the capabilities of Foundry, creating a more cohesive development experience.

2. Multi-Agent Workflows

Both Foundry and Microsoft Agent Framework support the creation of multi-agent workflows. This feature allows developers to orchestrate complex, multi-step processes, enhancing the capabilities of AI applications. By enabling multiple agents to work together, organizations can automate intricate workflows that would be challenging to manage with a single agent.

3. Identity and Security

Security is a paramount concern in the development of AI applications. The Microsoft Agent Framework utilizes Microsoft Entra ID for managing agent identities, ensuring secure authentication and authorization. With Foundry you get this and many other infrastructure features out of the box.

4. Development Flexibility

The combination of the Microsoft Agent Framework and Microsoft Foundry offers developers significant flexibility in how they create agents. They can choose to build agents using the Microsoft Agent Framework and have them hosted in Foundry, or they can create AI applications directly with Foundry. This flexibility is essential for meeting the diverse needs of businesses and organizations.

For more on this, be sure to watch my video interview of Bruno Capuano and Jon Galloway, both of Microsoft, where, among other things, Bruno demonstrates how easy it is to have Foundry host a Microsoft Agent Framework application.

Note: Microsoft strongly suggests using secrets and not putting these values in config.json

What is config.json?

The config.json file is a widely used configuration file format in many programming environments, particularly in JavaScript and .NET applications. It serves as a simple way to store application settings, such as API endpoints, feature flags, and other non-sensitive configurations. The structure of a config.json file is straightforward, making it easy for developers to read and modify.

Example of config.json

{
  "ApiUrl": "https://api.example.com",
  "FeatureFlag": true
}

Advantages of Using config.json

1. Simplicity and Accessibility: One of the primary advantages of using config.json is its simplicity. Developers can easily read and modify the file, making it an excellent choice for local development and testing environments. This ease of access allows for rapid iteration and debugging.
2. Version Control: Configuration files can be included in version control systems like Git. This feature is beneficial for tracking changes over time, allowing teams to collaborate effectively and maintain a history of configuration changes.
   
   Note, if you have secret values (such as keys) you do not want them in version control. One solution is to add them to your .gitignore file. A better solution is not to have them in config.json in the first place.
   
3. No Additional Setup Required: Unlike secrets management systems, which may require additional setup and configuration, using config.json typically involves minimal overhead. Developers can start using it right away without needing to integrate with external services.

Disadvantages of Using config.json

1. Security Risks: The most significant drawback of using config.json is its lack of security for sensitive data. If sensitive information, such as passwords or API keys, is stored in this file, it can be easily accessed by anyone with access to the codebase. This poses a substantial risk, especially in production environments.
2. Accidental Exposure: Including config.json in version control can lead to accidental exposure of sensitive data. Developers must be diligent about ensuring that sensitive information is excluded from version control, which can be challenging.
3. Limited to Non-Sensitive Data: While config.json is suitable for general configuration, it is not designed for managing sensitive information securely. Developers must find alternative methods for handling sensitive data, which can complicate the development process.

Secrets Management in Microsoft Agent Framework

What is Secrets Management?

The Microsoft Agent Framework provides a robust secrets management system designed to securely manage sensitive information such as credentials, API keys, and other secrets. This system is particularly useful for applications deployed in production environments where security is paramount.

Example of Secrets Management

Using Azure Key Vault, developers can securely store and retrieve secrets. Here’s a simple example of how to access a secret using the Azure SDK:

var secretClient = new SecretClient(new Uri("https://<your-key-vault-name>.vault.azure.net/"), new DefaultAzureCredential());
KeyVaultSecret secret = await secretClient.GetSecretAsync("MySecret");
string secretValue = secret.Value;

Advantages of Using Secrets Management

1. Enhanced Security: The primary advantage of using a secrets management system is its built-in security features. Secrets are encrypted and access-controlled, ensuring that sensitive data is not exposed in the codebase. This level of security is essential for protecting sensitive information in production environments.
2. Centralized Management: Secrets management systems like Azure Key Vault allow for centralized management of secrets across multiple applications. This centralization simplifies the process of updating and rotating secrets, reducing the risk of outdated or compromised credentials.
3. Integration with Azure Services: The Microsoft Agent Framework’s secrets management seamlessly integrates with other Azure services, providing a cohesive environment for managing application secrets. This integration enhances the overall security posture of applications deployed in the Azure ecosystem.

Disadvantages of Using Secrets Management

1. Complexity: Implementing a secrets management system can introduce additional complexity in setup and management compared to using a simple configuration file. Developers must familiarize themselves with the secrets management system and its APIs, which may require additional time and resources.
2. Cost: Depending on the chosen secrets management solution, there may be associated costs. For example, using Azure Key Vault incurs charges based on the number of operations performed and the amount of data stored. Organizations must weigh these costs against the benefits of enhanced security.
3. Learning Curve: For teams unfamiliar with secrets management practices, there may be a learning curve involved in adopting a new system. Training and documentation may be necessary to ensure that all team members understand how to use the system effectively.

Key Trade-offs

When deciding between config.json and secrets management in the Microsoft Agent Framework, developers must consider several key trade-offs:

1. Security vs. Convenience: Using config.json is convenient for non-sensitive configurations but poses security risks for sensitive data. In contrast, the Microsoft Agent Framework’s secrets management is secure but may require more setup and management effort.
2. Development vs. Production: config.json is often more suitable for development environments, where rapid iteration is essential. However, for production environments, where security is a priority, leveraging secrets management is advisable.
3. Version Control: Configuration files can be versioned easily, allowing for tracking changes over time. However, secrets should never be included in version control to prevent accidental exposure, necessitating a different approach for managing sensitive data.

Conclusion

Choosing between config.json and secrets management in the Microsoft Agent Framework ultimately depends on the specific needs of your application. For general configuration values that do not involve sensitive information, config.json remains a practical choice, provided that developers are diligent about handling sensitive data appropriately. However, for applications that require the management of sensitive information, leveraging the secrets management capabilities of the Microsoft Agent Framework is advisable to ensure security and compliance.

In summary, understanding the differences and trade-offs between these two approaches is crucial for developers aiming to build secure and efficient applications. By carefully considering the specific requirements of your project, you can make an informed decision that balances convenience, security, and maintainability.

Bzzzz Still your turn.

This is the approach I use in the demonstration program to log the invocation of the Tavily search tool.

What is Middleware?

To review, middleware is a software layer that acts as an intermediary between different software applications or components. In the context of the Microsoft Agent Framework, middleware enhances the interaction between agents and their underlying systems, allowing developers to intercept, modify, and log messages and operations. This capability is crucial for debugging, monitoring, and extending the functionality of agents.

ChatClient Middleware

Purpose

ChatClient Middleware is specifically designed to intercept calls made to an IChatClient implementation. This middleware is particularly useful for logging, modifying, or inspecting the raw messages exchanged between the agent and the underlying language model (LLM). By utilizing ChatClient Middleware, developers can gain insights into the communication flow, which is essential for debugging and improving the agent’s performance.

Use Case

A common use case for ChatClient Middleware is logging all messages sent and received by the agent. This can help developers understand how the agent interacts with users and the LLM, allowing for better optimization of responses and overall user experience.

Example

Here’s a simple example of how to implement ChatClient Middleware in C#:

var chatClient = new AIProjectClient(new Uri("your-uri"), new DefaultAzureCredential())
    .GetProjectOpenAIClient()
    .GetProjectResponsesClient()
    .AsIChatClient(deploymentName);

var middlewareEnabledChatClient = chatClient
    .AsBuilder()
    .Use(getResponseFunc: CustomChatClientMiddleware, getStreamingResponseFunc: null)
    .Build();

In this example, CustomChatClientMiddleware would be a function that processes the messages before they are sent to or after they are received from the LLM. This middleware can log the messages, modify them, or even implement additional logic based on the content of the messages.

Agent Middleware

Purpose

Agent Middleware operates at a higher level than ChatClient Middleware. It allows for the interception of all agent runs, enabling developers to inspect and modify the input and output of the agent’s operations. This middleware is essential for managing the overall behavior of the agent, including session management, identity tracking, and token budget management.

Use Case

A typical use case for Agent Middleware is collecting information about the agent’s session or identity. For instance, if an agent needs to maintain context across multiple interactions or manage its resource usage effectively, Agent Middleware would be the appropriate choice.

Example

Here’s how you might implement Agent Middleware in C#:

var agent = new ChatClientAgent(middlewareEnabledChatClient, instructions: "You are a helpful assistant.");

In this example, the ChatClientAgent is initialized with the middleware-enabled chat client. The agent can now leverage the capabilities of both ChatClient and Agent Middleware to enhance its functionality.

Function-Invocation Middleware for a Single Agent

Function-invocation middleware can be applied to both ChatClient and Agent Middleware. This type of middleware allows for the interception of function calls executed by the agent, enabling developers to inspect and modify inputs and outputs. This capability is particularly useful for logging, debugging, and implementing additional logic based on the agent’s operations.

Example of Function-Invocation Middleware in C

Here’s a simple example of how to implement function-invocation middleware for a single agent:

public class FunctionInvocationMiddleware
{
    public async Task InvokeAsync(AgentRunContext context, Func<AgentRunContext, Task> next)
    {
        // Before the function call
        Console.WriteLine($"Before function call: {context.FunctionName}");

        // Call the next middleware in the pipeline
        await next(context);

        // After the function call
        Console.WriteLine($"After function call: {context.Result}");
    }
}

// Usage
var agent = new ChatClientAgent(middlewareEnabledChatClient, instructions: "You are a helpful assistant.")
    .AsBuilder()
    .Use(FunctionInvocationMiddleware.InvokeAsync)
    .Build();

In this example, the FunctionInvocationMiddleware class defines an InvokeAsync method that logs the function name before and after the function call. This middleware can be used to track the execution flow of the agent’s operations, providing valuable insights into its behavior.

In the demonstration program we modify the creation of the ResearcherAgent to ad this bit of code:

.Use(async (agent, context, next, cancellationToken) =>
        {
            if (context.Function.Name == tavilyTool.Name)
            {
                _logger.LogInformation(
                    "Researcher invoking Tavily tool '{Tool}' with arguments {Arguments}",
                    context.Function.Name,
                    context.Arguments);
            }

            return await next(context, cancellationToken);
        })

That’s the only change necessary to have the middleware capture the calls to Tavily tool by the Researcher agent. None of the other agents change.

The trace from this change looks like this:

[trace] → execute_tool tavily_search
info: BlogWriter.ResearcherAgent[0]
      Researcher invoking Tavily tool 'tavily_search' with arguments [query, ChatClient middleware vs Agent middleware Microsoft Agent Framework]
[trace] ← execute_tool tavily_search (1796 ms)

Summary

In summary, the Microsoft Agent Framework offers two distinct types of middleware: ChatClient Middleware and Agent Middleware.

• ChatClient Middleware focuses on the interaction with the chat client, allowing for the logging and modification of messages exchanged with the LLM.
• Agent Middleware deals with the overall operations of the agent, enabling developers to manage session information, identity, and resource usage.

Additionally, function-invocation middleware can be implemented to intercept function calls, allowing for detailed control over the agent’s behavior.

Memory Architecture

1. Short-Term Memory: This component tracks ongoing interactions and is typically volatile, meaning it is cleared after the session ends. It allows the agent to respond to immediate queries and maintain a fluid conversation.
2. Long-Term Memory: In contrast, long-term memory retains information across sessions, enabling the agent to remember details that can enhance future interactions. This memory is crucial for building a relationship with users, as it allows the agent to provide continuity and context in conversations.

Context Providers

In MAF, long-term memory is managed through ContextProviders. These components allow agents to access relevant past interactions, user preferences, and other contextual information. By leveraging ContextProviders, agents can provide more relevant responses based on historical data, making interactions feel more natural and engaging.

Integration with Databases

To effectively manage long-term memory, AI agents require a reliable storage solution. Databases play a critical role in this process. Popular databases such as Neo4j and Azure Cosmos DB are commonly used to store long-term memory.

• Neo4j: This graph database allows for the storage of entities extracted from conversations. It enables the classification and linking of these entities back to the original messages, facilitating complex queries and relationships.
• Azure Cosmos DB: This database offers a unified solution for memory systems, providing speed and scalability essential for AI agents. Its multi-model capabilities allow for the storage of various data types, making it a versatile choice for long-term memory management.

Also, please see my post on Agent Memory for .NET

Real-World Use Cases

The integration of databases into the Microsoft Agent Framework opens up a plethora of possibilities for AI agents. Here are some real-world use cases that illustrate the benefits of long-term memory:

Personalized Recommendations

With long-term memory an AI agent can remember user preferences and past interactions to suggest tailored options. If a user frequently inquires about travel destinations, the agent can store this information and offer relevant suggestions in future conversations. This not only enhances user satisfaction but also fosters a sense of connection between the user and the agent.

Contextual Awareness

Long-term memory enables agents to maintain context over multiple sessions. For example, if a user mentions a specific project during one interaction, the agent can recall details about that project in subsequent conversations. This continuity enhances the overall user experience, as it allows for more meaningful and relevant interactions. Users are more likely to engage with an agent that remembers their interests and past discussions.

Supporting Data and Quotes

According to a blog post on Neo4j, “Long-term memory consists of entities which can be automatically extracted from conversations using the LLM and classified with the POLE+O schema.” This highlights the need for a systematic approach to data storage that allows for easy retrieval and classification.

POLE+O schema is a framework used to analyze systems by breaking them into five interacting dimensions:

• P)eople – Human actors
• O)bjects – tangible or digital artifacts
• L)ocation – spatial or contextual setting
• E)vents – actions that trigger change
• O)rganization – the structure or procedural layer that connects everything

In essence, this is the Who, What, Where, When and How.

A Microsoft Community Hub article states, “Long-term memory is typically shared across sessions,” emphasizing the necessity of a database that can persist data beyond individual interactions. This persistence is crucial for building a comprehensive understanding of user preferences and behaviors.

When an agent interacts with a user, it can save important information (like preferences or past conversations) in a database. The next time the user interacts with the agent, it can retrieve this information to provide a more personalized experience. This process is fundamental to creating a seamless and engaging interaction between users and AI agents.

Code Example

To illustrate how an AI agent might store and retrieve user preferences using a database, here’s a simple example in C#:

public class UserPreferences
{
    public string UserId { get; set; }
    public string Preference { get; set; }
}

public class MemoryDatabase
{
    private List<UserPreferences> preferencesStore = new List<UserPreferences>();

    public void SavePreference(string userId, string preference)
    {
        preferencesStore.Add(new UserPreferences { UserId = userId, Preference = preference });
    }

    public List<string> GetPreferences(string userId)
    {
        return preferencesStore.Where(p => p.UserId == userId).Select(p => p.Preference).ToList();
    }
}

// Usage
var memoryDb = new MemoryDatabase();
memoryDb.SavePreference("user123", "likes travel");
var userPreferences = memoryDb.GetPreferences("user123");

In this code snippet, we define a UserPreferences class to represent user preferences and a MemoryDatabase class to manage the storage and retrieval of these preferences. The SavePreference method allows the agent to store user preferences, while the GetPreferences method retrieves them for future interactions.

Conclusion

The integration of databases into the Microsoft Agent Framework is crucial for enabling long-term memory in AI agents. By leveraging structured data storage, agents can provide personalized and contextually aware interactions, significantly enhancing user experience. As AI technology continues to advance, the ability to remember and learn from past interactions will become increasingly important, making the role of databases in AI development more critical than ever. By understanding and implementing these concepts, developers can create more intelligent and responsive AI agents that truly understand and cater to user needs.

Overview of the Frameworks

LangChain

LangChain is a comprehensive framework designed for building applications that utilize large language models. It provides a rich set of tools for creating chains, retrievers, and tool-calling agents, making it particularly suitable for rapid prototyping and a wide array of use cases. The framework is primarily Python-based, which aligns well with the preferences of many data scientists and AI developers.

LangGraph

LangGraph, on the other hand, is a lower-level orchestration framework that focuses on building stateful multi-agent systems. It is tailored for workflows that require loops, persistence, and cyclic reasoning, making it ideal for complex agent architectures. LangGraph is particularly useful in scenarios where explicit state management is crucial, allowing developers to create intricate workflows that can adapt and respond to changing conditions.

These are not incompatible and many (most?) AI applications using one will use the other as well. I started my demonstration program starting here using LangChain and LangGraph and then translated it to C# starting here, and finally migrated it to Microsoft Agent Framework starting here.

Microsoft Agent Framework

The Microsoft Agent Framework is a robust solution designed specifically for .NET environments and Azure integration. It supports multi-agent orchestration and is built for enterprise-grade applications, providing first-class support for C# and Python. This framework is particularly advantageous for organizations that are heavily invested in the Microsoft ecosystem, as it offers seamless integration with Azure services and tools.

Key Differences

1. Ecosystem Fit

• LangChain: This framework is favored by teams that require flexibility and rapid iteration, especially in Python environments. Its extensive library of integrations (over 1,000) allows developers to quickly adapt and extend their applications.
• LangGraph: LangGraph is the choice for applications that necessitate explicit state management and complex workflows. It excels in scenarios where persistent memory and human-in-the-loop control are essential.
• Microsoft Agent Framework: This framework is ideal for .NET developers and organizations that rely on Azure services.

2. Architecture

• LangChain: The architecture of LangChain emphasizes building agent capabilities, focusing on the skills layer. It is designed for rapid development, allowing developers to create and iterate on applications quickly.
• LangGraph: In contrast, LangGraph serves as the control layer, defining how agents think and manage workflows. It employs structured logic, such as state machines and loops, to facilitate complex decision-making processes.
• Microsoft Agent Framework: This framework orchestrates multiple agents into a cohesive system, making it particularly suitable for enterprise applications that require coordination among various agents.

3. Integration and Support

• LangChain: With its vast ecosystem of integrations, LangChain is versatile and can be applied to a wide range of applications, from simple chatbots to complex data processing pipelines.
• LangGraph: Designed for complex, cloud-agnostic workflows, LangGraph excels in scenarios that require checkpointing and debugging, making it a powerful tool for developers working on intricate systems.
• Microsoft Agent Framework: The deep integration with Azure services and the focus on enterprise-level orchestration make the Microsoft Agent Framework a strong choice for organizations looking to leverage cloud capabilities in their applications.

Use Cases

LangChain

LangChain is particularly well-suited for projects that require quick iterations and flexibility. Some common use cases include:

• Retrieval-Augmented Generation (RAG) Pipelines: These pipelines combine retrieval mechanisms with generative models to produce contextually relevant responses.
• Multi-Step Workflows: LangChain can efficiently manage workflows that involve multiple steps, allowing for seamless transitions between tasks.

LangGraph

LangGraph shines in applications that require persistent memory and complex task automation. Typical use cases include:

• Human-in-the-Loop Control: Scenarios where human oversight is necessary, such as in decision-making processes that require validation or input from users.
• Complex Task Automation: Applications that involve intricate workflows, such as automated customer support systems or multi-agent negotiation platforms.

Microsoft Agent Framework

The Microsoft Agent Framework is ideal for enterprise applications, particularly in sectors that demand robust orchestration and compliance. Use cases include:

• Healthcare Applications: Systems that require secure and compliant handling of sensitive patient data, where multiple agents must work together to provide accurate and timely information.
• Financial Services: Applications that need to adhere to strict regulatory requirements while managing complex workflows involving multiple agents.

Example Code Snippets

To illustrate the capabilities of each framework, here are simple code snippets demonstrating their usage.

LangChain (Python)

from langchain import LLMChain, OpenAI

# Create a simple LLM chain
llm = OpenAI(model="text-davinci-003")
chain = LLMChain(llm=llm, prompt="What is the capital of France?")
response = chain.run()
print(response)  # Output: Paris

LangGraph (Python)

from langgraph import StateMachine

# Define a simple state machine
sm = StateMachine()
sm.add_state("start", on_enter=lambda: print("Starting..."))
sm.add_state("end", on_enter=lambda: print("Ending..."))
sm.add_transition("start", "end")
sm.run("start")  # Output: Starting...

Microsoft Agent Framework (C#)

using Microsoft.AgentFramework;

// Create a simple agent
var agent = new Agent("MyAgent");
agent.OnMessageReceived += (sender, message) => {
    Console.WriteLine($"Received: {message.Content}");
};
agent.Start();

Conclusion

Choosing between LangChain, LangGraph, and the Microsoft Agent Framework ultimately depends on your specific needs, including the programming environment, the complexity of the agent workflows, and the level of integration required with cloud services. Each framework has its strengths, making them suitable for different types of AI applications.

• LangChain is ideal for rapid development and flexibility in Python environments.
• LangGraph excels in scenarios requiring complex workflows and state management.
• Microsoft Agent Framework is the best choice for enterprise applications, particularly for organizations leveraging the Microsoft ecosystem.

That said, I suspect that the driving factor will be which environment the developer (team?) is comfortable with. .NET developers will be driven to Microsoft Agent Framework and most others will use LangChain/LangGraph or another Python framework.

What is OpenTelemetry?

OpenTelemetry is a set of APIs, libraries, agents, and instrumentation that enables developers to collect telemetry data from their applications. This data can include traces, metrics, and logs, which are essential for monitoring application performance and diagnosing issues. OpenTelemetry is designed to be vendor-agnostic, allowing developers to send their telemetry data to various backends for analysis.

Key Trends and Innovations in OpenTelemetry and MAF

1. Built-in Support for OpenTelemetry

One of the most significant advancements in the Microsoft Agent Framework is its built-in support for OpenTelemetry. This integration allows developers to automatically emit spans and metrics without the need for manual wrapping of every agent call. This feature simplifies the process of adding observability to applications, enabling developers to focus on building functionality rather than instrumentation.

2. Semantic Conventions

OpenTelemetry employs semantic conventions that help structure telemetry data in a meaningful way. In the context of AI agents, the OpenTelemetry GenAI semantic conventions ensure that the telemetry data collected is relevant and useful for analysis. This structured approach aids developers in understanding the performance and behavior of their agents more effectively.

3. Support for Multi-Agent Systems

The Microsoft Agent Framework is designed to support multi-agent architectures, allowing for complex interactions and behaviors to be monitored effectively. This capability is particularly beneficial in scenarios where multiple agents work together to complete tasks, as it provides a comprehensive view of the system’s performance.

By integrating OpenTelemetry, developers can monitor agent performance and interactions, gaining insights into usage patterns, error rates, and overall system efficiency. This data can be invaluable for optimizing the application and enhancing user experience.

Challenges in Integration

While the integration of OpenTelemetry with the Microsoft Agent Framework offers numerous benefits, there are challenges that developers may encounter. One notable issue is the missing Activity Events. Reports indicate that while spans and metrics are emitted correctly, some expected Activity Events, such as gen_ai.client.inference.operation.details, are not being generated. This limitation can restrict the granularity of observability, making it difficult to gain a complete understanding of agent interactions and performance.

Code Example: Setting Up OpenTelemetry in a Microsoft Agent Framework Application

To illustrate how to set up OpenTelemetry in a Microsoft Agent Framework application, consider the following code example:

using Microsoft.Extensions.AI;

// Create OpenAI client
var endpoint = Environment.GetEnvironmentVariable("AZURE_OPENAI_ENDPOINT");
var apiKey = Environment.GetEnvironmentVariable("AZURE_OPENAI_API_KEY");
var deploymentName = "gpt-4o-mini";

using var client = new AzureOpenAIClient(new Uri(endpoint), new AzureKeyCredential(apiKey))
    .GetChatClient(deploymentName)
    .AsIChatClient()
    .AsBuilder()
    .UseOpenTelemetry(sourceName: "MyAgent", configure: (cfg) => cfg.EnableSensitiveData = true)
    .Build();

var thread = client.GetNewThread();
logger.LogInformation("Agent created successfully with ID: {AgentId}", client.Id);

Explanation of the Code

1. Environment Variables: The code begins by retrieving the Azure OpenAI endpoint and API key from environment variables. This approach ensures that sensitive information is not hard-coded into the application.
2. Creating the OpenAI Client: An instance of AzureOpenAIClient is created using the retrieved endpoint and API key. This client is responsible for interacting with the OpenAI service.
3. Using OpenTelemetry: The UseOpenTelemetry method is called on the chat client builder. This method configures OpenTelemetry for the agent, allowing it to emit telemetry data. The sourceName parameter is set to “MyAgent,” and sensitive data emission is enabled.
4. Creating a New Thread: Finally, a new thread is created for the chat client, and a log message is generated to confirm the successful creation of the agent.

You can use OpenTelemetry on each agent, or you can instrument the model calls. To do the latter you only need to update Program.cs

IChatClient llm = openAIClient
    .GetChatClient(modelName)
    .AsIChatClient()
    .AsBuilder()
    .UseFunctionInvocation()
    .UseOpenTelemetry(sourceName: "BlogWriter.Agents")
    .Use(inner => new TokenCapChatClient(inner, maxTotalTokens: 10000))
    .Build();

Per agent allows you to capture which agent ran and agent-level timing. Per model call allows you to track model name, token usage, tool calls and per-round-trip latency.

Conclusion

Integrating OpenTelemetry with the Microsoft Agent Framework significantly enhances the observability of AI agents, providing developers with valuable insights into their performance and interactions. The built-in support for OpenTelemetry, along with semantic conventions and multi-agent system capabilities, makes it a robust choice for monitoring complex applications.

RAG – A quick example

RAG in detail

Deeper into RAG

The R in RAG

PEAS for Agentic AI

AI Reasoning and Planning

REACT and Agents in AI

Creating a multi-agent system (Python & LangChain/LangGraph) part 1 of 6

Migrating Agentic Code Python -> C# Part 1 of 6

Dependency Injection & Microsoft Agentic Framework

Migrating C# to Microsoft Agentic Framework

Transparency in Agentics

Ensuring Agent Safety

Logging & OpenTelemetry in Microsoft Agentic Framework

Activity Source in Microsoft Agentic Framework

Middleware in Microsoft Agentic Framework

Limit token usage in Microsoft Agentic Framework

Using OpenTelemetry in Microsoft Agent Framework

LangChain/LangGraph vs. Microsoft Agent Framework

Long term memory in Microsoft Agent Framework

Agent vs ChatClient Middleware

Managing Secrets

Microsoft Agent Framework and Foundry

Agent Memory for .NET

About Neo4j

Note: many of these blog posts had initial research and drafts done by the Blog Writer multi-agent application that serves as a demo for these articles. All were then edited by me.