This blog post was created with the help of AI tools. Yes, I used a bit of magic from language models to organize my thoughts and automate the boring parts, but the geeky fun and the in C# are 100% mine.

Hi!

Apple Silicon users, please try this C# sample and let us know whether speech generation and playback work on your Mac!

ElBruno.Speech 1.6.0-preview.5 keeps setup simple: call services.AddOllamaSpeech() for Microsoft.Extensions.AI.ITextToSpeechClient. It defaults to local Ollama and the phonon model. Let’s turn a sentence into a WAV file, with local inference handled by Ollama.

Conceptual flow, not a recording of generated audio. Image generated with AI.

1. Get ready

Use .NET 8 or 10 and a supported Apple Silicon Mac running Ollama with MLX. Ollama’s speech announcement recommends version 0.40.11 or later.

ollama pull phonon
dotnet new console -n HelloPhonon -f net10.0
cd HelloPhonon
dotnet add package ElBruno.Speech.Tts.Ollama --version 1.6.0-preview.5
dotnet add package ElBruno.Speech.Audio --version 1.6.0-preview.5
dotnet add package Microsoft.Extensions.DependencyInjection --version 10.0.13

For .NET 8, replace net10.0 with net8.0. Make sure Ollama is running.

Windows friends: the C# client can call a supported remote Mac, but local Ollama speech currently requires MLX. Windows/Linux server support is pending upstream, with no promised date.

2. Replace Program.cs

The MEAI speech API is experimental. The pragma below explicitly opts into it for this preview.

#pragma warning disable MEAI001
using ElBruno.Speech;
using ElBruno.Speech.Audio;
using ElBruno.Speech.Tts.Ollama;
using Microsoft.Extensions.AI;
using Microsoft.Extensions.DependencyInjection;
var services = new ServiceCollection();
services.AddOllamaSpeech();
using var provider = services.BuildServiceProvider();
var speech = provider.GetRequiredService<ITextToSpeechClient>();
using var pcm = new MemoryStream();
await foreach (var update in speech.GetStreamingAudioAsync(
    "Hello from C sharp! My app has a voice thanks to Ollama Phonon.",
    new TextToSpeechOptions { AudioFormat = "pcm" }))
{
    foreach (var audio in update.Contents.OfType<DataContent>())
        await pcm.WriteAsync(audio.Data);
}
byte[] samples = pcm.ToArray();
if (samples.Length == 0 || samples.Length % 2 != 0)
    throw new InvalidDataException("Expected complete PCM16 audio samples.");
WavWriter.Write("hello.wav",
    new AudioFormat(24000, 1, AudioSampleFormat.Int16), samples);
Console.WriteLine($"Saved: {Path.GetFullPath("hello.wav")}");

Run it:

dotnet run

On a supported Mac, listen with:

afplay hello.wav

The client receives PCM chunks incrementally. This short example buffers them, then writes a WAV with the final sizes: 24 kHz, mono, signed 16-bit PCM. It does not play audio while streaming.

No configuration is required for local Ollama: the defaults are http://localhost:11434/, model phonon, the model’s default voice, and WAV output. This example requests PCM per call for streaming. Using a remote Mac? Replace the registration with services.AddOllamaSpeech(new OllamaSpeechOptions { Endpoint = new Uri("http://YOUR-MAC:11434/") });. The library resolves the speech route automatically. Keep that server on a trusted network; this example does not configure authentication or expose it.

3. Try the complete demo

The full C# sample also saves complete WAV and streamed PCM/WAV files, validates their format, rejects silent output, and supports optional playback.

Honest status: 49 synthetic provider/sample test cases pass on each of .NET 8 and .NET 10, including no-options registration, default request values, and URL normalization for complete and streaming requests. My Windows host could not load Phonon because MLX was unavailable, so I have not generated or listened to real Phonon speech yet. Automated checks are not proof of pronunciation or sound quality.

Community testing invitation, not proof of working audio. Image generated with AI.

If you test it, share your Mac chip, macOS and Ollama versions, whether hello.wav plays, and how it sounds in issue #106. Please include any error message, but no secrets or private audio.

NuGet package | Release notes | Ollama speech docs

This blog post was created with the help of AI tools. Yes, I used a bit of magic from language models to organize my thoughts and automate the boring parts, but the geeky fun and the in C# are 100% mine.

Hi!

While building the USB demo for the upcoming New Year, New Bot series, I had two small robots connected to one Windows PC. Windows showed several COM ports, but the names did not tell me which cable belonged to which robot.

That matters before connecting to a device, and especially before choosing a firmware target. Two different boards can report the same USB vendor and product IDs. A COM number is a current Windows assignment, not a permanent identity.

The solution: read metadata, then unplug one cable

Instead of opening each serial port and guessing, this small console app reads Windows Plug and Play metadata: the COM assignment, device description, USB IDs, parent and location. It never opens a port, toggles DTR/RTS, resets a board or flashes firmware.

The diagram uses synthetic IDs. Notice that every cable has a clear, continuous path from the PC inventory to its device. The app helps narrow the match; disconnecting one cable and watching for REMOVED confirms which physical device it is.

The code: one file, no project scaffolding

The standalone sample targets .NET 10 and uses System.Management to ask Windows for present ports. It reads the COM assignment from the device’s registry metadata:

var query = new ObjectQuery(
    "SELECT * FROM Win32_PnPEntity WHERE PNPClass = 'Ports' AND Present = TRUE");
using var searcher = new ManagementObjectSearcher(
    new ManagementScope(@"root\cimv2"),
    query,
    new EnumerationOptions { Timeout = TimeSpan.FromSeconds(5), ReturnImmediately = false });
using var devices = searcher.Get();
foreach (ManagementObject device in devices)
{
    using (device)
    {
        string? instanceId = device["PNPDeviceID"] as string;
        if (string.IsNullOrWhiteSpace(instanceId))
            throw new InvalidOperationException("A present port has no PnP instance ID.");
        using var key = Registry.LocalMachine.OpenSubKey(
            $@"SYSTEM\CurrentControlSet\Enum\{instanceId}\Device Parameters",
            writable: false);
        string? port = key?.GetValue("PortName") as string;
        // The app validates the COM name, reads optional metadata, and reports it.
    }
}

The full code also handles ports disappearing during a query, orders ports numerically, and reports errors explicitly. --watch compares snapshots and reports added, removed or changed devices. --json makes the snapshot usable by another tool.

From the sample folder, with a stable .NET 10 SDK installed:

dotnet run SerialPorts.cs
dotnet run SerialPorts.cs -- --watch
dotnet run SerialPorts.cs -- --json
dotnet run SerialPorts.cs -- --self-test

For two look-alike devices, run watch mode, unplug just one cable, wait for REMOVED, reconnect it and label that physical robot. Repeat for the other device. Do not use the inventory as an automatic firmware-target selector. The synthetic self-test checks parsing and device changes without WMI, the registry or hardware.

This is a small building block for the upcoming New Year, New Bot series, where we are exploring a desk companion built with StackChan and .NET.

Image note: the hero photo was provided by Bruno. The diagram is a hand-authored vector with synthetic port and USB ID values. Other draft illustrations are not used in this shorter version.

References

• Public console sample and instructions
• .NET file-based apps
• Win32_PnPEntity

This blog post was created with the help of AI tools. Yes, I used a bit of magic from language models to organize my thoughts and automate the boring parts, but the geeky fun and the in C# are 100% mine.

Hi!

After my first post about s1-mini, a reader asked if the library could take the SRT files he generates for his podcast. Then he noticed the catch: the model only rewrites text, it has no idea what a timestamp is.

That is fine, because the model does not need to know. The real goal is bigger than cleaning an existing SRT: start with an audio recording and end with a subtitle file. That takes two local models, each doing the one thing it is good at:

Inference runs on your machine. Model files are downloaded on first use; audio is not uploaded.

Why not just give audio to s1-mini? s1-mini never hears audio and never sees timestamps. Whisper owns the timing. s1-mini only ever receives short pieces of text. The library keeps the two together.

Setup

dotnet new console -n PodcastToSrt
cd PodcastToSrt
dotnet add package ElBruno.S1Mini --version 0.2.1
dotnet add package ElBruno.Whisper --version 0.9.0

The first run downloads the Whisper model and the s1-mini ONNX model, then caches both.

Step 1: transcribe the audio with timestamps

Timestamps are opt-in in ElBruno.Whisper. If you forget EnableTimestamps, Segments comes back null:

using ElBruno.Whisper;
using var whisper = await WhisperClient.CreateAsync(new WhisperOptions
{
    // Whisper Tiny (English) keeps spoken fillers like "um" and "uh",
    // which is exactly what s1-mini is designed to clean up.
    Model = KnownWhisperModels.All[0],
    EnableTimestamps = true,
});
var transcription = await whisper.TranscribeAsync("episode.wav");

transcription.Segments is a list of TranscriptionSegment with Start, End (both TimeSpan) and Text.

The snippets below show an illustrative progression for the synthetic clip from the real-run example. They explain the shape of each result; Whisper’s exact transcript and segment boundaries can vary.

[00:00:00.000 - 00:00:11.452]
So I'm welcome back to the the show. Today we are talking about, uh, local transcription in dot net. We will meet on Thursday no wait Friday at ten thirty.

Step 2: turn the segments into subtitle cues

SubtitleCue.FromTimes formats the timing line for you (00:00:01,000 for SRT, 00:00:01.000 for WebVTT):

using ElBruno.S1Mini.Normalization;
var format = SubtitleFormat.Srt;
var rawCues = transcription.Segments!
    .Select(s => SubtitleCue.FromTimes(s.Start, s.End, s.Text.Trim(), format))
    .ToList();
var rawSrt = SubtitleParser.Write(rawCues, format);

At this point you already have a valid, timestamped, but still messy subtitle file.

For example, the same segment is now an SRT cue. The text is still uncleaned, but its timing is ready to pass through unchanged:

1
00:00:00,000 --> 00:00:11,452
So I'm welcome back to the the show. Today we are talking about, uh, local transcription in dot net. We will meet on Thursday no wait Friday at ten thirty.

Step 3: clean the text, keep the timing

using var normalizer = await TranscriptNormalizer.CreateAsync();
var cleaned = await normalizer.NormalizeSubtitlesAsync(rawSrt, format);
await File.WriteAllTextAsync("episode.srt", cleaned);

NormalizeSubtitlesAsync parses the cues, normalizes each cue’s text, and writes the file back. Timing lines are carried through as raw strings, so they cannot drift. Markup tags such as <i> are stripped, multi-line cues are joined into one line, and cues that are pure filler stay in place with empty text (or are removed if you pass dropEmptyCues: true, in which case SRT is renumbered).

After cleanup, the text is easier to read while the cue timing stays the same:

1
00:00:00,000 --> 00:00:11,452
So I'm welcome back to the show. Today we are talking about a local transcription in .NET. We will meet on Friday at 10:30.

The whole thing in one file

This is the PodcastToSrt sample, trimmed:

using ElBruno.S1Mini.Normalization;
using ElBruno.Whisper;
var input = args[0];
var format = args.Contains("--vtt") ? SubtitleFormat.WebVtt : SubtitleFormat.Srt;
var output = Path.ChangeExtension(input, format == SubtitleFormat.Srt ? ".srt" : ".vtt");
using var whisper = await WhisperClient.CreateAsync(new WhisperOptions
{
    Model = KnownWhisperModels.All[0],
    EnableTimestamps = true,
});
var transcription = await whisper.TranscribeAsync(input);
var rawCues = transcription.Segments!
    .Select(s => SubtitleCue.FromTimes(s.Start, s.End, s.Text.Trim(), format))
    .ToList();
using var normalizer = await TranscriptNormalizer.CreateAsync();
var cleaned = await normalizer.NormalizeSubtitlesAsync(
    SubtitleParser.Write(rawCues, format), format);
await File.WriteAllTextAsync(output, cleaned);

Run it:

dotnet run -- episode.wav            # writes episode.srt
dotnet run -- episode.wav --vtt      # writes episode.vtt

A real run

I tested the end-to-end pipeline with a generated WAV file named pod.wav in the system temporary directory. I created it with Windows System.Speech.Synthesis.SpeechSynthesizer, speaking a short script with deliberate fillers, a stutter and a self-correction. It is a synthetic test clip, not a podcast recording:

“So um welcome back to the the show. Today we are talking about, uh, local transcription in dot net. We will meet on Thursday no wait Friday at ten thirty.”

The output file:

1
00:00:00,000 --> 00:00:11,452
So I'm welcome back to the show. Today we are talking about a local transcription in .NET. We will meet on Friday at 10:30.

The stutter is gone, “dot net” became .NET, the self-correction resolved to Friday, and “ten thirty” became 10:30. Note the So I'm welcome back: that mistake came from Whisper Tiny’s transcription, and s1-mini cleans text, it does not fix words the speech model got wrong. A bigger Whisper model helps there, at the cost of keeping fewer fillers for s1-mini to clean.

This short clip produced a single segment. It did not establish complete-podcast handling or accurate acoustic timing: Whisper 0.9.0 preprocesses at most 30 seconds per call and can return a full-duration fallback segment. The updated repository sample uses explicit windows to process long recordings. The compact one-file example above is only suitable for a short WAV, not a complete podcast.

Real Spanish podcasts need a different path

The updated PodcastToSrt sample includes two complete NTN 459 recordings: voice-only Recording.m4a and edited ntn459.mp3, with music. These replace the synthetic clip as the real-audio evaluation fixtures, not as automated unit-test inputs.

The original sample selected whisper-tiny.en, an English-only model. Spanish needs a multilingual model such as whisper-base or whisper-small, without the .en suffix:

dotnet run --project src\samples\PodcastToSrt -- `
  src\samples\PodcastToSrt\assets\ntn459\Recording.m4a `
  artifacts\podcast\voice-es.srt --model whisper-base --language es
dotnet run --project src\samples\PodcastToSrt -- `
  src\samples\PodcastToSrt\assets\ntn459\ntn459.mp3 `
  artifacts\podcast\edited-es.vtt --model whisper-small --language es --vtt

s1-mini v1 is English-only too. Spanish is preserved without cleanup by default. This separates two questions: how well Whisper recognizes Spanish, and whether normalization preserves that transcription. Changing the cleanup prompt does not make s1-mini a supported Spanish model.

The sample also offers --language auto, but the pinned Whisper package does not return reliable detected-language metadata. That mode is explicitly labeled unconfirmed and skips cleanup. In the tested excerpt it generated a repeated “Commission” loop rather than a usable transcript. Prefer an explicit --language es for these recordings.

To intentionally produce English subtitles from Spanish audio, add --translate. Whisper translates, then s1-mini cleans the English result. To compare unsupported Spanish cleanup, opt into --cleanup experimental. Both modes retain the pre-cleanup subtitle file for inspection.

For compressed and long recordings, the sample uses FFmpeg and non-overlapping windows of at most 25 seconds, retaining original offsets and silence. JSON reports distinguish possible full-window fallback timings from timestamp spans. A tested, dependency-specific byte-decoding adapter also fixes the Spanish accent encoding omitted by Whisper 0.9.0; it does not improve speech recognition.

The sample README documents the opt-in comparison commands and limitations. Music, technical names, sentence boundaries and late-episode coverage need review; valid SRT syntax alone is not a quality benchmark.

The real-audio evaluation processed the complete voice recording into 59 cues and the edited episode into 61 cues, reaching both decoded ends. All cues were coarse full-window spans, so this proves full-file processing, not accurate speech alignment. Spanish technical names still need review, and no human-scored WER/CER is claimed.

WebVTT works the same way

Pass SubtitleFormat.WebVtt (or --vtt in the sample). The parser keeps WebVTT cue identifiers and cue settings like align:start, and skips NOTE, STYLE and REGION blocks.

Already have an SRT?

Skip steps 1 and 2. Read your file and call NormalizeSubtitlesAsync. The format is auto-detected (content starting with WEBVTT is WebVTT, everything else is SRT):

var srt = await File.ReadAllTextAsync("episode.srt");
var cleaned = await normalizer.NormalizeSubtitlesAsync(srt);

Testing without a model

The automated unit tests do not consume an audio file. They use a fake IChatClient to test subtitle parsing, writing, timestamp formatting and normalization. The manual end-to-end run described above used the generated pod.wav file. The TranscriptNormalizer(IChatClient) constructor makes the subtitle logic easy to test offline with a fake chat client:

var fake = new FakeChatClient();
fake.EnqueueResponse("So hello there everyone.");
fake.EnqueueResponse("");
fake.EnqueueResponse("Welcome back.");
using var normalizer = new TranscriptNormalizer(fake);
var output = await normalizer.NormalizeSubtitlesAsync(srt);
Assert.Equal(3, fake.CallCount);

The parser and writer are plain code, so the timestamp guarantees are checked with exact string comparisons.

Known limitations

• Cues are cleaned one at a time. The model is stateless, so it never sees neighboring cues. When a segment ends mid-sentence, punctuation and capitalization are less accurate than on full paragraphs. A future version could merge segments into sentences, clean them, and redistribute the text across the original time ranges.
• Timing needs validation. Cleanup preserves numeric timing values, not independently verified alignment. Whisper may return native spans or full-window fallback timing. Removing words does not create word-level alignment; coarse full-window cues are not production-ready subtitle segmentation.
• Transcription errors stay. s1-mini normalizes text. It does not correct words Whisper misheard.

Links

• NuGet: ElBruno.S1Mini
• GitHub: elbruno/ElBruno.S1Mini
• Model: superwhisper/s1-mini

This blog post was created with the help of AI tools. Yes, I used a bit of magic from language models to organize my thoughts and automate the boring parts, but the geeky fun and the in C# are 100% mine.

Hi!

Sometimes I ask an AI model a very small question:

Should this request go to billing or support?

And the model replies with a paragraph. Then another paragraph. Then a very polite explanation of why billing departments exist.

Nice conversation.

Not exactly what my application needed.

For classification, routing and evaluation, I often want a different shape of answer: a label, a score, or the probability of a proposition. Something I can use from C# without asking a chat model to please, pretty please, return only JSON.

That is the idea behind ElBruno.AI.Decisions.

Less chat. More decisions.

The libraries are on NuGet, and the code is on GitHub.

What is a decision model?

A chat model is useful when you need generated text. A decision model is designed for workloads such as classification and evaluation, returning structured decision results.

In this SDK, the shared API has three operations:

No hidden conversion from a probability into a business decision. If your application needs a threshold or a human-review queue, you define it.

That last part matters.

The model can help choose a route. It should not quietly become your business policy.

Four packages, one shared API

The project started around Jev. With Microsoft-Decision-1 available in Foundry and native decision support in Ollama, the original name became too narrow.

So the repository is now ElBruno.AI.Decisions, with four packages:

Install a provider package and NuGet brings in the core abstractions automatically.

The examples below target .NET 10 and version 0.6.2. This is a community SDK, not an official Microsoft, Ollama or TypeSafe AI SDK.

Quickstart 1: Microsoft-Decision-1 in Foundry

First, deploy Microsoft-Decision-1 in your Foundry project.

Important detail: this model is from Microsoft, not OpenAI. The invocation endpoint is:

https://<resource>.services.ai.azure.com/providers/microsoft/v1/systemone

Not /chat/completions. Not /mai/v1/decisions.

Use your deployment name in the Model setting. That name may be different from the catalog name Microsoft-Decision-1.

Create a console app and install the package:

dotnet new console --framework net10.0 --name FoundryDecisionsDemo
cd FoundryDecisionsDemo
dotnet add package ElBruno.AI.Decisions.Foundry --version 0.6.2

For this quickstart, use API-key authentication through an environment variable. Replace the resource and deployment placeholders in Program.cs:

using ElBruno.AI.Decisions.Foundry;
using var client = new FoundryDecisionClient(new FoundryDecisionOptions
{
    Endpoint = new Uri("https://<resource>.services.ai.azure.com"),
    Model = "<deployment-name>",
    ApiKey = Environment.GetEnvironmentVariable("FOUNDRY_API_KEY")
        ?? throw new InvalidOperationException("Set FOUNDRY_API_KEY.")
});
var teams = new Dictionary<string, string?>
{
    ["billing"] = "Invoices and payments",
    ["support"] = "Technical problems"
};
var result = await client.ChooseAsync(
    "Please correct my invoice.",
    "Which team should handle this request?",
    teams);
Console.WriteLine($"Team: {result.Choice}");
Console.WriteLine($"Label probability: {result.Probabilities[result.Choice]:P1}");
Console.WriteLine($"Confidence: {result.Confidence:P1}");

Set FOUNDRY_API_KEY in your shell using your secret-management workflow, then run dotnet run. Do not put the key in source control.

Passing the resource root automatically appends the SystemOne path. You can also pass the complete invocation URL.

Without an API key, the client uses AzureCliCredential after az login. The live smoke test used API-key authentication; CLI authentication has not been smoke-tested.

An illustrative provider response looks like this:

{
  "answers": {
    "decision": {
      "type": "choice",
      "choice": "billing",
      "probabilities": { "billing": 0.99, "support": 0.01 },
      "confidence": 0.95
    }
  }
}

Which gives output like:

Team: billing
Label probability: 99.0%
Confidence: 95.0%

Formatting follows your current culture. Values depend on the model and input.

Notice that the selected-label probability and confidence are different numbers. Keep them separate.

Full runnable sample: FoundryDecision.

Quickstart 2: local decisions with Ollama

Now the local version.

You need Ollama v0.35.0 or later, a running local server, and an actual System One decision model.

Not every model installed in Ollama is a decision model. A chat model does not become one because we ask nicely.

For example:

ollama pull nimble
dotnet new console --framework net10.0 --name OllamaDecisionsDemo
cd OllamaDecisionsDemo
dotnet add package ElBruno.AI.Decisions.Ollama --version 0.6.2

With Ollama running, replace Program.cs:

using ElBruno.AI.Decisions.Ollama;
using var client = new OllamaDecisionClient(new OllamaDecisionOptions
{
    Endpoint = new Uri("http://localhost:11434/"),
    Model = "nimble"
});
var teams = new Dictionary<string, string?>
{
    ["billing"] = "Invoices and payments",
    ["support"] = "Technical problems"
};
var result = await client.ChooseAsync(
    "Please correct my invoice.",
    "Which team should handle this request?",
    teams);
Console.WriteLine($"Team: {result.Choice}");
Console.WriteLine($"Label probability: {result.Probabilities[result.Choice]:P1}");
Console.WriteLine($"Confidence: {result.Confidence:P1}");

Run dotnet run. No API key is required for the local endpoint.

The SDK calls:

http://localhost:11434/v1/systemone

It sends native state and named questions. No chat prompt, no letter mapping, no first-token logprob workaround.

Here is an illustrative response, not the result of a live local-model test:

{
  "model": "nimble",
  "answers": {
    "decision": {
      "type": "choice",
      "choice": "billing",
      "probabilities": { "billing": 0.9, "support": 0.1 },
      "confidence": 0.531
    }
  },
  "usage": { "input_tokens": 174, "output_tokens": 1 }
}

Example output:

Team: billing
Label probability: 90.0%
Confidence: 53.1%

Ollama confidence measures distribution concentration:

1 - H(p) / ln(N)

It is not calibrated correctness and it is not the probability of the selected label.

Full runnable sample: OllamaLocal.

Same client, different questions

The Foundry and Ollama clients both implement IDecisionClient. With either client from the examples above, scoring and assessment use the same methods:

var score = await client.ScoreAsync(
    "Restart the app, check the logs, and include the error message.",
    "How actionable is this support answer?",
    new[]
    {
        "No useful action",
        "Some guidance, but important details are missing",
        "Clear steps with useful diagnostic information"
    });
Console.WriteLine($"Expected rubric level: {score.Score:F2}");
var assessment = await client.AssessAsync(
    "My card was charged twice for the same order.",
    "This request concerns a payment problem.");
Console.WriteLine($"Payment problem probability: {assessment.Probability:P1}");

Score uses an ordered rubric, lowest level first. With these three levels, the result is a fractional, zero-based position between 0 and 2. It is not automatically normalized to 0-1, and it is not rounded to an integer.

Assess returns a proposition probability. If the application needs a cutoff, make that policy explicit and validate it on representative data.

Quickstart 3: native Jev questions

The Jev package also exposes its native typed API, useful when you need structured state, named questions and provider-specific metadata.

dotnet new console --framework net10.0 --name JevDecisionsDemo
cd JevDecisionsDemo
dotnet add package ElBruno.AI.Decisions.Jev --version 0.6.2

Set JEV_API_KEY through your secret-management workflow and replace Program.cs:

using ElBruno.AI.Decisions.Jev;
using var client = new JevClient(new JevClientOptions
{
    ApiKey = Environment.GetEnvironmentVariable("JEV_API_KEY")
        ?? throw new InvalidOperationException("Set JEV_API_KEY."),
    DefaultModel = JevModels.Version1_13_0
});
var question = new JevQuestionKey<JevNoulAnswer>("relevant");
var request = new JevDecisionRequest(
    "A .NET developer asks about dependency injection.")
    .WithQuestion(
        question,
        new JevNoulQuestion("Is this about .NET development?"));
var response = await client.EvaluateAsync(request);
Console.WriteLine($"Relevance: {response.GetAnswer(question).Probability:P1}");

Run dotnet run with a valid service credential and access to the configured model. This quickstart demonstrates the SDK contract; live Jev compatibility remains unverified.

For shared-API routing and the rest of the Jev examples, start with the repository samples.

Where would I use this?

A few practical starting points:

• Route a support request to billing or technical support.
• Classify GitHub issues into a fixed set of labels.
• Score an answer against an explicit completeness rubric.
• Assess whether a retrieved document is relevant to a question.
• Choose which tool or workflow should handle a request.

The useful pattern is not “replace every chat model.”

It is use the right output shape for the task.

If you need a paragraph, use a chat model. If you need a decision distribution, start with a decision model.

Important: early access is still early access

Version 0.6.2 is published on NuGet, with a separate README for each package. That is not a production-readiness certificate.

Here is the current status:

The shared Score API supports 2-10 rubric levels. Ollama text requests are bounded at 64 KiB. Model-specific choice limits still apply.

Confidence is not a safety guarantee. A clean probability distribution is not proof that the model understood your business.

Measure on your own data. Keep a human-review path where appropriate. And do not turn an illustrative example into a benchmark slide.

Try the samples without credentials

The repository samples include an explicit offline mode:

git clone https://github.com/elbruno/ElBruno.AI.Decisions.git
cd ElBruno.AI.Decisions
dotnet run --project samples\12-FoundryDecision -- --offline
dotnet run --project samples\11-OllamaLocal -- --offline

These use synthetic responses. Great for exploring the API, not for measuring real model quality.

For real deployments, use the configuration and user-secrets guide.

Links

• Repository, source and samples
• NuGet core package
• Foundry package
• Ollama package
• Jev package
• Microsoft-Decision-1 introduction
• Microsoft’s Foundry endpoint example
• Ollama decision guide
• Ollama System One API
• TypeSafe AI documentation

Wrap-up

I like this direction because it makes a common application need explicit: sometimes we need AI to decide, not to talk.

Foundry when I want a hosted Microsoft model. Ollama when I want a local decision model. Jev when I want its native typed workflow.

One shared API where it makes sense, provider-specific features where they matter.

Less chat. More decisions. And hopefully, fewer paragraphs explaining why my invoice belongs to billing.

Happy coding!

El Bruno

This blog post was created with the help of AI tools. Yes, I used a bit of magic from language models to organize my thoughts and automate the boring parts, but the geeky fun and the in C# are 100% mine.

Draft for future publication and promotion. The DataIngestion adapter is available as 0.9.3-preview.1. The Markdown and ZIP examples use the current repository source, without assuming those features are already available in a stable release.

Hi!

Converting a PDF to Markdown is great. But when you are preparing documents for an AI pipeline, the next question is: what do I do with that Markdown afterward?

A document is more than a string. It has titles, sections, tables, lists, code examples, and references to its source. Flatten everything too early, and you lose information that could help you decide how to split it, enrich it, or display a citation.

I added three pieces to ElBruno.MarkItDotNet to support that next step: structured Markdown import with Markdig, a ZIP connector that does not extract files to disk, and an optional adapter for Microsoft.Extensions.DataIngestion.

The goal is not to implement an entire AI pipeline in one library. It is to connect its stages well.

First: What Is Available and How to Try It

The ElBruno.MarkItDotNet.DataIngestion adapter targets .NET 8 and .NET 10. Its Microsoft dependency is in preview and stays isolated from CoreModel.

To try the published adapter in a new application:

dotnet new console -n AdapterDemo -f net10.0
Set-Location AdapterDemo
dotnet add package ElBruno.MarkItDotNet.DataIngestion --version 0.9.3-preview.1

This command uses the .NET 10 SDK. The adapter also includes a .NET 8 target. CoreModel comes in as a dependency of the adapter. You do not need to install Microsoft’s full pipeline package to call ToIngestionDocument().

To try all three pieces together, the repository includes a runnable demo that uses project references:

git clone https://github.com/elbruno/ElBruno.MarkItDotNet.git
Set-Location ElBruno.MarkItDotNet
dotnet run --project src\samples\MarkdownIngestionDemo\MarkdownIngestionDemo.csproj --framework net8.0

The demo creates its own test ZIP containing Markdown, JSON, and text, processes the entries, and deletes the temporary archive when it finishes. It does not need AI services, API keys, or private documents.

1. A Minimal Adapter Example Using Only NuGet

Let’s start small: build a document and adapt it.

This complete program can replace Program.cs in AdapterDemo:

using ElBruno.MarkItDotNet.CoreModel;
using ElBruno.MarkItDotNet.DataIngestion;
var model = new Document
{
    Id = "release-notes",
    Metadata = new DocumentMetadata
    {
        Title = "Release notes",
        SourceFormat = "generated"
    },
    Sections =
    [
        new DocumentSection
        {
            Id = "changes",
            Heading = new HeadingBlock
            {
                Id = "changes-heading",
                Text = "Changes",
                Level = 2
            },
            Blocks =
            [
                new ParagraphBlock
                {
                    Id = "intro",
                    Text = "Documents now have structure."
                }
            ]
        }
    ]
};
var ingestion = model.ToIngestionDocument();
Console.WriteLine($"Identifier: {ingestion.Identifier}");
foreach (var element in ingestion.EnumerateContent())
{
    Console.WriteLine($"{element.GetType().Name}: {element.GetMarkdown()}");
}

The result keeps the document identifier. The title becomes a level 1 header, while the section heading and paragraph become native Microsoft ingestion elements.

There is no hidden LLM call here. No embeddings or automatic indexing. This is a change of representation so that other components can consume the document.

2. Markdown Becomes More Than a String

In the repository demo, this Markdown is our input:

# Deployment guide
## Configuration
Use **environment variables** for configuration.
| Setting | Value |
| --- | --- |
| Mode | Preview |
## Checklist
- [x] Build the application
- [ ] Review the deployment
```csharp
Console.WriteLine("Ready!");
```
> Review before deploying.

The new import API is intentionally small:

using ElBruno.MarkItDotNet.Markdown;
var model = MarkdownDocumentParser.Parse(markdown, "docs/deployment.md");
Console.WriteLine($"Title: {model.Metadata.Title}");
Console.WriteLine($"Format: {model.Metadata.SourceFormat}");
Console.WriteLine($"Document ID: {model.Id}");

Markdig parses CommonMark and its advanced extensions. The parser maps the result to Document, DocumentSection, and CoreModel blocks.

An initial H1 becomes the document title. Other headings form the section hierarchy. Paragraphs, tables, lists, code, and block quotes have their own representations; unsupported block content is retained as raw Markdown.

The existing Markdown converter keeps its behavior: structured import is optional, not a mandatory replacement for Markdown pass-through.

Walk Sections Without Losing Their Hierarchy

Reading only model.Sections is not enough: sections can contain subsections. This helper walks both:

using ElBruno.MarkItDotNet.CoreModel;
static void PrintSections(
    IReadOnlyList<DocumentSection> sections,
    int depth = 0)
{
    foreach (var section in sections)
    {
        var indent = new string(' ', depth * 2);
        Console.WriteLine($"{indent}Section: {section.Heading?.Text ?? "(root)"}");
        foreach (var block in section.Blocks)
        {
            Console.WriteLine($"{indent}  {block.GetType().Name}, ID: {block.Id}");
        }
        PrintSections(section.SubSections, depth + 1);
    }
}
// Inside your program:
// PrintSections(model.Sections);

The example input produces ParagraphBlock, TableBlock, ListBlock, CodeBlock, and QuoteBlock instances.

Inspect a Checklist

In our example document, Checklist is a top-level section:

var checklist = model.Sections.Single(
    section => section.Heading?.Text == "Checklist");
var tasks = checklist.Blocks.OfType<ListBlock>().Single();
foreach (var item in tasks.Items)
{
    Console.WriteLine($"{item.Text}: checked={item.IsChecked}");
}

IsChecked is nullable: true and false represent task items; null identifies an ordinary list item.

3. Save the Structure and Render Markdown Again

Serializing the model to JSON is a useful way to inspect its full structure:

using System.Text.Json;
var json = JsonSerializer.Serialize(
    model,
    new JsonSerializerOptions { WriteIndented = true });
Console.WriteLine(json);

Blocks use polymorphic discriminators such as $type. The JSON shows which content is a paragraph, a table, or a code block, along with its IDs and source references.

We can also render Markdown again:

using ElBruno.MarkItDotNet.CoreModel;
var rendered = MarkdownRenderer.Render(model);
Console.WriteLine(rendered);

There is an important distinction: OriginalMarkdown retains the syntax of the block, including bold text, links, or code fences. The renderer prioritizes that value when it is present.

This is not a promise of byte-for-byte reproduction of the entire file: spacing between blocks, line endings, and the title may be normalized. Editing Text also does not automatically invalidate OriginalMarkdown.

If I want to replace a parsed paragraph, I make that decision explicit:

var paragraph = model.Sections
    .SelectMany(section => section.Blocks)
    .OfType<ParagraphBlock>()
    .First();
var edited = paragraph with
{
    Text = "Use managed configuration for production.",
    OriginalMarkdown = null
};
Console.WriteLine(edited.Text);

with creates a new block. To render the updated document, you must include that block in a new section or document. The original block does not change.

4. ZIP as a Document Source, Not a Temporary Folder

A ZIP often arrives as a documentation bundle: guides, notes, JSON files, or reports. The connector exposes its files as SourceDocument instances through IDocumentSource.

using ElBruno.MarkItDotNet.Connectors;
using Microsoft.Extensions.Logging.Abstractions;
var source = new ZipArchiveConnector(
    new ZipArchiveConnectorOptions
    {
        ArchivePath = "documents.zip",
        MaxArchiveSizeBytes = 10 * 1024 * 1024,
        MaxEntrySizeBytes = 2 * 1024 * 1024,
        MaxTotalUncompressedBytes = 20 * 1024 * 1024,
        MaxEntries = 100
    },
    NullLogger<ZipArchiveConnector>.Instance);
using var timeout = new CancellationTokenSource(TimeSpan.FromSeconds(30));
await foreach (var document in source.GetDocumentsAsync(timeout.Token))
{
    Console.WriteLine(document.Name);
    Console.WriteLine(document.Source);
    Console.WriteLine(document.Metadata[SourceMetadataKeys.ArchiveEntryPath]);
    await using var content = await document.OpenReadAsync(timeout.Token);
    // Consume the stream here, before leaving this iteration.
}

Enumeration discovers the files. OpenReadAsync() opens their content on demand. The returned stream keeps the ZIP archive open until the stream is disposed.

Directory entries are not returned as documents. Entry names serve as references and are not turned into extraction paths.

For applications that use dependency injection:

using ElBruno.MarkItDotNet.Connectors;
using Microsoft.Extensions.DependencyInjection;
var services = new ServiceCollection();
services.AddZipArchiveConnector(options =>
{
    options.ArchivePath = "documents.zip";
    options.MaxEntries = 100;
    options.MaxEntrySizeBytes = 2 * 1024 * 1024;
});
using var provider = services.BuildServiceProvider();
var source = provider.GetRequiredService<IDocumentSource>();

This DI example also requires Microsoft.Extensions.DependencyInjection, the concrete container. The connectors package provides registration extensions and abstractions, not that container.

Limits Matter

Entries exceeding the per-entry limit are skipped with a warning. If the archive size, entry count, or eligible total exceeds its limit, the connector rejects the operation.

Reads have an additional per-entry cap. The total limit is not a shared counter across concurrent or repeated reads; it is a discovery-time check based on declared sizes.

This should not be presented as universal protection against malicious ZIP archives. There is no recursive extraction of nested ZIPs, decompression CPU budget, or configurable compression-ratio limit. In production, I would still use concurrency limits, isolation, and application-specific policies for untrusted files.

5. ZIP, Conversion, Parsing, and Adaptation: The Complete Flow

The four stages can be connected without writing intermediate files:

using ElBruno.MarkItDotNet;
using ElBruno.MarkItDotNet.DataIngestion;
using ElBruno.MarkItDotNet.Markdown;
var converter = new MarkdownConverter();
var allowedExtensions = new HashSet<string>(StringComparer.OrdinalIgnoreCase)
{
    ".md", ".txt", ".json"
};
await foreach (var document in source.GetDocumentsAsync(timeout.Token))
{
    var extension = Path.GetExtension(document.Name);
    if (!allowedExtensions.Contains(extension))
    {
        Console.WriteLine($"Skipped unsupported document: {document.Name}");
        continue;
    }
    await using var content = await document.OpenReadAsync(timeout.Token);
    var convertedMarkdown = await converter.ConvertAsync(
        content, extension, timeout.Token);
    var structured = MarkdownDocumentParser.Parse(
        convertedMarkdown, document.Source);
    var ingestionDocument = structured.ToIngestionDocument();
    Console.WriteLine(
        $"{document.Name}: {ingestionDocument.EnumerateContent().Count()} elements");
}

This snippet reuses source and timeout from the earlier example. The complete program is in the demo.

The allowlist is an application decision. We can extend it to formats supported by registered converters, but a ZIP does not magically make every file inside it compatible.

There is also a traceability detail: sourcePath includes the ZIP archive and entry, but the parser’s offsets refer to the converted Markdown, not equivalent positions in the original PDF or DOCX.

One more distinction: no intermediate files does not mean constant memory usage. The converter produces a string and the parser builds a tree; the ZIP connector avoids extracting every entry to disk.

6. Native Tables, Not Just Text with Pipes

The adapter maps a table to IngestionDocumentTable with a cell matrix:

using Microsoft.Extensions.DataIngestion;
var ingestion = model.ToIngestionDocument();
var table = ingestion.EnumerateContent()
    .OfType<IngestionDocumentTable>()
    .Single();
Console.WriteLine($"Rows: {table.Cells.GetLength(0)}");
Console.WriteLine($"Columns: {table.Cells.GetLength(1)}");
Console.WriteLine($"Header: {table.Cells[0, 0]?.Text}");
Console.WriteLine($"Value: {table.Cells[1, 1]?.Text}");

For our input, the first row contains Setting and Value; the second contains Mode and Preview.

This makes the structure available to downstream components that need to handle tables differently from paragraphs.

7. IDs and Source References Travel Too

The adapter’s metadata uses the markitdotnet. prefix:

var root = ingestion.Sections.Single();
Console.WriteLine(root.Metadata["markitdotnet.documentId"]);
Console.WriteLine(root.Metadata["markitdotnet.sourceFormat"]);
foreach (var element in ingestion.EnumerateContent())
{
    if (element.Metadata.TryGetValue("markitdotnet.blockId", out var blockId))
    {
        Console.WriteLine($"Block: {blockId}");
    }
    if (element.Metadata.TryGetValue("markitdotnet.sourceOffset", out var offset))
    {
        Console.WriteLine($"Markdown offset: {offset}");
    }
    if (element.Metadata.TryGetValue("markitdotnet.sourcePath", out var path))
    {
        Console.WriteLine($"Source: {path}");
    }
}

Document metadata is stored on the root section. Block references are stored on the corresponding element; PageNumber is also copied when present.

The parser generates a deterministic ID for the same content and source path. That does not mean the ID stays the same after editing the file, moving it, or changing the converted Markdown.

Lists, block quotes, and code blocks do not all have dedicated native equivalents in this adapter: they are represented as paragraphs with Markdown and block-type metadata. The internal hierarchy of a list or quote is not mapped to a hierarchy of Microsoft ingestion elements.

Why an Optional Package?

Because not every consumer needs the same thing.

If I want to convert a file to Markdown, I keep using the main package. If I want to inspect Markdown structure, I use the optional parser. If I want to integrate CoreModel with Microsoft components, I add DataIngestion.

The adapter references Microsoft.Extensions.DataIngestion.Abstractions. It does not parse Markdown again with Microsoft’s reader or force the base library to depend on a preview API.

That gives us a clear boundary of responsibility: MarkItDotNet prepares and represents documents; your application decides how to use them.

Try It and Follow the Project

The complete example for this article is in MarkdownIngestionDemo. You can run it on .NET 8 or .NET 10 by changing --framework.

The Markdown import guide, ZIP connector guide, and adapter guide cover each API separately.

Find the source and packages here:

• ElBruno.MarkItDotNet repository.
• DataIngestion adapter 0.9.3-preview.1 on NuGet.
• Microsoft.Extensions.DataIngestion.

If you are preparing documents for AI with C#, give the demo a try. Sometimes the most useful next step is not another model call: it is preserving more of the structure your documents already have.

Cheers!

Bruno

Materials for Future Promotion

Short title: Markdown + ZIP + DataIngestion: Structured Documents with C#.

Excerpt: ElBruno.MarkItDotNet adds structured Markdown import, ZIP reading without extraction, and an optional adapter for Microsoft.Extensions.DataIngestion. A runnable C# demo connects all three pieces.

Social post: A document is more than a string. With ElBruno.MarkItDotNet, you can preserve sections, tables, and source references, read ZIP entries without extracting them to disk, and adapt CoreModel to IngestionDocument. The adapter is in preview, and the repo includes a complete demo. Code: https://github.com/elbruno/ElBruno.MarkItDotNet

Images: Original hero and pipeline diagram in PNG, with editable SVG sources in images/. When publishing outside GitHub, upload both images and replace this draft’s relative paths with the blog’s image URLs.

Editorial note: This article was prepared with AI assistance; examples were checked against the repository APIs. Verify the published Markdown and Connectors versions before replacing the source-based setup with NuGet installation instructions.

This blog post was created with the help of AI tools. Yes, I used a bit of magic from language models to organize my thoughts and automate the boring parts, but the geeky fun and the in C# are 100% mine.

Hi!

EmbeddingGemma 2 Integration

ElBruno.LocalEmbeddings.EmbeddingGemma provides local text-only embeddings from Google’s EmbeddingGemma 2 model through IEmbeddingGenerator<string, Embedding<float>>.

The base model is multimodal, but this integration only handles strings. Image, audio, and video inputs need separate preprocessors and are not supported by this package.

Install and use

dotnet add package ElBruno.LocalEmbeddings.EmbeddingGemma

using ElBruno.LocalEmbeddings.EmbeddingGemma;
using ElBruno.LocalEmbeddings.EmbeddingGemma.Options;
await using var generator = await EmbeddingGemmaEmbeddingGenerator.CreateAsync(
    new EmbeddingGemmaOptions
    {
        OutputDimensions = 768,
        MaxSequenceLength = 8192
    });
var query = EmbeddingGemmaPrompts.CreateSearchQuery("Which planet is known as the Red Planet?");
var passage = EmbeddingGemmaPrompts.CreateDocument(
    "Mars is known for its reddish appearance.",
    title: "Mars");
var vectors = await generator.GenerateAsync([query, passage]);

For a runnable semantic-search example that compares a query against several documents, run:

dotnet run --project src/Samples/EmbeddingGemmaConsoleApp

The sample requests 256-dimensional vectors and ranks its documents by cosine similarity. See EmbeddingGemmaConsoleApp for prerequisites and customization.

Task prefixes are not added automatically. The helper methods use Google’s recommended retrieval formats:

• Query: task: search result | query: {query}
• Document: title: {title or none} | text: {content}

For symmetric tasks, use the same task prefix for every input (for example, task: sentence similarity | query: {text}). Avoid comparing vectors created for incompatible tasks or different output dimensions.

Configuration

Dependency injection is available from ElBruno.LocalEmbeddings.EmbeddingGemma.Extensions:

services.AddEmbeddingGemmaEmbeddings(options =>
{
    options.OutputDimensions = 256;
    options.CacheDirectory = @"D:\model-cache";
});

The DI factory initializes the model synchronously on first resolution. In async-first applications, create the generator with CreateAsync and register the instance as a singleton instead.

Model files, revision, and licensing

The package downloads three files:

• onnx-community/embeddinggemma-2-ONNX, revision daa72c51243991dfcaf9f9137d2c573d8f7790c0: Q4 text graph and its external weight data.
• google/embeddinggemma-2, revision 914f7f89142e33e77833254d9c9b90c3cef7303b: SentencePiece tokenizer.model.

Each file is checked against its pinned SHA-256 digest before it is used. The Q4 ONNX external weights are approximately 166 MiB (174 MB), the graph is below 1 MiB, and the tokenizer model is approximately 4.7 MB. The export includes Microsoft ONNX Runtime custom quantization operators and is tested with ONNX Runtime 1.24.4.

EmbeddingGemma 2 is published under Google’s Apache 2.0 Gemma 4 license terms. The exporter repository indicates Apache 2.0 as well. Review the source model’s license and terms for your distribution and use case.

Offline and local model use

After a successful download, set EnsureModelDownloaded = false to avoid network access:

await using var generator = await EmbeddingGemmaEmbeddingGenerator.CreateAsync(
    new EmbeddingGemmaOptions { EnsureModelDownloaded = false });

Alternatively, point ModelPath at a directory containing the pinned model layout. Local paths are loaded directly; they are not downloaded or checksum-verified automatically.

Output dimensions and context

Google supports 768-dimensional output and Matryoshka truncation to 512, 256, or 128 dimensions. This integration takes the leading dimensions and normalizes the truncated vector. Use the same dimension for every vector in an index.

Input length is capped at 8,192 tokens, including BOS and EOS. Longer inputs are truncated by the tokenizer. Chunk longer documents before embedding when complete-document retrieval matters.

The model-backed integration test is opt-in because it requires the large ONNX artifacts. With .NET 8 selected, set EMBEDDINGGEMMA_DOWNLOAD_INTEGRATION=1 before running the EmbeddingGemma test project to download and verify the pinned model, then execute an end-to-end embedding check. Without this variable, tests use a model already present in EMBEDDINGGEMMA_MODEL_PATH or the default cache and skip the integration check when none is available.

Troubleshooting

• Model download/hash error: Confirm access to Hugging Face and retry; downloaded files are pinned and verified, so a mismatched artifact is rejected.
• Missing external data: Keep model_q4.onnx_data beside the ONNX graph under the onnx directory.
• Local path error: Verify tokenizer.model, onnx/model_q4.onnx, and onnx/model_q4.onnx_data exist.
• Memory pressure or slow startup: The Q4 graph and external weights total about 166 MiB; including the tokenizer, the first download is about 171 MiB. Expect higher temporary memory requirements while ONNX Runtime loads the graph. Reduce batch size and sequence length for constrained machines.
• Different search quality: Use matching query/document task formats and keep index/query dimensions the same.

References

• Google EmbeddingGemma 2 documentation
• Google model repository
• Pinned ONNX export
• Hugging Face SentencePiece tokenizer API

This blog post was created with the help of AI tools. Yes, I used a bit of magic from language models to organize my thoughts and automate the boring parts, but the geeky fun and the in C# are 100% mine.

Hi!

TL;DR

I added ElBruno.LocalLLMs.Decisions to the library (shipped in v0.22.0). It runs the Laya decision model family locally, in-process, on ONNX Runtime. No Python. No server. No API key.

• Typed answers, not prose: choice, score, and yes/no — each with a full probability distribution.
• One forward pass. Asking 4 questions costs about what asking 1 costs. Measured: ~590 ms for 4 questions.
• Non-autoregressive. It doesn’t generate an answer token by token. It reads your text once and reads the probabilities straight off the output.
• Honest warning up front: the public checkpoints are not reliably calibrated. The ranking is good. The confidence number needs your own validation. I’ll show you exactly where it breaks.

dotnet add package ElBruno.LocalLLMs.Decisions

The problem: I was using a sledgehammer to answer a yes/no question

Here’s the thing I kept doing, and I bet you do it too.

I have a support ticket. I want to know which team should get it. So I call a chat model:

“You are a helpful classifier. Given the following ticket, respond with exactly one of: billing, technical, sales. Respond with only the word, no explanation. Ticket: …”

And then I pray. Because what comes back might be billing. Or Billing. Or "billing". Or Based on the content, this appears to be a billing issue. Or, my personal favourite, a beautifully formatted markdown table explaining its reasoning when I explicitly asked for one word.

So I write a parser. Then I write a retry. Then I write a fallback. And all of that is wrapped around a model that is generating text token by token to tell me one of three things I already knew the shape of.

That’s System Two thinking — slow, deliberate, sequential — used for a System One job.

System One is the fast one. It’s the “that’s a snake” reflex before your brain says the word snake. Recognition, not reasoning. And when your job is “put this in one of three buckets,” recognition is all you need.

That’s exactly what Laya is built for.

What “one forward pass” actually means

A normal LLM answering your question does this:

read prompt → predict token → append → predict token → append → ... → stop

Every token is another trip through the network. That’s why latency scales with how much it decides to say.

Laya does this:

read prompt (with [MASK] markers where the answers go) → ONE pass → read probabilities

The answer positions are marked in the input. After a single pass, the probabilities for every one of your questions are sitting right there in the output tensor. There’s no sampling, no temperature roulette, no “it decided to be chatty today.”

Which means batching is nearly free. Four questions get four sets of markers in the same sequence, and you still pay for one pass.

Show me the code

Minimal version — one question:

using ElBruno.LocalLLMs.Decisions;
using var client = new LayaOnnxDecisionClient();
var result = await client.ChooseAsync(
    "My invoice charged me twice for the same subscription month and I want my money back.",
    new[] { "billing", "technical", "sales" },
    "Which team should handle this support ticket?");
Console.WriteLine(result.Choice);        // billing
Console.WriteLine(result.Probability);   // 0.984

First run downloads the model from HuggingFace (~800 MB) and caches it. Every run after that is local and offline.

Option descriptions matter more than you’d think

This is the single highest-leverage thing in the whole API, and it’s easy to miss. Instead of bare labels, pass a dictionary with descriptions:

var teams = new Dictionary<string, string?>
{
    ["billing"]   = "Payment, invoice and refund problems",
    ["technical"] = "Bugs, outages, API errors and anything with a stack trace",
    ["sales"]     = "Pricing, plan comparisons, upgrades and new purchases"
};

The word technical on its own is nearly meaningless to a classifier. “Bugs, outages, API errors and anything with a stack trace” gives it something to actually match against. The accuracy difference is not subtle.

The real thing: 4 questions, 1 pass

var result = await client.EvaluateAsync(
    new DecisionRequest(ticket)
        .Choose("team", teams, "Which team should handle this support ticket?")
        .Score("urgency", urgencyLevels, "How urgent is this ticket?")
        .Ask("refund", "The customer is asking for a refund.")
        .Ask("angry",  "The customer is angry or frustrated."));
ChoiceResult      team    = result.Choice("team");
ScoreResult       urgency = result.Score("urgency");
ProbabilityResult refund  = result.Probability("refund");
ProbabilityResult angry   = result.Probability("angry");
// Gate on the probability, not the label — so ambiguous tickets go to a human
// instead of being confidently misrouted.
string route = team.ChoiceOrNull(0.6) ?? "human-review";

Here’s the actual output from the LocalDecisions sample on my machine:

------------------------------------------------------------------------------
My invoice charged me twice for the same subscription month and I want my m…
  route   : billing        (billing @ 98.4%)
  urgency : urgent, answer within an hour
            score 1.67 of 3
  refund  : yes  (p = 0.940)
  angry   : yes  (p = 0.768)
  distribution: billing 98%  sales 1%  technical 1%
  244 input tokens, 586 ms, model 'elbruno/laya-onnx'
------------------------------------------------------------------------------
The export endpoint returns a 500 every time I pass more than 50 ids. Here …
  route   : technical      (technical @ 73.7%)
  urgency : urgent, answer within an hour
            score 1.82 of 3
  refund  : no   (p = 0.412)
  angry   : no   (p = 0.002)
  distribution: technical 74%  billing 15%  sales 11%
  260 input tokens, 595 ms, model 'elbruno/laya-onnx'
------------------------------------------------------------------------------
Hi! Just wondering what the difference is between the Pro and Team plans be…
  route   : sales          (sales @ 64.9%)
  urgency : normal, answer within a day
            score 0.94 of 3
  refund  : no   (p = 0.000)
  angry   : no   (p = 0.000)
  distribution: sales 65%  billing 20%  technical 15%
  248 input tokens, 526 ms, model 'elbruno/laya-onnx'

Look at that middle one. refund: no (p = 0.412) — the model is genuinely unsure, and it tells me it’s unsure. A chat model would have just said “no” with total conviction. Having the number is the whole point.

And angry: p = 0.002 on the stack-trace ticket versus 0.768 on the double-billing one. That’s a sentiment signal I got for free, in the same pass, because I asked.

Now the part where I’m honest with you

I could stop the post here and it would be a nice launch announcement. But I ran into something while building this that you need to know about, because it would absolutely bite you in production.

Laya’s public checkpoints are not reliably calibrated.

Laya fits a separate temperature for each (question type, option count) bucket — choice:2, choice:3-5, choice:6-10, choice:11+. That temperature divides the logits before the softmax.

The English checkpoint ships choice:11+ = 0.1006.

Laya’s own code says the minimum valid temperature is 0.5. Dividing by 0.1006 multiplies the logits roughly tenfold before the softmax — which flattens every distribution into a spike at whatever won.

So this package clamps it to 0.5 and — this is the important bit — tells you it did:

var result = await client.ChooseAsync(ticket, twelveDepartments, "Route this ticket");
if (result.CalibrationClamped)
{
    // The ranking is still meaningful. The confidence number is not.
    logger.LogWarning("Confidence for {Choice} is not a fitted calibration.", result.Choice);
}

It’s a per-answer flag, not a global one, because a single checkpoint can have perfectly good temperatures for some buckets and garbage for others.

But here’s what the clamp does not buy you

I wrote in the docs, at first, that clamping “keeps the number honest.” Then I actually ran it, and I was wrong. So I went back and fixed the docs — and the HuggingFace model card.

Same ticket, same model, only the number of options changes:

Both picked the right department. But the 12-option answer reports 100.0% confidence — after clamping. Because 0.5 still doubles the logits. Clamping prevents the catastrophic tenfold sharpening; it does not give you a calibrated number.

What that means in practice:

• Use the ranking. Choice and the relative order of Probabilities are trustworthy.
• Don’t use the magnitude for choices with 11+ options.
• ChoiceOrNull(threshold) will never abstain in that bucket — a saturated probability clears every threshold you’d plausibly set.
• Prefer fewer options. Splitting one 12-way choice into two smaller ones keeps you in a bucket the checkpoint actually fitted correctly.

The fp16 gotcha

One more, smaller one. The checkpoint stores fp16 weights. Ask a question alone vs. batched with three others and you get slightly different numbers, because batched matrix multiplications accumulate in a different order:

Deltas land around 1e-6 to 1e-4. Way too small to flip a routing decision. But big enough to ruin your day if you assert exact equality in tests (use a 1e-3 tolerance), cache on the probability value (key on the label instead), or hash results for change detection (round first).

Batching is still the right default. Just don’t treat the low decimals as stable.

What I had to build to make this work in .NET

There was no ONNX build of Laya when I started. There is now — I ported one and published it as elbruno/laya-onnx (mirroring the Apache-2.0 original).

The .NET side had to reimplement, faithfully, what the Python reference does:

The whole thing is covered by 79 tests, including parity checks against the reference outputs — because “I ported the math correctly” is a claim that deserves evidence, not vibes.

Wiring it into DI is one line:

builder.Services.AddLocalDecisions();

When to use this (and when not to)

Reach for it when:

• You’re classifying, routing, triaging, moderating, or scoring
• You need the answer fast and cheap and offline
• You want a probability, not a sentence
• You’re asking the same shaped question thousands of times

Don’t reach for it when:

• You need something written, summarised, or explained
• The task genuinely needs multi-step reasoning
• You need calibrated confidence over 11+ options and can’t fit your own temperature

It’s not a replacement for a chat model. It’s the thing you put in front of one, so the expensive model only sees the cases that actually need it.

Getting started

1. Install:dotnet add package ElBruno.LocalLLMs.Decisions
2. Run a sample — the model downloads and caches on first use:dotnet run --project src/samples/LocalDecisions
3. See the caveats live:dotnet run --project src/samples/DecisionCalibration
4. Fit your own threshold on your own labelled data. The 0.5 default is a starting point, not a decision boundary.

What’s next

• Surfacing the escalation head (act_logits) — Laya has a signal for “this needs a human” that I read but don’t expose yet
• int8 export and a proper benchmark against the fp16 one
• Multilingual checkpoints — the English one stays confident on non-Latin scripts while being wrong, which is the worst combination

Building this changed how I think about where LLMs belong in an app. Not every question deserves a conversation. Some questions just need an answer, in 600 milliseconds, on your own machine, with a number attached so you know how much to trust it.

And honestly? Finding that 0.1006 temperature — and then discovering my own fix didn’t fully fix it — was the most fun part. That’s the difference between shipping a wrapper and shipping something you’ve actually looked inside.

— Bruno

Full guide: Decisions Guide · Samples: LocalDecisions · DecisionCalibration · Model: elbruno/laya-onnx · Original: NandhaKishorM/laya

Made with by Bruno Capuano (ElBruno)

This blog post was created with the help of AI tools. Yes, I used a bit of magic from language models to organize my thoughts and automate the boring parts, but the geeky fun and the in C# are 100% mine.

The Aspire Demo We Didn’t Get to Show Live: Watching an Agent Harness Work

We ran out of time during Session 4 of MafClaw to show the Aspire dashboard demo. And this is one of those demos that really benefits from seeing it, not just hearing “yes, we have observability.”

So here is the walkthrough we didn’t get to do live: take a small C# agent, let Microsoft Agent Framework’s Harness call a tool, and follow that work through traces, structured logs, and metrics in Aspire.

No finance application to understand first. No AppHost. No Docker requirement. Just a console sample and the standalone dashboard.

Start with one question and one tool

Sample 12, 12-observability-aspire, asks a deliberately simple question:

What is this workshop lesson about?

The agent has one ordinary C# function, get_lesson_topic. It returns a synthetic workshop topic. The interesting part is not the answer; it is what happens between the question and the answer.

The model requests the tool, the Harness executes it and sends its result back, and the model produces the final response. We make one RunAsync call. We do not write that model/tool loop ourselves.

This is a new variant, not a replacement for the earlier observability samples:

The complete finance application is still the final example, after these concepts have been introduced on their own.

Two terminals, then dotnet run

You need .NET 10, the Aspire CLI, and the MafClaw source. This walkthrough uses Aspire CLI 13.5.1. The normal sample uses your configured Microsoft Foundry model, so complete the shared setup once from the repository root:

.\tools\configure-user-secrets.ps1
az login --output none

If your Session 4 model settings are already configured, do not set them again just for this sample. It shares the existing settings store. Live model inference can incur charges. In the first terminal:

aspire dashboard run

Open the login URL printed by the CLI and keep its browser token private. The default browser endpoint is http://localhost:18888; the OTLP/HTTP receiver is on http://localhost:4318. They are different ports for different jobs.

In the second terminal, from the public repository root:

cd .\session-04\samples\12-observability-aspire
dotnet run

That’s the demo command. No --live, no save/resume arguments, and no dashboard configuration file to write.

Select mafclaw-sample12 in the dashboard. The console prints the trace ID so you can find the exact run instead of guessing which request to open.

About the screenshots: these were captured after the session, not during the broadcast. They use explicitly scripted model responses (--fixture) to keep the screenshots reproducible and free of live deployment identifiers. The Harness, C# tool execution, OpenTelemetry export, and Aspire records are real. The short timings are not live model latency or a performance benchmark.

1. Traces: who did what?

Post-session fixture capture: five real spans, including the model/tool/model sequence inside the Harness invocation.

Expand the trace and read it from the outside in:

• lesson.run is the sample’s root operation.
• invoke_agent is the MAF agent invocation.
• The first chat is the model turn that asks to use the tool.
• execute_tool get_lesson_topic is the actual C# tool execution.
• The second chat processes the returned result.

That is the part I wanted to pause on during the stream. The final answer doesn’t tell us whether a function actually ran. The trace gives us the execution path and the parent/child relationships.

In a live run, this also helps separate model latency from tool latency. A fixture still proves orchestration, but its near-instant model turns cannot tell us how fast the configured model will be.

2. Structured logs: give a message its context

The completion log links back to the same trace shown above.

The application emits one intentionally boring message:

logger.LogInformation("Lesson completed with {ToolCalls} tool call.", toolCalls);

Because that log is written while lesson.run is active, it carries the trace and span identifiers. Click the trace link in Aspire and you return to the work that produced the message.

This is much more useful than a disconnected “done!” line. The log answers “what did the application report?” and the trace lets us investigate “what happened around it?”

Also notice what the message does not contain: the user prompt, the model’s answer, or the tool payload.

3. Metrics: count the behavior, not the prose

The counter is incremented inside the real C# tool. Aspire’s Table view shows a value of 1 for this run.

The counter is small enough to explain without leaving the file:

var toolCounter = meter.CreateCounter<long>(
    "lesson.tool.calls",
    description: "Actual lesson-tool invocations.");

Inside GetLessonTopic, we call:

toolCounter.Add(1);

This is a measurement of an invocation, not the model claiming that it called a tool. Each run of this short console sample creates a fresh in-process counter; it is not a durable total across process restarts.

Aspire also lists SDK instruments such as operation duration and token usage when available. Missing token data does not mean zero tokens. Scripted inference is not evidence of real model usage or cost.

Where the instrumentation comes from

The integration is visible in Program.cs, rather than hidden behind the final application’s factory. These two excerpts show the important boundaries:

using var observedModel = model.AsBuilder()
    .UseOpenTelemetry(
        sourceName: sourceName,
        configure: telemetry => telemetry.EnableSensitiveData = false)
    .Build();

Microsoft.Extensions.AI observes individual model calls. We use that client to construct the Harness with AsHarnessAgent, then instrument the outer agent invocation:

var agent = harness.AsBuilder()
    .UseOpenTelemetry(
        sourceName,
        telemetry => telemetry.EnableSensitiveData = false)
    .Build();

The Harness’s automatic telemetry wrapper is disabled in this sample so the explicit agent wrapper is the one we teach, not a duplicate. Tool invocation instrumentation comes from the SDK. OpenTelemetry then subscribes to those sources and exports the data:

using var traces = Sdk.CreateTracerProviderBuilder()
    .SetResourceBuilder(resource)
    .AddSource(sourceName, "Microsoft.Extensions.AI")
    .AddOtlpExporter(options => ConfigureExport(options, "traces"))
    .Build();

The metric and log providers use the same service name. The exporter helper sends HTTP/protobuf to /v1/traces, /v1/metrics, and /v1/logs on the local receiver.

So the responsibilities are separate: the Harness orchestrates, the SDK instruments, OpenTelemetry exports, and Aspire displays. Starting the dashboard alone does not instrument an application.

Two details that matter in a short demo

Flush before the process exits. A console program can finish before batched telemetry has been sent. The sample explicitly flushes traces and metrics and disposes the log provider. It also observes exporter failures. A successful answer followed by a failed export is not reported as an observability success.

ASPIRE EXPORT PASS means the run and flushes completed without a reported exporter failure. It is still worth opening Aspire and checking the retained records, as we did here.

Keep content capture off, but don’t call that complete redaction. Both wrappers set EnableSensitiveData = false. Operational metadata, including model identifiers or server addresses in live runs, can still appear. Use synthetic questions, keep the dashboard local, and inspect screenshots before sharing them.

The screenshots came from a separate loopback-only dashboard with anonymous access for the capture. That is why Aspire’s unsecured-endpoint warning is visible. The normal aspire dashboard run command above keeps its default browser-token protection; do not expose an anonymous dashboard to a network.

If you see an answer but no telemetry

First, make sure the dashboard was started before the sample. Then check that the exporter points at 4318, not the UI’s 18888. A stale OTEL_EXPORTER_OTLP_ENDPOINT environment variable can override the sample’s default; clear it or set it to the matching loopback HTTP base address.

The standalone dashboard stores telemetry in memory. Stopping the console doesn’t remove the exported run, but restarting the dashboard does. It is a telemetry viewer here, not an AppHost managing the console process.

For an offline rehearsal, dotnet run -- --fixture uses scripted inference while keeping the real tool and telemetry pipeline. It still needs a running dashboard or collector. That switch is for reproducible checks; the normal teaching flow remains dotnet run.

The demo’s real takeaway

An agent’s answer is the beginning of the investigation, not the whole story. With this small sample we can see which component called the tool, how the model turns fit together, and whether the log and counter agree with the trace.

We didn’t get to show that live, but now you can run it at your own pace: start the dashboard, run the sample, and follow one request all the way through.

The series materials are in the MafClaw repository.
See the Sample 12 source and walkthrough
and the Aspire standalone dashboard documentation
for the complete setup and troubleshooting details.

This blog post was created with the help of AI tools. Yes, I used a bit of magic from language models to organize my thoughts and automate the boring parts, but the geeky fun and the in C# are 100% mine.

Hi!

During the live Q&A in the second session of the From Model to Agent: The Agent Framework Harness, Live in C# series on September 10, 2026, someone in the audience asked a practical question:

What happens if an agent requests approval, but the user does not answer? Can the approval expire or retry?

That question is easy to overlook in a happy-path demo. We usually show the agent requesting a tool, the user selecting Approve or Deny, and the workflow continuing. Real applications also need to handle silence, invalid answers, repeated approval requests, cancellation, and unavailable input.

So, after the session, I created a new sample: 22 – Approval Retries and Timeouts.

The sample adds a bounded approval policy to a Microsoft Agent Framework agent running with Harness:

• five seconds to answer each approval request
• five attempts maximum
• y or yes approves immediately
• n or no denies immediately
• missing or invalid input retries
• cancellation and input failures deny the action
• exhausting the attempts denies automatically
• one denial remains in effect for the rest of the prompt
• repeated approval rounds from the model are also bounded

The important principle is simple:

Silence is not consent.

About the MafClaw series

MafClaw is a four-part Microsoft Reactor live-coding series where we build a C# agent incrementally with .NET, Microsoft Agent Framework, and Harness.

The running sample is a finance education assistant. We start with a model, turn it into an agent, add tools and planning, introduce safe access to data, require human approval for side effects, add memory, and then move toward more advanced and production-ready scenarios.

• Microsoft Reactor series page
• Sample repository
• Series introduction

You can also follow the four sessions on the .NET YouTube channel:

1. Meet Your Claw: A Harness in Three Lines of C#
2. Working With Your Data, Safely: Files, Approvals and Memory
3. Scaling the Claw: Skills, Shell, CodeAct and Background Agents
4. Production Ready: Observability, Governance and Deployment

The audience question that inspired this sample came from session 2, where we explored file boundaries, human approval, and agent memory.

The scenario

The user asks the agent:

Buy 10 shares of MSFT.

This is a simulated trade. No real transaction is created, but it represents a side effect that should not execute only because a model requested it.

The tool is wrapped with ApprovalRequiredAIFunction:

internal static class ApprovalGateTools
{
    [Description("Places a demo-only simulated trade after the Harness approval flow allows it.")]
    public static string RequestSimulatedTradeOrder(
        [Description("Trade side, for example buy or sell.")] string side,
        [Description("Ticker symbol, for example MSFT.")] string symbol,
        [Description("Number of shares.")] int shares)
    {
        var normalizedSide = side.Trim().ToLowerInvariant();
        var normalizedSymbol = symbol.Trim().ToUpperInvariant();
        if (shares <= 0)
        {
            return "Denied: share quantity must be greater than zero.";
        }
        return $"Approved: simulated {normalizedSide} order for {shares} shares " +
               $"of {normalizedSymbol}. This is not a real transaction.";
    }
    public static AIFunction RequestSimulatedTrade { get; } =
        new ApprovalRequiredAIFunction(
            AIFunctionFactory.Create(
                RequestSimulatedTradeOrder,
                "request_simulated_trade"));
}

The model can request request_simulated_trade, but Harness pauses before executing the C# function. The host application, not the model, owns the approval decision.

Configure a bounded approval policy

The sample configures the policy with a small block:

const int maxApprovalAttempts = 5;
var approvalTimeout = TimeSpan.FromSeconds(5);
var approvalPolicy = new TimedApprovalPolicy(
    maxApprovalAttempts,
    approvalTimeout);

These numbers are intentionally short for a live demo. A production application should choose deadlines and retry limits that match its user experience and the risk of the requested action.

The policy validates its configuration:

public TimedApprovalPolicy(
    int maxAttempts,
    TimeSpan timeout,
    Func<TimeSpan, CancellationToken, Task<string?>>? readLineAsync = null)
{
    if (maxAttempts <= 0)
    {
        throw new ArgumentOutOfRangeException(
            nameof(maxAttempts),
            "Approval attempts must be greater than zero.");
    }
    if (timeout <= TimeSpan.Zero)
    {
        throw new ArgumentOutOfRangeException(
            nameof(timeout),
            "Approval timeout must be greater than zero.");
    }
    this.maxAttempts = maxAttempts;
    this.timeout = timeout;
    this.readLineAsync = readLineAsync ?? ReadConsoleLineWithTimeoutAsync;
}

The injectable input reader is useful beyond this console sample. It keeps the approval policy separate from the user interface and makes the decision logic deterministic to test.

Retry missing or invalid input

The approval loop retries only when the user has not made a valid decision:

for (var attempt = 1; attempt <= maxAttempts; attempt++)
{
    Console.Write(
        $"Approve tool {toolName}({arguments})? [y/n] " +
        $"Attempt {attempt}/{maxAttempts}, " +
        $"{timeout.TotalSeconds:0}s timeout: ");
    var input = await readLineAsync(timeout, cancellationToken);
    if (input is null)
    {
        Console.WriteLine(attempt == maxAttempts
            ? "Timed out. Approval denied after the final attempt."
            : "Timed out. Retrying approval.");
        continue;
    }
    input = input.Trim();
    if (input.Equals("y", StringComparison.OrdinalIgnoreCase) ||
        input.Equals("yes", StringComparison.OrdinalIgnoreCase))
    {
        return new ApprovalDecision(
            true,
            $"Approved by console user on attempt {attempt}.");
    }
    if (input.Equals("n", StringComparison.OrdinalIgnoreCase) ||
        input.Equals("no", StringComparison.OrdinalIgnoreCase))
    {
        return new ApprovalDecision(
            false,
            $"Denied by console user on attempt {attempt}.");
    }
    Console.WriteLine(attempt == maxAttempts
        ? "Invalid response. Approval denied after the final attempt."
        : "Invalid response. Enter y or n; retrying approval.");
}
return new ApprovalDecision(
    false,
    $"Denied automatically after {maxAttempts} unanswered or invalid attempts.");

There is an important distinction here:

• No response is not a denial yet, so the policy may ask again.
• Invalid input is not approval, so the policy may ask again.
• Explicit n is a decision, so the policy stops immediately.
• Explicit y is consent, so the policy stops immediately.
• No valid answer after the limit becomes an automatic denial.

Retries must never turn uncertainty into approval.

Do not block forever on console input

Console.ReadLine() waits indefinitely and cannot provide the timeout behavior we need for an interactive console.

The sample checks for available keyboard input until the deadline:

var input = new StringBuilder();
var startedAt = Stopwatch.GetTimestamp();
while (Stopwatch.GetElapsedTime(startedAt) < timeout)
{
    cancellationToken.ThrowIfCancellationRequested();
    if (!Console.KeyAvailable)
    {
        await Task.Delay(50, cancellationToken);
        continue;
    }
    var key = Console.ReadKey(intercept: true);
    if (key.Key == ConsoleKey.Enter)
    {
        Console.WriteLine();
        return input.ToString().Trim();
    }
    if (!char.IsControl(key.KeyChar))
    {
        input.Append(key.KeyChar);
        Console.Write(key.KeyChar);
    }
}
Console.WriteLine();
return null;

The complete implementation also supports redirected input and distinguishes an input timeout from caller cancellation. Cancellation, console I/O errors, and unavailable input are converted into denied decisions instead of allowing the tool to run.

Make denial sticky for the prompt

Limiting keyboard attempts is not enough.

After receiving a denial, the model could theoretically request the same tool again when the agent continues. A new request must not silently reopen the approval process during the same user prompt.

The console runner records whether any approval has already been denied:

var approvalRound = 0;
var approvalDeniedForPrompt = false;
while (true)
{
    var approvalRequests = response.Messages
        .SelectMany(message => message.Contents)
        .OfType<ToolApprovalRequestContent>()
        .ToList();
    if (approvalRequests.Count == 0)
    {
        return;
    }
    approvalRound++;
    if (approvalRound > MaxApprovalRoundsPerPrompt)
    {
        Console.WriteLine(
            "Approval denied: the agent exceeded the approval round limit " +
            "for this prompt.");
        return;
    }
    var approvalResponses = new List<AIContent>();
    foreach (var request in approvalRequests)
    {
        var functionCall = request.ToolCall as FunctionCallContent;
        var toolName = functionCall?.Name ?? request.ToolCall.CallId;
        var arguments = functionCall?.Arguments is null
            ? string.Empty
            : string.Join(
                ", ",
                functionCall.Arguments.Select(
                    item => $"{item.Key}: {item.Value}"));
        var decision = approvalDeniedForPrompt
            ? new ApprovalDecision(
                false,
                "Denied automatically because another approval was already " +
                "denied for this prompt.")
            : await approvalPolicy.RequestApprovalAsync(toolName, arguments);
        approvalDeniedForPrompt |= !decision.Approved;
        approvalResponses.Add(
            request.CreateResponse(decision.Approved, decision.Message));
    }
    response = await agent.RunAsync(
        [new ChatMessage(ChatRole.User, approvalResponses)],
        session);
}

This adds two separate limits:

1. The human gets a bounded number of attempts to answer one approval request.
2. The agent gets a bounded number of approval rounds for one user prompt.

Once a request is denied, later approval requests in that prompt are denied automatically. The model cannot negotiate its way around the user’s answer.

Demo paths

Run the sample:

dotnet run --project `
  .\session-02\samples\22-approval-retries-timeouts\MafClaw.Sample22.csproj

Then use the same agent prompt for each path:

Buy 10 shares of MSFT.

Approved

Answer before the deadline:

y

The tool executes and returns a simulated result.

Explicitly denied

Answer:

n

The request is denied immediately. It does not consume the remaining attempts.

Invalid input followed by approval

Answer:

maybe

The policy rejects the value and asks again. Answer y on the next attempt to approve.

No response

Do not type anything.

Each attempt expires after five seconds. After the fifth attempt, the request is denied automatically and the simulated trade does not execute.

Why this matters outside the demo

Human approval is often presented as a yes/no dialog, but the operational behavior around that dialog is part of the security boundary.

A real approval design should answer questions such as:

• How long is approval valid?
• What happens when the approver is unavailable?
• Can an expired request be retried?
• How many retries are allowed?
• Does an explicit denial remain final?
• Can the agent immediately ask again?
• What happens if multiple tools request approval together?
• What audit data identifies the user, tool, arguments, decision, and time?
• Does the approval authorize exactly one execution?

The sample is deliberately small, but it demonstrates the core rule: the host application owns these policies. The language model should not decide how long to wait, how many times to retry, or whether silence means approval.

Try the complete sample

The complete source is available in the public MafClaw repository:

• Sample 22 – Approval retries and timeouts
• Session 2 samples
• MafClaw repository

And if you want to see the audience discussion that prompted this additional scenario, watch Working With Your Data, Safely: Files, Approvals and Memory on the .NET YouTube channel.

This blog post was created with the help of AI tools. Yes, I used a bit of magic from language models to organize my thoughts and automate the boring parts, but the geeky fun and the in C# are 100% mine.

Hi!

Ok so I spend the full morning with this, so let’s better write it down for Future Bruno:

The deployment was green. The hosted agent was active. The Microsoft Foundry Playground could reach it. There were no authentication failures, container crashes, or model errors.

There was just one problem: when I asked the C# agent, “Who are you?”, it answered:

I’m ChatGPT, an AI assistant created by OpenAI.

That was not the behavior defined in the source code. The C# sample was explicitly instructed to identify itself as the C# hosted agent. The same problem appeared in the Go and C++ hosted agents.

My first suspicion was stale build output or an aggressive container cache. The real cause was simpler and more useful: I had deployed from the wrong Git revision.

The setup

You can find the working samples in the microsoft-foundry-hosted-agents repository. For a focused walkthrough of the C# sample, see From dotnet run to Foundry Hosted Agent in 3 Lines of C# on the .NET Blog.

The repository contains three Microsoft Foundry hosted-agent samples:

Each agent has language-specific instructions. For example, the C# agent is configured with:

instructions: "You are the C# hosted agent sample, running on .NET 10 with Microsoft Agent Framework. " +
              "When greeted or asked who you are, identify yourself as the C# hosted agent. Keep your answers brief."

The Go and C++ samples have equivalent instructions for their own runtimes.

Despite that, all three deployed agents returned the generic model identity.

Why the obvious evidence was misleading

Several facts initially suggested a packaging problem:

• The source files contained the correct language-specific instructions.
• All deployed agent versions were active.
• Each endpoint returned a successful model response.
• The issue reproduced in a completely new session.
• For the C# sample, inspecting the deployed build output revealed the older prompt: You are a friendly assistant.

That last detail looked like proof of a stale compiler artifact. It was proof that the deployed artifact was old, but not proof that the compiler or deployment service had incorrectly cached it.

The Git reflog and timestamps completed the picture.

The timeline that exposed the cause

The original versions were deployed between 11:30 and 11:54:

The language-specific instructions arrived in the working tree with that 12:04 branch switch and pull. The deployments had already happened.

The local source looked correct during the investigation because I was looking at main. Microsoft Foundry was correctly running the artifacts produced earlier from the feature branch.

There was no mysterious cloud cache. I had successfully deployed exactly the wrong code.

Confirming the diagnosis

Before changing anything, I reproduced the behavior outside the Playground. This ruled out old browser conversation history.

For the C# Responses agent:

azd ai agent invoke --new-session --new-conversation "who are you?"

For the Go and C++ Invocations agents:

azd ai agent invoke --new-session "who are you?"

All three returned the generic identity.

Next, I checked the deployed version:

azd ai agent show --output json

Then I compared the repository state:

git branch --show-current
git status --short
git rev-parse HEAD
git rev-parse origin/main
git reflog --date=iso

The reflog was the decisive evidence. It showed that the deployments preceded the switch to the revision containing the new instructions.

The fix

I verified that:

1. the active branch was main;
2. HEAD matched origin/main;
3. the source contained the intended instructions; and
4. the local builds contained the intended prompt.

I then redeployed all three samples. Microsoft Foundry created new immutable agent versions:

Each corrected deployment was tested with a new session. The C# Responses agent also used a new conversation to guarantee that no previous conversation history influenced the result.

A safer deployment gate

The key lesson is that a successful build and a successful deployment answer different questions:

• Build succeeded: this revision can be compiled.
• Deployment succeeded: these packaged bytes can run.
• Behavioral smoke test succeeded: the active deployment behaves like the revision I intended to release.

All three checks are necessary.

1. Verify the Git revision

Run this before packaging or deploying:

git fetch origin
$branch = git branch --show-current
$head = git rev-parse HEAD
$remote = git rev-parse origin/main
Write-Host "Branch: $branch"
Write-Host "HEAD:   $head"
Write-Host "Remote: $remote"
git status --short
if ($branch -ne "main") {
    throw "Deployment must run from main."
}
if ($head -ne $remote) {
    throw "Local HEAD does not match origin/main."
}

Whether deployments must come from main, a release branch, or a tag is a team decision. The important part is to make the rule explicit and fail before deployment when it is not satisfied.

For production releases, a tag or immutable commit SHA is safer than relying only on a branch name:

$expectedCommit = "<approved-commit-sha>"
$actualCommit = git rev-parse HEAD
if ($actualCommit -ne $expectedCommit) {
    throw "Workspace is not at the approved release commit."
}

2. Build and test from that revision

Use the repository’s normal validation commands before deployment:

dotnet build .\MAF-Agents-Samples.slnx
Push-Location .\04-MAF-Agent-GO-Hosted
go test ./...
Pop-Location
Push-Location .\06-Foundry-Agent-CPP-Hosted
cmake --preset debug
cmake --build --preset debug
ctest --preset debug
Pop-Location

This catches source and dependency problems, but it still does not prove that the intended revision reached Microsoft Foundry.

3. Record the commit with the deployment

Capture the SHA in deployment logs:

$commit = git rev-parse HEAD
Write-Host "Deploying commit $commit"
azd deploy --no-prompt

In a CI/CD pipeline, preserve this value as release metadata or an artifact. Being able to answer “Which commit produced agent version 7?” turns a long investigation into a quick comparison.

4. Verify the active immutable version

After deployment:

azd ai agent show --output json

Confirm that:

• the new version is active;
• the protocol is correct;
• the model deployment and environment variables are correct; and
• the endpoint points to the expected agent.

Do not assume that a completed azd deploy means the endpoint has already switched to the version you intended.

5. Test a distinguishing behavior

Hello is a weak smoke test. Almost any functioning model can answer it.

Choose a prompt whose expected response distinguishes the new release from the old one. In this case, identity was ideal:

azd ai agent invoke --new-session "who are you?"

For a Responses agent, reset both forms of state:

azd ai agent invoke --new-session --new-conversation "who are you?"

Other useful distinguishing checks include:

• a newly added tool;
• a changed refusal or safety rule;
• a version-specific response marker;
• a newly supported data source; or
• a corrected edge case.

The smoke test should fail when the previous release is still active.

6. Evaluate the behavior you actually care about

The original evaluation descriptions called these agents friendly, general-purpose assistants. That criterion would pass for both the incorrect and corrected deployments.

The evaluation criteria now explicitly require each agent to identify itself as the C#, Go, or C++ hosted sample. This turns identity from an informal expectation into a regression check.

Evaluations should describe the behavior that makes the release distinct—not only generic qualities such as helpfulness, relevance, and brevity.

CI/CD is the long-term solution

Manual deployment from a developer workstation makes the active branch, uncommitted changes, local build outputs, and shell environment part of the release process.

A CI/CD deployment should instead:

1. start from an explicit commit or signed tag;
2. require a clean checkout;
3. run builds and tests;
4. log the commit SHA;
5. deploy through azd;
6. wait for the new immutable agent version to become active;
7. invoke a release-specific smoke test in fresh state; and
8. run a small regression evaluation suite.

That process does not merely automate deployment. It makes the deployed source traceable and reproducible.

What I would check next time

If deployed behavior does not match local source, I would investigate in this order:

1. Reproduce with a new session and, for Responses, a new conversation.
2. Check the active agent version.
3. Print the current branch and commit SHA.
4. Compare HEAD with the approved release SHA.
5. Check git status for uncommitted changes.
6. Inspect the packaged artifact or container image.
7. Only then investigate build and container caching.

This order starts with the cheapest and most common explanations. It also separates three different sources of stale behavior:

• stale conversation state;
• stale deployed version; and
• stale or unintended source revision.

Final takeaway

The cloud platform did exactly what I asked it to do. The mistake was assuming that the source visible in my editor was the source I had deployed ten minutes earlier.

The most important deployment question is not:

Did the deployment succeed?

It is:

Can I prove which revision is active, and can I prove that it behaves like that revision?

An immutable version number, a recorded Git SHA, and one distinguishing smoke test provide that proof.

This blog post was created with the help of AI tools. Yes, I used a bit of magic from language models to organize my thoughts and automate the boring parts, but the geeky fun and the in C# are 100% mine.

Hi

My favorite demo for this library is intentionally small:

That any any any any collapse is the interesting part. This is more than deleting a list of filler words. It is transcript normalization: turning the rough text emitted by speech-to-text into something a human would actually write.

superwhisper/s1-mini is a 0.6B-parameter Qwen3 fine-tune with one job: ASR transcript normalization. It is not a chat model. It is not a speech model. It does not listen to audio. It takes an existing raw speech-to-text transcript and rewrites it as clean written text.

ElBruno.S1Mini packages that workflow for .NET developers. It downloads an ONNX conversion from Hugging Face on first use, runs it locally with ONNX Runtime GenAI, and exposes a small C# API for normalizing transcripts. Nothing leaves the machine.

The current NuGet package is ElBruno.S1Mini 0.1.1. It targets net8.0 and net10.0, and the C# code is MIT licensed.

The basic C# sample

Install the package:

dotnet add package ElBruno.S1Mini

Then normalize a transcript:

using ElBruno.S1Mini.Normalization;
using var normalizer = await TranscriptNormalizer.CreateAsync();
var cleaned = await normalizer.NormalizeAsync(
    "you don't have any any any any change at all?");
Console.WriteLine(cleaned);
// You don't have any change at all.

On first use, the library downloads elbruno/s1-mini-onnx from Hugging Face into %LOCALAPPDATA%/ElBruno/S1Mini/models. The default model subfolder is int4, about 390 MB on disk, and subsequent runs use the cached copy.

The main API is deliberately small:

using ElBruno.S1Mini.Normalization;
using var normalizer = await TranscriptNormalizer.CreateAsync();
var cleaned = await normalizer.NormalizeAsync(
    "so um i need to like send the the report by uh friday no wait make that thursday",
    new TranscriptNormalizerOptions
    {
        Styling = TranscriptStyling.SemiFormal,
        Structure = TranscriptStructure.Prose,
        Context = TranscriptContext.General,
        MaxTokens = 1024,
    });
Console.WriteLine(cleaned);
// So I need to send the report by Thursday.

You can also configure the local model cache:

using ElBruno.S1Mini;
using ElBruno.S1Mini.Normalization;
var s1Options = new S1MiniOptions
{
    CacheDirectory = @"C:\models",
    EnsureModelDownloaded = true,
};
using var normalizer = await TranscriptNormalizer.CreateAsync(s1Options);

Or register it with dependency injection:

using ElBruno.S1Mini;
builder.Services.AddTranscriptNormalizer(options =>
{
    options.CacheDirectory = @"C:\models";
});

The package also exposes S1MiniClient, an IChatClient implementation for Microsoft.Extensions.AI, but the warning still matters: s1-mini is not a general chat model. TranscriptNormalizer is the safer layer because it builds the prompt shape the model expects.

What the model actually cleaned today

These outputs were measured with the library today using greedy decoding and default options:

The examples show several useful behaviors in a compact set:

• filler removal: um, uh
• stutter and repetition collapse: we we we, the the, any any any any
• self-correction resolution: option b i mean option c becomes option C
• spoken-to-written numerals and currency: twenty five dollars and fifty cents becomes $25.50
• date and time formatting: three thirty on tuesday the tenth becomes 3:30 on Tuesday the 10th

One honest caveat is visible in the currency example: the model kept the colloquial like in The total was like $25.50. That is a good reminder of the boundary. This is a normalizer, not a rewriter. It cleans speech artifacts, punctuation, casing, numbers, dates, and corrections. It does not always turn casual phrasing into formal prose.

The live microphone sample

The repository includes a Windows-only sample at src/samples/LiveMicTranscription. Its pipeline is:

microphone (NAudio, 16 kHz mono)
  -> Silero VAD v5 (ElBruno.Realtime.SileroVad, ~2 MB)
  -> Whisper (ElBruno.Whisper)
  -> s1-mini (ElBruno.S1Mini)

The console UI is built with Spectre.Console. The sample supports live microphone capture, but it also supports repeatable runs:

dotnet run --project src/samples/LiveMicTranscription -- --save-audio
dotnet run --project src/samples/LiveMicTranscription -- --wav recordings

--save-audio records detected utterances. --wav <file|folder> replays a saved file or folder through the identical pipeline. That matters because live microphone testing is otherwise not repeatable. Every take is a different performance.

Caption: The live microphone sample loads Whisper, Silero VAD, and s1-mini locally before listening.

The pipeline shape is not decorative. It came from debugging a real failure mode.

The filler-word paradox

The whole point of s1-mini is removing fillers like um and uh. The first version of the sample used a simple energy-threshold voice activity detector. That sounds reasonable until you remember what fillers are acoustically: low-energy sounds near the noise floor.

The threshold gate removed the exact words the model was supposed to clean. Whisper never received them. s1-mini then had nothing to remove, and the model looked broken.

The fix was switching to a neural VAD: Silero VAD v5 via ElBruno.Realtime.SileroVad. Silero is small, around 2 MB, but it classifies speech instead of only measuring loudness. Quiet fillers survive long enough to reach Whisper.

There was a second, subtler bug. Silero returns multiple speech segments. If you concatenate only those segments, you delete the quiet audio between them. That mangles prosody and can make Whisper lose the very hesitation you wanted to preserve.

The correct approach is to use Silero’s timestamps to cut one contiguous slice: first segment start to last segment end, with about 400 ms of padding. The same spoken sentence produced this A/B result:

Caption: The useful view is side-by-side: raw ASR transcript first, normalized text second.

A common explanation is that Whisper strips disfluencies because it was trained on subtitles. That was tested here and was false for this setup. Whisper Tiny and Base both retained 4/4 fillers verbatim. Larger Whisper models do tidy more aggressively, so the sample defaults matter when you are testing cleanup behavior.

Bonus: the ONNX migration and how s1-mini works internally

The ONNX model used by the package is elbruno/s1-mini-onnx, an explicitly unofficial, unaffiliated, non-endorsed derivative of superwhisper/s1-mini. The upstream model is Apache-2.0 with a naming clause. The C# code in this repository is MIT. The downloaded weights remain under the upstream Apache-2.0 license.

The conversion exists so .NET apps can run s1-mini locally through Microsoft.ML.OnnxRuntimeGenAI 0.15.1. The package uses INT4 by default, and that is the variant to use today.

The Hugging Face repository also publishes an fp16/ folder, but that variant is broken on CPU with onnxruntime-genai 0.15.1. Every prompt fails with a shape mismatch in the GQA repeat_kv Reshape node. This is an ONNX Runtime builder/graph-optimizer bug for FP16 + GQA on the CPU execution provider, not something specific to this conversion. S1MiniOptions.ModelSubPath defaults to int4 for exactly this reason.

There is another runtime edge case worth documenting: temperature=0. The model should run with greedy decoding. Callers should be able to pass Temperature = 0f. But onnxruntime-genai‘s native runtime crashes with an integer divide-by-zero if a literal temperature=0.0 is passed into native search options, even when do_sample=False.

ElBruno.S1Mini guards that at the runtime layer. Temperature <= 0 is treated as the library-wide greedy contract, and the native temperature option is omitted. That keeps greedy decoding safe without changing it into sampling.

Internally, TranscriptNormalizer sends the exact system prompt expected by the model:

You are a text normalizer for speech-to-text transcripts. The input begins with a control line
specifying the styling, structure, and context settings; clean the transcript to match those
settings and output only the cleaned text.

Then it builds a one-line control header followed by the raw transcript:

[Styling: semi-formal] [Structure: prose] [Context: general]
so um i need to like send the the report by uh friday no wait make that thursday

That control line is what steers the model’s behavior. The prompt is rendered with the Qwen3 non-thinking chat format, ported verbatim from the model’s own chat_template.jinja and verified byte-for-byte against the real model. The assistant generation prompt ends with an empty thinking block:

<|im_start|>assistant
<think>
</think>

That selects Qwen3 non-thinking mode, which is what s1-mini expects.

The control values are useful, but they are not all equally literal. Empirical testing showed:

That table is important because API names can imply more than the model actually guarantees. The docs should reflect measured behavior, not the model card we wish we had.

Finally, the upstream quality number should be attributed correctly: Superwhisper reports 94.8% token accuracy on 7,519 held-out English cases. That is their measurement, not an independent benchmark from this project. This is English only, v1.

One further caveat that is easy to get wrong. That figure was measured on Superwhisper’s official GGUF Q4_K_M build — their model card calls it “the build the published accuracy was measured on.” This library runs a different quantization, the INT4 ONNX conversion, and its accuracy has not been separately measured. So 94.8% is a real number about the reference build, not a verified property of the weights this package downloads. Quoting it as the latter would be overclaiming.

If you are not on .NET, those official GGUF builds are the better starting point — they work with llama.cpp, Ollama, and LM Studio directly. The ONNX conversion exists specifically because ONNX Runtime GenAI is what .NET can consume.

For .NET developers, the practical result is straightforward: if you already have raw ASR text and want local cleanup, ElBruno.S1Mini gives you a small API, a cached local model, and a pipeline that stays on your machine.

Links:

• GitHub: https://github.com/elbruno/ElBruno.S1Mini
• NuGet: https://www.nuget.org/packages/ElBruno.S1Mini
• Upstream model: https://huggingface.co/superwhisper/s1-mini
• Official GGUF builds (llama.cpp / Ollama / LM Studio): https://huggingface.co/superwhisper/s1-mini-GGUF
• ONNX model: https://huggingface.co/elbruno/s1-mini-onnx

GitHub Copilot app
        ↓
Custom Azure OpenAI provider
        ↓
Azure OpenAI v1 Responses API
        ↓
Model deployed in Microsoft Foundry

Model:            gpt-5.6-sol
Deployment name:  gpt-5.6-sol
Deployment type:  GlobalStandard
Wire API:         Responses

Models + endpoints →  → Details

https://.services.ai.azure.com/openai/v1/responses

https://.services.ai.azure.com/openai/v1/

Resource host
https://.services.ai.azure.com

API base path
/openai/v1/

Operation
responses

https://.services.ai.azure.com/openai/v1/responses

Settings → Model providers → Add provider → Azure OpenAI

Display name:   Azure OpenAI
Endpoint:       https://.services.ai.azure.com/openai/v1/
API version:    v1
Wire API:       responses
API key:        
Custom headers: {}

https://.openai.azure.com/openai/v1/

https://.services.ai.azure.com/openai/v1/

v1

responses

POST /openai/v1/responses

{
  "model": "gpt-5.6-sol",
  "input": "Hi!"
}

API key resource == endpoint resource

{}

Display name:      gpt-5.6-sol
Wire model:        
Reasoning effort:  High

### Display name

This is the friendly name displayed by the GitHub Copilot app.

It does not have to match the Azure model or deployment name:

“`text

GPT-5.6-sol on Azure

Production reasoning model

Foundry GPT-5.6

“`

### Wire model

This is the identifier sent in the `model` property of the request.

Azure OpenAI routes requests using the **deployment name**, not only the catalog model name.

For example, if Foundry contains:

“`text

Catalog model:    gpt-5.6-sol

Deployment name:  gpt-5-6-sol-prod

“`

configure:

“`text

Display name:  gpt-5.6-sol

Wire model:    gpt-5-6-sol-prod

“`

The app displays the friendly model name but sends:

“`json

{

  “model”: “gpt-5-6-sol-prod”

}

“`

When the display name already matches the deployment name, Wire model can remain empty.

### Token limits

The model editor also supports optional prompt and output token limits.

Leave these fields empty unless you want the app to apply a limit that is lower than the deployment capability. Azure quotas and model limits continue to apply independently.

### Reasoning effort

Select the reasoning levels accepted by the model deployment.

For this example, I enabled:

“`text

High

“`

Reasoning effort affects how much internal reasoning the model can use before returning its answer. Higher effort can improve complex planning and analysis, while increasing latency and token consumption.

## Step 4: save and verify the provider

After saving the provider and model, the Model providers page displays the Azure OpenAI provider and its configured model.

Provider endpoint
https://.services.ai.azure.com/openai/v1/

Wire API
responses

Wire model
gpt-5.6-sol

Authentication
Azure OpenAI resource key

Hi!

Microsoft Foundry deployment
        ↓ deployment name

GitHub Copilot app model
        ↓ Wire model

Azure OpenAI provider
        ├── Endpoint: .../openai/v1/
        ├── API version: v1
        ├── Wire API: responses
        └── Authentication: resource API key

This blog post was created with the help of AI tools. Yes, I used a bit of magic from language models to organize my thoughts and automate the boring parts, but the geeky fun and the in C# are 100% mine.

TL;DR

• KnownModels.MagenticBrain is now ready for local .NET usage with ElBruno.LocalLLMs.
• ONNX package is available at:
  • https://huggingface.co/elbruno/MagenticBrain-onnx
• Original model source:
  • https://huggingface.co/microsoft/MagenticBrain
• This support is designed to power end-to-end agentic app experiences such as:
  • https://github.com/elbruno/ElBruno.MagenticUI

Why this model is important (and what it is designed for)

MagenticBrain is not just another general-purpose chat model. It is designed for agent orchestration: planning multi-step tasks, selecting tools, chaining tool calls across rounds, and deciding when to terminate with a final answer.

That design matters because many app scenarios need more than one prompt/one response:

• file + web research workflows
• iterative tool usage with state between turns
• “do the task, then submit result” orchestration patterns

In short, MagenticBrain is built for agentic execution loops, which is why it is a strong fit for .NET + LocalChatClient + MagenticUI experiences.

When Microsoft Research introduced MagenticLite, MagenticBrain, and Fara1.5, they framed a practical path for local agentic workflows:

• Official post:
  • https://www.microsoft.com/en-us/research/blog/magenticlite-magenticbrain-fara1-5-an-agentic-experience-optimized-for-small-models/

For this repo, this post marks the concrete implementation milestone: MagenticBrain is now a first-class supported model in ElBruno.LocalLLMs.

Why this matters for MagenticUI scenarios

The target is a clean .NET workflow where orchestration and model inference stay in the same stack:

1. Use KnownModels.MagenticBrain in LocalChatClient.
2. Keep tool-calling and multi-round logic in C#.
3. Reuse the same model path for local multi-agent UX in MagenticUI-style applications.

This is exactly the scenario behind:

• https://github.com/elbruno/ElBruno.MagenticUI

Basic usage: MagenticBrain in C#

1. Install

dotnet add package ElBruno.LocalLLMs --version 0.20.4

2. Create a local MagenticBrain client

using ElBruno.LocalLLMs;
using Microsoft.Extensions.AI;
var options = new LocalLLMsOptions
{
    Model = KnownModels.MagenticBrain,
    EnsureModelDownloaded = true,
    Temperature = 0.7f,
    MaxSequenceLength = 32768
};
using var client = await LocalChatClient.CreateAsync(options);

3. Use it in an agentic loop with tools

var response = await client.GetResponseAsync(
[
    new ChatMessage(ChatRole.System, "You are an agentic assistant. Use tools and call submit when done."),
    new ChatMessage(ChatRole.User, "List project files and summarize README.")
],
new ChatOptions
{
    Tools = tools
});

Repo samples you can run now

• src/samples/MagenticBrainAgent
  • OmniAgent-style round loop, tool calls, and submit stop-signal.
• src/samples/MagenticUIServer
  • Local multi-agent web sample with SignalR + React UI.
• docs/magentic-ui-dotnet.md
  • Architecture and setup details for MagenticUI .NET flow.

No extra sample project is required for this post: the existing MagenticBrain and MagenticUI samples already cover the runnable story.

Sample app screenshots (MagenticUIServer)

The following screenshots illustrate the sample app flow using the Magentic UI client and agent stream model:

You can run the sample from:

• src/samples/MagenticUIServer/MagenticUIServer
• src/samples/MagenticUIServer/MagenticUIServer/ClientApp

Relevant links

https://github.com/elbruno/ElBruno.MagenticUI

NuGet: https://www.nuget.org/packages/ElBruno.LocalLLMs

Repository: https://github.com/elbruno/ElBruno.LocalLLMs

Supported models reference:

https://github.com/elbruno/ElBruno.LocalLLMs/blob/main/docs/supported-models.md

Auto-download guide:

https://github.com/elbruno/ElBruno.LocalLLMs/blob/main/docs/auto-download.md

Official Microsoft announcement:

https://www.microsoft.com/en-us/research/blog/magenticlite-magenticbrain-fara1-5-an-agentic-experience-optimized-for-small-models/

Original MagenticBrain model:

https://huggingface.co/microsoft/MagenticBrain

Published ONNX package used by this repo:

https://huggingface.co/elbruno/MagenticBrain-onnx

This blog post was created with the help of AI tools. Yes, I used a bit of magic from language models to organize my thoughts and automate the boring parts, but the geeky fun and the in C# are 100% mine.

TL;DR

• Fara1.5-9B now works end-to-end in C# with ElBruno.LocalLLMs.
• The validated multimodal ONNX package is published at:
  • https://huggingface.co/elbruno/Fara1.5-9B-onnx
• You can run image+text inference locally with LocalVisionChatClient and KnownModels.Fara15_9B.
• This closes the loop for end-to-end local agent experiences in:
  • https://github.com/elbruno/ElBruno.MagenticUI

When Microsoft Research introduced MagenticLite, MagenticBrain, and Fara1.5, it made a strong case for practical local agentic experiences:

• Original announcement: https://www.microsoft.com/en-us/research/blog/magenticlite-magenticbrain-fara1-5-an-agentic-experience-optimized-for-small-models/

In plain words: smaller, specialized models can deliver real agentic behavior when the tooling is right.

For this repo, that “tooling is right” moment means: Fara is now actually runnable end-to-end in .NET, not just listed as an idea.

Why this matters for C# end-to-end

The goal of ElBruno.LocalLLMs is to make local model usage feel native in .NET. With Fara support in place:

1. You can auto-download the model from Hugging Face.
2. Run vision prompts with LocalVisionChatClient.
3. Keep everything in a C# workflow without external service dependencies.

That is especially important for app-level experiences like:

• https://github.com/elbruno/ElBruno.MagenticUI

where “agent loop + UI + model inference” should stay coherent in one stack.

Basic usage: Fara in C#

1. Install

dotnet add package ElBruno.LocalLLMs --version 0.20.3

2. Minimal Fara vision client

using ElBruno.LocalLLMs;
using Microsoft.Extensions.AI;
var options = new LocalLLMsOptions
{
    Model = KnownModels.Fara15_9B,
    EnsureModelDownloaded = true,
    ExecutionProvider = ExecutionProvider.Cpu
};
await using var client = new LocalVisionChatClient(options);

3. Ask Fara about an image

var response = await client.GetResponseAsync(
[
    new ChatMessage(ChatRole.User, "Describe the image in one sentence.")
],
new VisionChatOptions
{
    ImagePaths = [@"C:\images\screen.png"],
    MaxOutputTokens = 64
});
Console.WriteLine(response.Text);

Reusing existing Fara sample code

Fara-specific sample code is already in the repo:

• src/samples/FaraVisionAgent

This sample already covers:

• auto-download from elbruno/Fara1.5-9B-onnx
• local --model-path override
• --image-path flow with VisionChatOptions

If you want to start from runnable code instead of snippets, use that sample first.

Validation snapshot

Focused validation for Fara support passed:

• Fara unit tests: 39/39
• Fara Hugging Face checks: 3/3
• Fara vision lifecycle E2E: pass
  • download -> inference -> cache hit -> delete

Detailed report:

• docs/tests/2026-07-28-15-run-results.md
• Issue tracking (closed): #35

Relevant links

• NuGet: https://www.nuget.org/packages/ElBruno.LocalLLMs
• Repo: https://github.com/elbruno/ElBruno.LocalLLMs
• Fara ONNX package: https://huggingface.co/elbruno/Fara1.5-9B-onnx
• Microsoft announcement:
  • https://www.microsoft.com/en-us/research/blog/magenticlite-magenticbrain-fara1-5-an-agentic-experience-optimized-for-small-models/
• MagenticUI .NET app:
  • https://github.com/elbruno/ElBruno.MagenticUI

 This blog post was created with the help of AI tools. The geeky fun and the in C# are 100% mine.

TL;DR

Someone asked me to add Inkling — Thinking Machines’ shiny new open-weights model — to ElBruno.LocalLLMs. I did the evaluation, and the answer is: not for now. 

Inkling is a 975B-parameter, Mixture-of-Experts, multimodal (text + image + audio) model. My library runs local LLMs in .NET through ONNX Runtime GenAI, and Inkling hits every wall at once:

• MoE routing isn’t supported by the ONNX GenAI builder.
• Multimodal in/out has no path through the text-only conversion pipeline.
• Size: ~490 GB+ of weights even at INT4 — this is a data-center model, not a laptop model.
• NVFP4 numerics + custom architecture = no conversion path exists.

So I documented it honestly as Not Viable in blocked-models.md, pointed folks at hosted APIs for Inkling itself, and recommended small local alternatives (Phi-4, Qwen2.5-32B, DeepSeek-R1-Distill-Qwen-14B). If you want it on your own hardware today — you’d need a rack, not a ThinkPad.

The Longer Story (thanks Copilot!)

Hi!

Every now and then someone opens an issue or drops me a message: “Hey Bruno, can you add <shiny new model> to your library?” I love those messages. It means people are actually using ElBruno.LocalLLMs to run models locally in .NET, and they want the latest and greatest.

This time the request was Inkling, from Thinking Machines. And it is a fascinating model. But after digging in, I had to give the answer nobody likes to hear: not right now. Let me walk you through why — because the “why” is genuinely interesting, and it says a lot about the difference between a frontier cloud model and a local model.

First, what is Inkling?

Inkling is a general-purpose, natively multimodal model. It takes text, images, and audio as input and produces text output. Under the hood:

On paper? Gorgeous. It’s an impressive piece of engineering, and it benchmarks right alongside the big closed models. But almost every one of those beautiful properties is a brick wall for local inference.

What my library actually does

Quick reminder on how ElBruno.LocalLLMs works: it lets you run local LLMs in .NET through the standard IChatClient interface, powered by ONNX Runtime GenAI. To add a model, I:

1. Convert its weights to ONNX GenAI format (usually via optimum / the onnxruntime-genai builder).
2. Register a ModelDefinition in KnownModels.cs.
3. Ship it so you can dotnet add package and go.

The whole design assumes a model you can realistically download and run on your own machine — a laptop, a workstation, maybe a single GPU. That assumption is the crux of the problem.

The four walls

Wall #1 — Mixture of Experts. ONNX Runtime GenAI’s model builder doesn’t support MoE routing (or maybe I am missing something). It can’t express “pick 6 of 256 experts per token.” This is the same blocker that keeps Mixtral and DeepSeek-V3 out of the library. Inkling just has a lot more experts.

Wall #2 — Multimodal in/out. My conversion path is text-generation only (text-generation-with-past). Inkling’s vision patch encoder and discrete audio tokenizer have nowhere to go in that pipeline. Even if the text backbone converted cleanly, the image and audio front-ends wouldn’t.

Wall #3 — Size. This is the big one. 975B parameters is roughly ~2 TB in BF16. Quantize it all the way down to INT4 and you’re still looking at ~490 GB+ of weights — plus multiple terabytes of RAM just to run the conversion. For inference you’d need hundreds of GB of VRAM across multiple data-center GPUs. That’s not “local” by any definition I can ship to you.

Wall #4 — Numerics + custom architecture. Inkling ships in BF16 and NVFP4 (an NVIDIA 4-bit format, not ONNX). Combined with its custom multimodal MoE design, there’s simply no export path today.

Any one of these would be enough to block it. Inkling has all four. It’s essentially a bigger, multimodal sibling of DeepSeek-V3 (671B MoE), which I’d already marked as Not Viable for the same reasons.

So what did I actually do?

I didn’t want this to be a silent “no.” So I:

• Ran the full evaluation and wrote it up in docs/blocked-models.md with a proper blocker table.
• Added a heads-up (with a link) in docs/supported-models.md so nobody wastes an afternoon trying to convert it.
• Logged it in the changelog.

And most importantly — alternatives:

•  Want Inkling specifically? Use it via a hosted API (the Tinker cookbook or a third-party inference provider). That’s what it’s built for.
•  Want something great that runs locally in .NET today? Try Phi-4 (14B), Qwen2.5-32B, or DeepSeek-R1-Distill-Qwen-14B. All native ONNX, all ready to go with KnownModels.

The takeaway

Not every frontier model is meant to run on your machine — and that’s okay. Part of maintaining a local LLM library is being honest about where “local” ends and “data center” begins. Inkling is a spectacular model. It’s just not a local one.

If the ONNX Runtime GenAI builder adds MoE + multimodal support down the line, I’ll revisit. Until then, Inkling stays on the “not viable” list — documented, explained, and with good alternatives right next to it.

Thanks for the request, whoever you are. Keep ’em coming.

This blog post was created with the help of AI tools. I used them to organize ideas and write a first draft, but the workflow, the code, the testing, and the enthusiasm are mine.

Hi!

I like reading AI news. I do not like opening seven tabs, comparing five versions of the same announcement, and wondering whether the most important developer update was hidden under an article about a robot making coffee.

So I built a small project: Weekly AI News Digest.

It is a static site that collects relevant AI and technology stories, turns them into a readable digest, and publishes the result with GitHub Pages. The interesting part is not the HTML page. The interesting part is that the workflow is driven by GitHub Agentic Workflows.

In other words: an agent researches, curates, writes the page, and proposes the change. GitHub still keeps the dangerous part of the process behind a controlled pull request.

That balance is the whole point.

Generated illustration. It represents the workflow concept and is not a screenshot of the product.

I did not want another news scraper

A normal news scraper is easy to imagine:

1. Fetch RSS feeds.
2. Print links.
3. Add a sad README.md.
4. Promise to improve it later.

The problem is that a list of links is not a digest.

For a weekly briefing, I want context:

• What happened?
• Why should a developer care?
• Which stories are actually important?
• Which ones are related to GitHub, Copilot, developer tools, security, or AI platforms?

That last question matters to me. I wanted a GitHub-first lens. Not because GitHub is the only place where AI happens, but because it is where a lot of us build, review, ship, automate, and occasionally ask a coding agent to refactor half the repository before coffee.

The result is a small, focused site instead of an infinite stream.

The workflow is a Markdown file

The core of the project lives in .github/workflows/weekly-news-digest.md.

Yes, a Markdown file.

GitHub Agentic Workflows use a Markdown-based definition: frontmatter describes the workflow permissions, schedule, network access, and safe outputs; the body tells the agent what to do.

For this digest, the instructions are intentionally specific:

• read seven approved RSS feeds;
• look back over the most recent 14 days;
• select 15 useful stories;
• include at least seven GitHub Changelog stories when available;
• cap the number of stories from every other source;
• write a TL;DR and a “Why it matters” explanation for every selected story;
• produce a separate five-item GitHub-only highlights section;
• render a complete static page at docs/index.html.

This is an important detail about agentic workflows: the prompt is not decoration.

The prompt is the product specification.

If I only write “find interesting AI news,” I get a very creative interpretation of interesting. If I write the sources, time window, allocation rules, output shape, and quality bar, I get something I can review.

Not magic. Just better constraints.

From seven feeds to one readable page

The workflow begins with a small set of sources:

• GitHub Changelog
• TechCrunch AI
• MIT Technology Review
• Hacker News
• Ars Technica
• The Verge
• VentureBeat AI

The agent extracts the title, URL, source, publication date, and plain-text excerpt for each recent entry. Then it curates the list.

Generated illustration. It represents the source-to-digest concept and is not a screenshot of the product.

The curation rules deliberately make GitHub the default lens. At least seven of the 15 selected stories come from GitHub Changelog when that many are available. Other sources cannot dominate the page.

That does not mean “ignore the rest of the industry.” It means that, when I open this page, I should quickly see the developer-platform changes that may affect my daily work.

The generated page also includes a five-bullet TL;DR — GitHub highlights block at the top. Those bullets are only allowed to use GitHub sources. This is a small rule, but it prevents a common AI-summary problem: mixing unrelated sources into a summary that sounds coherent but loses provenance.

The UI has opinions too

The generated site is a self-contained HTML page. No framework, no CDN, no build pipeline that needs a different build pipeline to explain the first build pipeline.

It uses a GitHub-inspired dark theme, with system, light, and dark modes. It also has:

• source filter chips;
• tag filters;
• full-text search across titles, sources, summaries, impact notes, and tags;
• a live “Showing X of 15 stories” counter;
• responsive cards for reading on a desktop or a phone.

The source filter starts with GitHub-related sources selected.

Again, that is intentional. A visitor can expand the view, but the default answer to “what changed this week?” starts close to the tools many developers already use.

Product screenshot captured locally on July 10, 2026 at http://127.0.0.1:4173/. It shows the generated digest with the default GitHub-first filter.

Small UX decisions like this are useful because they turn a generated page into something that feels curated. The agent can produce information. The product still needs an opinion about how people should consume it.

The agent does not get to write directly to main

This is the part I care about most.

The workflow does not hand an agent unrestricted repository write access and hope everybody has a quiet afternoon.

The agent runs with read-only repository permissions in a sandboxed environment. It can research the approved network locations and prepare output, but the actual repository write is handled by a separate safe-outputs pull-request step.

The path looks like this:

Agent (read-only, sandboxed)
  -> agent output artifact
  -> threat detection
  -> safe-outputs pull request
  -> review / auto-merge rule
  -> GitHub Pages deployment

The workflow creates a pull request with the digest and automated labels. A separate automation can merge that trusted digest PR. GitHub Pages then serves the docs/ folder from main.

This is a much better model than “an AI wrote directly to production.”

Generated illustration. It represents the safety and delivery model and is not a screenshot of the product.

The agent is useful where it has an advantage: reading, comparing, summarizing, and drafting. The platform keeps the repository mutation in a narrow, auditable path.

That is how I want agentic automation to work: capable by default, but not casually omnipotent.

Why use an agent instead of a normal script?

Traditional scripts are still great.

If the task is “parse this XML,” “rename these files,” or “run these tests,” I want deterministic code. A script is faster, cheaper, easier to test, and far less likely to develop a sudden interest in the history of computer science.

But curation is different.

Choosing the 15 stories that matter from a large group of feeds is not only filtering by date or keyword. It involves relevance, duplication, developer impact, and context. Writing a concise explanation of why a release matters also benefits from language and reasoning.

That is a reasonable place to use an agent.

The pattern I am using is simple:

• use code for deterministic work;
• use an agent for judgment and language;
• put the agent behind clear boundaries;
• make the output easy to inspect;
• keep the final delivery path controlled by the platform.

You do not need an agent for every cron job. You need one when the workflow has a real reasoning step and you can state the expected output clearly.

Things I learned while building it

Constraints improve the results

The most useful instructions were not the poetic ones. They were the measurable ones:

• 14-day lookback;
• 15 stories;
• at least seven GitHub items;
• at most four items from any other source;
• exactly five GitHub-only top highlights;
• specific fields for each card.

An agent with a vague task will fill the ambiguity. An agent with a clear task will spend more time doing useful work.

Generated HTML can still be a product

The page is generated, but it is not a random document dump. The template has visual rules, responsive behavior, accessibility-friendly labels, theme persistence, and filters.

Static HTML is still one of the most reliable deployment targets on the internet.

A pull request is a great integration boundary

The pull request gives me a diff, history, labels, checks, and a place to stop the automation if something looks wrong.

It is also familiar. You do not need a new dashboard to understand what changed. You can use the collaboration system that is already part of your development workflow.

What I want to add next

The digest is intentionally small, but there are many directions to explore:

• better duplicate detection across syndications;
• recurring-topic summaries;
• a history page for comparing weeks;
• optional user-selected source profiles;
• richer visual explanations of the workflow;
• screenshots that document the actual filtering and theme experience.

For now, the goal is simpler: open one page, catch up on useful news, and understand why it matters.

The project is open source on GitHub: elbruno/weekly-ai-news-digest.

Source screenshot captured from GitHub Changelog on July 10, 2026. It documents an input source, not a feature of this project.

If you build your own agentic workflow, my recommendation is not “give the agent all the permissions.”

Start with a narrow problem. Describe the output. Keep the boundary visible. Let the agent do the part where it is good.

And let your pull request do the part where you are good.

This blog post was created with the help of AI tools. Yes, I used a bit of magic from language models to organize my thoughts and automate the boring parts, but the geeky fun and the in C# are 100% mine.

Hi!

If you have been playing with Foundry Local to run AI models on your own machine, you probably ran into the same thing I did: you never really know what is running.

You load a model from the CLI. Your C# app loads another one through the SDK. A Python script spins up its own. A .NET Aspire sample launches a proxy. And five minutes later you have several models loaded across several ports, eating memory, and you have no idea which one belongs to what.

Foundry Local does not run on a single fixed port, and the foundry CLI only sees what the CLI itself started. Everything an app loads directly through the SDK is, basically, invisible.

So I built a small tool to fix that: Foundry Local Monitor.

It is a tiny Windows tray app that watches your machine and tells you, in real time, which Foundry Local instances are running, which models are loaded, and on which device (CPU, GPU, CUDA, DirectML…). When a model loads or unloads — from any app — you get a notification.

What it does

• Lives in your system tray, out of the way.
• Discovers all Foundry Local instances on your machine, no matter which language or SDK started them.
• Shows the models loaded by each instance, grouped by process.
• Tells you the device backend for each model (CPU / GPU / CUDA / TensorRT / DirectML / WinML) with a colored badge.
• Fires a toast notification when a model is loaded or unloaded.
• Has a handy button to open the process location on disk.

Install it

It is published as a .NET global tool, so one line and you are done:

dotnet tool install -g ElBruno.FoundryLocalMonitor

Then run it:

foundrylocalmon

That’s it. Look for the icon in your system tray. Right-click for options, double-click to open the main window with the Status, Loaded Models, and Available Models tabs.

To update later:

dotnet tool update -g ElBruno.FoundryLocalMonitor

A quick example: the scenario it detects

Here is the kind of code the monitor is designed to see. This is a plain .NET console app using the Microsoft.AI.Foundry.Local SDK — no CLI involved. The SDK spins up its own OpenAI-compatible REST server, and that is exactly what the monitor finds and shows in real time.

using Microsoft.AI.Foundry.Local;
using Microsoft.Extensions.Logging.Abstractions;
// The SDK hosts an internal OpenAI-compatible server on this address.
// The C# SDK default is 55588 (Python uses 55589).
var config = new Configuration
{
    AppName = "MyFoundryApp",
    Web     = new Configuration.WebService { Urls = "http://127.0.0.1:55588" }
};
await FoundryLocalManager.CreateAsync(config, NullLogger.Instance);
var manager = FoundryLocalManager.Instance;
// Start the web service — this is the endpoint the monitor discovers.
await manager.StartWebServiceAsync();
var endpoint = manager.Urls?.FirstOrDefault();
Console.WriteLine($"Foundry Local is serving at {endpoint}");