Skip to content

Use Microsoft.Extensions.DataIngestion in AI Chat Web template #7023

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
MackinnonBuck merged 35 commits into from
Nov 10, 2025
Merged

Use Microsoft.Extensions.DataIngestion in AI Chat Web template #7023

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
f8d2fe5
Add Markdown support
MackinnonBuck Nov 5, 2025
3631508
Remove PDF support
MackinnonBuck Nov 5, 2025
b311c28
Revert "Remove PDF support"
MackinnonBuck Nov 5, 2025
fd6b2fc
Add 'Example_GPS_Watch.md'
MackinnonBuck Nov 5, 2025
6da6a21
Add MEDI dependencies
MackinnonBuck Nov 5, 2025
3a41be8
Revert "[MEDI] Remove collection key type workaround (#7010)"
MackinnonBuck Nov 6, 2025
3c3a27c
MEDI integration into chat template
MackinnonBuck Nov 6, 2025
8c41874
Remove PdfPig dependency
MackinnonBuck Nov 6, 2025
da477f5
Fix citation + normalize identifier path
MackinnonBuck Nov 6, 2025
eb7b2cd
Undo changes to `M.E.DI.csproj`
MackinnonBuck Nov 6, 2025
7f758af
Update snapshots
MackinnonBuck Nov 7, 2025
3955026
Merge branch 'main' into mbuck/chat-template-data-ingestion
jeffhandley Nov 7, 2025
28ec490
Update DataIngestion unit tests to handle keys as either strings or g…
jeffhandley Nov 7, 2025
8108e80
Update SK and fix MEDI version
MackinnonBuck Nov 7, 2025
5296eea
Remove SK workaround
MackinnonBuck Nov 7, 2025
0f56e53
Fix sandbox paths to allow running tests multiple times
MackinnonBuck Nov 7, 2025
1a65d68
Reliable data ingestion
MackinnonBuck Nov 7, 2025
2bffe17
Enable MEDI tracing
MackinnonBuck Nov 7, 2025
012d08e
Simplify log message
MackinnonBuck Nov 7, 2025
baa407d
Add `PdfPigReader` for non-Aspire template
MackinnonBuck Nov 7, 2025
9f9921c
Invert PdfPigReader exclusion condition
MackinnonBuck Nov 7, 2025
9c9c596
Merge remote-tracking branch 'origin/main' into mbuck/chat-template-d…
MackinnonBuck Nov 7, 2025
aa9d026
Use Markitdown MCP
MackinnonBuck Nov 7, 2025
a799f0b
Update snapshots
MackinnonBuck Nov 7, 2025
960bbc5
Undo changes to `IngestionPipelineTests.cs`
MackinnonBuck Nov 7, 2025
537474d
Update src/ProjectTemplates/Microsoft.Extensions.AI.Templates/src/Cha…
MackinnonBuck Nov 7, 2025
45c07c9
Update snapshots
MackinnonBuck Nov 7, 2025
b51f32b
Improve template execution test failure output
jeffhandley Nov 9, 2025
537b616
Support .NET 10 in aichatweb, using it by default
jeffhandley Nov 8, 2025
491e808
Merge branch 'main' into mbuck/chat-template-data-ingestion
jeffhandley Nov 10, 2025
84cfea6
Show a message when loading documents by loading docs as a separate tool
jeffhandley Nov 10, 2025
0467a99
disable the incremental ingestion
adamsitnik Nov 10, 2025
cfc845f
map every PDF page to a single section
adamsitnik Nov 10, 2025
855cfd2
drop SK dependency
adamsitnik Nov 10, 2025
17ba491
Add system prompt instructions for calling the LoadDocuments tool. Fi…
jeffhandley Nov 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions ...tes/src/ChatWithCustomData/ChatWithCustomData-CSharp.Web/Components/Pages/Chat/Chat.razor
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,10 @@
{
statefulMessageCount = 0;
messages.Add(new(ChatRole.System, SystemPrompt));
chatOptions.Tools = [AIFunctionFactory.Create(SearchAsync)];
chatOptions.Tools = [
AIFunctionFactory.Create(LoadDocumentsAsync),
AIFunctionFactory.Create(SearchAsync)
];
}

private async Task AddUserMessageAsync(ChatMessage userMessage)
Expand Down Expand Up @@ -106,7 +109,14 @@
await chatInput!.FocusAsync();
}

[Description("Searches for information using a phrase or keyword")]
[Description("Loads the documents needed for performing searches. Must be completed before a search can be executed, but only needs to be completed once.")]
private async Task LoadDocumentsAsync()
{
await InvokeAsync(StateHasChanged);
await Search.LoadDocumentsAsync();
}

[Description("Searches for information using a phrase or keyword. Relies on documents already being loaded.")]
private async Task<IEnumerable<string>> SearchAsync(
[Description("The phrase to search for.")] string searchPhrase,
[Description("If possible, specify the filename to search that file only. If not provided or empty, the search includes all files.")] string? filenameFilter = null)
Expand Down
13 changes: 13 additions & 0 deletions ...tWithCustomData/ChatWithCustomData-CSharp.Web/Components/Pages/Chat/ChatMessageItem.razor
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,19 @@ else if (Message.Role == ChatRole.Assistant)
</div>
</div>
}
else if (content is FunctionCallContent { Name: "LoadDocuments"})
{
<div class="assistant-search">
<div class="assistant-search-icon">
<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor">
<path stroke-linecap="round" stroke-linejoin="round" d="M19.5 14.25v-2.625a3.375 3.375 0 0 0-3.375-3.375h-1.5A1.125 1.125 0 0 1 13.5 7.125v-1.5a3.375 3.375 0 0 0-3.375-3.375H8.25m0 12.75h7.5m-7.5 3H12M10.5 2.25H5.625c-.621 0-1.125.504-1.125 1.125v17.25c0 .621.504 1.125 1.125 1.125h12.75c.621 0 1.125-.504 1.125-1.125V11.25a9 9 0 0 0-9-9Z" />
</svg>
</div>
<div class="assistant-search-content">
Loading relevant documents (this will take a minute)...
</div>
</div>
}
else if (content is FunctionCallContent { Name: "Search" } fcc && fcc.Arguments?.TryGetValue("searchPhrase", out var searchPhrase) is true)
{
<div class="assistant-search">
Expand Down
6 changes: 4 additions & 2 deletions ...Templates/src/ChatWithCustomData/ChatWithCustomData-CSharp.Web/Services/SemanticSearch.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,12 @@ public class SemanticSearch(
{
private Task? _ingestionTask;

public async Task LoadDocumentsAsync() => await ( _ingestionTask ??= dataIngestor.IngestDataAsync(ingestionDirectory, searchPattern: "*.*"));

public async Task<IReadOnlyList<IngestedChunk>> SearchAsync(string text, string? documentIdFilter, int maxResults)
{
_ingestionTask ??= dataIngestor.IngestDataAsync(ingestionDirectory, searchPattern: "*.*");
await _ingestionTask;
// Ensure documents have been loaded before searching
await LoadDocumentsAsync();

var nearest = vectorCollection.SearchAsync(text, maxResults, new VectorSearchOptions<IngestedChunk>
{
Expand Down
14 changes: 12 additions & 2 deletions ...ureOpenAI_Qdrant_Aspire.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/Chat.razor
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,10 @@
{
statefulMessageCount = 0;
messages.Add(new(ChatRole.System, SystemPrompt));
chatOptions.Tools = [AIFunctionFactory.Create(SearchAsync)];
chatOptions.Tools = [
AIFunctionFactory.Create(LoadDocumentsAsync),
AIFunctionFactory.Create(SearchAsync)
];
}

private async Task AddUserMessageAsync(ChatMessage userMessage)
Expand Down Expand Up @@ -106,7 +109,14 @@
await chatInput!.FocusAsync();
}

[Description("Searches for information using a phrase or keyword")]
[Description("Loads the documents needed for performing searches. Must be completed before a search can be executed, but only needs to be completed once.")]
private async Task LoadDocumentsAsync()
{
await InvokeAsync(StateHasChanged);
await Search.LoadDocumentsAsync();
}

[Description("Searches for information using a phrase or keyword. Relies on documents already being loaded.")]
private async Task<IEnumerable<string>> SearchAsync(
[Description("The phrase to search for.")] string searchPhrase,
[Description("If possible, specify the filename to search that file only. If not provided or empty, the search includes all files.")] string? filenameFilter = null)
Expand Down
13 changes: 13 additions & 0 deletions ...drant_Aspire.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/ChatMessageItem.razor
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,19 @@ else if (Message.Role == ChatRole.Assistant)
</div>
</div>
}
else if (content is FunctionCallContent { Name: "LoadDocuments"})
{
<div class="assistant-search">
<div class="assistant-search-icon">
<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor">
<path stroke-linecap="round" stroke-linejoin="round" d="M19.5 14.25v-2.625a3.375 3.375 0 0 0-3.375-3.375h-1.5A1.125 1.125 0 0 1 13.5 7.125v-1.5a3.375 3.375 0 0 0-3.375-3.375H8.25m0 12.75h7.5m-7.5 3H12M10.5 2.25H5.625c-.621 0-1.125.504-1.125 1.125v17.25c0 .621.504 1.125 1.125 1.125h12.75c.621 0 1.125-.504 1.125-1.125V11.25a9 9 0 0 0-9-9Z" />
</svg>
</div>
<div class="assistant-search-content">
Loading relevant documents (this will take a minute)...
</div>
</div>
}
else if (content is FunctionCallContent { Name: "Search" } fcc && fcc.Arguments?.TryGetValue("searchPhrase", out var searchPhrase) is true)
{
<div class="assistant-search">
Expand Down
6 changes: 4 additions & 2 deletions ...web.AzureOpenAI_Qdrant_Aspire.verified/aichatweb/aichatweb.Web/Services/SemanticSearch.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,12 @@ public class SemanticSearch(
{
private Task? _ingestionTask;

public async Task LoadDocumentsAsync() => await ( _ingestionTask ??= dataIngestor.IngestDataAsync(ingestionDirectory, searchPattern: "*.*"));

public async Task<IReadOnlyList<IngestedChunk>> SearchAsync(string text, string? documentIdFilter, int maxResults)
{
_ingestionTask ??= dataIngestor.IngestDataAsync(ingestionDirectory, searchPattern: "*.*");
await _ingestionTask;
// Ensure documents have been loaded before searching
await LoadDocumentsAsync();

var nearest = vectorCollection.SearchAsync(text, maxResults, new VectorSearchOptions<IngestedChunk>
{
Expand Down
14 changes: 12 additions & 2 deletions ...rationTests/Snapshots/aichatweb.Basic.verified/aichatweb/Components/Pages/Chat/Chat.razor
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,10 @@
{
statefulMessageCount = 0;
messages.Add(new(ChatRole.System, SystemPrompt));
chatOptions.Tools = [AIFunctionFactory.Create(SearchAsync)];
chatOptions.Tools = [
AIFunctionFactory.Create(LoadDocumentsAsync),
AIFunctionFactory.Create(SearchAsync)
];
}

private async Task AddUserMessageAsync(ChatMessage userMessage)
Expand Down Expand Up @@ -106,7 +109,14 @@
await chatInput!.FocusAsync();
}

[Description("Searches for information using a phrase or keyword")]
[Description("Loads the documents needed for performing searches. Must be completed before a search can be executed, but only needs to be completed once.")]
private async Task LoadDocumentsAsync()
{
await InvokeAsync(StateHasChanged);
await Search.LoadDocumentsAsync();
}

[Description("Searches for information using a phrase or keyword. Relies on documents already being loaded.")]
private async Task<IEnumerable<string>> SearchAsync(
[Description("The phrase to search for.")] string searchPhrase,
[Description("If possible, specify the filename to search that file only. If not provided or empty, the search includes all files.")] string? filenameFilter = null)
Expand Down
13 changes: 13 additions & 0 deletions .../Snapshots/aichatweb.Basic.verified/aichatweb/Components/Pages/Chat/ChatMessageItem.razor
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,19 @@ else if (Message.Role == ChatRole.Assistant)
</div>
</div>
}
else if (content is FunctionCallContent { Name: "LoadDocuments"})
{
<div class="assistant-search">
<div class="assistant-search-icon">
<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor">
<path stroke-linecap="round" stroke-linejoin="round" d="M19.5 14.25v-2.625a3.375 3.375 0 0 0-3.375-3.375h-1.5A1.125 1.125 0 0 1 13.5 7.125v-1.5a3.375 3.375 0 0 0-3.375-3.375H8.25m0 12.75h7.5m-7.5 3H12M10.5 2.25H5.625c-.621 0-1.125.504-1.125 1.125v17.25c0 .621.504 1.125 1.125 1.125h12.75c.621 0 1.125-.504 1.125-1.125V11.25a9 9 0 0 0-9-9Z" />
</svg>
</div>
<div class="assistant-search-content">
Loading relevant documents (this will take a minute)...
</div>
</div>
}
else if (content is FunctionCallContent { Name: "Search" } fcc && fcc.Arguments?.TryGetValue("searchPhrase", out var searchPhrase) is true)
{
<div class="assistant-search">
Expand Down
6 changes: 4 additions & 2 deletions ....IntegrationTests/Snapshots/aichatweb.Basic.verified/aichatweb/Services/SemanticSearch.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,12 @@ public class SemanticSearch(
{
private Task? _ingestionTask;

public async Task LoadDocumentsAsync() => await ( _ingestionTask ??= dataIngestor.IngestDataAsync(ingestionDirectory, searchPattern: "*.*"));

public async Task<IReadOnlyList<IngestedChunk>> SearchAsync(string text, string? documentIdFilter, int maxResults)
{
_ingestionTask ??= dataIngestor.IngestDataAsync(ingestionDirectory, searchPattern: "*.*");
await _ingestionTask;
// Ensure documents have been loaded before searching
await LoadDocumentsAsync();

var nearest = vectorCollection.SearchAsync(text, maxResults, new VectorSearchOptions<IngestedChunk>
{
Expand Down
14 changes: 12 additions & 2 deletions ...s/aichatweb.BasicAspire.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/Chat.razor
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,10 @@
{
statefulMessageCount = 0;
messages.Add(new(ChatRole.System, SystemPrompt));
chatOptions.Tools = [AIFunctionFactory.Create(SearchAsync)];
chatOptions.Tools = [
AIFunctionFactory.Create(LoadDocumentsAsync),
AIFunctionFactory.Create(SearchAsync)
];
}

private async Task AddUserMessageAsync(ChatMessage userMessage)
Expand Down Expand Up @@ -106,7 +109,14 @@
await chatInput!.FocusAsync();
}

[Description("Searches for information using a phrase or keyword")]
[Description("Loads the documents needed for performing searches. Must be completed before a search can be executed, but only needs to be completed once.")]
private async Task LoadDocumentsAsync()
{
await InvokeAsync(StateHasChanged);
await Search.LoadDocumentsAsync();
}

[Description("Searches for information using a phrase or keyword. Relies on documents already being loaded.")]
private async Task<IEnumerable<string>> SearchAsync(
[Description("The phrase to search for.")] string searchPhrase,
[Description("If possible, specify the filename to search that file only. If not provided or empty, the search includes all files.")] string? filenameFilter = null)
Expand Down
13 changes: 13 additions & 0 deletions ....BasicAspire.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/ChatMessageItem.razor
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,19 @@ else if (Message.Role == ChatRole.Assistant)
</div>
</div>
}
else if (content is FunctionCallContent { Name: "LoadDocuments"})
{
<div class="assistant-search">
<div class="assistant-search-icon">
<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor">
<path stroke-linecap="round" stroke-linejoin="round" d="M19.5 14.25v-2.625a3.375 3.375 0 0 0-3.375-3.375h-1.5A1.125 1.125 0 0 1 13.5 7.125v-1.5a3.375 3.375 0 0 0-3.375-3.375H8.25m0 12.75h7.5m-7.5 3H12M10.5 2.25H5.625c-.621 0-1.125.504-1.125 1.125v17.25c0 .621.504 1.125 1.125 1.125h12.75c.621 0 1.125-.504 1.125-1.125V11.25a9 9 0 0 0-9-9Z" />
</svg>
</div>
<div class="assistant-search-content">
Loading relevant documents (this will take a minute)...
</div>
</div>
}
else if (content is FunctionCallContent { Name: "Search" } fcc && fcc.Arguments?.TryGetValue("searchPhrase", out var searchPhrase) is true)
{
<div class="assistant-search">
Expand Down
6 changes: 4 additions & 2 deletions ...apshots/aichatweb.BasicAspire.verified/aichatweb/aichatweb.Web/Services/SemanticSearch.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,12 @@ public class SemanticSearch(
{
private Task? _ingestionTask;

public async Task LoadDocumentsAsync() => await ( _ingestionTask ??= dataIngestor.IngestDataAsync(ingestionDirectory, searchPattern: "*.*"));

public async Task<IReadOnlyList<IngestedChunk>> SearchAsync(string text, string? documentIdFilter, int maxResults)
{
_ingestionTask ??= dataIngestor.IngestDataAsync(ingestionDirectory, searchPattern: "*.*");
await _ingestionTask;
// Ensure documents have been loaded before searching
await LoadDocumentsAsync();

var nearest = vectorCollection.SearchAsync(text, maxResults, new VectorSearchOptions<IngestedChunk>
{
Expand Down
14 changes: 12 additions & 2 deletions ...aichatweb.Ollama_Qdrant.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/Chat.razor
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,10 @@
{
statefulMessageCount = 0;
messages.Add(new(ChatRole.System, SystemPrompt));
chatOptions.Tools = [AIFunctionFactory.Create(SearchAsync)];
chatOptions.Tools = [
AIFunctionFactory.Create(LoadDocumentsAsync),
AIFunctionFactory.Create(SearchAsync)
];
}

private async Task AddUserMessageAsync(ChatMessage userMessage)
Expand Down Expand Up @@ -106,7 +109,14 @@
await chatInput!.FocusAsync();
}

[Description("Searches for information using a phrase or keyword")]
[Description("Loads the documents needed for performing searches. Must be completed before a search can be executed, but only needs to be completed once.")]
private async Task LoadDocumentsAsync()
{
await InvokeAsync(StateHasChanged);
await Search.LoadDocumentsAsync();
}

[Description("Searches for information using a phrase or keyword. Relies on documents already being loaded.")]
private async Task<IEnumerable<string>> SearchAsync(
[Description("The phrase to search for.")] string searchPhrase,
[Description("If possible, specify the filename to search that file only. If not provided or empty, the search includes all files.")] string? filenameFilter = null)
Expand Down
13 changes: 13 additions & 0 deletions ...llama_Qdrant.verified/aichatweb/aichatweb.Web/Components/Pages/Chat/ChatMessageItem.razor
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,19 @@ else if (Message.Role == ChatRole.Assistant)
</div>
</div>
}
else if (content is FunctionCallContent { Name: "LoadDocuments"})
{
<div class="assistant-search">
<div class="assistant-search-icon">
<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor">
<path stroke-linecap="round" stroke-linejoin="round" d="M19.5 14.25v-2.625a3.375 3.375 0 0 0-3.375-3.375h-1.5A1.125 1.125 0 0 1 13.5 7.125v-1.5a3.375 3.375 0 0 0-3.375-3.375H8.25m0 12.75h7.5m-7.5 3H12M10.5 2.25H5.625c-.621 0-1.125.504-1.125 1.125v17.25c0 .621.504 1.125 1.125 1.125h12.75c.621 0 1.125-.504 1.125-1.125V11.25a9 9 0 0 0-9-9Z" />
</svg>
</div>
<div class="assistant-search-content">
Loading relevant documents (this will take a minute)...
</div>
</div>
}
else if (content is FunctionCallContent { Name: "Search" } fcc && fcc.Arguments?.TryGetValue("searchPhrase", out var searchPhrase) is true)
{
<div class="assistant-search">
Expand Down
6 changes: 4 additions & 2 deletions ...shots/aichatweb.Ollama_Qdrant.verified/aichatweb/aichatweb.Web/Services/SemanticSearch.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,12 @@ public class SemanticSearch(
{
private Task? _ingestionTask;

public async Task LoadDocumentsAsync() => await ( _ingestionTask ??= dataIngestor.IngestDataAsync(ingestionDirectory, searchPattern: "*.*"));

public async Task<IReadOnlyList<IngestedChunk>> SearchAsync(string text, string? documentIdFilter, int maxResults)
{
_ingestionTask ??= dataIngestor.IngestDataAsync(ingestionDirectory, searchPattern: "*.*");
await _ingestionTask;
// Ensure documents have been loaded before searching
await LoadDocumentsAsync();

var nearest = vectorCollection.SearchAsync(text, maxResults, new VectorSearchOptions<IngestedChunk>
{
Expand Down
14 changes: 12 additions & 2 deletions ...pshots/aichatweb.OpenAI_AzureAISearch.verified/aichatweb/Components/Pages/Chat/Chat.razor
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,10 @@
{
statefulMessageCount = 0;
messages.Add(new(ChatRole.System, SystemPrompt));
chatOptions.Tools = [AIFunctionFactory.Create(SearchAsync)];
chatOptions.Tools = [
AIFunctionFactory.Create(LoadDocumentsAsync),
AIFunctionFactory.Create(SearchAsync)
];
}

private async Task AddUserMessageAsync(ChatMessage userMessage)
Expand Down Expand Up @@ -106,7 +109,14 @@
await chatInput!.FocusAsync();
}

[Description("Searches for information using a phrase or keyword")]
[Description("Loads the documents needed for performing searches. Must be completed before a search can be executed, but only needs to be completed once.")]
private async Task LoadDocumentsAsync()
{
await InvokeAsync(StateHasChanged);
await Search.LoadDocumentsAsync();
}

[Description("Searches for information using a phrase or keyword. Relies on documents already being loaded.")]
private async Task<IEnumerable<string>> SearchAsync(
[Description("The phrase to search for.")] string searchPhrase,
[Description("If possible, specify the filename to search that file only. If not provided or empty, the search includes all files.")] string? filenameFilter = null)
Expand Down
13 changes: 13 additions & 0 deletions ...atweb.OpenAI_AzureAISearch.verified/aichatweb/Components/Pages/Chat/ChatMessageItem.razor
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,19 @@ else if (Message.Role == ChatRole.Assistant)
</div>
</div>
}
else if (content is FunctionCallContent { Name: "LoadDocuments"})
{
<div class="assistant-search">
<div class="assistant-search-icon">
<svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor">
<path stroke-linecap="round" stroke-linejoin="round" d="M19.5 14.25v-2.625a3.375 3.375 0 0 0-3.375-3.375h-1.5A1.125 1.125 0 0 1 13.5 7.125v-1.5a3.375 3.375 0 0 0-3.375-3.375H8.25m0 12.75h7.5m-7.5 3H12M10.5 2.25H5.625c-.621 0-1.125.504-1.125 1.125v17.25c0 .621.504 1.125 1.125 1.125h12.75c.621 0 1.125-.504 1.125-1.125V11.25a9 9 0 0 0-9-9Z" />
</svg>
</div>
<div class="assistant-search-content">
Loading relevant documents (this will take a minute)...
</div>
</div>
}
else if (content is FunctionCallContent { Name: "Search" } fcc && fcc.Arguments?.TryGetValue("searchPhrase", out var searchPhrase) is true)
{
<div class="assistant-search">
Expand Down
6 changes: 4 additions & 2 deletions ...ts/Snapshots/aichatweb.OpenAI_AzureAISearch.verified/aichatweb/Services/SemanticSearch.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,12 @@ public class SemanticSearch(
{
private Task? _ingestionTask;

public async Task LoadDocumentsAsync() => await ( _ingestionTask ??= dataIngestor.IngestDataAsync(ingestionDirectory, searchPattern: "*.*"));

public async Task<IReadOnlyList<IngestedChunk>> SearchAsync(string text, string? documentIdFilter, int maxResults)
{
_ingestionTask ??= dataIngestor.IngestDataAsync(ingestionDirectory, searchPattern: "*.*");
await _ingestionTask;
// Ensure documents have been loaded before searching
await LoadDocumentsAsync();

var nearest = vectorCollection.SearchAsync(text, maxResults, new VectorSearchOptions<IngestedChunk>
{
Expand Down
Loading