Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .github/copilot-instructions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# NuGet Publishing

Packages are published via GitHub Actions triggered by git tags. Each component workflow (e.g. `mevd.yml`) owns its build, test, and pack steps; on tag pushes it calls the generic `sign-and-publish.yml` reusable workflow to sign and publish to NuGet.org.

## Tag format

- `<component>/v<version>` — release all packages in a component (e.g. `mevd/v1.0.0-preview.1`)
- `<component>/<subcomponent>/v<version>` — release a single package (e.g. `mevd/PgVector/v1.0.0-preview.1`)

The subcomponent name must match the directory name under the component's `src/` folder exactly (e.g. `PgVector`, `Qdrant`, `InMemory`).

Preview/prerelease versions use standard NuGet prerelease suffixes in the tag (e.g. `v1.0.0-preview.1`).

## Release flow

Pushing a matching tag triggers the component workflow, which runs the full test suite, packs NuGet packages, then hands them to `sign-and-publish.yml` for signing (Azure Key Vault) → GitHub Release → NuGet.org (gated by manual approval on the `nuget-release-gate` environment).

## Adding a new component

Create a `<component>.yml` workflow that:
1. Triggers on its own tag prefix (e.g. `mycomp/v*`)
2. Handles all component-specific build and test logic
3. On tag pushes, calls `package.yml` then `sign-and-publish.yml`
1 change: 1 addition & 0 deletions .github/workflows/SignClientFileList.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
**/CommunityToolkit.*
153 changes: 130 additions & 23 deletions .github/workflows/mevd.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@ name: MEVD
on:
push:
branches: [main]
tags:
- "mevd/v*" # All providers: mevd/v1.0.0-preview.1
- "mevd/*/v*" # Single provider: mevd/PgVector/v1.0.0-preview.1
paths:
- 'MEVD/**'
- 'Shared/**'
Expand All @@ -27,6 +30,52 @@ env:
DOTNET_CLI_TELEMETRY_OPTOUT: true

jobs:
discover:
name: Discover providers
runs-on: ubuntu-latest
outputs:
all_providers: ${{ steps.scan.outputs.all_providers }}
# Release-specific outputs (only meaningful on tag pushes)
is_release: ${{ steps.tag.outputs.is_release }}
version: ${{ steps.tag.outputs.version }}
pack_target: ${{ steps.tag.outputs.pack_target }}
steps:
- uses: actions/checkout@v4
with:
sparse-checkout: MEVD/src
sparse-checkout-cone-mode: false

- name: Scan provider directories
id: scan
run: |
# All providers from src (excluding Shared)
all=$(ls -d MEVD/src/*/ | xargs -I{} basename {} | grep -v '^Shared$' | jq -R . | jq -sc .)
echo "all_providers=$all" >> $GITHUB_OUTPUT

- name: Parse release tag
id: tag
if: startsWith(github.ref, 'refs/tags/')
run: |
echo "is_release=true" >> $GITHUB_OUTPUT

tag="${GITHUB_REF_NAME}"
# Strip leading "mevd/" prefix
rest="${tag#mevd/}"

if [[ "$rest" == v* ]]; then
# All providers: mevd/v1.0.0-preview.1
version="${rest#v}"
echo "pack_target=MEVD/MEVD.slnf" >> $GITHUB_OUTPUT
else
# Single provider: mevd/PgVector/v1.0.0-preview.1
provider="${rest%%/v*}"
version="${rest#*/v}"
echo "pack_target=MEVD/src/$provider/$provider.csproj" >> $GITHUB_OUTPUT
fi

echo "version=$version" >> $GITHUB_OUTPUT
echo "Parsed tag '$tag' → version='$version'"

build:
name: Build
runs-on: ubuntu-latest
Expand All @@ -38,27 +87,24 @@ jobs:
with:
global-json-file: global.json

- uses: actions/cache@v4
with:
path: ~/.nuget/packages
key: nuget-${{ hashFiles('Directory.Packages.props') }}
restore-keys: nuget-

- name: Build
run: dotnet build MEVD/MEVD.slnf

unit-tests:
name: 'Unit: ${{ matrix.provider }} (${{ matrix.os }})'
needs: build
needs: [discover, build]
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest]
provider:
- InMemory
- AzureAISearch
- CosmosMongoDB
- PgVector
- Pinecone
- Qdrant
- Redis
- SqliteVec
- Weaviate
provider: ${{ fromJson(needs.discover.outputs.all_providers) }}
steps:
- uses: actions/checkout@v4

Expand All @@ -67,29 +113,29 @@ jobs:
with:
global-json-file: global.json

- uses: actions/cache@v4
with:
path: ~/.nuget/packages
key: nuget-${{ hashFiles('Directory.Packages.props') }}
restore-keys: nuget-

- name: Test
run: dotnet test MEVD/test/${{ matrix.provider }}.UnitTests

# Conformance tests run on Ubuntu only: most providers use Testcontainers with Linux Docker
# images, which aren't available on Windows GitHub-hosted runners.
conformance-tests:
name: 'Conformance: ${{ matrix.provider }}'
needs: build
needs: [discover, build]
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
provider:
# AzureAISearch and CosmosMongoDB require cloud instances and are not run in CI.
# - AzureAISearch
# - CosmosMongoDB
- InMemory
- PgVector
- Pinecone
- Qdrant
- Redis
- SqliteVec
- Weaviate
provider: ${{ fromJson(needs.discover.outputs.all_providers) }}
# AzureAISearch and CosmosMongoDB require cloud instances and can't run in CI.
exclude:
- provider: AzureAISearch
- provider: CosmosMongoDB
steps:
- uses: actions/checkout@v4

Expand All @@ -98,5 +144,66 @@ jobs:
with:
global-json-file: global.json

- uses: actions/cache@v4
with:
path: ~/.nuget/packages
key: nuget-${{ hashFiles('Directory.Packages.props') }}
restore-keys: nuget-

- name: Test
run: dotnet test MEVD/test/${{ matrix.provider }}.ConformanceTests

# --- Release pipeline (only on tag pushes) ---

package:
name: Package
if: startsWith(github.ref, 'refs/tags/')
needs: [discover, unit-tests, conformance-tests]
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
global-json-file: global.json

- uses: actions/cache@v4
with:
path: ~/.nuget/packages
key: nuget-${{ hashFiles('Directory.Packages.props') }}
restore-keys: nuget-

- name: Pack NuGet packages
run: >
dotnet pack ${{ needs.discover.outputs.pack_target }}
-c Release
-o ./nuget
-p:PackageVersion=${{ needs.discover.outputs.version }}
-p:CI=true

- name: Upload NuGet packages as artifacts
uses: actions/upload-artifact@v4
with:
name: nuget-packages
if-no-files-found: error
path: ./nuget

- name: List NuGet packages in step summary
run: |
echo "## NuGet packages:" >> $GITHUB_STEP_SUMMARY
find nuget -type f -name '*.nupkg' -exec basename {} \; >> $GITHUB_STEP_SUMMARY

- name: Upload signing file list
uses: actions/upload-artifact@v4
with:
name: nuget-list
if-no-files-found: error
path: ${{ github.workspace }}/.github/workflows/SignClientFileList.txt

release:
name: Release
if: startsWith(github.ref, 'refs/tags/')
needs: package
uses: ./.github/workflows/release.yml
secrets: inherit
104 changes: 104 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
name: Sign and publish NuGet packages

on:
workflow_call:
secrets:
SIGN_KEY_VAULT_URL:
required: true
SIGN_CLIENT_ID:
required: true
SIGN_CLIENT_SECRET:
required: true
SIGN_TENANT_ID:
required: true
SIGN_CERTIFICATE:
required: true
NUGET_PACKAGE_PUSH_TOKEN:
required: true

jobs:
sign:
name: Sign
runs-on: windows-latest
permissions:
id-token: write
steps:
- name: Setup .NET
uses: actions/setup-dotnet@v4
with:
dotnet-version: 10.0.x

- name: Download packages
uses: actions/download-artifact@v4
with:
name: nuget-packages
path: ${{ github.workspace }}/packages

- name: Download signing file list
uses: actions/download-artifact@v4
with:
name: nuget-list
path: ${{ github.workspace }}

- name: Install signing tool
run: dotnet tool install --tool-path ./tools sign --version 0.9.1-beta.23356.1

- name: Sign packages
run: >
./tools/sign code azure-key-vault
**/*.nupkg
--base-directory "${{ github.workspace }}/packages"
--file-list "${{ github.workspace }}/SignClientFileList.txt"
--timestamp-url "http://timestamp.digicert.com"
--publisher-name ".NET Foundation"
--description "AI Community Toolkit"
--description-url "https://github.com/CommunityToolkit/AI"
--azure-key-vault-url "${{ secrets.SIGN_KEY_VAULT_URL }}"
--azure-key-vault-client-id ${{ secrets.SIGN_CLIENT_ID }}
--azure-key-vault-client-secret "${{ secrets.SIGN_CLIENT_SECRET }}"
--azure-key-vault-tenant-id ${{ secrets.SIGN_TENANT_ID }}
--azure-key-vault-certificate "${{ secrets.SIGN_CERTIFICATE }}"
--verbosity Information

- name: Upload signed packages
uses: actions/upload-artifact@v4
with:
name: signed-nuget-packages
if-no-files-found: error
path: ${{ github.workspace }}/packages/**/*.nupkg

release:
name: GitHub Release
needs: sign
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Download signed packages
uses: actions/download-artifact@v4
with:
name: signed-nuget-packages

- name: Create GitHub Release
uses: softprops/action-gh-release@v2
with:
files: ./*.nupkg

publish-nuget:
name: Publish to NuGet
needs: sign
runs-on: ubuntu-latest
environment:
name: nuget-release-gate
steps:
- name: Download signed packages
uses: actions/download-artifact@v4
with:
name: signed-nuget-packages

- name: Push to NuGet.org
run: >
dotnet nuget push ./*.nupkg
--source "https://api.nuget.org/v3/index.json"
--api-key ${{ secrets.NUGET_PACKAGE_PUSH_TOKEN }}
--skip-duplicate
11 changes: 11 additions & 0 deletions Directory.Build.props
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,19 @@
<ImplicitUsings>enable</ImplicitUsings>
<Nullable>enable</Nullable>
<ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
<IsPackable>false</IsPackable>
</PropertyGroup>

<PropertyGroup>
<ContinuousIntegrationBuild Condition="'$(CI)' == 'true'">true</ContinuousIntegrationBuild>
<EmbedUntrackedSources>true</EmbedUntrackedSources>
<DebugType>embedded</DebugType>
</PropertyGroup>

<ItemGroup Condition="'$(IsPackable)' == 'true'">
<PackageReference Include="Microsoft.SourceLink.GitHub" PrivateAssets="all" IncludeAssets="runtime; build; native; contentfiles; analyzers" />
</ItemGroup>

<Import Project="$(MSBuildThisFileDirectory)eng/LegacySupport.props" />

</Project>
4 changes: 4 additions & 0 deletions Directory.Packages.props
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,10 @@
<PackageVersion Include="sqlite-vec" Version="0.1.7-alpha.2.1" />
</ItemGroup>

<ItemGroup Label="Build">
<PackageVersion Include="Microsoft.SourceLink.GitHub" Version="10.0.102" />
</ItemGroup>

<ItemGroup Label="Test">
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="18.4.0" />
<PackageVersion Include="Microsoft.Extensions.Configuration" Version="10.0.6" />
Expand Down
3 changes: 3 additions & 0 deletions MEVD/src/Directory.Build.props
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,9 @@
<Import Project="$([MSBuild]::GetPathOfFileAbove('Directory.Build.props', '$(MSBuildThisFileDirectory)../'))" />

<PropertyGroup>
<IsPackable>true</IsPackable>
<GenerateDocumentationFile>true</GenerateDocumentationFile>

<NoWarn>$(NoWarn);MEVD9000,MEVD9001</NoWarn> <!-- Experimental MEVD connector-facing APIs -->
<InjectCallerAttributesOnLegacy>true</InjectCallerAttributesOnLegacy>
<InjectTrimAttributesOnLegacy>true</InjectTrimAttributesOnLegacy>
Expand Down
Loading