Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 74 additions & 12 deletions 02-E2E-Model-Development-on-Foundry/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,27 +2,89 @@

Livestream starting soon! **Click below to register.**

[![Reactor Livestream](./assets/poster-quest2.png)](https://www.youtube.com/live/GHH50rDlLn0?si=-i3hPYq1o6H271_z)
[![Reactor Livestream](./assets/poster-quest2.png)](https://developer.microsoft.com/reactor/events/26773/)

# Building AI Agents E2E On Microsoft Foundry

_In the previous quest, you used Foundry Local to deploy and use models locally, to build on-device AI solutions. But what if you want to build enterprise-grade AI solutions at cloud-scale? This is where Microsoft Foundry comes in._
In this Quest, you'll explore the end-to-end model development journey for designing and developing AI agents with Microsoft Foundry models. You'll start at the Foundry portal and learn to setup a project, then select and deploy the right model for the task. Then, you'll move to code, and learn to observe, optimize, and protect, your AI agent using techniques like fine-tuning, tracing, evaluations, and AI red-teaming - using the Microsoft Foundry SDK. By the end of this quest, you should have a good intuition for the capabilities of the Microsoft Foundry platform and have a sandbox you can use to explore these further.

In this Quest, you'll explore the end-to-end workflow for designing and developing AI solutions with _Microsoft Foundry models_ in the cloud. You'll learn to select models (using catalog, leaderboards, and playground), customize them (with fine-tuning) - then observe their performance and assess their quality and safety (with tracing, evaluations, and red-teaming). By completing this quest, you'll get a better intuition for what it takes to build trustworthy AI using Microsoft Foundry.
## Scenario

Let's set the stage with a popular real-world use case.
Let's use a popular real-world scenario to motivate the quest. Imagine you are an AI engineer working for Zava, a fictitious enterprise retail company. You have been asked to build Cora, your new customer service AI agent that answers shoppers' questions about products in [this sample catalog](./assets/assets.csv).

Imagine you are an AI engineer working for _Zava_, a fictitious enterprise retail organization selling home improvement products to DIY customers. You have been asked to build "Cora" - a customer service agent that answers shopper questions in-store and online. In this quest, we'll keep it simple and have the agent answer questions about the products in a provided catalog.
Cora needs to meet three requirements:

The AI Agent must meet three objectives:
1. **Be polite and helpful** in interactions. Think tone and response format.
2. **Be cost-effective** to operate. Think latency, compute and token costs.
3. **Be trustworthy** in responses. Ensure safety, quality and accuracy.

1. **Be polite and helpful** in customer interactions. _Meet specific response tone and format_.
1. **Be cost-effective** in operations. _Minimize latency and token costs in usage_.
1. **Be trustworthy** in responses. _Ensure safety, quality & accuracy in responses_.
How do we go from plan to prototype and production, to meet these goals?

_How do we go from these requirements - to plan, prototype and production?_
## Steps To Complete The Quest
### 1. Pre-Requisites

To complete the quest, you will need:

## Developer Journey
1. an active Azure subscription - with *Owner* or *Contributor* access.
2. a personal GitHub account - with Codespaces quota.

Coming soon!
### 2. Launch Codespaces

1. Open a new browser tab and log into your GitHub account.
2. Now navigate to [this link](https://aka.ms/foundry/e2e-codespaces) to see the page below. Click create codespace.
![Codespaces](./assets/codespaces.png)
3. This will open up a new browser tab with a VS Code editor session loading.
4. Wait till the VS Code editor has loaded - and you see an active terminal. This takes a few minutes.
5. Your local environment is ready. ✅


### 3. Authenticate with Azure

1. Type this command into the VS Code terminal when ready. Then complete the prompts using the device code, to log into your Azure account.
```bash
az login
```
1. Verify that you are logged in using this command. You should see your Azure profile.
```bash
az account show
```
1. Your local environment is connected to Azure. ✅



### 4. Complete the Quest

The repoistory is setup with a `devcontainer.json` that installs all the required depdencencies and gives you access to a free tier of GitHub Copilot.

Open the `README.md` in that repository to see the various _tasks_ available for that quest.

- **Task 1:** Understand Foundry capabilities
- **Task 2:** Setup a Foundry project
- **Task 3:** Select a base model
- **Task 4:** Customize the base model
- **Task 5:** Design the AI Agent
- **Task 6:** Evaluate the agent responses
- **Task 7:** Trace the agent execution
- **Task 8:** Run a Red-Teaming scan
- **Task 9:** Teardown and cleanup

**Important Notes:**

1. **Some tasks like fine-tuning can take a lot of time to complete** - in this case, you should be able to go on to the next task, then return to review results here when the job completes.
1. **Tasks like fine-tuning, evaluations and red-teaming have added costs and constraints** - for instance, we need to use a specific model in a specific region. If this is not viable in your subscription, then treat this as a _read-only_ task and explore the code and results from our run.
1. **Teardown the project when done** - to prevent unexpected costs from model or compute use.

This quest has a lot of steps to complete, many of which cannot be completed in a single hour. And that's okay. The main objective is to give you intuition for what an end-to-end journey would involve - and provide code snippets and results from a sample run, to help you connect theory to practice.

**Focus on completing just a few tasks today**. Validate your development envrionment and get familiar with the Microsoft Foundry UI and SDK. Then, read through the rest using the screenshots from our sample execution run to get an understanding of the purpose and process for each step.

**Explore the rest at your own pace later**. Create a fork of the repo, then launch a new Codespaces instance on that fork and work through the steps in order. Try customizing the data to suit a different scenario. Or change the code to try a different evaluator or attack strategy for red-teaming. Use your fork as a sandbox for building a deeper understanding of these capabilities with hands-on experiments.


### Return to the Build-a-thon

Once you have completed this quest and get an intuitive sense for end-to-end development with Microsoft Foundry, return to the main Build-a-thon repository to continue with the next quests.

## AI Note

This quest was partially created with the help of AI. The author reviewed and revised the content to ensure accuracy and quality.
50 changes: 50 additions & 0 deletions 02-E2E-Model-Development-on-Foundry/assets/assets.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
name,sku,price,description,stock_level,image_path,main_category,subcategory
Premium Interior Latex Flat,PFIP000001,40.0,"High-quality flat interior paint with excellent coverage and hide, perfect for ceilings and low-traffic areas.",19,paint_&_finishes_interior_paint_premium_interior_latex_flat_20250620_192214.png,PAINT & FINISHES,INTERIOR PAINT
Interior Eggshell Paint,PFIP000002,44.0,"Durable eggshell finish paint with subtle sheen, ideal for living rooms and bedrooms with easy cleanup.",80,paint_&_finishes_interior_paint_interior_eggshell_paint_20250620_192249.png,PAINT & FINISHES,INTERIOR PAINT
Interior Semi-Gloss Paint,PFIP000003,47.0,"Washable semi-gloss interior paint for kitchens, bathrooms, and trim work with moisture resistance.",2,paint_&_finishes_interior_paint_interior_semi_gloss_paint_20250620_192336.png,PAINT & FINISHES,INTERIOR PAINT
Zero VOC Interior Paint,PFIP000004,52.0,Environmentally friendly zero-VOC paint for healthy indoor air quality in all living spaces.,25,paint_&_finishes_interior_paint_zero_voc_interior_paint_20250620_192411.png,PAINT & FINISHES,INTERIOR PAINT
One-Coat Interior Paint,PFIP000005,50.0,High-hide one-coat paint formula that covers most colors in a single application for time savings.,47,paint_&_finishes_interior_paint_one_coat_interior_paint_20250620_192445.png,PAINT & FINISHES,INTERIOR PAINT
Exterior Latex Paint Satin,PFEP000006,50.0,"Weather-resistant exterior paint with satin finish for siding, trim, and outdoor surfaces.",93,paint_&_finishes_exterior_paint_exterior_latex_paint_satin_20250620_192519.png,PAINT & FINISHES,EXTERIOR PAINT
Exterior Acrylic Paint,PFEP000007,57.0,Premium acrylic exterior paint with fade resistance and superior adhesion for long-lasting protection.,74,paint_&_finishes_exterior_paint_exterior_acrylic_paint_20250620_192554.png,PAINT & FINISHES,EXTERIOR PAINT
Universal Bonding Primer,PFPR000011,32.0,"High-adhesion primer for all surfaces including wood, metal, drywall, and previously painted surfaces.",96,paint_&_finishes_primer_universal_bonding_primer_20250620_192814.png,PAINT & FINISHES,PRIMER
Stain-Blocking Primer,PFPR000012,40.0,"Heavy-duty primer that blocks water stains, crayon marks, and other discoloration from bleeding through.",42,paint_&_finishes_primer_stain_blocking_primer_20250620_192849.png,PAINT & FINISHES,PRIMER
Drywall Primer,PFPR000014,29.0,Fast-drying primer designed for new drywall with excellent sealing properties and quick recoat time.,42,paint_&_finishes_primer_drywall_primer_20250620_192959.png,PAINT & FINISHES,PRIMER
Synthetic Brush Set,PFBR000016,16.0,Professional synthetic bristle brushes for latex paints with excellent paint pickup and smooth finish.,86,paint_&_finishes_brushes_synthetic_brush_set_20250620_193107.png,PAINT & FINISHES,BRUSHES
Natural Bristle Brush Set,PFBR000017,23.0,Traditional natural bristle brushes for oil-based paints and stains with superior flow and leveling.,40,paint_&_finishes_brushes_natural_bristle_brush_set_20250620_193149.png,PAINT & FINISHES,BRUSHES
Angled Brush Set,PFBR000018,20.0,Precision angled brushes for cutting in edges and detail work with comfortable ergonomic handles.,28,paint_&_finishes_brushes_angled_brush_set_20250620_193229.png,PAINT & FINISHES,BRUSHES
Foam Brush Set,PFBR000019,9.0,"Disposable foam brushes for small projects, touch-ups, and applying stains and polyurethane.",8,paint_&_finishes_brushes_foam_brush_set_20250620_193303.png,PAINT & FINISHES,BRUSHES
Roller Frame 9-inch,PFRL000021,10.0,Heavy-duty roller frame with threaded handle attachment and cage design for various cover types.,90,paint_&_finishes_rollers_roller_frame_9_inch_20250620_193410.png,PAINT & FINISHES,ROLLERS
Microfiber Roller Covers,PFRL000022,14.0,High-quality microfiber roller covers for smooth finishes with minimal lint and excellent paint capacity.,57,paint_&_finishes_rollers_microfiber_roller_covers_20250620_193446.png,PAINT & FINISHES,ROLLERS
Nap Roller Cover Set,PFRL000023,12.0,Assorted nap roller covers from 1/4-inch to 1-inch for different surface textures and paint types.,52,paint_&_finishes_rollers_nap_roller_cover_set_20250620_193520.png,PAINT & FINISHES,ROLLERS
Mini Roller Kit 4-inch,PFRL000024,8.0,"Compact roller kit for small areas, touch-ups, and detail work with frame and covers included.",62,paint_&_finishes_rollers_mini_roller_kit_4_inch_20250620_193555.png,PAINT & FINISHES,ROLLERS
Extension Pole 4-foot,PFRL000025,15.0,Telescoping extension pole for reaching high areas and reducing ladder use during painting projects.,34,paint_&_finishes_rollers_extension_pole_4_foot_20250620_193630.png,PAINT & FINISHES,ROLLERS
Metal Paint Tray 9-inch,PFPT000042,10.0,Durable metal paint tray with deep well and ribbed ramp for proper roller loading.,70,paint_&_finishes_paint_trays_metal_paint_tray_9_inch_20250620_194611.png,PAINT & FINISHES,PAINT TRAYS
Paint Tray Liner Set,PFPT000043,5.0,Disposable plastic liners that fit standard paint trays for quick cleanup and color changes.,0,paint_&_finishes_paint_trays_paint_tray_liner_set_20250620_194645.png,PAINT & FINISHES,PAINT TRAYS
Canvas Drop Cloth 9x12,PFDC000046,20.0,Heavy-duty canvas drop cloth for floor protection during painting with absorbent cotton material.,53,paint_&_finishes_drop_cloths_canvas_drop_cloth_9x12_20250620_194828.png,PAINT & FINISHES,DROP CLOTHS
Plastic Drop Cloth,PFDC000047,5.0,Lightweight plastic sheeting for furniture and floor protection with easy disposal after use.,60,paint_&_finishes_drop_cloths_plastic_drop_cloth_20250620_194905.png,PAINT & FINISHES,DROP CLOTHS
Pre-Taped Masking Film,PFDC000050,10.0,Plastic film with pre-applied masking tape for quick coverage of windows and trim work.,97,paint_&_finishes_drop_cloths_pre_taped_masking_film_20250620_195046.png,PAINT & FINISHES,DROP CLOTHS
Orbital Sander 1/4 Sheet,PTSA000011,67.0,Variable-speed orbital sander with dust collection system for smooth wood finishing.,43,power_tools_sanders_orbital_sander_14_sheet_20250620_182913.png,POWER TOOLS,SANDERS
Random Orbit Sander 5-inch,PTSA000012,92.0,Random orbital action sander with hook-and-loop pad system for swirl-free finishes.,45,power_tools_sanders_random_orbit_sander_5_inch_20250620_182949.png,POWER TOOLS,SANDERS
Palm Sander,PTSA000013,52.0,Compact palm sander with ergonomic design for detail work and tight corners.,27,power_tools_sanders_palm_sander_20250621_105614.png,POWER TOOLS,SANDERS
Finishing Hammer 13oz,HTHM041300,25.0,Lightweight finishing hammer with smooth face for trim work and delicate construction tasks.,75,hand_tools_hammers_finishing_hammer_13oz_20250620_181314.png,HAND TOOLS,HAMMERS
Phillips Screwdriver Set,HTSD010001,16.0,Complete Phillips head screwdriver set with magnetic tips and comfortable cushion grips in multiple sizes.,79,hand_tools_screwdrivers_phillips_screwdriver_set_20250620_181424.png,HAND TOOLS,SCREWDRIVERS
Flathead Screwdriver Set,HTSD020001,14.0,Professional flathead screwdrivers with precision-machined tips and ergonomic handles for various applications.,95,hand_tools_screwdrivers_flathead_screwdriver_set_20250620_181457.png,HAND TOOLS,SCREWDRIVERS
Needle-Nose Pliers 6-inch,HTPL000006,15.0,Long-nose pliers with serrated jaws for gripping small objects and working in tight spaces.,71,hand_tools_pliers_needle_nose_pliers_6_inch_20250621_093109.png,HAND TOOLS,PLIERS
Tape Measure 25-foot,HTMT000013,14.0,Heavy-duty tape measure with standout blade and magnetic tip for one-person measuring.,50,hand_tools_measuring_tools_tape_measure_25_foot_20250621_110945.png,HAND TOOLS,MEASURING TOOLS
Level 24-inch,HTMT000015,27.0,Precision aluminum level with three vials for accurate horizontal and vertical alignment.,100,hand_tools_measuring_tools_level_24_inch_20250621_111113.png,HAND TOOLS,MEASURING TOOLS
Speed Square 7-inch,HTMT000014,10.0,Aluminum speed square for quick angle cuts and layout work with degree markings.,42,hand_tools_measuring_tools_speed_square_7_inch_20250621_111029.png,HAND TOOLS,MEASURING TOOLS
Wood Screw Assortment,HWSC000001,12.0,Comprehensive collection of wood screws in various lengths from 1/2-inch to 3-inch with Phillips and Robertson drives.,14,hardware_screws_wood_screw_assortment_20250621_112459.png,HARDWARE,SCREWS
Drywall Screws 1-5/8 inch,HWSC000002,8.0,Fine-thread drywall screws with sharp points for quick penetration and secure attachment to metal studs.,71,hardware_screws_drywall_screws_1_58_inch_20250620_195131.png,HARDWARE,SCREWS
Finish Nails 2-inch,HWNA000007,12.0,Small-head finish nails for trim work and cabinetry where nail heads will be countersunk and filled.,48,hardware_nails_finish_nails_2_inch_20250620_195356.png,HARDWARE,NAILS
Common Nail Assortment,HWNA000006,16.0,Traditional common nails from 6d to 16d for general construction and framing applications.,72,hardware_nails_common_nail_assortment_20250621_112543.png,HARDWARE,NAILS
Flat Washer Assortment SAE,HWWA000016,8.0,Standard flat washers in SAE sizes to distribute load and prevent bolt head pull-through.,57,hardware_washers_flat_washer_assortment_sae_20250620_195901.png,HARDWARE,WASHERS
Wire Nut Assortment,ELWRN001,8.0,Color-coded wire nuts in multiple sizes for various wire gauge combinations.,82,electrical_wire_nuts_wire_nut_assortment_20250620_210513.png,ELECTRICAL,WIRE NUTS
Vinyl Electrical Tape,ELET000026,4.0,Standard black electrical tape for insulating wire connections and splices.,95,electrical_electrical_tape_vinyl_electrical_tape_20250620_211346.png,ELECTRICAL,ELECTRICAL TAPE
PTFE Thread Tape White,PLTFL001,2.0,Standard white PTFE tape for sealing water pipe threads.,86,plumbing_teflon_tape_ptfe_thread_tape_white_20250620_214238.png,PLUMBING,TEFLON TAPE
Safety Net for Storage,SOOVR005,35.0,Protective net to prevent stored items from falling from overhead areas.,77,storage_&_organization_overhead_storage_safety_net_for_storage_20250620_223003.png,STORAGE & ORGANIZATION,OVERHEAD STORAGE
Old Work Box,ELJB000014,6.0,Retrofit electrical box for adding outlets to existing walls without wall removal.,82,electrical_junction_boxes_old_work_box_20250620_210403.png,ELECTRICAL,JUNCTION BOXES
Extension Pole 4-foot,PFRL000025,15.0,Telescoping extension pole for reaching high areas and reducing ladder use during painting projects.,34,paint_&_finishes_rollers_extension_pole_4_foot_20250620_193630.png,PAINT & FINISHES,ROLLERS
Lineman's Pliers 9-inch,HTPL000007,25.0,Heavy-duty lineman's pliers with cutting edges and insulated grips for electrical work.,18,hand_tools_pliers_linemans_pliers_9_inch_20250620_182023.png,HAND TOOLS,PLIERS
Cordless Drill 18V Li-Ion,PTDR000001,115.0,"Professional cordless drill with lithium-ion battery, variable speed control, and LED work light.",3,power_tools_drills_cordless_drill_18v_li_ion_20250620_182306.png,POWER TOOLS,DRILLS
Hammer Drill 1/2-inch,PTDR000002,160.0,Corded hammer drill with percussion action for drilling into concrete and masonry materials.,77,power_tools_drills_hammer_drill_12_inch_20250620_182340.png,POWER TOOLS,DRILLS
Right Angle Drill,PTDR000003,138.0,Compact right-angle drill for tight spaces with 3/8-inch chuck and reversible motor.,5,power_tools_drills_right_angle_drill_20250620_182416.png,POWER TOOLS,DRILLS
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ hide:

## 🎟️ Registration

[**Register now!**](https://developer.microsoft.com/reactor/series/S-1628)
[**Register now!**](https://developer.microsoft.com/reactor/series/S-1628//?wt.mc_id=github_S-1628_webpage_reactor)

Sign up and browse the livestream schedule to register for sessions you're interested in.

Expand Down