Add the cratespro crate and import the tugraph module.#1440
Conversation
Signed-off-by: hongwangliu-nju <1649209217@qq.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Pull Request Overview
This PR migrates cratespro crate modules to the extensions/cratespro/common directory and adds TuGraph module import logic. The changes consolidate Rust crate analysis and graph database functionality under a unified structure.
Key Changes:
- Migrates cratespro modules to extensions/cratespro/common for better organization
- Adds TuGraph graph database integration for crate dependency analysis
- Implements Kafka-based message processing for repository imports
- Introduces search functionality with embedding support and PostgreSQL integration
Reviewed Changes
Copilot reviewed 77 out of 77 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| extensions/cratespro/import_tugraph/ | New TuGraph import service with Kafka consumer and graph import logic |
| extensions/cratespro/common/tudriver/ | TuGraph client wrapper for graph database operations |
| extensions/cratespro/common/search/ | Search module with AI embedding and full-text search capabilities |
| extensions/cratespro/common/repo_import/ | Repository import logic with version parsing and Kafka handling |
| extensions/cratespro/common/model/ | Data models for TuGraph, GitHub, and repository sync operations |
| extensions/cratespro/common/database/ | Database entity definitions and ORM mappings |
| extensions/cratespro/common/data_transporter/ | API handlers and Redis caching for crate data transport |
Comments suppressed due to low confidence (1)
extensions/cratespro/common/data_transporter/src/handler.rs:1
- Misspelled field name 'decription' should be 'description'. This appears to be a copy of the same typo from crate_info.rs
use std::collections::HashSet;
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
| count += 1; | ||
| if count == 10000 { | ||
| import_driver.context.write_tugraph_import_files().await; | ||
| let _ =import_driver.context.import_from_env_vars().await; |
There was a problem hiding this comment.
Missing space after '=' in variable assignment. Should be 'let _ = import_driver.context.import_from_env_vars().await;'
| let _ =import_driver.context.import_from_env_vars().await; | |
| let _ = import_driver.context.import_from_env_vars().await; |
|
|
||
| // 处理description,将多行字符串转换为单行,用\n替换换行符 | ||
| let description = parsed["package"] | ||
| .get("decription") |
There was a problem hiding this comment.
Misspelled field name 'decription' should be 'description'
| .get("decription") | |
| .get("description") |
| env::var("NEW_CRATES_DIR").unwrap_or_else(|_| CLONE_CRATES_DIR.to_string()); | ||
| let split_crates_dir = | ||
| env::var("SPLIT_CRATES_DIR").unwrap_or_else(|_| CLONE_CRATES_DIR.to_string()); |
There was a problem hiding this comment.
[nitpick] Both variables use the same fallback constant CLONE_CRATES_DIR, but split_crates_dir should likely have its own dedicated fallback path to avoid confusion
Migrate the crates in cratespro to extensions/cratespro/common, and add the logic of parsing and importing graph to extensions/cratespro