Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 0 additions & 12 deletions .github/ISSUE_TEMPLATE/pre-task.md

This file was deleted.

2 changes: 1 addition & 1 deletion .github/ISSUE_TEMPLATE/r2cn.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ assignees: ''
2. 所有 __PR__ 提交必须通过 `GitHub Actions` 自动化测试,提交 __PR__ 后请关注 `GitHub Actions` 结果;
3. 代码注释均需要使用英文;

[__PR 提交地址__] 提交到 [git-internal](https://github.com/web3infra-foundation/git-internal) 仓库的 `main` 分支 `` 目录;
[__PR 提交地址__] 提交到 [git-internal](https://github.com/web3infra-foundation/git-internal) 仓库的 `main` 分支 `src` 目录;

[__开发指导__]
1. 认领任务参考 [r2cn 开源实习计划 - 任务认领与确认](https://r2cn.dev/docs/student/assign);
Expand Down
11 changes: 11 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file

version: 2
updates:
- package-ecosystem: "cargo" # See documentation for possible values
directory: "/" # Location of package manifests
schedule:
interval: "daily"
33 changes: 17 additions & 16 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -7,41 +7,42 @@ license = "MIT"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
sea-orm = { version = "1.1.12", features = ["sqlx-sqlite"] }
flate2 = { version = "1.1.1", features = ["zlib"] }
serde = { version = "1.0.219", features = ["derive"] }
bstr = "1.12.0"
hex = "0.4.3"
thiserror = "2.0.12"
thiserror = "2.0.17"
tracing = "0.1.41"
tracing-subscriber = "0.3.20"
sha1 = "0.10.6"
colored = "3.0.0"
chrono = { version = "0.4.41", features = ["serde"] }
tracing-subscriber = "0.3.19"
uuid = { version = "1.17.0", features = ["v4"] }
threadpool = "1.8.1"
num_cpus = "1.17.0"
dashmap = "6.1.0"
tokio = { version = "1.45.1", features = ["fs"] }
lru-mem = "0.3.0"
bincode = { version = "2.0.1", features = ["serde"] }
byteorder = "1.5.0"
futures-util = "0.3.31"
bytes = "1.10.1"
axum = { version = "0.8.4", features = ["macros", "json"] }
memchr = "2.7.4"
encoding_rs = "0.8.35"
rayon = "1.10.0"
reqwest = "0.12.18"
rayon = "1.11.0"
reqwest = "0.12.23"
ring = "0.17.14"
serde_json = "1.0.140"
serde_json = "1.0.145"
ahash = "0.8.12"
diffs = "0.5.1"
libc = "0.2.172"
zstd-sys = { version = "2.0.15+zstd.1.5.7", features = ["experimental"] }
libc = "0.2.177"
zstd-sys = { version = "2.0.16+zstd.1.5.7", features = ["experimental"] }
sea-orm = { version = "1.1.17", features = ["sqlx-sqlite"] }
flate2 = { version = "1.1.4", features = ["zlib"] }
serde = { version = "1.0.228", features = ["derive"] }
chrono = { version = "0.4.42", features = ["serde"] }
uuid = { version = "1.18.1", features = ["v4"] }
tokio = { version = "1.47.1", features = ["fs"] }
bincode = { version = "2.0.1", features = ["serde"] }
axum = { version = "0.8.6", features = ["macros", "json"] }


[dev-dependencies]
tokio = { version = "1.45.1", features = ["full"] }
tokio = { version = "1.47.1", features = ["full"] }
tokio-util = { version = "0.7.15", features = ["io"] }
quickcheck = "1.0.3"
rand = "0.9.2"
Expand Down
51 changes: 49 additions & 2 deletions src/errors.rs
Original file line number Diff line number Diff line change
@@ -1,97 +1,144 @@
//! Error types for the Git-Internal crate.
//!
//! This module defines a unified error enumeration used across object parsing,
//! pack encoding/decoding, index handling, caching, and streaming. It integrates
//! with `thiserror` to provide rich `Display` implementations and error source
//! chaining where applicable.
//!
//! Notes:
//! - Each variant carries contextual details via its message payload.
//! - Variants cover parse/validation, I/O, encoding/decoding, network/auth,
//! and custom errors.

use thiserror::Error;

#[derive(Error, Debug)]
/// Unified error enumeration for the Git-Internal library.
///
/// - Used across object parsing, pack encode/decode, index, caching and streams.
/// - Implements `std::error::Error` via `thiserror`.
pub enum GitError {
/// Invalid or unsupported git object type name.
#[error("The `{0}` is not a valid git object type.")]
InvalidObjectType(String),

/// Malformed or unsupported blob object encoding.
#[error("The `{0}` is not a valid git blob object.")]
InvalidBlobObject(String),

/// Malformed tree object.
#[error("Not a valid git tree object.")]
InvalidTreeObject,

/// Invalid tree entry (mode/name/hash).
#[error("The `{0}` is not a valid git tree item.")]
InvalidTreeItem(String),

/// Tree contains no entries.
#[error("`{0}`.")]
EmptyTreeItems(String),

/// Invalid commit signature type.
#[error("The `{0}` is not a valid git commit signature.")]
InvalidSignatureType(String),

/// Malformed commit object.
#[error("Not a valid git commit object.")]
InvalidCommitObject,

/// Commit parse or validation failed.
#[error("Invalid Commit: {0}")]
InvalidCommit(String),

/// Malformed tag object.
#[error("Not a valid git tag object: {0}")]
InvalidTagObject(String),

/// Malformed note object.
#[error("Not a valid git note object: {0}")]
InvalidNoteObject(String),

/// Malformed or unsupported index (.idx) file.
#[error("The `{0}` is not a valid idx file.")]
InvalidIdxFile(String),

/// Malformed or unsupported pack file.
#[error("The `{0}` is not a valid pack file.")]
InvalidPackFile(String),

/// Invalid pack header magic or version.
#[error("The `{0}` is not a valid pack header.")]
InvalidPackHeader(String),

/// Malformed or unsupported git index file.
#[error("The `{0}` is not a valid index file.")]
InvalidIndexFile(String),

/// Invalid git index header.
#[error("The `{0}` is not a valid index header.")]
InvalidIndexHeader(String),

/// Invalid CLI or function argument.
#[error("Argument parse failed: {0}")]
InvalidArgument(String),

/// I/O error from underlying reader or writer.
#[error("IO Error: {0}")]
IOError(#[from] std::io::Error),

/// Invalid SHA1 hash formatting or value.
#[error("The {0} is not a valid Hash value ")]
InvalidHashValue(String),

/// Delta object reconstruction error.
#[error("Delta Object Error Info:{0}")]
DeltaObjectError(String),

/// Object not fully populated for packing.
#[error("The object to be packed is incomplete ,{0}")]
UnCompletedPackObject(String),

/// Invalid decoded object info.
#[error("Error decode in the Object ,info:{0}")]
InvalidObjectInfo(String),

#[error("Can't found Hash value :{0} from current file")]
NotFountHashValue(String),
/// Hash not found in current file context.
#[error("Cannot find Hash value: {0} from current file")]
NotFoundHashValue(String),

/// Failed to encode object to bytes.
#[error("Can't encode the object which id [{0}] to bytes")]
EncodeObjectError(String),

/// Text encoding or UTF-8 conversion error.
#[error("UTF-8 conversion error: {0}")]
ConversionError(String),

/// Invalid path when locating parent tree.
#[error("Can't find parent tree by path: {0}")]
InvalidPathError(String),

/// Failed to encode pack entries.
#[error("Can't encode entries to pack: {0}")]
PackEncodeError(String),

/// Object missing from caches or storage.
#[error("Can't find specific object: {0}")]
ObjectNotFound(String),

/// Repository not found.
#[error("Repository not found")]
RepoNotFound,

/// Unauthorized access.
#[error("UnAuthorized: {0}")]
UnAuthorized(String),

/// Network communication error.
#[error("Network Error: {0}")]
NetworkError(String),

/// Generic custom error for miscellaneous failures.
#[error("{0}")]
CustomError(String),
}
33 changes: 32 additions & 1 deletion src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,4 +1,35 @@
//! Git-Internal is a library for encoding and decoding Git Pack format files or streams.
//! Git-Internal: a high-performance Rust library for Git objects and pack files—encode/decode, delta (ref/offset/zstd),
//! caching, streaming, and sync/async pipelines.
//!
//! Goals
//! - Provide high-performance parsing and generation of Git pack format and Git objects.
//! - Support both file-based and streaming inputs of Git Pack.
//! - Offer synchronous and asynchronous APIs for multi-threaded and Tokio runtimes.
//!
//! Core Capabilities
//! - Decoding: base objects and delta objects (Ref-Delta, Offset-Delta, ZstdDelta).
//! - Encoding: serial and parallel pipelines; offset deltas and Zstd-based deltas.
//! - Streaming: `decode_stream` for `Stream<Bytes>`; `decode_async` decodes in a new thread and sends entries.
//! - Caching & memory: LRU-based cache; `MemSizeRecorder` tracks heap usage; optional `mem_limit` to bound memory.
//! - Utilities: SHA1, zlib, delta, zstdelta toolkits.
//!
//! Modules
//! - `internal::pack`: decode/encode, caches, waitlists, parallel pipelines, helpers.
//! - `internal::object`: Blob/Tree/Commit/Tag/Note objects, type enum, object trait.
//! - `internal::zlib`: compression/decompression stream utilities.
//! - `delta` and `zstdelta`: delta algorithms and rebuild helpers.
//! - `errors`: unified error types.
//! - `hash`: SHA1 helpers.
//! - `utils`: common utilities (e.g., `CountingReader`).
//!
//! Typical Usage
//! - Offline large-file decode: `Pack::decode_async(reader, sender)` decodes in a thread and sends `Entry`s.
//! - Stream decode: `Pack::decode_stream(stream, sender)` consumes `ReaderStream` under Tokio.
//! - Parallel encode: `encode_async` / `parallel_encode` build packs from many objects.
//!
//! Test Data
//! - Located under `tests/data/`, includes real pack files and object sets.

pub mod errors;
pub mod hash;
pub mod internal;
Expand Down