Skip to content

Support lazy parsing to reduce memory usage#8

Closed
Copilot wants to merge 4 commits intomasterfrom
copilot/support-lazy-parsing
Closed

Support lazy parsing to reduce memory usage#8
Copilot wants to merge 4 commits intomasterfrom
copilot/support-lazy-parsing

Conversation

Copy link
Contributor

Copilot AI commented Oct 9, 2025

Overview

This PR implements lazy parsing support to reduce memory usage when parsing Go source code. Lazy parsing defers the evaluation of type declarations (constants, variables, and types) until they are actually accessed, making it particularly useful when working with large codebases or when only a subset of types needs to be analyzed.

Motivation

When parsing large Go packages, eagerly parsing all type declarations can consume significant memory, especially if only a small subset of types are actually used. This implementation addresses that by deferring parsing until necessary.

Changes

New Feature: WithLazyParsing() Option

Enable lazy parsing mode:

import "github.com/wzshiming/gotype"

// Create an importer with lazy parsing enabled
imp := gotype.NewImporter(gotype.WithLazyParsing())

// Parse source code - types are only fully parsed when accessed
scope, err := imp.ImportSource("mypackage", []byte(sourceCode))
if err != nil {
    // handle error
}

// Accessing a type triggers lazy parsing of that specific declaration
myType, ok := scope.ChildByName("MyType")

Implementation Details

  • typeLazy wrapper: New type that wraps AST declarations and defers parsing until first access
  • Thread-safe: Uses sync.Once to ensure safe concurrent access during lazy initialization
  • Selective parsing: Functions and imports are still parsed eagerly as they are lightweight and needed for dependency resolution
  • Backward compatible: Default behavior unchanged; lazy parsing is opt-in

Performance Improvements

Benchmark results with 100 type declarations:

  • Eager parsing: 464,198 B/op, 9,179 allocs/op
  • Lazy parsing: 339,396 B/op, 6,679 allocs/op
  • Improvement: ~27% reduction in both memory usage and allocations

Memory usage test with 1,000 type declarations:

  • Eager parsing: 3,724,016 bytes
  • Lazy parsing: 2,492,040 bytes
  • Improvement: ~33% memory reduction

Testing

  • ✅ All existing tests pass without modification
  • ✅ Added comprehensive test suite for lazy parsing functionality
  • ✅ Race detector passes with no issues
  • ✅ Added benchmarks demonstrating memory improvements
  • ✅ Added example program in cmd/lazy_example/

Documentation

  • Updated README.md with lazy parsing usage and benefits
  • Updated README_cn.md with Chinese documentation
  • Added example program demonstrating the feature

Example

See cmd/lazy_example/main.go for a complete example comparing eager vs lazy parsing performance.

Closes #[issue_number]

Original prompt

This section details on the original issue you should resolve

<issue_title>Support lazy parsing</issue_title>
<issue_description>Reduce memory usage by parsing source code only when needed</issue_description>

Comments on the Issue (you are @copilot in this section)

Fixes #7

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 3 commits October 9, 2025 08:25
Co-authored-by: wzshiming <6565744+wzshiming@users.noreply.github.com>
Co-authored-by: wzshiming <6565744+wzshiming@users.noreply.github.com>
Co-authored-by: wzshiming <6565744+wzshiming@users.noreply.github.com>
Copilot AI changed the title [WIP] Add support for lazy parsing to reduce memory usage Support lazy parsing to reduce memory usage Oct 9, 2025
Copilot AI requested a review from wzshiming October 9, 2025 08:32
@wzshiming wzshiming closed this Oct 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support lazy parsing

2 participants