Skip to content
Closed
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
d32cf62
Introduce BalanceHeuristicExpand
schneems Jan 12, 2022
7324ef1
New direction BlockNode and IndentTree
schneems Jan 17, 2022
ddcc646
Initial tree building
schneems Jan 26, 2022
3682b6b
Handle invalid nodes
schneems Jan 26, 2022
a7d0fe2
Tree building perf commit
schneems Jan 26, 2022
6fd4e36
Reintroduce and fix InsertionSort for performance
schneems Jan 26, 2022
24cde0f
Fix missing deleted logic
schneems Jan 26, 2022
32eb0b3
Build tree recording
schneems Jan 27, 2022
20656d7
Slightly faster initial sorting
schneems Jan 27, 2022
b1acaf5
Tiny perf change to CodeLine#initialize
schneems Jan 27, 2022
763dec4
Remove unused code
schneems Jan 27, 2022
4ab2b70
Pull out inner/outer nodes
schneems Jan 27, 2022
5ba79c0
Rename @inner to @parents
schneems Jan 27, 2022
60d796e
Figure out search algorithm
schneems Jan 27, 2022
083e733
REXE test case
schneems Jan 28, 2022
c260450
Add more test cases
schneems Jan 28, 2022
22f9fc0
Add another test
schneems Jan 28, 2022
f2b8878
Fix problem with my logic for "inner/outer" check
schneems Jan 28, 2022
b7a6c65
Add misindentation test
schneems Jan 29, 2022
01c2b8c
Refactor decision logic into node
schneems Jan 29, 2022
1fa20e1
Initial idea to search by "diagnosing" a node
schneems Jan 31, 2022
16a109b
Fix BlockNode.from_parents above/below
schneems Feb 1, 2022
72cd93e
Fix BlockNode.from_parents that have 1 parent
schneems Feb 1, 2022
c5dbd27
Handle multiple case
schneems Feb 1, 2022
db85b5b
First indent tree search
schneems Feb 2, 2022
1a34697
Standardrb --fix
schneems Feb 3, 2022
6185b90
More search cases
schneems Feb 3, 2022
036373b
Follow multiple errors with IndentSearch
schneems Feb 5, 2022
f977054
Implement tracing on new search
schneems Feb 6, 2022
62bff09
Move IndentSearch to a proper file
schneems Feb 7, 2022
b8a1549
Add missing frozen string magic comment
schneems Feb 7, 2022
4301fab
Move BlockRecorder to proper file
schneems Feb 7, 2022
53eb266
Document IndentSearch
schneems Feb 7, 2022
ea6986e
Move journey to proper file
schneems Feb 7, 2022
ccd9816
Document and refactor IndentTree
schneems Feb 7, 2022
93bab24
Document BlockNode and update BlockDocument docs
schneems Feb 8, 2022
f3a83a5
Refactor logic to Diagnose class
schneems Feb 8, 2022
e06af0c
Fix problem with one bad internal node
schneems Feb 8, 2022
a666da7
Fix block reporter
schneems Feb 8, 2022
011e8a0
Diagnose docs, rename problems & fix one edge case
schneems Feb 8, 2022
70087ed
Refactor Diagnose into smaller methods & rename
schneems Feb 8, 2022
745dc3b
Move diagnose node to its own file
schneems Feb 8, 2022
194ee1b
standardrb --fix
schneems Feb 8, 2022
e0256d1
Finish extraction of Diagnose logic
schneems Feb 8, 2022
c0d8625
Remove unused BalanceHueristicExpand
schneems Feb 9, 2022
e5a70a9
WIP Integration with One failing test
schneems Feb 9, 2022
fc872fb
WIP 8 total failing tests
schneems Feb 9, 2022
026aa1b
WIP Initial BlockNodeContext
schneems Feb 10, 2022
60b4ed3
Update block building
schneems Feb 15, 2022
ee2c9fd
WIP WIP WIP????
schneems Feb 16, 2022
2303179
WIPPPPPPPPPPPPPPPPPPPPPPPPPP
schneems Feb 16, 2022
be9b4c4
WIP
schneems May 20, 2022
02285d2
Fix nested pseudo pair case
schneems Jun 4, 2022
ef535d5
192 examples, 2 failures, 1 pending
schneems Jun 7, 2022
fc7a62c
Check if one node can be pruned from multiple
schneems Jun 7, 2022
91b94a4
Move test out of ruby CLI
schneems Jun 8, 2022
c3b30da
Porting examples from existing tests
schneems Jun 8, 2022
4dfa1b2
This is a difficult problem
schneems Jul 26, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Tree building perf commit
Currently spending 40% of time building the tree in popping off nodes

```
40.12% 	0.35% 	3.16 	0.03 	0.00 	3.13 23373 	DeadEnd::PriorityQueue#pop
```

Spending 9% of time enqueueing nodes. It might be worth it to revisit using an insertion sort (with bsearch_index) to see if it's faster than a heap implementation as we're guaranteeing to search through all elements in the queue.

Currently perf for tree building our target case is:

```
$ be rspec spec/unit/indent_tree_spec.rb:7
Run options: include {:locations=>{"./spec/unit/indent_tree_spec.rb"=>[7]}}

DeadEnd::IndentTree
  WIP syntax_tree.rb.txt for performance validation

Finished in 0.75664 seconds (files took 0.14354 seconds to load)
1 example, 0 failures
```

Main has the full search around 1.2~1.1 seconds with the same time to load). So our budget to hit equal perf is 0.45 seconds. I'm hoping we can beat that by a bunch, but we've not even touched ripper yet and it's *supposed* to be the expensive part.
  • Loading branch information
schneems committed May 20, 2022
commit a7d0fe2ac980aaa75cb677664af1ad7c3ab35abe
22 changes: 15 additions & 7 deletions lib/dead_end/block_document.rb
Original file line number Diff line number Diff line change
Expand Up @@ -59,14 +59,9 @@ def call
self
end

def capture(node: , captured: )
inner = []
inner.concat(Array(captured))
inner << node
inner.sort_by! {|block| block.start_index }

def capture_all(inner)
lines = []
indent = node.indent
indent = inner.first.indent
lex_diff = LexPairDiff.new_empty
inner.each do |block|
lines.concat(block.lines)
Expand Down Expand Up @@ -98,6 +93,15 @@ def capture(node: , captured: )
now
end

def capture(node: , captured: )
inner = []
inner.concat(Array(captured))
inner << node
inner.sort_by! {|block| block.start_index }

capture_all(inner)
end

def eat_above(node)
return unless now = node&.eat_above

Expand Down Expand Up @@ -126,5 +130,9 @@ def pop
def peek
@queue.peek
end

def inspect
"#<DeadEnd::BlockDocument:0x000000010b375lol >"
end
end
end
2 changes: 0 additions & 2 deletions lib/dead_end/block_node.rb
Original file line number Diff line number Diff line change
Expand Up @@ -10,8 +10,6 @@ def initialize(lines: , indent: , next_indent: nil, lex_diff: nil)
@indent = indent
@next_indent = next_indent
@lines = lines
@left = nil
@right = nil
@inner = []

@start_index = lines.first.index
Expand Down
20 changes: 2 additions & 18 deletions lib/dead_end/indent_tree.rb
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,6 @@ def initialize(document: )

def call
reduce
# loop do
# requeue
# if document.queue.length >= @last_length
# break
# else
# @last_length = document.queue.length
# reduce
# end
# end

self
end
Expand All @@ -45,21 +36,14 @@ def reduce
break if below.leaning == :right
end

blocks.delete(original)
if !blocks.empty?
node = document.capture(node: original, captured: blocks)
if blocks.length != 1
node = document.capture_all(blocks)
document.queue << node
end
end
self
end

def requeue
document.each do |block|
document.queue << block
end
end

def to_s
@document.to_s
end
Expand Down
13 changes: 13 additions & 0 deletions spec/unit/indent_tree_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,19 @@

module DeadEnd
RSpec.describe IndentTree do
it "WIP syntax_tree.rb.txt for performance validation" do
file = fixtures_dir.join("syntax_tree.rb.txt")
lines = file.read.lines
lines.delete_at(768 - 1)
source = lines.join

debug_perf do
code_lines = CleanDocument.new(source: source).call.lines
document = BlockDocument.new(code_lines: code_lines).call
tree = IndentTree.new(document: document).call
end
end

it "invalid if/else end with surrounding code" do
source = <<~'EOM'
class Foo
Expand Down