0.5.0-rc2 Up to 2x performance regression loading precompiled packages

[ufechner7]

Julia 0.4.6

tic(); using PyPlot; toc()
elapsed time: 2.569278583 seconds

Julia 0.5.0-rc2

tic(); using PyPlot; toc();
elapsed time: 5.360103935 seconds

This happens with precompiled packages, therefore - to my understanding - the root cause cannot be, that the new llvm version is slower compiling.
It would be very nice, if this regression could be fixed.
Computer: Linux 64 bit, Ubuntu 14.04, i7-3770 CPU @ 3.40GHz × 4

[yuyichao]

Precompile doesn't remove codegen overhead. Although it can't explain all the difference, it seems that at least 50% of the time is indeed spent in llvm.

[ufechner7]

Some more results. They are not consistent: Loading Gtk (master branch) is even faster on 0.5, compared to 0.4.6. The slow down factor of loading JuMP is 1.5 .

Julia 0.4.6
tic(); using PyPlot; toc()
elapsed time: 2.569278583 seconds

julia> tic(); using Gtk; toc();
elapsed time: 1.424508958 seconds

julia> tic(); using JuMP; toc();
elapsed time: 0.617230441 seconds


Julia 0.5.0-rc2

tic(); using PyPlot; toc();
elapsed time: 5.360103935 seconds

julia> tic(); using Gtk; toc();
elapsed time: 0.943486983 seconds

julia> tic(); using JuMP; toc();
elapsed time: 0.910472513 seconds

[ufechner7]

Why is precompilation not removing the codegen overhead? I thought, that this is exactly the purpose of precompilation?

[tkelman]

we don't currently save native code in the .ji files, so llvm still has work to do

[ufechner7]

Which data format is stored in the .ji files?

[yuyichao]

Serialized (inferred) AST.

[vtjnash]

half is due to llvm. the other half is due to jl_recache_types. the worklist in that function is optimized for hundreds of items (as it had on v0.4). on v0.5, it now often has hundreds of thousands.

[ufechner7]

Why does it have so many items now?

[ViralBShah]

From a user perspective, it would be nice if PyPlot loading time is as before or lesser. I hope there is something we can do here.