High CPU usage for building large sites

[damithc]

When working with the CS2103 website with the latest master, the 'serve' command seems to be more CPU intensive than before, often going near 100% and freezing up the entire computer at times. I didn't notice such a problem in previous releases (although I was previously using the -o option most of the time). Was there any recent changes to the site building that could be causing this?

EDIT: can be replicated at v2.14.1 too. It's just that I didn't notice it because I was using the lazy loading option all the time.

Not a show stopper as CS2103 is an unusually large site, but it is always good to optimize the build process if possible.

[damithc]

It's the Generating pages... phase that pushes CPU usage to 100%

[ang-zeyu]

If I recall, on my laptop with a similar configuration to the above I consistently hit 100% usage quite a couple of versions back, though it never 'freezed'. Could it just be thrashing? (your system's memory seems quite full)

[damithc]

If I recall, on my laptop with a similar configuration to the above I consistently hit 100% usage quite a couple of versions back, though it never 'freezed'. Could it just be thrashing? (your system's memory seems quite full)

I see. The freeze is only momentary, so it could be just a temporary bottleneck. Yes, it could have happened a few versions ago. I remember there was a jump in the total build time sometime back too. Could have happened at the same time.

[damithc]

Looks like MarkBind is spawning nearly 30 JVMs during the build process. I thought removing all puml files might stop the JVMs being spawned, but it didn't.

[ang-zeyu]

are there any other potential reproduction conditions? I've tried editing a few puml files with markbind serve on the site but couldn't reproduce it

[damithc]

are there any other potential reproduction conditions? I've tried editing a few puml files with markbind serve on the site but couldn't reproduce it

Not related to puml files. It happens even if there are no puml files as all. Just run CS2103 website as markbind serve and watch the Task Manager. Note that the JVMs show up only at certain points of the build cycle. Seems like it happens twice in a build cycle (i.e., two peaks in the CPU usage); one of them happens near to the end of the build process, just before the progress bar starts filling up (progress bar is not visible if using git bash).

[damithc]

I could not replicate it with any other site though. Can replicate with CS2103 site or se-book site (i.e., cd into the book sub folder of CS2103 site and run markbind serve). But it happens even if I delete all puml files in CS2103. Probably related to scale of the site? May be node uses a pool of threads if the payload is high?

[ang-zeyu]

ok got it as well (had to expand to view the child processes under cmd.exe), though removing all <puml> tags (not puml files, could you confirm?) worked for me, which makes sense as we don't have any other java apps
The high number of jvms is likely due to the high reuse nature of the site, which is expected.

I tried on the ay1920s1 version of the site as well, and there were also a number of jvms, although lesser (perhaps due to less reuse), which may be the reason why it appears to have only recently surfaced.

Will look into a fix

[damithc]

ok got it as well (had to expand to view the child processes under cmd.exe), though removing all <puml> tags (not puml files, could you confirm?) worked for me, which makes sense as we don't have any other java apps

You are right. It goes away if there are no puml tags. Good catch. I must have done it wrongly when I tried it last time.

Good if we can fix it though. We only have <10 puml diagrams at the moment but I was hoping to add more this semester. Live preview is still manageable using the lazy loading option but deploying is going to be a b*tch (as it needs a full build) when we have more puml diagrams.

[ang-zeyu]

no worries, should be a pretty simple fix 😀