Replace vscode built-ins by extension pack

[marcdumais-work]

What it does

This PR removes most of the vscode built-ins, that were previously listed individually, and replaces them with the built-in extensions pack, that we recently produced. That pack contains all built-ins, so Blueprint will now provide, OOTB, provide very similar feature as VS Code, including grammars that enable syntax highlight in many languages and e.g. themes.

There is one extension ATM that does not work as expected from the pack, that we pin to a specific version known to work better: vscode.git, and one extension that we exclude: vscode.extension-editing.

update: we added a second one, vscode.markdown-language-features, that we now pin to an earlier version, that's known to work well.

Also included in this PR is the switch from the Theia-specific git extension, @theia/git, to the vscode equivalent, vscode.git/vscode.git-ui. This brings Blueprint closer to VS Code, and enables using the very popular extension eamodio/vscode-gitlens.

Known problems

I think the below issues are now addressed - I'm keeping them for reference.

• Currently, the extensions, that are part of a pack, that's itself listed in the app's package.json, are resolved at runtime, when the user first starts the app. It takes a minute or two for all built-ins to download, and everything falls back on its feet.
  • We are working on a mechanism to optionally resolve extension packs at buildtime.
  • done - extensions part of a pack are resolved at built-time
• contrary to my expectation, a very recent version of built-ins, pulled through the pack, gets pulled: 1.57.0-next.0988e056b2d. I expected a version that about matches our vscode extensions API level, ~1.50.0 IIRC. Maybe some recent improvements are still on master, not part of @theia v1.13.0 that this draft PR currently uses.
  • Fixed on master. Thanks @vince-fugnitto
• extensions that are explicitly mentioned in package.json, like markdown-language-features are ignored at runtime, in favor of the version fetched for the pack. Maybe it's a matter of activation order? e.g. last activated wins? For whatever reason, the pack version wins, which breaks e.g. markdown language features
  • fixed - the issue was the label provided along with the extensions. It needs to match exactly publisher.id, else it will unzip in a different folder and it's a matter of luck which version is picked at startup.
• in packaged form, the built-in extensions pack currently installs the extensions under /tmp/vscode-unpacked. This means the extensions will potentially be installed more than onc[WIP] re
  • no longer the case since the extensions part of a pack are resolved at built-time - they end-up in the plugins folder, in an unzipped form.
• a few extensions do not currently behave correctly. We might need an exclusion mechanism, to black-list extensions that currently cause problems, so they are ignored.
  • not less needed ATM since we found working versions for all builtins (a few are pinned to older versions than the pack)
    • One extension, part of the vscode built-ins pack, is not starting correctly: vscode.extension-editing
      
    • -> fixed by use of Theia v1.17.1 that includes PR below, and adding an entry to ignore the misbehaving extension
    • PR: cli: introduce exclude id list for extension-packs theia#9956
• Until git: consider switching to builtin extensions #61 is done, the built-in git extension will potentially clash with Theia's
  • update: as suggested, I included this change in this PR. This takes care of the clash - it's now possible to install and use Gitlens

How to test

Build packages and verify that everything seem to work as well as before. Extensions that were known to be "sensitive" and that may benefit from a bit more tests are the ones that handle json and markdown files.

To confirm that more features are now available through the vscode built-in extensions pack, you can try to switch theme, using command preferences: color theme, and chose e.g. "dark (visual studio)".

Review checklist

• as an author, I have thoroughly tested my changes and carefully followed the review guidelines

Reminder for reviewers

• as a reviewer, I agree to behave in accordance with the review guidelines

[vince-fugnitto]

I confirmed (from bb168db) that the following pull-request eclipse-theia/theia#9486 fixes the issue related to vscode-builtins pulling versions which exceed the version specified by the API version (we should fetch the version that matches our API for vscode-builtins for the same reasons mentioned in the pull-request).

I verified for both:

• blueprint started with yarn start.
• blueprint packaged with yarn package (AppImage).

I verified with the best way I knew how, to reference my pull-request locally by updating:

https://github.com/eclipse-theia/theia-blueprint/blob/151e0e70f4f9a984f933856c6072f8dba38d9a88/applications/electron/package.json#L74

to: (local dependency reference specific to my environment)

"@theia/vsx-registry": "file:../../../theia/packages/vsx-registry/",

[marcdumais-work]

Thanks @vince-fugnitto . So we should be is pretty good shape, once we merge your referenced PR. Hopefully in time for theia 1.14.0 next week.

[marcdumais-work]

I will soon push a version with an updated yarn.lock, to benefit from this recently merged, relevant PR

[marcdumais-work]

I expect #157 will make it possible to exclude the one currently known misbehaving vscode built-in extension (vscode.extension-editing), pulled by the pack, making this PR finally merge-able.

[jfaltermeier]

Looks good to me. Git-Support, Json-Editing, Java, Typescript all worked fine in my small test.
One thing that surprised me was that the default association for markdown files was the preview for me. Is this expected?
Editing markdown files with Open with -> Code Editor works fine however.

[marcdumais-work]

Editing markdown files with Open with -> Code Editor works fine however.

That is a bit strange. So far, I can only open markdown source by explicitly doing "open-with -> code editor". I will investigate.

[marcdumais-work]

@jfaltermeier Markdown should behave better - I have pinned an older version that's known to work well. Before, we had two extensions providing markdown support - I have removed the one that's Theia-specific, so the one from vscode is always used. The removed one used to provide the "open-with" entries when an .md file was selected.

[marcdumais-work]

I have rebased to the latest master branch, to benefit from Theia v1.17.2.

[jfaltermeier]

👍

[marcdumais-work]

Thanks for the review @jfaltermeier

[marcdumais-work]

@brianking With this PR merged, we now have the full set of vscode built-in extensions (minus one or two we explicitly exclude), bundled in Blueprint. This is a nice milestone, feature-wise, I think.

This means we now have:

• a wide set of grammars, covering a wide array of programming and scripting languages, that will automatically provide "syntax highlighting"
  e.g. Windows/DOS batch files (.bat):
  

• Various new themes are now available, accessible from the command palette: "preferences: color theme"
  
  e.g. "Red":
  
  "Abyss":
  

See here, under section "bundled extensions", for the full list of built-in extensions that are part of the pack: https://open-vsx.org/extension/eclipse-theia/builtin-extension-pack/1.50.1

All this good content is available from the download page, version 1.17.2 and later.

[brianking]

@marcdumais-work This is great, and something we should be making noise about.

Would it make sense to wait for Beta 2 to be released, then we can do a blog post and / or some social posts?

[koegel]

@marcdumais-work This is great, and something we should be making noise about.

Would it make sense to wait for Beta 2 to be released, then we can do a blog post and / or some social posts?

Yes, it would make sense to wait for the beta 2 release imho. @jfaltermeier is planning on building and publishing the release on September 21st. It will also contain some interesting fixes for the update function.

[brianking]

Yes, it would make sense to wait for the beta 2 release imho. @jfaltermeier is planning on building and publishing the release on September 21st. It will also contain some interesting fixes for the update function.

Was Beta 2 officially released?

[koegel]

Sorry for the late reply, you probably know by now but just for the sake of completeness:
https://eclipsesource.com/blogs/2021/09/27/eclipse-theia-blueprint-beta-2-is-released/