improve local content repository development

[tripodsan]

it should be possible to quickly render content from the local git repository, if it matches the current origin.

for example:

strains:
  - name: default
    content:  https://github.com/adobe/developer.adobe.com.git#master
    code:  https://github.com/adobe/developer.adobe.com.git#master
    static:  https://github.com/adobe/developer.adobe.com.git/htdocs#master

When running hlx up in a local checkout of https://github.com/adobe/developer.adobe.com.git, the content and static content should be served from the local repo instead from github.

Also, it should be possible to disable this feature, or even specify more repositories that should be emulated.

Suggest

hlx up will have a new flag --git or --git-serve or --mount or --repo which takes 1 directory argument that must point to a local checkout of a content repository. hlx up will then start a git-server on those directory and map all repository references in the strains.

[tripodsan]

Example

1. checkout 2 repositories, hlxtest and hlxtest-content
2. using this config:

definitions:
  repos:
    - &defaultRepo https://github.com/tripodsan/hlxtest.git#master
    - &contentRepo https://github.com/tripodsan/hlxtest-content.git#master

strains:
  - name: api
    url: https://www.99productrules.com/api/general
    code: *defaultRepo
    content: *contentRepo
    static: *defaultRepo

  - name: default
    url: https://www.99productrules.com
    code: *defaultRepo
    content: *defaultRepo
    static: *defaultRepo

3. start the server like this:

$ hlx up --log-level=debug --host=www.99productrules.com --local-repo=. --local-repo=../hlxtest-content

now http://localhost:3000/index.html is served from the local checkout of hlxtest and
http://localhost:3000/api/general/README.html is served from the local checkout of hlxtest-content:

[hlx] debug: git emulating ssh://git@github.com/tripodsan/hlxtest.git via http://127.0.0.1:52270/helix/github.com--tripodsan--hlxtest.git#master of ./
[hlx] debug: git emulating ssh://git@github.com/tripodsan/hlxtest-content.git via http://127.0.0.1:52270/helix/github.com--tripodsan--hlxtest-content.git#master of ../hlxtest-content
...
[hlx] debug: fetching Markdown from http://127.0.0.1:52270/raw/helix/github.com--tripodsan--hlxtest/master/index.md
...
[hlx] debug: fetching Markdown from http://127.0.0.1:52270/raw/helix/github.com--tripodsan--hlxtest-content/master/README.md
...

[tripodsan]

see some docu in https://github.com/adobe/helix-cli/blob/mountable-git/README.md#development---serving-local-content

[stefan-guggisberg]

Another option would be to start git-server manually to serve any local repositories you want and then specify the localhostrepo urls in helix-config.yaml.

[koraa]

@tripodsan That seems like a very complicated commandline o start a server for local development.
I think that the use case of hlx up is pretty much exclusively serving local content (possibly from a dirty git tree) before actually publishing, so isn't there a way to accomplish this using just hlx up? Maybe using a flag in the config or something…

We could also consider just using submodules in some way to represent the other repositories? Or just any way to make sure the resources from all repositories are available locally?

[tripodsan]

Another option would be to start git-server manually to serve any local repositories you want and then specify the localhostrepo urls in helix-config.yaml.

this is exactly what we want to avoid :-) the developer should not need to alter the config just for local development. also, explaining the git-server config would be yet-another complication.

[tripodsan]

so isn't there a way to accomplish this using just hlx up? Maybe using a flag in the config or something…

well, you can use .env, as explained in https://github.com/adobe/helix-cli/blob/mountable-git/README.md#development---serving-local-content :

HLX_HOST=www.project-helix.io
HLX_LOCAL_REPO=., ../welcome-api
HLX_LOG_LEVEL=debug

We could also consider just using submodules in some way to represent the other repositories?

and then? you still have the problem that the content url in a strain points to a github repository.

[kptdobe]

I like the idea, we just need to make sure it aligns on the "development" process and progression and to "the strain concept":

1. when a developer starts, no helix-config, hlx up takes the local content + code (no need for params)
2. on first hlx deploy / publish, he gets a first strain that he needs to understand and setup properly. helix-config contains now a "publish strain" which points to whatever combination of content repo + code repo.
3. then back locally, he wants then to continue coding / change content and this is where he needs "something" where you proposal would be needed.

Whatever transition from 1 to 2 and from 2 to 3 required, we should give to the developer a setup that allows to simply go back to local development (without or without local content).

Also, previously, we said we could create "local" strains which obviously disappeared (at least from https://github.com/adobe/developer.adobe.com/blob/master/helix-config.yaml), which I think is good to avoid the multiplication of strains.

The only thing I do not like: --local-repo.

Why local-repo ? If the goal of this param is to allow me to override the urls in the repo section, I would expect to be able to specify a git url too (like I want to test my local setup with the local code repo and another git repo). It happened to me several times actually, testing a code repo with a random git repo. I had to edit the helix-config.yaml file.
Also, what if I only want to override the 2 repo property... ?
I would prefer something like --repo.defaultRepo=. --repo.contentRepo=../hlxtest-content just to be more explicit (less magic).

[filmaj]

Agree with @kptdobe. Also, --repo.defaultRepo=. almost feels like it should be an implicit default?

[tripodsan]

I would prefer something like --repo.defaultRepo=. --repo.contentRepo=../hlxtest-content just to be more explicit (less magic).

but this is not how it works. all it does, it emulates (hijacks, if you will) a github repository. this is simple to understand. and you have a checkout of that repo, so you point to it, and it will be emulated.

also, why would it know about contentRepo or defaultRepo? those are arbitrary aliases in the config.

If the goal of this param is to allow me to override the urls in the repo section,

no, it isn't. it is about using local git repositories instead of github.

[tripodsan]

then back locally, he wants then to continue coding / change content and this is where he needs "something" where you proposal would be needed.

yes. this would be done by having --local-repo=..

Also, --repo.defaultRepo=. almost feels like it should be an implicit default?

@filmaj maybe, but then, how do I turn it off? --no-local-repo=. ? I think it is important to also be able to test the remote content.

alternative names

• --git-serve
• --github-for
• --hijack
• --intercept

One additional feature that is missing would be the ability to override the origin. with the --local-repo you only specify the directory, and it uses the origin to define the intercepted github repository. but if you want to test a local checkout for a different repository, without fiddling with the remote origin, we could do something like:

hlx up --local-repo= ./:https://github.com/adobe/project-helix-alternative.git

then a strain with the content/static url of https://github.com/adobe/project-helix-alternative.git would be hijacked to serve the local content from ./

[kptdobe]

I do not think "this is simple to understand". Especially with your latest comments, I do not get it anymore. This feels like tweaking the system while as an end-user, I need something more abstract and aligned to the strains concept or what I have defined in my helix-config.yaml.

also, why would it know about contentRepo or defaultRepo? those are arbitrary aliases in the config.

This is the point, as a user, I define those aliases, I know them, I know what they mean... I just want to "temporarily" override them.

the ability to override the origin

As an end-user, I do not want to override the origin. The origin trick was already defined multiple times as a mysterious and confusing fallback ;)

From your example: --host=www.99productrules.com --local-repo=. --local-repo=../hlxtest-content

how does the thing determine that the first local-repo is for codeRepo git url and the second is for the contentRepo ?