WIP: Use grunt for assets, remove django pipeline, remove ruby, use bower for client-side deps…

[felipemontoya]

This PR was created at the Open edX Con hackathon. This PR continues the work already done in #6026.

Here’s a list of what it does:

• Use grunt for client-side assets management
• Use bower for client-side assets dependencies
• Remove the django pipeline
• Use libsass instead of ruby sass

Cool things:

• This PR allows you to run everything client-side related on the host (and not on the devstack).
• If ran on the host machine, compiles all LMS assets in 1.8 seconds.
• Remove ruby from edx-platform

TODO:

• Remove the JS pipeline
• Update font awesome to 4.3.0 (keep an eye on Upgrade: Font Awesome 4.3.0 #6875)

[openedx-webhooks]

Thanks for the pull request, @felipemontoya! I've created OSPR-363 to keep track of it in JIRA. JIRA is a place for product owners to prioritize feature reviews by the engineering development teams.

Feel free to add as much of the following information to the ticket:

• supporting documentation
• edx-code email threads
• timeline information ('this must be merged by XX date', and why that is)
• partner information ('this is a course on edx.org')
• any other information that can help Product understand the context for the PR

All technical communication about the code itself will still be done via the Github pull request interface. As a reminder, our process documentation is here.

[felipemontoya]

@tusbar, @sarina, @singingwolfboy, @benpatterson, @talbs and anyone else who is looking at this PR and I didn't ping. This is mainly a rebase of the previous work #6026.

I still have to update neat to 1.7.1 and the font awesome version. Then probably @tusbar will be ready to continue the work on the JS pipeline.

[sarina]

@felipemontoya - I am not really too sure how to start reviewing this, but perhaps I can work on helping get a passing build.

In the past two weeks we've had a lot of test infrastructure change, so you'll have to rebase in order for us to get a build that's even slightly useful, unfortunately.

Also I may be misremembering but I think @ormsbee is another person excited by this PR and perhaps can give a cycle to giving a look at it.

[sarina]

Also @felipemontoya fontawesome has been recently updated, see #6055 and #6875

[felipemontoya]

@sarina I just rebased to master and ran the test locally. The python test run fine and the js was missing a task to compile the assets using grunt so I created it. @tusbar are there plans to have a different build command for js after removing the js pipeline?

Now the js test suite runs, but there are some failing js test and I could use some help to figure them out.

[sarina]

@felipemontoya looks like there's still some problems with assets. See the console output of the failing shards, here's one for example:

15:38:10 ValueError: The file 'js/dashboard/donation.js' could not be found with 
<pipeline.storage.PipelineFinderStorage object at 0x574fa90>.

On the new Jenkins, you can get to the shard output by first clicking on "Build Graph", then clicking on the second box ("edx-platform-all-tests-pr #(prnum) (hash)")

[sarina]

@felipemontoya I also think the Javascript console output is worrying. It says that a bunch of files cannot be found and then it is just hanging.

[felipemontoya]

@sarina doing python manage.py lms --settings=bok_choy collectstatic --noinput > /dev/null already gets me the same results as the Jenkins shard, but running paver test_js does not. I don't get the missing files warning. So I will have to look deep into that. Maybe cleaning the repository will do. I'll give it a look.

[sarina]

Yeah it just looks like various Javascript files aren't being loaded in and so tests are failing because they can't find the required JS files.

[sarina]

I'm not positive how to resolve that. I can see if DB can provide some guidance - right now he's really busy, though, working on cutting the new Birch release, so he may not be free for another week or two.

[sarina]

Our pipeline is a total mess and we're beginning some concentrated effort to completely redo it, getting rid of django-pipeline in the process, in the coming 3 months.

With that said, there are a lot of wide-ranging implications around changing the pipeline. We will look at this body of work for inspiration, but will be starting on a new branch, so I am going to close this. When we have a pull request available, we will solicit input from the community on the mailing list.