Skip to content

Commit 1197274

Browse files
RishabhJain2018RishabhJain2018
authored
TravisCI and Frontend Update: Enhance TravisCI configuration for improved test reporting and coverage uploads (Cloud-CV#4780)
* Enhance TravisCI configuration for improved test reporting and coverage uploads - Updated frontend test command in .travis.yml to include npm install and additional reporters. - Modified backend test command to generate JUnit XML reports. - Added Codecov CLI installation and commands to upload test results for both backend and frontend. - Updated karma configuration to include JUnit reporter for frontend tests. * Add npm configuration file and update dependencies in package.json - Introduced a new .npmrc file for project-specific npm settings. - Updated various dependencies in package.json to their latest versions for improved performance and security. - Enhanced Dockerfile configurations for both development and production environments to optimize build processes. - Added Docker Compose Bake configuration for faster builds. - Refactored gulpfile.js for better readability and functionality. * Optimize npm configurations in Dockerfiles and .npmrc for improved performance - Updated .npmrc to enable progress and set loglevel to verbose. - Increased fetch timeout and retry settings for better reliability. - Enhanced Dockerfiles for both development and production environments with optimized npm configurations. - Ensured global and project dependencies are installed with progress and verbose logging for better visibility during builds. * Enhance CI configurations and Docker setup for improved testing and performance - Updated .npmrc for CI builds with optimized fetch settings and logging. - Modified .travis.yml to start Xvfb for headless Chrome testing. - Enhanced docker-compose.yml with Chrome environment variables and resource limits. - Improved karma.conf.js for better test execution in CI, including timeout adjustments. - Updated Dockerfile to set Chrome environment variables for Travis CI. * Refactor CI and Docker configurations for streamlined testing - Removed Xvfb startup from .travis.yml as it is no longer needed. - Eliminated healthcheck from docker-compose.yml to simplify service configuration. - Added xvfb package installation in Dockerfile to ensure availability for headless testing. * Update karma configuration to use chrome-launcher for Chrome executable path - Replaced puppeteer with chrome-launcher in karma.conf.js for improved Chrome path resolution. - Added chrome-launcher as a dependency in package.json. * Refactor ESLint and Docker configurations for improved consistency and performance - Updated .eslintrc to use 'readonly' for global variables for better clarity. - Modified gulpfile.js to ensure compatibility with the latest gulp-sass version. - Simplified Chrome executable path resolution in karma.conf.js by removing chrome-launcher dependency. - Updated package.json to downgrade 'del' dependency and adjust 'glob' and 'glob-stream' versions. - Enhanced Dockerfile configurations for both development and production environments, including the installation of additional packages and improved npm install commands. - Refactored SCSS files for better readability and consistency in styling.
1 parent 5bd0d87 commit 1197274

13 files changed

Lines changed: 422 additions & 272 deletions

File tree

‎.eslintrc‎

Lines changed: 22 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -3,27 +3,29 @@ eslint config file.
33
*/
44

55
{
6-
"env": {
7-
"es6": 1,
8-
"browser": 1
9-
},
6+
"env": {
7+
"es6": 1,
8+
"browser": 1
9+
},
1010

11-
"globals": {
12-
"angular":1,
13-
"jquery": 1,
14-
"$": 1,
15-
"_": 1,
16-
"moment": true
17-
},
11+
"globals": {
12+
"angular": "readonly",
13+
"jquery": "readonly",
14+
"$": "readonly",
15+
"_": "readonly",
16+
"moment": "readonly"
17+
},
1818

19-
"extends": "eslint:recommended",
19+
"extends": "eslint:recommended",
2020

21-
"rules": {
22-
"no-console": "warn",
23-
"semi": ["error",
24-
"always", {
25-
"omitLastInOneLineBlock": true
26-
}
27-
],
28-
},
21+
"rules": {
22+
"no-console": "warn",
23+
"semi": [
24+
"error",
25+
"always",
26+
{
27+
"omitLastInOneLineBlock": true
28+
}
29+
]
30+
}
2931
}

‎.npmrc‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# npm configuration for CI builds
2+
fetch-timeout=600000
3+
fetch-retry-mintimeout=20000
4+
fetch-retry-maxtimeout=120000
5+
progress=false
6+
loglevel=warn
7+
maxsockets=50
8+
prefer-offline=true
9+
audit=false
10+
fund=false

‎.travis.yml‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -42,11 +42,11 @@ jobs:
4242
- docker-compose --profile worker_py3_7 --profile worker_py3_8 --profile worker_py3_9 --profile statsd build || travis_terminate 1;
4343

4444
# Frontend Tests
45-
- docker-compose run nodejs bash -c "gulp dev && karma start --single-run && gulp staging" || travis_terminate 1;
45+
- docker-compose run nodejs bash -c "npm install && gulp dev && karma start --single-run --reporters=junit,coverage && gulp staging" || travis_terminate 1;
4646

4747
# Backend Tests
4848
- docker-compose run -e DJANGO_SETTINGS_MODULE=settings.test django python manage.py flush --noinput || travis_terminate 1;
49-
- docker-compose run -e DJANGO_SETTINGS_MODULE=settings.test django pytest --cov . --cov-config .coveragerc --cov-report=xml:coverage-backend.xml || travis_terminate 1;
49+
- docker-compose run -e DJANGO_SETTINGS_MODULE=settings.test django pytest --cov . --cov-config .coveragerc --cov-report=xml:coverage-backend.xml --junitxml=junit-backend.xml -o junit_family=legacy || travis_terminate 1;
5050

5151
# Check Code Quality
5252
- docker-compose run -e DJANGO_SETTINGS_MODULE=settings.dev -e VERBOSE=1 django bash -c "
@@ -63,6 +63,15 @@ jobs:
6363
echo 'All code quality checks passed!'" || travis_terminate 1;
6464

6565
after_success:
66+
# Install Codecov CLI
67+
- pip install codecov-cli
68+
69+
# Upload backend test results to Codecov
70+
- codecovcli do-upload --report-type test_results --file junit-backend.xml
71+
72+
# Upload frontend test results to Codecov (if available)
73+
- codecovcli do-upload --report-type test_results --file coverage/frontend/TEST-*.xml || true
74+
6675
# Upload backend coverage to Codecov
6776
- bash <(curl -s https://codecov.io/bash) -f coverage-backend.xml -F backend
6877

‎docker-compose.yml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,13 +73,23 @@ services:
7373
dockerfile: docker/dev/nodejs/Dockerfile
7474
environment:
7575
NODE_ENV: development
76+
CHROME_BIN: /usr/bin/google-chrome
77+
DISPLAY: :99.0
7678
ports:
7779
- "8888:8888"
7880
- "35729:35729"
7981
volumes:
8082
- .:/code
8183
- /code/node_modules
8284
- /code/bower_components
85+
deploy:
86+
resources:
87+
limits:
88+
memory: 2G
89+
cpus: '1.0'
90+
reservations:
91+
memory: 1G
92+
cpus: '0.5'
8393

8494
statsd-exporter:
8595
hostname: statsd

‎docker/dev/docker.env‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,9 @@
1+
# Docker Compose Bake configuration for faster builds
2+
COMPOSE_BAKE=true
3+
COMPOSE_BAKE_ARGS="--build-arg PIP_NO_CACHE_DIR=1"
4+
DOCKER_BUILDKIT=1
5+
COMPOSE_DOCKER_CLI_BUILD=1
6+
17
AWS_ACCESS_KEY_ID=x
28
AWS_SECRET_ACCESS_KEY=x
39
AWS_DEFAULT_REGION=us-east-1

‎docker/dev/nodejs/Dockerfile‎

Lines changed: 51 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,47 +1,76 @@
11
# Base stage with common dependencies
2-
FROM node:14.21.3-bullseye AS base
2+
FROM node:18.19.0-bullseye AS base
3+
4+
# Set environment variables for better npm performance
5+
ENV NODE_ENV=production
6+
ENV NPM_CONFIG_LOGLEVEL=verbose
7+
ENV NPM_CONFIG_PROGRESS=true
8+
ENV NPM_CONFIG_FETCH_TIMEOUT=600000
9+
ENV NPM_CONFIG_FETCH_RETRY_MINTIMEOUT=20000
10+
ENV NPM_CONFIG_FETCH_RETRY_MAXTIMEOUT=120000
311

412
# Install Python2, symlink, and C++ build tools for node-gyp/node-sass
513
RUN apt-get update && \
614
apt-get install -y --no-install-recommends \
715
python2 \
816
python-is-python2 \
917
make \
10-
g++ && \
18+
g++ \
19+
curl \
20+
wget \
21+
gnupg \
22+
ca-certificates \
23+
xvfb && \
1124
rm -rf /var/lib/apt/lists/*
1225

1326
WORKDIR /code
1427

15-
# Add dependencies
16-
ADD ./package.json /code
17-
ADD ./bower.json /code
18-
ADD ./gulpfile.js /code
19-
ADD ./.eslintrc /code
20-
ADD ./karma.conf.js /code
21-
22-
# Install Prerequisites
23-
RUN npm install -g bower gulp gulp-cli
24-
RUN npm link gulp
25-
RUN npm cache clean -f
26-
RUN npm install
27-
RUN npm install -g karma-cli
28-
RUN npm install -g qs
29-
RUN bower install --allow-root
28+
# Copy package files first for better caching
29+
COPY package.json bower.json gulpfile.js .eslintrc karma.conf.js .npmrc ./
30+
31+
# Configure npm with better settings for Docker builds
32+
RUN npm config set fetch-timeout 600000 && \
33+
npm config set fetch-retry-mintimeout 20000 && \
34+
npm config set fetch-retry-maxtimeout 120000 && \
35+
npm config set progress true && \
36+
npm config set loglevel verbose && \
37+
npm config set maxsockets 50
38+
39+
# Install global dependencies first with optimizations and progress
40+
RUN npm install -g bower gulp gulp-cli karma-cli qs --no-audit --no-fund --progress --verbose && \
41+
npm link gulp && \
42+
npm cache clean -f
43+
44+
# Install project dependencies with optimizations and progress
45+
RUN npm install --only=production --no-audit --no-fund --prefer-offline --progress --verbose
3046

3147
# AMD64-specific stage
3248
FROM base AS amd64
33-
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
34-
RUN sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list'
35-
RUN apt-get update && apt-get install -yq google-chrome-stable libxss1
49+
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - && \
50+
sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' && \
51+
apt-get update && apt-get install -yq google-chrome-stable libxss1 && \
52+
rm -rf /var/lib/apt/lists/*
3653

3754
# ARM64-specific stage
3855
FROM base AS arm64
39-
RUN apt-get update && apt-get install -yq chromium libxss1
40-
RUN ln -sf /usr/bin/chromium /usr/bin/google-chrome
56+
RUN apt-get update && apt-get install -yq chromium libxss1 && \
57+
ln -sf /usr/bin/chromium /usr/bin/google-chrome && \
58+
rm -rf /var/lib/apt/lists/*
4159

4260
# Final stage - automatically selects the right architecture
4361
FROM ${TARGETARCH:-amd64}
4462

63+
# Set Chrome environment variables for Travis CI
64+
ENV CHROME_BIN=/usr/bin/google-chrome
65+
ENV CHROME_PATH=/usr/bin/google-chrome
66+
ENV DISPLAY=:99.0
67+
68+
# Copy source code
69+
COPY . /code/
70+
71+
# Install bower components after source code is copied
72+
RUN bower install --allow-root
73+
4574
# Set common command and expose port
4675
CMD ["gulp", "dev:runserver"]
4776
EXPOSE 8888

‎docker/prod/nodejs/Dockerfile‎

Lines changed: 42 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
FROM node:14.21.3-bullseye AS base
1+
FROM node:18.19.0-bullseye
22

33
ARG NODE_ENV
44

@@ -8,33 +8,57 @@ RUN apt-get update && \
88
python2 \
99
python-is-python2 \
1010
make \
11-
g++ && \
11+
g++ \
12+
curl \
13+
wget \
14+
gnupg \
15+
ca-certificates \
16+
xvfb && \
1217
rm -rf /var/lib/apt/lists/*
1318

1419
WORKDIR /code
1520

16-
# Add dependencies
17-
ADD ./package.json /code
18-
ADD ./bower.json /code
19-
ADD ./gulpfile.js /code
20-
ADD ./.eslintrc /code
21-
ADD ./karma.conf.js /code
22-
23-
# Install Prerequisites
24-
RUN npm install -g bower gulp gulp-cli
25-
RUN npm install phantomjs-prebuilt
26-
RUN npm link gulp
27-
RUN npm cache clean -f
28-
RUN npm install
29-
RUN npm install -g qs
21+
# Copy package files first for better caching
22+
COPY package.json bower.json gulpfile.js .eslintrc karma.conf.js .npmrc ./
23+
24+
# Configure npm with better settings for Docker builds
25+
RUN npm config set fetch-timeout 600000 && \
26+
npm config set fetch-retry-mintimeout 20000 && \
27+
npm config set fetch-retry-maxtimeout 120000 && \
28+
npm config set progress true && \
29+
npm config set loglevel verbose && \
30+
npm config set maxsockets 50
31+
32+
# Install global dependencies
33+
RUN npm install -g bower gulp gulp-cli karma-cli qs --no-audit --no-fund --progress --verbose && \
34+
npm link gulp && \
35+
npm cache clean -f
36+
37+
# Install project dependencies with optimizations
38+
RUN npm install --only=production --no-audit --no-fund --prefer-offline --progress --verbose
39+
40+
# Copy frontend source
41+
COPY frontend /code/frontend
42+
43+
# Install bower components after source code is copied
3044
RUN bower install --allow-root
31-
ADD frontend /code/frontend
45+
46+
# Install Chrome for testing (if needed)
47+
RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - && \
48+
sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' && \
49+
apt-get update && apt-get install -yq google-chrome-stable libxss1 && \
50+
rm -rf /var/lib/apt/lists/*
51+
52+
# Set Chrome environment variables for production
53+
ENV CHROME_BIN=/usr/bin/google-chrome
54+
ENV CHROME_PATH=/usr/bin/google-chrome
55+
ENV DISPLAY=:99.0
3256

3357
RUN gulp ${NODE_ENV}
3458

3559
FROM nginx:1.13-alpine
3660
# Adding NODE_ENV here as well since this is a multistage build
3761
ARG NODE_ENV
3862
COPY docker/prod/nodejs/nginx_${NODE_ENV}.conf /etc/nginx/conf.d/default.conf
39-
COPY --from=base /code /code
63+
COPY --from=0 /code /code
4064
COPY /ssl /etc/ssl

0 commit comments

Comments
 (0)