From 19e90446f71a7a439f5270be8532ed2af1ae9da0 Mon Sep 17 00:00:00 2001 From: Janardhan Pulivarthi Date: Sun, 31 Oct 2021 11:19:43 +0530 Subject: [PATCH] Add preliminary changes as main branch --- .asf.yaml | 4 ++-- .github/workflows/applicationTests.yml | 4 ++-- .github/workflows/build.yml | 4 ++-- .github/workflows/componentTests.yml | 4 ++-- .github/workflows/documentation.yml | 4 ++-- .github/workflows/functionsTests.yml | 4 ++-- .github/workflows/python.yml | 4 ++-- README.md | 14 +++++++------- docker/testsysds.Dockerfile | 2 +- docs/_includes/header.html | 2 +- .../api/python/sources/guide/federated.rst.txt | 2 +- docs/css/bootstrap.min.css | 4 ++-- docs/index.md | 2 +- docs/site/algorithms-factorization-machines.md | 4 ++-- docs/site/windows-source-installation.md | 4 ++-- pom.xml | 2 +- scripts/staging/google-cloud/README.md | 4 ++-- scripts/staging/onnx/onnx_systemds/README.md | 2 +- .../onnx/onnx_systemds_docs/onnx_systemds.rst | 2 +- .../onnx_systemds_design.rst | 18 +++++++++--------- src/assembly/bin/README.md | 2 +- .../python/docs/source/guide/federated.rst | 2 +- 22 files changed, 47 insertions(+), 47 deletions(-) diff --git a/.asf.yaml b/.asf.yaml index e9a4c8201e4..297788062ae 100644 --- a/.asf.yaml +++ b/.asf.yaml @@ -49,12 +49,12 @@ github: protected_branches: - master: + main: # Do not allow merge commits # by allowing linear history required_linear_history: true staging: profile: beta - whoami: master + whoami: main outputdir: docs diff --git a/.github/workflows/applicationTests.yml b/.github/workflows/applicationTests.yml index ddcb85e3bda..d1fdcb80b10 100644 --- a/.github/workflows/applicationTests.yml +++ b/.github/workflows/applicationTests.yml @@ -30,7 +30,7 @@ on: - 'src/main/python/docs/**' - 'dev/**' branches: - - master + - main pull_request: paths-ignore: - 'docs/**' @@ -39,7 +39,7 @@ on: - 'src/main/python/docs/**' - 'dev/**' branches: - - master + - main jobs: applicationsTests: diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index b80f4feede2..4fe60b6e086 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -30,7 +30,7 @@ on: - 'src/main/python/docs/**' - 'dev/**' branches: - - master + - main pull_request: paths-ignore: - 'docs/**' @@ -39,7 +39,7 @@ on: - 'src/main/python/docs/**' - 'dev/**' branches: - - master + - main jobs: build: diff --git a/.github/workflows/componentTests.yml b/.github/workflows/componentTests.yml index b729d49bc8e..276a27804f9 100644 --- a/.github/workflows/componentTests.yml +++ b/.github/workflows/componentTests.yml @@ -30,7 +30,7 @@ on: - 'src/main/python/docs/**' - 'dev/**' branches: - - master + - main pull_request: paths-ignore: - 'docs/**' @@ -39,7 +39,7 @@ on: - 'src/main/python/docs/**' - 'dev/**' branches: - - master + - main jobs: componentTests: diff --git a/.github/workflows/documentation.yml b/.github/workflows/documentation.yml index 81ca26acda4..848224c0dec 100644 --- a/.github/workflows/documentation.yml +++ b/.github/workflows/documentation.yml @@ -27,13 +27,13 @@ on: - '*.html' - 'dev/**' branches: - - master + - main pull_request: paths-ignore: - '*.html' - 'dev/**' branches: - - master + - main jobs: documentation1: diff --git a/.github/workflows/functionsTests.yml b/.github/workflows/functionsTests.yml index 5838d7e3527..ae9a1cd9f5b 100644 --- a/.github/workflows/functionsTests.yml +++ b/.github/workflows/functionsTests.yml @@ -30,7 +30,7 @@ on: - 'src/main/python/docs/**' - 'dev/**' branches: - - master + - main pull_request: paths-ignore: - 'docs/**' @@ -39,7 +39,7 @@ on: - 'src/main/python/docs/**' - 'dev/**' branches: - - master + - main jobs: applicationsTests: diff --git a/.github/workflows/python.yml b/.github/workflows/python.yml index 99b40f1299d..5106063063b 100644 --- a/.github/workflows/python.yml +++ b/.github/workflows/python.yml @@ -30,7 +30,7 @@ on: - 'src/main/python/docs/**' - 'dev/**' branches: - - master + - main pull_request: paths-ignore: - 'docs/**' @@ -39,7 +39,7 @@ on: - 'src/main/python/docs/**' - 'dev/**' branches: - - master + - main jobs: applicationsTests: diff --git a/README.md b/README.md index 963aafea9cb..c48f03cc02b 100644 --- a/README.md +++ b/README.md @@ -39,10 +39,10 @@ heterogeneous and nested schema. **Status and Build:** SystemDS is renamed from SystemML which is an **Apache Top Level Project**. To build from source visit [SystemDS Install from source](https://apache.github.io/systemds/site/install.html) -[![Build](https://github.com/apache/systemds/workflows/Build/badge.svg?branch=master&event=push)](https://github.com/apache/systemds/actions?query=workflow%3A%22Build%22+branch%3Amaster+event%3Apush) -[![Documentation](https://github.com/apache/systemds/workflows/Documentation/badge.svg?branch=master&event=push)](https://github.com/apache/systemds/actions?query=workflow%3ADocumentation+branch%3Amaster+event%3Apush) -[![Component Test](https://github.com/apache/systemds/workflows/Component%20Test/badge.svg?branch=master&event=push)](https://github.com/apache/systemds/actions?query=workflow%3A%22Component+Test%22+branch%3Amaster+event%3Apush) -[![Application Test](https://github.com/apache/systemds/workflows/Application%20Test/badge.svg?branch=master&event=push)](https://github.com/apache/systemds/actions?query=workflow%3A%22Application+Test%22+branch%3Amaster+event%3Apush) -[![Function Test](https://github.com/apache/systemds/workflows/Function%20Test/badge.svg?branch=master&event=push)](https://github.com/apache/systemds/actions?query=workflow%3A%22Function+Test%22+branch%3Amaster+event%3Apush) -[![Python Test](https://github.com/apache/systemds/workflows/Python%20Test/badge.svg?branch=master&event=push)](https://github.com/apache/systemds/actions?query=workflow%3A%22Python+Test%22+branch%3Amaster+event%3Apush) -[![Federated Python Test](https://github.com/apache/systemds/workflows/Federated%20Python%20Test/badge.svg?branch=master&event=push)](https://github.com/apache/systemds/actions?query=workflow%3A%22Federated+Python+Test%22+branch%3Amaster+event%3Apush) +[![Build](https://github.com/apache/systemds/workflows/Build/badge.svg?branch=main&event=push)](https://github.com/apache/systemds/actions?query=workflow%3A%22Build%22+branch%3Amain+event%3Apush) +[![Documentation](https://github.com/apache/systemds/workflows/Documentation/badge.svg?branch=main&event=push)](https://github.com/apache/systemds/actions?query=workflow%3ADocumentation+branch%3Amain+event%3Apush) +[![Component Test](https://github.com/apache/systemds/workflows/Component%20Test/badge.svg?branch=main&event=push)](https://github.com/apache/systemds/actions?query=workflow%3A%22Component+Test%22+branch%3Amain+event%3Apush) +[![Application Test](https://github.com/apache/systemds/workflows/Application%20Test/badge.svg?branch=main&event=push)](https://github.com/apache/systemds/actions?query=workflow%3A%22Application+Test%22+branch%3Amain+event%3Apush) +[![Function Test](https://github.com/apache/systemds/workflows/Function%20Test/badge.svg?branch=main&event=push)](https://github.com/apache/systemds/actions?query=workflow%3A%22Function+Test%22+branch%3Amain+event%3Apush) +[![Python Test](https://github.com/apache/systemds/workflows/Python%20Test/badge.svg?branch=main&event=push)](https://github.com/apache/systemds/actions?query=workflow%3A%22Python+Test%22+branch%3Amain+event%3Apush) +[![Federated Python Test](https://github.com/apache/systemds/workflows/Federated%20Python%20Test/badge.svg?branch=main&event=push)](https://github.com/apache/systemds/actions?query=workflow%3A%22Federated+Python+Test%22+branch%3Amain+event%3Apush) diff --git a/docker/testsysds.Dockerfile b/docker/testsysds.Dockerfile index 161b86a1896..5d0ac56b8b0 100644 --- a/docker/testsysds.Dockerfile +++ b/docker/testsysds.Dockerfile @@ -22,7 +22,7 @@ FROM ubuntu:20.04 # Install Maven -# Credit https://github.com/Zenika/alpine-maven/blob/master/jdk8/Dockerfile +# Credit https://github.com/Zenika/alpine-maven/blob/7623e76e95af5973fe8397a9cabf17c4eb931ec1/jdk8/Dockerfile # InstallR Guide: https://cran.r-project.org/ WORKDIR /usr/src/ diff --git a/docs/_includes/header.html b/docs/_includes/header.html index 9b91a67095c..2cbdc35d8a0 100644 --- a/docs/_includes/header.html +++ b/docs/_includes/header.html @@ -55,7 +55,7 @@
  • ML Algorithms Reference
  • Other:
    • -
  • Contributing to SystemDS 🡕
    • +
  • Contributing to SystemDS 🡕
  • GitHub 🡕
    • diff --git a/docs/api/python/sources/guide/federated.rst.txt b/docs/api/python/sources/guide/federated.rst.txt index 237214fb65e..051ad03ae83 100644 --- a/docs/api/python/sources/guide/federated.rst.txt +++ b/docs/api/python/sources/guide/federated.rst.txt @@ -32,7 +32,7 @@ Start Federated worker To start a federated worker, you first have to setup your environment variables. A simple guide to do this is in the SystemDS Repository_. -.. _Repository: https://github.com/apache/systemds/tree/master/bin/ +.. _Repository: https://github.com/apache/systemds/tree/main/bin/ If that is setup correctly simply start a worker using the following command. Here the ``8001`` refer to the port used by the worker. diff --git a/docs/css/bootstrap.min.css b/docs/css/bootstrap.min.css index ac5561a0505..a445f551d5d 100644 --- a/docs/css/bootstrap.min.css +++ b/docs/css/bootstrap.min.css @@ -1,7 +1,7 @@ /*! * Bootstrap v3.3.6 (http://getbootstrap.com) * Copyright 2011-2015 Twitter, Inc. - * Licensed under MIT (https://github.com/twbs/bootstrap/blob/master/LICENSE) + * Licensed under MIT (https://github.com/twbs/bootstrap/blob/main/LICENSE) */ @@ -231,7 +231,7 @@ th { } -/*! Source: https://github.com/h5bp/html5-boilerplate/blob/master/src/css/main.css */ +/*! Source: https://github.com/h5bp/html5-boilerplate/blob/main/dist/css/style.css */ @media print { *, diff --git a/docs/index.md b/docs/index.md index 2db46fbcbc2..3e543b94433 100644 --- a/docs/index.md +++ b/docs/index.md @@ -43,5 +43,5 @@ Various forms of documentation for SystemDS are available. - Instructions on python can be found at [Python Documentation](./api/python/index) - The [JavaDOC](./api/java/index) contains internal documentation of the system source code. - [Install from Source](./site/install) guides through setup from git download to running system. -- If you want to contribute take a look at [Contributing](https://github.com/apache/systemds/blob/master/CONTRIBUTING.md) +- If you want to contribute take a look at [Contributing](https://github.com/apache/systemds/blob/main/CONTRIBUTING.md) - [R to DML](./site/dml-vs-r-guide) walks through the basics of converting a script from R to dml. diff --git a/docs/site/algorithms-factorization-machines.md b/docs/site/algorithms-factorization-machines.md index a6e6cc2a206..373df0b690e 100644 --- a/docs/site/algorithms-factorization-machines.md +++ b/docs/site/algorithms-factorization-machines.md @@ -136,7 +136,7 @@ criterion is the minimal least square error on $D$. ### Usage -The `train()` function in the [fm-regression.dml](https://github.com/apache/systemml/blob/master/scripts/staging/fm-regression.dml) script, takes in the input variable matrix and the corresponding target vector with some input kept for validation during training. +The `train()` function in the [fm-regression.dml](https://github.com/apache/systemml/blob/main/scripts/staging/fm-regression.dml) script, takes in the input variable matrix and the corresponding target vector with some input kept for validation during training. ``` java train = function(matrix[double] X, matrix[double] y, matrix[double] X_val, matrix[double] y_val) @@ -196,7 +196,7 @@ loss or logit loss. ### Usage -The `train` function in the [fm-binclass.dml](https://github.com/apache/systemml/blob/master/scripts/staging/fm-binclass.dml) +The `train` function in the [fm-binclass.dml](https://github.com/apache/systemml/blob/main/scripts/staging/fm-binclass.dml) script, takes in the input variable matrix and the corresponding target vector with some input kept for validation during training. This script also contain `train()` and `predict()` function as in the case of regression. diff --git a/docs/site/windows-source-installation.md b/docs/site/windows-source-installation.md index 5e25b512af3..d503c0100b1 100644 --- a/docs/site/windows-source-installation.md +++ b/docs/site/windows-source-installation.md @@ -58,13 +58,13 @@ Ex. Corresponding to the CUDA version, `C:\Program Files\cudnn-9.0-windows10-x64 ### Getting Apache SystemDS Source Code SystemDS source code is available from [github.com/apache/systemds](https://github.com/apache/systemds) by either cloning or -downloading a zip file (based on a branch) into ``. The default is the *master* branch. +downloading a zip file (based on a branch) into ``. The default is the *main* branch. ```bash git clone https://github.com/apache/systemds systemds ``` -The master branch contains the source code which will be used to create the next major version of Apache SystemDS. +The main branch contains the source code which will be used to create the next major version of Apache SystemDS. _**Speed Tip:**_ If the complete repository history isn't needed then using a shallow clone (`git clone --depth 1`) will save significant time. diff --git a/pom.xml b/pom.xml index 4aeb4f540f5..e31cfe8a47c 100644 --- a/pom.xml +++ b/pom.xml @@ -1114,7 +1114,7 @@ 1.6 - + com.google.protobuf protobuf-java diff --git a/scripts/staging/google-cloud/README.md b/scripts/staging/google-cloud/README.md index 5f40f748e44..06a36dd87ef 100644 --- a/scripts/staging/google-cloud/README.md +++ b/scripts/staging/google-cloud/README.md @@ -93,7 +93,7 @@ Note: `workerConfig` and `secondaryWorkerConfig` will be present. ### SSH into the cluster -SSH into the cluster (master node) would provide fine grained control of the cluster. +SSH into the cluster (primary node) would provide fine grained control of the cluster. ```sh gcloud compute ssh ${CLUSTERNAME}-m --zone=us-central1-c @@ -109,7 +109,7 @@ For example, gcloud dataproc clusters list --region=us-central1 ``` -to exit the cluster master instance +to exit the cluster primary instance ```sh logout diff --git a/scripts/staging/onnx/onnx_systemds/README.md b/scripts/staging/onnx/onnx_systemds/README.md index 5b855ded8c9..42a8fce2116 100644 --- a/scripts/staging/onnx/onnx_systemds/README.md +++ b/scripts/staging/onnx/onnx_systemds/README.md @@ -19,4 +19,4 @@ limitations under the License. # onnx-systemds -A tool for importing/exporting [onnx](https://github.com/onnx/onnx/blob/master/docs/IR.md) graphs into/from SystemDS DML scripts. +A tool for importing/exporting [onnx](https://github.com/onnx/onnx/blob/main/docs/IR.md) graphs into/from SystemDS DML scripts. diff --git a/scripts/staging/onnx/onnx_systemds_docs/onnx_systemds.rst b/scripts/staging/onnx/onnx_systemds_docs/onnx_systemds.rst index 78fda902cb6..b0d6b63f1dc 100644 --- a/scripts/staging/onnx/onnx_systemds_docs/onnx_systemds.rst +++ b/scripts/staging/onnx/onnx_systemds_docs/onnx_systemds.rst @@ -29,7 +29,7 @@ Prerequisites to run onnx-systemds you need to: - install `onnx `_: `Installation instructions `_ -- `set up the environment `_ +- `set up the environment `_ Usage ------ diff --git a/scripts/staging/onnx/onnx_systemds_docs/onnx_systemds_design.rst b/scripts/staging/onnx/onnx_systemds_docs/onnx_systemds_design.rst index acc15c68c0b..1d07800efe2 100644 --- a/scripts/staging/onnx/onnx_systemds_docs/onnx_systemds_design.rst +++ b/scripts/staging/onnx/onnx_systemds_docs/onnx_systemds_design.rst @@ -25,18 +25,18 @@ Design This document describes the initial design of `onnx-systemds` For dealing with different operator-set versions of onnx the current strategy is to use the -`converter provided by onnx `_ to convert to a common version. +`converter provided by onnx `_ to convert to a common version. However, the converter does not support adapters for all op-sets/operators so this conversion will fail for many models. On the onnx repository you can find a list of -`currently supported adapters `_ +`currently supported adapters `_ Goals ----- - - Support for importing `operators of the ONNX base definition `_ - - Support for importing `operators defined by ONNX-ML `_ + - Support for importing `operators of the ONNX base definition `_ + - Support for importing `operators defined by ONNX-ML `_ - Support for exporting DML script to ONNX graphs @@ -50,11 +50,11 @@ Onnx - Operators Onnx includes several very simple and also more complex operators. When implementing an operator it's best to have a look at the -`operator schemas `_, +`operator schemas `_, which precisely define the inputs, outputs and attributes of the operation. Besides the standard onnx definition, there also exists onnx-ML the operator schemas for which are defined in a -`separate document `_. +`separate document `_. It is an extension of the standard onnx format, however currently only onnx standard operators are supported. Onnx - Files @@ -62,7 +62,7 @@ Onnx - Files Onnx uses the `ProtoBuf format `_. It specifies this representation in several ``.proto``/``.proto3`` -`files `_ again with dedicated files for onnx-ML. +`files `_ again with dedicated files for onnx-ML. These files are helpful to understand the underlying structure and values that are possible. Protobuf creates the underlying structure such that you can access elements of the onnx graph as if they were @@ -167,7 +167,7 @@ Implementing new operators ---------------------------- When implementing an operator it's best to have a look at the -`operator schemas `_ +`operator schemas `_ which exactly define the inputs, outputs and attributes of the operation It is also nice to have a test-model to work with, to generate one refer to @@ -184,7 +184,7 @@ Testing models --------------- onnx provides a convenient way for -`creating models `_ +`creating models `_ using helper functions in python. All current test-models are produced like this (see ``tests/onnx/test_models``). Creating a Testcase diff --git a/src/assembly/bin/README.md b/src/assembly/bin/README.md index 0706cccfef2..4310f334745 100644 --- a/src/assembly/bin/README.md +++ b/src/assembly/bin/README.md @@ -23,7 +23,7 @@ limitations under the License. SystemDS is a versatile system for the end-to-end data science lifecycle from data integration, cleaning, and feature engineering, over efficient, local and distributed ML model training, to deployment and serving. To this end, we aim to provide a stack of declarative languages with R-like syntax for (1) the different tasks of the data-science lifecycle, and (2) users with different expertise. These high-level scripts are compiled into hybrid execution plans of local, in-memory CPU and GPU operations, as well as distributed operations on Apache Spark. In contrast to existing systems - that either provide homogeneous tensors or 2D Datasets - and in order to serve the entire data science lifecycle, the underlying data model are DataTensors, i.e., tensors (multi-dimensional arrays) whose first dimension may have a heterogeneous and nested schema. -**Documentation:** [SystemDS Documentation](https://github.com/apache/systemds/tree/master/docs) +**Documentation:** [SystemDS Documentation](https://github.com/apache/systemds/tree/main/docs) ## Getting started diff --git a/src/main/python/docs/source/guide/federated.rst b/src/main/python/docs/source/guide/federated.rst index 237214fb65e..051ad03ae83 100644 --- a/src/main/python/docs/source/guide/federated.rst +++ b/src/main/python/docs/source/guide/federated.rst @@ -32,7 +32,7 @@ Start Federated worker To start a federated worker, you first have to setup your environment variables. A simple guide to do this is in the SystemDS Repository_. -.. _Repository: https://github.com/apache/systemds/tree/master/bin/ +.. _Repository: https://github.com/apache/systemds/tree/main/bin/ If that is setup correctly simply start a worker using the following command. Here the ``8001`` refer to the port used by the worker.