Improve license parsing by shr-project · Pull Request #279 · ros-infrastructure/superflore

shr-project · 2020-09-18T17:47:42Z

Fixes #271

Corresponding changes for catkin_pkg:
ros-infrastructure/catkin_pkg#296
and rosdistro:
ros/rosdistro#26601

tests/test_utils.py

superflore/utils.py

Regenerate with superflore modified to better parse the license values as implemented in: ros-infrastructure/superflore#279 Recipes generated by superflore for all packages in ROS distribution noetic. This pull request was generated by running the following command: superflore-gen-oe-recipes --dry-run --no-branch --ros-distro noetic --output-repository-path . --upstream-branch HEAD

…5146 Regenerate with superflore modified to better parse the license values as implemented in: ros-infrastructure/superflore#279 Recipes generated by superflore for all packages in ROS distribution melodic. This pull request was generated by running the following command: superflore-gen-oe-recipes --dry-run --no-branch --ros-distro melodic --output-repository-path . --upstream-branch HEAD

…3600 Regenerate with superflore modified to better parse the license values as implemented in: ros-infrastructure/superflore#279 Recipes generated by superflore for all packages in ROS distribution dashing. This pull request was generated by running the following command: superflore-gen-oe-recipes --dry-run --no-branch --ros-distro dashing --output-repository-path . --upstream-branch HEAD

…171906 Regenerate with superflore modified to better parse the license values as implemented in: ros-infrastructure/superflore#279 Recipes generated by superflore for all packages in ROS distribution eloquent. This pull request was generated by running the following command: superflore-gen-oe-recipes --dry-run --no-branch --ros-distro eloquent --output-repository-path . --upstream-branch HEAD

Regenerate with superflore modified to better parse the license values as implemented in: ros-infrastructure/superflore#279 Recipes generated by superflore for all packages in ROS distribution foxy. This pull request was generated by running the following command: superflore-gen-oe-recipes --dry-run --no-branch --ros-distro foxy --output-repository-path . --upstream-branch HEAD

…5327 Regenerate with superflore modified to better parse the license values as implemented in: ros-infrastructure/superflore#279 Recipes generated by superflore for all packages in ROS distribution rolling. This pull request was generated by running the following command: superflore-gen-oe-recipes --dry-run --no-branch --ros-distro rolling --output-repository-path . --upstream-branch HEAD

…5146 Regenerate with superflore modified to better parse the license values as implemented in: ros-infrastructure/superflore#279 Recipes generated by superflore for all packages in ROS distribution melodic. This pull request was generated by running the following command: superflore-gen-oe-recipes --dry-run --no-branch --ros-distro melodic --output-repository-path . --upstream-branch HEAD

…3600 Regenerate with superflore modified to better parse the license values as implemented in: ros-infrastructure/superflore#279 Recipes generated by superflore for all packages in ROS distribution dashing. This pull request was generated by running the following command: superflore-gen-oe-recipes --dry-run --no-branch --ros-distro dashing --output-repository-path . --upstream-branch HEAD

…171906 Regenerate with superflore modified to better parse the license values as implemented in: ros-infrastructure/superflore#279 Recipes generated by superflore for all packages in ROS distribution eloquent. This pull request was generated by running the following command: superflore-gen-oe-recipes --dry-run --no-branch --ros-distro eloquent --output-repository-path . --upstream-branch HEAD

Regenerate with superflore modified to better parse the license values as implemented in: ros-infrastructure/superflore#279 Recipes generated by superflore for all packages in ROS distribution foxy. This pull request was generated by running the following command: superflore-gen-oe-recipes --dry-run --no-branch --ros-distro foxy --output-repository-path . --upstream-branch HEAD

…5327 Regenerate with superflore modified to better parse the license values as implemented in: ros-infrastructure/superflore#279 Recipes generated by superflore for all packages in ROS distribution rolling. This pull request was generated by running the following command: superflore-gen-oe-recipes --dry-run --no-branch --ros-distro rolling --output-repository-path . --upstream-branch HEAD

…5146 Regenerate with superflore modified to better parse the license values as implemented in: ros-infrastructure/superflore#279 Recipes generated by superflore for all packages in ROS distribution melodic. This pull request was generated by running the following command: superflore-gen-oe-recipes --dry-run --no-branch --ros-distro melodic --output-repository-path . --upstream-branch HEAD Signed-off-by: Martin Jansa <martin.jansa@lge.com>

…3600 Regenerate with superflore modified to better parse the license values as implemented in: ros-infrastructure/superflore#279 Recipes generated by superflore for all packages in ROS distribution dashing. This pull request was generated by running the following command: superflore-gen-oe-recipes --dry-run --no-branch --ros-distro dashing --output-repository-path . --upstream-branch HEAD Signed-off-by: Martin Jansa <martin.jansa@lge.com>

…171906 Regenerate with superflore modified to better parse the license values as implemented in: ros-infrastructure/superflore#279 Recipes generated by superflore for all packages in ROS distribution eloquent. This pull request was generated by running the following command: superflore-gen-oe-recipes --dry-run --no-branch --ros-distro eloquent --output-repository-path . --upstream-branch HEAD Signed-off-by: Martin Jansa <martin.jansa@lge.com>

Regenerate with superflore modified to better parse the license values as implemented in: ros-infrastructure/superflore#279 Recipes generated by superflore for all packages in ROS distribution foxy. This pull request was generated by running the following command: superflore-gen-oe-recipes --dry-run --no-branch --ros-distro foxy --output-repository-path . --upstream-branch HEAD Signed-off-by: Martin Jansa <martin.jansa@lge.com>

…5327 Regenerate with superflore modified to better parse the license values as implemented in: ros-infrastructure/superflore#279 Recipes generated by superflore for all packages in ROS distribution rolling. This pull request was generated by running the following command: superflore-gen-oe-recipes --dry-run --no-branch --ros-distro rolling --output-repository-path . --upstream-branch HEAD

allenh1 · 2020-12-10T18:54:17Z

superflore/utils.py

+                       "zlib-acknowledgement",
+                       "ZPL-1.1",
+                       "ZPL-2.0",
+                       "ZPL-2.1"]


👀 @shr-project wow, that's got to be all of them now.

allenh1

@shr-project this is above and beyond what I was looking for. Thanks a million! 👍 from me.

allenh1 · 2020-12-10T18:58:08Z

@shr-project In which order should I merge #279, #280, #281?

Let me know and I'll start hitting merge buttons. :)

shr-project · 2020-12-10T19:03:15Z

@shr-project In which order should I merge #279, #280, #281?

Let me know and I'll start hitting merge buttons. :)

#281, #280, #279 :)

shr-project · 2020-12-11T00:44:03Z

Please DON'T SQUASH the commits when merging, it will be impossible to review and understand the commits after they are squashed.

* instead of trying to guess what the owner meant * exact string replacements are safer than parsing with regular expressions because the license value is often quite creative * these statistics from old regexp implementation are just from regenerated melodic recipes: A) clearly wrong replacements 1 "BSD & CreativeCommons-by-nc-4.0" -> "BSD & CC-BY-SA-3.0" 1 "BSD & GPLv3 & LGPLv3" -> "BSD & GPL-3 & LGPL-2" 1 "BSD,-Apache-2.0" -> "BSD-2" 1 "CreativeCommons-Attribution-NonCommercial-ShareAlike-4.0-International" -> "CC-BY-SA-3.0" 1 "LGPL3" -> "LGPL-2" 2 "Creative-Commons-BY-NC-ND-3.0" -> "CC-BY-NC-ND-4.0" 4 "GPLv3 & LGPLv3 & BSD" -> "GPL-3 & LGPL-2 & BSD" 17 "GPLv3" -> "GPL-3" 21 "LGPLv3" -> "LGPL-2" B) not clearly wrong, but guessing too much about version and often incorrectly (after inspection of the actual source files) 1 "Apache-License-2.0 & LGPL" -> "Apache-2.0 & LGPL-2" 1 "BSD & LGPL & GPL-for-sigblock" -> "BSD & LGPL-2 & GPL-1" 1 "LGPL & proprietary" -> "LGPL-2 & proprietary" 2 "BSD & Creative-Commons" -> "BSD & CC-BY-SA-3.0" 4 "Apache" -> "Apache-1.0" 5 "GNU-Lesser-General-Public-License-LGPL-" -> "LGPL-2" 15 "GPL" -> "GPL-1" 23 "LGPL" -> "LGPL-2" * and in some cases it was loosing some useful information: 1 "BSD,-Apache-2.0" -> "BSD-2" 1 "BSD,-LGPL" -> "BSD" 1 "BSD,-some-icons-are-licensed-under-the-GNU-Lesser-General-Public-License-LGPL-or-Creative-Commons-Attribution-Noncommercial-3.0-License" -> "BSD" 1 "BSD,GPL-because-of-list.h;-other-files-released-under-BSD,GPL" -> "BSD" 1 "BSD,LGPL,Apache-2.0" -> "BSD-2" 1 "BSD,LGPL,LGPL-amcl-" -> "BSD" 1 "GPL-because-of-list.h;-other-files-released-under-BSD" -> "GPL-1" 1 "GPLv2-with-linking-exception" -> "GPL-2" 2 "BSD,-GPL" -> "BSD" 8 "BSD-3-Clause" -> "BSD" * fixes: ros-infrastructure#271

* 'Apache' doesn't always mean version 1 in 'Apache-1.0' * 'GPL' doesn't always mean version 1 in 'GPL-1' * 'Mozilla Public License' doesn't always mean version 2.0 in 'MPL-2.0' * 'BSD License 2.0' doesn't always mean clause-2 in 'BSD-2' * 'Creative Commons' doesn't always mean Creative Commons Attribution Share Alike 3.0 in 'CC-BY-SA-3.0' * to be safe leave the orignal value and catkin_pkg package.xml validation will hopefully notify component developer to better specify the version based on the real source files - instead of superflore guessing what was meant by that * fixes: ros-infrastructure#271 Signed-off-by: Martin Jansa <martin.jansa@lge.com>

Signed-off-by: Martin Jansa <martin.jansa@lge.com>

…t original value in comment * print the original value from package.xml whenever the get_license function modified, this makes it easier to review what superflore did in the recipe and makes some more creative values more readable e.g. "HEBI C++ Software License (https://www.hebirobotics.com/softwarelicense)" instead of: "HEBI-C-Software-License-https-www.hebirobotics.com-softwarelicense-" or "N/A" instead of "N-A" Signed-off-by: Martin Jansa <martin.jansa@lge.com>

* replace spaces with dashes in these 4 cases which weren't mapped to valid SPDX (which would be without spaces) Signed-off-by: Martin Jansa <martin.jansa@lge.com>

herb-kuta-lge · 2020-12-13T08:32:15Z

@allenh1 Even though this PR affects more than the OpenEmbedded portions of the code, since you've already approved it, I'm going to merge it.

shr-project force-pushed the jansa/license branch from e9dcc46 to 61919e1 Compare September 18, 2020 18:51

allenh1 reviewed Sep 24, 2020

View reviewed changes

tests/test_utils.py Show resolved Hide resolved

allenh1 reviewed Sep 24, 2020

View reviewed changes

superflore/utils.py Show resolved Hide resolved

shr-project force-pushed the jansa/license branch 6 times, most recently from 579ccb5 to 7db3324 Compare November 6, 2020 04:08

shr-project mentioned this pull request Nov 6, 2020

Validate license values ros-infrastructure/catkin_pkg#296

Open

shr-project force-pushed the jansa/license branch from 7db3324 to 117ddb1 Compare November 9, 2020 13:08

shr-project mentioned this pull request Nov 9, 2020

[dunfell] Regenerate all recipes with modified superflore license parsing ros/meta-ros#762

Merged

shr-project force-pushed the jansa/license branch 4 times, most recently from f7a38ba to 1a60953 Compare November 13, 2020 18:59

shr-project force-pushed the jansa/license branch from 1a60953 to e359385 Compare December 4, 2020 02:01

shr-project force-pushed the jansa/license branch 2 times, most recently from 9852b10 to 34b5f21 Compare December 5, 2020 12:16

allenh1 reviewed Dec 10, 2020

View reviewed changes

allenh1 previously approved these changes Dec 10, 2020

View reviewed changes

herb-kuta-lge previously approved these changes Dec 10, 2020

View reviewed changes

shr-project dismissed stale reviews from herb-kuta-lge and allenh1 via 5881fd8 December 11, 2020 00:44

shr-project force-pushed the jansa/license branch 3 times, most recently from cfb2d70 to 4d9fa14 Compare December 11, 2020 15:54

herb-kuta-lge previously approved these changes Dec 12, 2020

View reviewed changes

shr-project added 5 commits December 12, 2020 16:01

tests: update tests to match with new utils.get_license behavior

71b7b82

Signed-off-by: Martin Jansa <martin.jansa@lge.com>

tests: update tests to match with using translate_license function

2f8fdc1

* replace spaces with dashes in these 4 cases which weren't mapped to valid SPDX (which would be without spaces) Signed-off-by: Martin Jansa <martin.jansa@lge.com>

shr-project dismissed herb-kuta-lge’s stale review via 2f8fdc1 December 13, 2020 00:01

shr-project force-pushed the jansa/license branch from 4d9fa14 to 2f8fdc1 Compare December 13, 2020 00:01

herb-kuta-lge approved these changes Dec 13, 2020

View reviewed changes

herb-kuta-lge merged commit 4105ad8 into ros-infrastructure:master Dec 13, 2020

shr-project deleted the jansa/license branch December 14, 2020 22:28

shr-project mentioned this pull request Sep 11, 2024

[scarthgap] License issues checking by SPDX License. ros/meta-ros#1216

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve license parsing#279

Improve license parsing#279
herb-kuta-lge merged 5 commits intoros-infrastructure:masterfrom
shr-project:jansa/license

shr-project commented Sep 18, 2020

Uh oh!

Uh oh!

Uh oh!

allenh1 Dec 10, 2020

Uh oh!

allenh1 left a comment

Uh oh!

allenh1 commented Dec 10, 2020

Uh oh!

shr-project commented Dec 10, 2020

Uh oh!

shr-project commented Dec 11, 2020

Uh oh!

herb-kuta-lge commented Dec 13, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

shr-project commented Sep 18, 2020

Uh oh!

Uh oh!

Uh oh!

allenh1 Dec 10, 2020

Choose a reason for hiding this comment

Uh oh!

allenh1 left a comment

Choose a reason for hiding this comment

Uh oh!

allenh1 commented Dec 10, 2020

Uh oh!

shr-project commented Dec 10, 2020

Uh oh!

shr-project commented Dec 11, 2020

Uh oh!

herb-kuta-lge commented Dec 13, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants