Fixing broken multipolygons

[Andygol]

OSM data consists of objects of different types. One of these types are multipolygons. (See more details http://wiki.openstreetmap.org/wiki/Relation:multipolygon)

Sometimes the data contains errors. It may be improperly placed tags and sometimes errors in the topology of the objects. They all lead to incorrect representation of features details for the data consumer or to a lack of objects on the map.

Objects formed using Multipolygons can be easily damaged by users during editing. As a result of the Data Team research a large number of such objects was found. See link

To help debug broken multipolygons, we have a map tool that can be used to check if its one of the bad ones and the location of the issues: http://osmlab.github.io/fixing-polygons-in-osm/map/

Goal: to fix corrupted multipolygons

Estimated features to fix: 2917

Estimated time to fix: To be calculated

Sources: Spreadsheet, Bing, experiences, local knowledge, http://osmlab.github.io/fixing-polygons-in-osm/map/

To work with Multipolygons best to use JOSM with plugins:

• Relation Toolbox - help to work with multipolygons
• UtilsPlugin2 (see wiki osm) - provide a lot of useful tools.
• DownloadAlong - Downloads data and raw GPS data along one or more ways
• GeoChat - live chat for coordination with other mappers

Setup

• Add Layers in JOSM:

  • OSMInspector Areas WMS (Enter URL > GET Layers > Select Geofabrik Tools > Image format: png) - http://tools.geofabrik.de/osmi/view/areas/wxs?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetCapabilities
  • mapbox-streets - tms[99]:https://api.mapbox.com/v4/mapbox.streets/{zoom}/{x}/{y}.png?access_token=pk.eyJ1IjoicHJhdGlreWFkYXYiLCJhIjoiMTA2YWUxNjRkNmFmZGQ4YzAxZWFiNDk0NDM1YjE1YjAifQ.4P6N5dNmA_WQXd3BsJvu5w

  
  * Adding WMS Layer

• Install plugins

Instruction

1. Choose broken multipolygon from the spreadsheet
2. Download it into JOSM. For manual downloading use menu File->Download Object… Ctrl+Shift+O (⌘+⇧+O)

1. Find a place with a problem
   
2. Download the data around it to avoid possible damage of data during the repair process Ctrl+Shift+↓ (⌘+⇧+↓)
3. Open relation in the Relation editor. Find the gaps and missing roles
   
4. Fix the multipolygon, adding the missing segments or removing unnecessary ones
5. As a result, you'll have closed inner and outer loops of the multipolygon
   
6. Run Validation to check your edits Shift+V (⇧+V)
7. If needed repeat steps 5 - 8

What to fix

• touching rings
• selfintersections
• unclosed ways

In case of old style tagging - put tags on the multipolygon instead it members

Changeset comment and source:

• Comment ---> Fixed broken multipolygon #https://github.com/mapbox/mapping/issues/222
• Source ---> Bing; OSMInspector

@mapbox/team-data

[geochetan]

Today (08/23) the @mapbox/team-data has fixed 180 invalid relations. Most of the relations in the spreadsheet are actually fixable. The overview of the stats are here.

Fixed issues	No error	rendering	won't fix	Grand total
125	4	17	34	180

[bsrinivasa]

On 24th and 26th of August, @mapbox/team-data has fixed 176 broken relations. Relations with tags water, waterway and wetland have been completely fixed along with relations containing more than 200 members with various tags.

Fixed issues	No error	rendering	won't fix	Grand total
156	0	10	11	176

Next actions

Few of the members from the team will continue to fix the remaining invalid relations from the spreadsheet on Monday(29th August).

[bsrinivasa]

The team has fixed 116 and has reviewed 147 broken relations for the day. Most of the relations with more than 100 members have been fixed so far. The statistics for the day below.

Fixed issues	No error	rendering	Won't fix	Grand total
116	0	3	26	147

Next actions

@batpad will be generating a new list of broken relations. Few of the members from the team will continue to fix the remaining invalid relations tomorrow from the spreadsheet using the same workflow.

[bsrinivasa]

Today, @mapbox/team-data has reviewed 65 invalid relations out of which 49 of them have been fixed. All relations consisting of more than 100 members have been fixed as of today. The statistics for the day has been tabulated below.

Fixed issues	No error	rendering	Won't fix	Grand total
49	0	1	15	65

Next actions

The significant relations which were broken/invalid which includes water, waterway, wetland, wood and huge boundaries have been fixed.
@batpad will be generating a new list of broken relations and the priorities will be based on the newly generated list.

[gundersen]

Great framing on this progress @bsrinivasa, it's really helpful to see the context in addition to the table break down of numbers.

[bsrinivasa]

Today, few from @mapbox/team-data has reviewed 71 invalid relations out of which 61 of them have been fixed. @batpad has generated an updatedGeoJSON which contains numerous invalid/broken relations that failed to render in Mapbox Streets layer. A list of these has been generated and has been classified in the spreadsheet. The statistics on the daily progress of reviewing these has been tabulated below.

Fixed issues	No error	rendering	Won't fix	Grand total
60	0	2	9	71

Next actions

Few members of the team will focus fixing relations from the freshly prepared list in the spreadsheet prioritising the features including water (tag: natural=water) tomorrow.

Here is the spreadsheet that has the lists of invalid / broken relations and the detailed statistics

[oini]

Relation 362725 is broken due to being unclosed. There were earlier attempts made by our team to fix this as can been seen in the earlier spreadsheet as well as the history of the relation and various of it's components, especially the point where the ring is unclosed.

Unsure why/how this is happening. It would be great to get some clarity on how we can make sure that our fixes actually work and does not cause any further breakage.

cc @maning @bsrinivasa

[bsrinivasa]

Today's highlights:

• The team has reviewed 68 relations and fixed 55 out of them
• All 66 invalid/broken relations with tag natural=water has been reviewed and most of them are fixed

Fixed issues	No error	rendering	Won't fix	Grand total
55	0	7	6	68

Next actions:

Members with more than 100 members and version numbers greater than 20 have been filtered and 92 relations have been shortlisted. The @mapbox/team-data will be focusing on reviewing these tomorrow.

Here is the updated spreadsheet that has the lists of invalid / broken relations and the detailed statistics.

[bsrinivasa]

Today's progress:

• The team has fixed 18 relations out of 30 being reviewed.

• The list was categorised based on relations with more than 100 members and version number higher than 20

• There are 91 invalid/broken relation with above filter out of which 30 has been reviewed for the day

• Per voice with @Andygol, he will be helping out in fixing few of these in the coming week.

  Fixed issues	Validated	Rendering	Won't fix	Grand total
  15	3	3	9	30

Next actions

The @mapbox/team-data will continue to fix these relations on Monday with the current prioritised list.

Here is the updated spreadsheet that has the lists of invalid / broken relations and the detailed statistics.

[bsrinivasa]

Highlights for the day

• Reviewing and fixing broken multi-polygons is now a part of daily product QA and is addressed by the assigned team for 1 hour every day.

• Today 8 relations have been reviewed out of which 2 have been fixed.

  Fixed issues	Validated	Rendering	Won't fix	Grand total
  2	0	2	4	8

Next actions

• @Andygol and 4 members from Peru team will be joining the product QA along with 4 members from Bengaluru data team.
• The @mapbox/team-data will continue to fix these relations with the current prioritised list.

Here is the updated spreadsheet that has the lists of invalid / broken relations as well the detailed statistics.

[bsrinivasa]

Progress update

• In the past 2 days, 20 relations have been reviewed out of which 11 have been fixed.

  Fixed issues	No error	Rendering	Won't fix	Grand total
  11	0	1	8	20

Next actions

• Few members from Peru team will start fixing these relations from today
• The @mapbox/team-data will spend an hour as part of product QA and continue to fix these relations with the current prioritised list.
• A new GeoJSON would be generated tomorrow once most of the relations in the list are reviewed.

Here is the updated spreadsheet that has the lists of invalid / broken relations as well the detailed statistics.

[bsrinivasa]

Progress update

• Today @mapbox/team-data has reviewed 146 broken multi-polygons of which 78 have been fixed.

• @mapbox/peru-office has started fixing these broken/invalid relations from today.

• The spreadsheet has been re-filtered for the relations with members greater than 25 and version higher than 5.

  Fixed issues	No error	Validated	Won't fix	Grand total
  78	45	0	20	143

Next actions

• A new GeoJSON of these broken multi-polygons has been generated by @batpad.
• A new prioritised list of these relations would be prepared by Monday and few members from @mapbox/team-data will spend an hour as part of product QA and continue to fix these relations.

Here is the current spreadsheet that has the lists of invalid / broken relations as well the detailed statistics.

[bsrinivasa]

Progress update

• The statistic and progress on fixing the broken multi-polygons will be updated weekly.
• A new spreadsheet has been generated which has the prioritised list of broken polygons that needs to fixed.
• The list has been filtered out for all the previously reviewed relation ids, type=boundary, relations with members > 25 and version number > 25

The total number of broken/invalid relations that has been reviewed in the past week (19th and 23rd September 2016) are as tabulated.

Fixed	No errors	Won't fix	Grand total
21	12	7	40

Next actions

• @batpad to continuously monitor the OSM for broken multi-polygon and facilitate @mapbox/team-data with a better workflow based on classification of errors responsible for these broken multipolygons
• @bsrinivasa to keep track of the list and update the stats on progress next week

Here is the spreadsheet that has the prioritised list of invalid / broken relations as well the detailed statistics of each list.