MapShaper exported GeoJSON vs ogr2ogr, compatibility issues with D3js

[wahlatlas]

I am wondering if MapShaper does an even cleverer job to projection than D3. The issues I have is converting a shapefile for Germany into GeoJSON: It displays perfectly in Mapshaper but I can't reproduce the same rendering with standard D3js code (like the example in Scott Murrays book, see at the end of this post),

What I get instead is this

However converting the same shapefile with ogr2ogr on the commandline works

ogr2ogr  -f "GeoJSON" -t_srs crs:84 germany.json vg250_L.shp

It might well be a misunderstanding of D3 on my part (wrong projection?), so forgive me beforehand. This is how I loaded the GeoJSON that I generated with the above ogr2ogr command

//Define map projection
var projection = d3.geo.mercator()
                    .center([14.5,52.5])
                    .scale(3000);

//Define path generator
var path = d3.geo.path()
                 .projection(projection);

//Create SVG element
var svg = d3.select("body")
            .append("svg")
            .attr("width", w)
            .attr("height", h);

//Load in GeoJSON data
d3.json("germany.json", function(json) {

    //Bind data and create one path per GeoJSON feature
    svg.selectAll("path")
       .data(json.features)
       .enter()
       .append("path")
       .attr("d", path);
});

The shapefile discussed in this example is from the open data portal for Germany: http://www.geodatenzentrum.de/auftrag1/archiv/vektor/vg250_kompakt/2014/vg250_2014-01-01.utm32s.shape.kompakt.zip

If I had one wish, it would be that Mapshaper generated some example D3js code that displays the GeoJSON the same way Mapshaper does itself.

[mbloch]

Here is why d3 is giving you abstract art instead of a map of Germany:

The original Shapefile contains projected coordinates. (The projection is Transverse Mercator; it's specified in the Shapefile's .prj file). Your ogr2ogr command converted the coordinates to the WGS84 coordinate system (with units of decimal degrees of latitude and longitude), which is what your d3 projection function expects.

Mapshaper doesn't support coordinate system transformations, so the GeoJSON file you generated with Mapshaper still contains projected coordinates, in units of meters. D3 is transforming the coordinates in meters as if they were lat-long coordinates.

You can solve this issue by using Mapshaper together with ogr2ogr. You could first convert to WGS84 using ogr2ogr and then edit your output file with Mapshaper.

I'll consider your suggestion to generate d3 code, thanks :)

[mbostock]

Your ogr2ogr command converted the coordinates to the WGS84 coordinate system (with units of decimal degrees of latitude and longitude), which is what d3 expects.

True, if you’re using a geographic projection like d3.geo.mercator. But you can also use the null projection if the coordinates are already in pixels (as in bl.ocks.org/9265467) or a custom projection to translate, scale and flip meters into pixels (such as with d3.geo.transform in bl.ocks.org/5663666).

[wahlatlas]

Thanks both of you for the instant response, I see clearer now!
(And if it weren't for your tools, I couldn't see at all)

[mbloch]

For to time being, I want to stay focused on data editing as opposed to map creation, so I'm going to say "no" to d3 output.

[rgdonohue]

@mbloch Matt, to further clarify, I'm thinking that the Mapshaper client still doesn't support coordinate system transformations (though you've added some projection support lately I see)? For instance, if I run this command on some cartographic boundary shapefiles, which are unprojected but NAD83:

mapshaper cb_2016_us_county_20m.shp -simplify dp 20% -o format=geojson us-counties.json

This output won't be as accurate as if I first convert the Shapefiles to WGS84 with ogr2ogr?

[mbloch]

Hi Rich,

The mapshaper client supports most of the transformations that ogr2ogr does. Ogr2ogr uses the Proj.4 library, and I ported most of Proj.4 to JavaScript for mapshaper (https://github.com/mbloch/mapshaper-proj), so mapshaper should be able to give very similar output in most cases.

Are you seeing differences? If so, could you show me exactly how you are running both programs?

NAD83 and WGS84 are very similar... ogr2ogr doesn't seem to apply any transition when run using the command below, it basically just replaces the NAD83 .prj with a WGS84 .prj file.

ogr2ogr -t_srs '+init=epsg:4326' output.shp cb_2016_us_county_20m/cb_2016_us_county_20m.shp

[rgdonohue]

Ah, I thought I was seeing a difference but wasn't testing without the simplification. Looks like they're the same whether I convert with ogr2ogr first or not.

curl -LOk http://www2.census.gov/geo/tiger/GENZ2016/shp/cb_2016_us_county_20m.zip
unzip cb_2016_us_county_20m.zip
mapshaper cb_2016_us_county_20m.shp -o format=geojson counties1.json
ogr2ogr counties_wgs84.shp  -t_srs "EPSG:4326" cb_2016_us_county_20m.shp
mapshaper counties_wgs84.shp -o format=geojson counties2.json

That's interesting about ogr2ogr. I thought it did an actual GCS transformation.

Anyway, great work on Mapshaper lately. I'm introducing students to it alongside command line/Node processing!

[dsantuc]

So, for the benefit of people like me, who are having the same problem as @wahlatlas and stumble across this thread, mapshaper does now support coordinate system transformations through the "-proj" command and, for SVG and TopoJSON output, the "width" output flag.

The above recommended solution of converting to WGS84 can now be done in mapshaper like this:

mapshaper vg250_L.shp -proj wgs84 -o format=geojson germany.json

But if the ultimate goal is to display in d3, you can change the coordinate system to pixels (as @mbostock suggested) and convert to TopoJSON like this:

mapshaper vg250_L.shp -o format=topojson width=800 germany.json

That way you don't need to do any projection in d3. (You'll also have to modify the d3 code above to use TopoJSON input.)

[mbloch]

@rgdonohue, glad to hear that you're using mapshaper in your class... please let me know what aspects of the program are confusing to students or could otherwise be improved :)