Saturday, November 24, 2012

Kartograph tutorial: Electoral map

Tomorrow I will be all the day at the polling station, since I've been chosen as a member.
The last two weeks I was playing with the amazing Kartograph software, so it was a good moment to experiment with electoral maps (the first time for me).

In this example, I will explain step by step how to create the map above. It's quite similar to this tutorial, but I want to continue in a new post, going interactive.
Kartograph creates vector SVG images, which you can edit later with Inkscape or Illustrator, so gives much more flexibility than systems generating PNG like files, much more difficult to modify.
As in all the posts, you can get all the files used in the example.
This example has two continuation posts:

Getting the data

As usual, getting the data is not so easy. The Catalan government has a good open data web site, where I found:
• A file with a really descriptive name, bm50mv33sh1fc1r170.zip, with all the administrative boundaries (provinces, comarques, and municipalities).
• Lots of files with election results. I choose the 2010 elections, since they where to the Catalan parliament, like the ones tomorrow. As you can see on the map, the party CiU won with a very big majority, so the map is not as interesting as it could be.
I have used the municipalities to draw the map because the result is more diverse than using bigger zones. Actually, the real  constituency is the province, but CiU won everywhere, and a plain blue map is quite boring.
So I've had to join the two files to get one file with the geometries and the results. The process is quite long and dirty (why didn't they use an id? I had to join with the names), so I won't explain how to do it, but put the result at the data files. You can find this file here.

Then, to decorate the map, I used the following files
• World boundaries from Natural Earth (ne_10m_admin_0_countries.zip), to draw the coast line outside Catalonia
• From VMAP, the layers Trees, Crops, and DepthContours, to decorate the map outside the electoral  constituencies.
Since the layers are worldwide, so very big, I have used these ogr commands to clip:
ogr2ogr -clipsrc -3 37 4 44 Trees2.shp Trees.shp
and to simplify:
ogr2ogr -simplify 10 munis.shp bm50mv33sh1fpm1r170.shp
Doing so, the time to generate the map is divided by five or more.

Installing Kartograph

Since we only need kartograph.py for this tutorial,  first, download it from the github page clicking at the zip icon.
In a linux system, uncompress and execute
python setup install
as a super user.
That's all, if you have the GDAL python bindings installed.

Creating the map

To create a map with Kartograph, you will need a configuration file in JSON format, which will have three basic sections:

Projection

To set the projection, there used to be a web page named Visual map configurator, that doesn't work any more. But don't worry, you can use the Map Projections page. Just choose the projection that fits you more, change the parameters and click the gear icon:
A dialog will open, and the lines that are interesting in this case are, in the image example, like:
"proj": "sinusoidal",
"lon0": 20
This will be translated in our json file as:
"proj": {
"id": "sinusoidal",
"lon0": 20
}

Bounds:

The part of the world we want to represent is set here. It's quite well explained at the documentation, but it can be a bit confusing, and not all the options work with all the projections.
In our example, I have used:
  "bounds": {
"mode": "bbox",
"data": [-0, 40, 4, 43],
"crop": [-3, 37, 5, 44]
} 

• mode: How the bounds are expressed. BBOX is the basic option, but you can also set it defining the points you want to enter in the map, or even the features in a layer. If the layers are in different projections, other modes can be a little tricky.
• data: In our case, the bounding box. In other modes, the layer name, the points, or whatever.
• crop: Is an optional label. Our svg will be clipped at the bounds set at data, but all the data in the files will be processed. If the files include all the world, this takes a long time, and generates much bigger SVG outputs. With crop, only the features inside the BBOX will be included.

Layers:

As the name suggests, the layers to include.
   "municipalities":{
"src": "./mun_out.shp"
}

There are also two special layer, graticule and sea. The first draws the meridians ans parallels, while the second does nothing more than giving a feature to draw the background:
   "background": {"special": "sea"},
"graticule":{ "special": "graticule", "latitudes": 1, "longitudes": 1}


All the layers  will be drawn in the order indicated at the json file, so this must be well chosen to select which layer hides what.

Styling

This is the nice part. Without styling, the SVG can be used directly with Inkscape or Kartograph.js, but is possible to generate styled maps directly with kartograph.py.
You can give the style either in the json file or in a separate css file, which seems cleaner. The names given to the layer are the ones to be used in the css as the id. So to give a style to the municipalities layer, add
#municipalities {
fill: #FFF;
stroke: #882222;
stroke-width: 0.5px;
stroke-opacity: 0.4;
}

The general options are at the documentation again. CSS for SVG is a little different from the one used in traditional html.
Since we want to paint the municipalities in a different color depending of the party who won the elections, we will use filters, like this one:
#municipalities[Winner=CiU]{
fill: #99edff;
}


It would be nice to compare different fields i.e. CiU > PSOE, but this is not possible (at least, I haven't found how to do it), so I had to calculate the winner and put it in a field (called Winner, as you can see in the example)

Drawing

There are two options to draw the map. A command line program is installed with the setup, called kartograph.
To draw the styled map, just type
kartograph elections.json --style elections.css -o elections.svg
But you can also include all this in a python program, so could generate the data and then the map. In our case, the code would be
from kartograph import Kartograph
import sys
K = Kartograph()
K.generate(cfg, outfile='elections.svg', format='svg', stylesheet=css) 

Finally, I edited the svg file with Inkscape to put the titles and legend. Is just to show that the idea is generating a base svg and from there, draw the pretty final map.

Configuration files

To draw the map in the example, I have used the following files:
elections.json
{
"proj": {
"id": "sinusoidal",
"lon0": 20
},
"layers": {
"background": {"special": "sea"},
"graticule":{ "special": "graticule", "latitudes": 1, "longitudes": 1, "styles": { "stroke-width": "0.3px" } },
"world":{
},
"trees":{
"src": "data/Trees2.shp",
"simplify": true
},
"crops":{
"src": "data/Crops2.shp",
"simplify": true
},
"depth": {
"src": "data/DepthContours2.shp",
"simplify": true
},
"municipalities":{
"src": "./mun_out.shp"
}
},
"bounds": {
"mode": "bbox",
"data": [-0, 40, 4, 43],
"crop": [-3, 37, 5, 44]
}
}

elections.css
#background {
fill: #e8f9fb;
stroke: none;
},
#world {
fill: #f5f3f2;
stroke: none;
},
#graticule {
stroke-width: 0.3px;
},
#municipalities {
fill: #FFF;
stroke: #882222;
stroke-width: 0.5px;
stroke-opacity: 0.4;
},
#municipalities-label {
font-family: Arial;
font-size: 13px;
},
#municipalities[Winner=CiU]{
fill: #99edff;
},
#municipalities[Winner=PSC-PSOE]{
fill: #ff9999;
},
#municipalities[Winner=ERC]{
fill: #EDE61A;
},
#depth {
stroke: #223366;
stroke-width: 0.5px;
stroke-opacity: 0.4;
},
#trees {
fill: #d2f8c0;
stroke: none;
},
#crops {
fill: #fcf8d8;
stroke: none;
}

What's next

If I have time, I'll try my first Kartograph.js example. From the svg generated, is possible to create cool interactive maps.

1. If I am merging two shape files into one svg file. The resultant svg is getting cut.

1. Steve, in the example up to five files are merged, but they stay as different layers.
You want to do the same or merging two shapefiles into one single layer in the SVG?
If you want, send me the files and configuration file at my email rveciana (at) gmail (dot) com

2. Hi Roger. Thank you for posting this. I'm learning Kartograph and worked off your example. Out of curiosity, did the background color modification you made actually change the background---or is that coloration due to the ocean-depth layers you have? I tried something similar (http://gis.stackexchange.com/questions/45583/background-sea-layer-not-coloring-in-kartograph-map) and found I was unable to actually change the color of the "sea."

I think that your problem is because of a known bug. I asked the author at GitHub and told me that in certain projection, this happens.

In my case, the color is defined at the background, not the ocean-depth. I define it with the special layer
"background": {"special": "sea"}
but if you change the projection, it stops working (in most of the options)

4. Hi Roger,

First thank for the posts, there is no much info about this library on the net. I am trying to work on spanish provinces svg maps, and i am not able to make it work. I am using this maps:
http://www.nbsp.es/2009/02/25/mapas-de-los-municipios-de-espana-en-vectorial/

I configure to load this map, and to add the following layer but it doesn't work.

Layers names must be id tags inside the svg file, right?

5. Hola Miguel!

Have you generated the file using kartograph.py?
The workflow is to take the GIS file (shapefile, GeoJSON, whatever you have) and run the kartograph.py to generate the SVG. You can't run it from an SVG file.

By the way, there is open data about the Spanish municipalitites in a proper format. Take a look at: http://centrodedescargas.cnig.es/CentroDescargas/equipamiento/lineas_limite.zip
I have a post explaining how to filter them by province: http://geoexamples.blogspot.com.es/2013/04/fast-tip-filtering-features-using-ogr.html
Use ogr2ogr with -where to get only the municipalities in Alicante.

6. Hey Roger,

Thanks for your reply. No, i didn't try with a svg generated by kartograph.py. I did have lot of problems to make kartograph.py run, i wrote to the author, and post it in stackoverflow (http://stackoverflow.com/questions/16297476/install-of-gdal-failed-for-kartograph-py-dll-load-failed ). I think is due to all the dependencias, and that i install a pack with geo tools, and now it's a mess.
I'll take a look to your links, and write back, thanks a lot.

ogr2ogr is one of those tools i had to install i think...

i'll write you soon

7. Hi again,

What environment are you using? (OS, python version). I'll try to set it up in another computer, the one i'm trying is driving me nuts.
It definitely seems that your way it's the right one to get maps for all municipalities by provincies, what is my target here. But i'm dealing with this gdal-python installation in windows.

BTW, i'd be nice to share other geo python thoughts by email, and also we can switch to spanish, i couldn't find yours, so mine is m_fiandor_@_gmail_.com (without the _)

8. Only Kartograph.py ? Kartograph.js not support convertion shp => svg?
You know alternative for convertion shp -> svg ?
I need for .net application web :/

Thanks, and great work in this blog.

9. GDAL doesn't support SVG creation (only reading), so you will have to use a specific library.
This one uses .NET:
http://geotoolsnet.sourceforge.net/Index.html

Hope it helps...

10. I have been recently taking a look at kartograph. I understand that we generate the svg with kartograph.py and then use the svg in kartograph.js to render maps in a web interface.

I want to generate something like this http://kartograph.org/showcase/animated-symbols/. But using world map instead. The problem is I am very new to mapping and geo data. Its super difficult for me to find how to generate a svg only with country outlines using kartograph.py.

There is an example to test kartograph.py which generates a world map svg but its close to 2MB in size. Is this normal? Wont it delay the initial loading of the graph with a 2 MB svg ( atleast in poor internet connection).

Also the example uses some .shp files to draw the map, where can i get those files.

In other simple words, I am trying to find a simple end to end example to how to use kartograph, your example is very close but still i dunno how to translate this in to a working example that i need

any pointers on how to generate the a simple world map with country outlines and what are layers, filters and explanation of few terms to get me started will be very helpful. Thanks in advanc

11. Hi Venkatesh,

I think that your problem is that you have too much detail in your data, so it's too big for your purpose. If you take a look at:
http://kartograph.org/docs/kartograph.py/#simplifying-map-features

You'll see that kartograph can do this for you.

I don't have the example you need exacly in kartograph, but the data and maybe some ideas are explained in this post (using D3 instead of kerograph):

http://geoexamples.blogspot.com/2013/02/mapping-with-d3js-canvas-or-svg.html

13. Hi, how do you identify the bounds of the area that you need for a map? is there any tool or what technique do you use

14. You can try with

http://www.informationfreeway.org/

Where you can see the coordinates at the lower right corner of the map. Once you know the bounding box, just try it in the Kartograph configuration file.

15. Hi Roger,

I want to generate a simple Ivory Coast's map. I could able to setup all the tools on windows and successfully generate a world.svg(very huge 11.3 MB).
here is the script:

from kartograph import Kartograph
config = {
"layers": [{
}]
}
K = Kartograph()
K.generate(config, outfile='mymap.svg')

But how can i create just the ivory coast map and not the whole world ??

Thanks!

16. Hi Sai,

Look at the bounds part. In my example, the bounds are

"bounds": {
"mode": "bbox",
"data": [-0, 40, 4, 43],
"crop": [-3, 37, 5, 44]
}

And you should limitate your map to the Ivory Coast zone. If you don't know the coordinates, try with:

http://www.informationfreeway.org/

I hope it helps

1. Thanks Roger. That helped!

I have one more problem. I also want to display the Atlantic ocean(just below ivory coast), for that i am adding the ocean layer(ne_50m_ocean.shp) but when i am trying to apply styles and color, quite shockingly, the styles are being applied to the land area instead of ocean!! I also tried with ne_10m_ocean.shp but the same problem persists.

Is there any other way to add a ocean and apply color to it ?

I didn't understand how did you added the ocean in your example.

Thanks!

17. Hi Roger,

Thanks for posting this tutorial. It's been helpful.

However, I am having trouble generating a map of France using a .shp file in Lambert Conformal Conic projection. When I run the program, I get an error: ERROR 6: No translation for Lambert_Conformal_Conic to PROJ.4 format is known. My .json config file looks like this:

{
"proj": {
"id": "lcc",
"lon0": 3,
"lat0": 46.5,
"lat1": 47.22794391363692,
"lon1": 44.36902841543616
},
"layers": {
"base": {
"src": "COMMUNE.shp",
"join": {
"group-by": {
"commune": "NOM_COMM"
},
"export-ids": false
}
}
}
}

From what I have read, Kartograph should be able to handle LCC projections. Is this correct? And if so, any idea how to fix the problem?

I have installed the most recent version of Kartograph from GitHub today and have successfully generated the world map test project. So I am fairly certain things are configured properly.

Thanks!

18. The layer that styles the sea is the background, since your data files contains data from the land, not the sea:

#background {
fill: #e8f9fb;
stroke: none;
}

1. Thanks Roger!

I found out that the problem was with the projection! Now that i have changed the projection from satellite to sinusoidal, it works!

19. Hi Curtis,

Lambert conformal should be included. In proj4 can handle it for sure, but I don't know if kartograph uses the same notation. The tutorial doesn't have docs for the projections, but the code has a file called proj4.py which is supposed to wrap any proj4 projection.
https://github.com/kartograph/kartograph.py/issues/34

Besides, when I tried projections different from the ones at the examples, I found problems for example with the background layer.

20. Thanks for the help, Roger. The file kartograph/proj/_initi_.py lists the abbreviation for Lambert Conformal Conical as 'lcc'. I modified my .json file accordingly but still no luck. I've submitted an issue on GitHub if you're interested in the fix:
https://github.com/kartograph/kartograph.py/issues/90

Thanks again!

Curtis

21. Hi Roger,

I have generated a colorful Ivory Coast's map but the when i try to load the SVG map with all the layers using kartograpgh.js, only a map without any CSS styling is displayed.. Why so? when I load the same SVG map using object tab of HTML5, map is displayed with all the colors.

Doesn't kartograph.js load the SVG maps with the original styling ?

22. No, you have to style it in a different way. I have an other entry at the blog explaining how to do it:
http://geoexamples.blogspot.com/2012/11/kartograph-tutorial-ii-going-interactive.html

23. Then what is the use of creating styled SVG maps using kartograpgh when we have to do it all over again in kartograph.js ?

24. Well, you don't have to style the svg as in this post, but as in the other. Actually, it's more or less the same, and you can make it interactive, so it's better.

1. Hmmm ok, thanks!

25. In my map, the graticule doesn't cover the whole map, it only covers up to 80%(just in the centre) and it looks weird..why? how can draw a graticule which covers the entire map?

26. Hi, I don't know what's happening in your case, but the docs (http://kartograph.org/docs/kartograph.py/#graticule) only explain that you can set how many lines you want in the graticule in longitude and latitude.
Maybe is a problem with the projection. I had problems with the background for certain projections, it's possible that there is something wrong with this too.

27. Yes, the docs doesn't contains much of information. No matter whichever projection is use, the result is same, no full graticule! I don't understand whats going wrong!!! Here is my JSON config file:

{

"proj": {
"id": "ortho",
"lon0": 0,
"lat0": 0
},

"layers": {
"background": {"special": "sea"},
"graticule": { "special": "graticule", "latitudes": 1, "longitudes": 1, "styles": { "stroke-width": "0.3px" } },

"Country": {
"filter": { "sov_a3": "CIV" }
},
"OtherCountries": {
"filter": { "sov_a3": ["BFA", "GHA", "GIN", "LBR", "MLI"] }
},
"States": {
"filter": { "admin": "Ivory Coast" }
},
"OtherStates": {
"filter": { "admin": ["Guinea", "Ghana", "Liberia", "Mali", "Burkina Faso" ] }
},
},
"riverlakeslines": {
"src": "data/ne_10m_rivers_lake_centerlines.shp"
},
"lakes": {
"src": "data/ne_10m_lakes.shp"
},
"waterareas": {
"src": "data/CIV_water_areas_dcw.shp"
}
},
"bounds": {
"mode": "bbox",
"data": [-8.712, 4.149, -2.373, 10.898 ]

}
}

Here it is, how it looks:
http://tinypic.com/r/qwy5hx/5

28. Try changing the bounds section to something similar to

"bounds": {
"mode": "bbox",
"data": [-0, 40, 4, 43],
"crop": [-3, 37, 5, 44]
}

Crop must cover a bigger region than data so more data than needed is processed to avoid problems at the map edges.

29. Hey there, i'm using kartograph.py *as a module* to generate an svg form multiple layers (2 .shp, 1 graticule and the background). My problem is that no matter how i order them, the graticule is never visible. Do u have any idea? A hint: since the layer argument uses a dictionary, is not possible for the items in the dict to be ordered in the way you specify. Is there any parameter for ordering?

30. Hi,

I've never used Kartograph in this way, to know this information... Anyway, I found that certain projections give a lot of problems with the background, so maybe there is tome bug with the graticule too. I would try changing the projection to test if this is the problem.

31. Hi Roger,

A few months ago I was able to use Kartograph.py to generate svg maps.
Now I can not. Running the test I get this result:

mio@mio-desktop:~/kartograph-test\$ kartograph world.json -o world.svg

> Traceback (most recent call last):
> File "/usr/local/bin/kartograph", line 9, in
'kartograph')()
> File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 337,
> File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2279,
> File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 1989,
> entry = __import__(self.module_name, globals(),globals(),
['__name__'])
> File
"/usr/local/lib/python2.7/dist-packages/kartograph/__init__.py", line 2,
in
> from kartograph import Kartograph
> File
"/usr/local/lib/python2.7/dist-packages/kartograph/kartograph.py", line
2, in
> from options import parse_options
> File "/usr/local/lib/python2.7/dist-packages/kartograph/options.py",
line 8, in
> import proj
> File
"/usr/local/lib/python2.7/dist-packages/kartograph/proj/__init__.py",
line 21, in
> from base import Proj
> File
"/usr/local/lib/python2.7/dist-packages/kartograph/proj/base.py", line
21, in
> from shapely.geometry import Polygon, LineString, Point,
MultiPolygon, MultiLineString, MultiPoint
> File
"/usr/lib/python2.7/dist-packages/shapely/geometry/__init__.py", line 4,
in
> from geo import box, shape, asShape, mapping
> File "/usr/lib/python2.7/dist-packages/shapely/geometry/geo.py",
line 5, in
> from point import Point, asPoint
> File "/usr/lib/python2.7/dist-packages/shapely/geometry/point.py",
line 7, in
> from shapely.geos import lgeos, DimensionError
> File "/usr/lib/python2.7/dist-packages/shapely/geos.py", line 100,
in
> geos_capi_version = geos_c_version = _geos_c_version()
> File "/usr/lib/python2.7/dist-packages/shapely/geos.py", line 98, in
_geos_c_version
> return tuple(int(n) for n in v.split('.'))
> File "/usr/lib/python2.7/dist-packages/shapely/geos.py", line 98, in

> return tuple(int(n) for n in v.split('.'))
> ValueError: invalid literal for int() with base 10: '2 r3921'
>

Ubuntu 12.10. Python 2.7.

Any tip for a possible solution?

Regards,

Marco Stara
Italy

1. Hello Marco,
I'm sorry, but I don't know where can be the problem... Maybe you changed the python version or something like that.

32. Hi,
I'm using all this for the first time. I have my data in R (which is convertible to other formats, I guess). I want to do a "chloropleth" map of this data (rainfall over India). Can you please suggest me a tutorial for a beginner??? I am hoping to use rjson if it is possible.

1. Hi,

I have used R just a few times, but:

To use the KArtograph library, you should have the elections result (or the value you want to show) as a field of the shp file. I have no idea on how to get that.

Using D3js, you can have the geometries in one hand (the geoJSON or TopoJSON file) and the results in the other, using a simple json or csv file. Converting the R file to a json one should be quite easy.

The same example using D3js can be found here: http://geoexamples.blogspot.com.es/2013/02/d3js-electoral-map.html

2. I appreciate that you replied Roger. Thank you so much. Ok let's say that I can convert the R data to JSON format. That is not an issue. I've tried to go through most of the tutorials I've found on the net. And the main problem is that I have my data in latlon format, while the rest of these tutorials have more like point-value pairs as the data.

For example, my data is a 2-dimensional array where the rows represent latitude and the columns represent longitude and just one type of data ie, rainfall. Can you help me understand how to use Kartograph on this type of data?

3. Hi,

Well, actually using kartograph in this case would be quite difficult. But you can do it. You should create a GeoJSON like this one:
[
{
"type": "Feature",
"geometry": {
"type": "Point",
"coordinates": [125.6, 10.1]
},
"properties": {
"precipitation": 23.3
}
}
]
With one entry for each point. Once you have this, you can transform it into a shapefile (either with online tools or using the program gdal_transform). Then, you can follow the Kartograph tutorials about drawing points.
I think that generating this kind of file shouldn't be very difficult with R