{"id":673,"date":"2013-02-15T08:46:09","date_gmt":"2013-02-15T08:46:09","guid":{"rendered":"http:\/\/drainthemainbrain.wordpress.com\/?p=673"},"modified":"2013-02-15T08:46:09","modified_gmt":"2013-02-15T08:46:09","slug":"how-to-make-a-fancy-map-visual","status":"publish","type":"post","link":"http:\/\/blog.rajarshidas.com\/?p=673","title":{"rendered":"How to Make a Fancy Map Visual"},"content":{"rendered":"

A week or two ago, Nikhil Kumar<\/a> showed me this awesome real-time cartogram of twitter feeds<\/a>. Since it was something I had no idea how to do, I thought it was a good time to learn. I decided to play around with geographical data visualizations and see what kind of graphics could be easily produced.<\/p>\n

First, some quick explanations, since I wasn’t familiar with these terms. A cartogram is a map where the area (or distance) is distorted to represent information about some other variable – in the above case, this is the volume of tweets. Another way to represent such data is through the use of choropleths where the areas are colored by the external variable as in this example<\/a>. There are other ways to demonstrate such data in 3-dimensional space but I focused on cartograms and choropleths.<\/p>\n

What I used<\/strong>
\nI used US foreign assistance data<\/a>, both economic and military, from 1946 to 2011 in constant 2011 dollar amounts. I had to remove data that was not properly attributed to a country. Also, there were some other countries that had to be excluded due to conflicts with the map (they didn’t exist at the time the map was produced). The total amount of foreign assistance to all countries (from 1946 to 2011) in 2011 dollars comes out to $2,086,027,401,858 and of this, about 20% was removed from data due to the issues mentioned above. Since the data has such high magnitudes, I decided to log the values to make it easier to deal with. Wiki commons has a\u00a0world map<\/a> available which I grabbed for this purpose.\u00a0Also, I used the fantastic color brewer<\/a> to make the visuals look good.<\/p>\n

And finally the following:<\/p>\n

    \n
  1. R – to manipulate dBase files<\/li>\n
  2. Scapetoad<\/a> – to generate cartograms<\/li>\n
  3. Quantum GIS (QGIS)<\/a> – to color cartogram shapefiles<\/li>\n
  4. Python (Beautiful Soup<\/a> and Python Imaging Library<\/a>) – to parse the SVG file, create a choropleth, and add legend\/text to images<\/li>\n
  5. GIMP (and David’s Batch Processor<\/a>) – to batch convert SVG to JPG<\/li>\n<\/ol>\n

    Process – Cartogram<\/strong>
    \nThough the underlying mathematics behind a cartogram is complex, the third party software, Scapetoad, makes it fairly easy to do. First, I had to match up the foreign assistance data to the world map shapefile, specifically the dBase file. This I did in R simply by adding a column to the dBase file with the foreign assistance data. Then, I used Scapetoad to create a cartogram using the modified shapefile.<\/p>\n

    The process behind creating a cartogram is the Gastner-Newman Diffusion Algorithm<\/a>. In this case, each country has an area defined by the map and a value for foreign assistance which can combine to give a density variable ($\/area). The algorithm works by allowing the “population somehow to ‘flow away’ from high density areas into low-density ones until the density is equalized everywhere”. This can be done via a diffusion process and then applying a Gaussian blur to the density, thus distorting the map.<\/p>\n

    Scapetoad generated a simple cartogram which I exported to a shapefile and opened with QGIS. Using QGIS, I colored the cartogram also based on the foreign assistance variable to generate the image below. The colors are split by 6 quantiles. The color could have been better utilized to display another variable, such as GDP.
    \n\"\"
    \nFor comparison, here is the version with original data (not logged)
    \n\"\"
    \nPersonally, I didn’t like the look of the cartogram, so I decided to create a choropleth as well.<\/p>\n

    Process – Choropleth<\/strong>
    \nSince the process behind a choropleth is more intuitive and easy to implement, I did this using Python. This helped when I wanted to run a batch process on every year separately. However, for the purposes of running just one set of data, QGIS actually can handle the work more efficiently and make a prettier end product.<\/p>\n

    The Python code is fairly straightforward. First, the map is read in as an SVG file and parsed using Beautiful Soup. The foreign assistance data is read in and split into 6 quantiles (same as the color scheme above). Each color change represent a magnitude change of 100x. Since an SVG filed is defined in XML format, a\u00a0style attribute can be added to each country in the file with the proper fill color. This was run for each year as well as the total to generate a series of SVG image files. I then used GIMP and DBP to convert these files to jpeg format for easier distribution. All files use the same quantiles which were based on total sums from 1946 to 2011, so the yearly files are not as segmented as they could be. Below, is the chart for total foreign assistance which can be compared with the cartograms above.
    \n\"\"
    \nAnd I made a quick video of yearly foreign assistance. I gotta learn how to make one of these     Economist\u00a0interactives<\/a>\u00a0next…
    \n[youtube=http:\/\/www.youtube.com\/watch?v=u1xpgWzx1gQ&rel=0]<\/p>\n","protected":false},"excerpt":{"rendered":"

    A week or two ago, Nikhil Kumar showed me this awesome real-time cartogram of twitter feeds. Since it was something I had no idea how to do, I thought it was a good time to learn. I decided to play around with geographical data visualizations and see what kind of graphics could be easily produced.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2,6],"tags":[8,10,15,24,25,26],"class_list":["post-673","post","type-post","status-publish","format-standard","hentry","category-data-analysis","category-programming","tag-cartogram","tag-choropleth","tag-data-visualization","tag-python","tag-qgis","tag-r"],"jetpack_featured_media_url":"","_links":{"self":[{"href":"http:\/\/blog.rajarshidas.com\/index.php?rest_route=\/wp\/v2\/posts\/673","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/blog.rajarshidas.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/blog.rajarshidas.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/blog.rajarshidas.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/blog.rajarshidas.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=673"}],"version-history":[{"count":0,"href":"http:\/\/blog.rajarshidas.com\/index.php?rest_route=\/wp\/v2\/posts\/673\/revisions"}],"wp:attachment":[{"href":"http:\/\/blog.rajarshidas.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=673"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/blog.rajarshidas.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=673"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/blog.rajarshidas.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=673"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}