Tools of the Trade
Posted on 27th Jun 2016
To create the visualisations on The First Say, I use a variety of software-based tools. While a pencil and paper is certainly handy for generating ideas, at some point, I have to switch over to software to do everything from cleaning up the data to publishing the final result.
Some data are best visualised as a simple bar chart while others require a little more complexity. I tend to think of visualisations on this spectrum depending on what I am trying to communicate and how much data I am trying to show. Each visualisation, or project, follows a fairly standard process that starts with exploring data and ends with uploading files and converting graphics to PDF for printing.
For exploring and cleaning up data, I use Microsoft Excel. I appreciate the lack of a learning curve I have with Excel and the simple text-based tools for extracting and organising data from PDFs. The relative simplicity of the data I have analysed thus far has made this possible but I foresee a future need to move to more sophisticated statistical software such as R or PSPP that are more suiting to exploring complex data. I acknowledge that there are some criticisms of Excel with regards to replication and data fidelity, both of which I try to address in each project.
I also use Excel to create simple visualisations such as Immeasurable Progress. It allows me to compose quick charts without any coding that I can easily edit to achieve a desired look. The difficult part for these graphs is the need to convert them to a web-friendly format (in most cases, SVG files) which I create and touch up using the free graphics editor Inkscape.
For the stuff in-between (a little more complex, but without interactivity), I am at a bit of a loss. I tried out Tableau Public for BGCSE Results but got frustrated by the lack of ownership. As long as you do not mind your work being stuck on their website, this is a great tool for anyone starting out. There are other online tools that either have the same limitation or are costly to use your own version. Right now, I am thinking of heading in two directions to see how they pan out. The first is back to Excel. I have been inspired recently by the work of Stephen Few and Jorge Camoes that prove that Excel can be used to make complicated and visually pleasing graphics. My main hiccup is coming up with an effective method to convert the result to a high-quality web format. I used Excel for What Are We Missing and found that it took twice as long to convert the graph to SVG that it did to create it in the first place. The second direction is to move on to R. The ggplot library can very easily meet many graphing needs, but it requires me to get back up to speed with R, so we will see how that goes. Recommendations are welcome, but be sure to keep in mind any potential cost and learning curve.
To create the maps in both Explore Island Data and Nation of Immigrants I used data from Natural Earth. I then edited the maps to achieve a balance between simplicity and accuracy. Too much detail and the map loads and animates slowly. Too little detail and you cannot make out countries or islands. I did most of the editing in qgis but also used a handful of online tools to simplify lines and convert file types.
Finally, I use JetBrains PhpStorm (besides Excel, the only software that I have to pay for) to code, design, and test all web-based work and FileZilla to upload files to DreamHost, The First Say's hosting service.
If you have any questions on how to use or install any of this software, let me know and I will try to point you in the right direction.