Plotting Data with GMT

Why use the Generic Mapping Tools?

The Generic Mapping Tools (GMT) package is an early, open-source plotting and spatial data analysis package that follows the philosophy that I try to teach in Environmental Informatics: Deal with complex data management problems by breaking them down into smaller more manageable pieces. The GMT package includes 18 modules that directly create plots, and an additional 45 modules that can do data processing. You only use the modules directly relevant to your current task to build a plot, or work with your data. There is a commonality in the command line options for all of the GMT modules that means if you learn one you have a pretty good idea how to use another. Each GMT module is called from the command line, so GMT plays well with anything else you can do at the command line, including shell scripting, python coding, AWK scripting, and Linux command line tools (grep, sed, cut, etc). Additional information can be found at GMT overview: History, philosophy, and usage.

I learned GMT as a graduate student, and have continued to use it throughout my career, teaching it to all of my friends and students. When it first came out, it was relatively unique. Now there is a lot of competition, so why do I continue to teach it?

GMT can plot anything. It can do simple x-y axis, contour maps, shaded relief maps, time series plots, pie charts, box-whisker plots, triangle plots (e.g., the soil triangle), polar plots, 3-D perspectives, you can build any kind of plot with GMT. Every plot, even a really complex one, is built up from a sequence of simple commands.
GMT draws exactly what what you tell it to draw. Other plotting packages (e.g. matplotlib) try to automatically make things fit, or rely on user interactions with a GUI interface (e.g., ArcMap, Excel), both of which make it hard to create a plot of consistent quality. With GMT, you have to define the plot size, the font size, the line colors and size, and none of that will change unless you change the command. You create a figure that is 3 inches by 2.5 inches with an 11 pt Helvetica font, then that is exactly what you will get when GMT is done. No rescaling, accidentally panning the view window, or using the wrong font when editing. Generating a figure panel with three maps of exactly the same size, using the same colorbar, is not a problem.
Plots built with GMT are fully repeatable. Capture all of the steps in a script and you can regenerate the same figure, or you can change the data and update the figure.
GMT is a basic GIS, including the ability to reproject data, do spatial analysis, resample and interpolate data, and when fully installed it includes global data sets for coastlines, lakes, rivers and geopolitical boundaries (see the pscoast module). In many cases it can be much faster than trying to do similar processing in a GUI reliant tool like ArcGIS.
GMT graphical defaults are more polished than many other packages, plus you can set defaults that are applied to everything you create on a system, or within a folder. So you can create a folder for your manuscript and your poster presentation, and customize fonts, colors, units, etc. independently for both.
Finally, GMT produces all figures as high-quality encapsulated postscript (EPS) files. This is a vector format, which means it contains everything needed to draw the image. So while you can create a figure with exactly the dimensions you want, a vector file can be rescaled anyway you want without losing information. EPS files can be embedded directly into Word documents, and will print at the full resolution of the printer. They are a preferred format for journal publications.

Why not do everything with GMT?

GMT does not automatically rescale axis to fit any data, so is not a great tool for graphical data analysis.
Graphical data analysis also benefits from having the plotting package integrated with the software tool (R, matplotlib, etc), so that you can slice and mix your data quickly to produce new figures.
There are times when interacting with the data using a mouse and GUI will result in a good figure much faster (e.g., drawing maps in ArcMap)
GMT is not always installed, and the start-up time can be daunting.

Introduction

This document introduces you to using GMT as a plotting package. It assumes a knowledge of Linux and Bash scripting, and access to the RCAC cluster computers (or a version of GMT installed on a Linux environment on you own computer). The official classic GMT tutorial (GMT 5.x) is very good, but a little more complete than is actually necessary for the Environmental Informatics course. This page uses parts of the official tutorial, interspersed with some of my own material to provide a less in-depth but hopefully more well-rounded introduction to the power of GMT.

Accessing GMT on the RCAC Cluster Computers

Some of the clusters have GMT loaded as a module, so the command "module load gmt" will load the GMT libraries. However, I have found it difficult to keep on top of RCAC to make sure that they have the latest patches installed, that they install all of the libraries (coastlines, rivers, etc) and that they have a version available on all of the clusters. Therefore, I continue to provide a full version of GMT to my students and others who are interested in using it while on the clusters. To get access to my version, you need to add the directory to your PATH. The PATH variable is what controls all folders that Linux will look in when asked to find a command. You can modify your PATH variable to include folders that you want included in the search process, something that is particularly useful as you start writing more of your own code or installing your own packages (for example, GMT).

To add a folder permanently to your search PATH, you need to edit (or create) the .bash_profile in your home directory. Most Unix/Linux programs will store settings and preferences in dot files in your home directory. At this point in the class, you probably have versions of .emacs, .mozilla, and .spyder. The .bash_profile file establishes settings for Bash whenever you login, and you may have a version from a previous lab exercise. Whether or not it already exists, use emacs to open the file ~/.bash_profile (the .bash_profile will have no effect on you if it is stored anywhere other than in your home directory).

To make GMT work:

If you already had a ~/.bash_profile and it contains a line starting with "PATH=$PATH...". Then you simply need to add the GMT path to the end of the existing search list, as follows:
```
PATH=$PATH:<...>:/depot/phig/apps/linux/GMT/bin
```
Otherwise, you need to add the following line to the new ~/.bash_profile file
```
PATH=$PATH:.:/depot/phig/apps/linux/GMT/bin
```
Finally, for either case, save the file and exit emacs. From the command line you can activate the changes to the \~/.bash_profile for the current shell by typing:
```
source ~/.bash_profile
```
The only way to apply the changes to all of your sessions is to log out of thinlinc completely (close all windows and terminals), and then log back into the cluster. Since the changes are in your .bash_profile, they will become permanent.

If all has gone well, you should now be able to type "gmt" at the command line prompt and get a message similar to the following:

[cherkaue@scholar-fe02] $ gmt

        GMT - The Generic Mapping Tools, Version 5.4.5 [64-bit] [20 cores]
(c) 1991-2019 Paul Wessel, Walter H. F. Smith, Remko Scharroo, Joaquim Luis, and Florian Wobbe

Supported in part by the US National Science Foundation (http://www.nsf.gov/)
and volunteers from around the world (see http://gmt.soest.hawaii.edu/).

This program comes with NO WARRANTY, to the extent permitted by law.
You may redistribute copies of this program under the terms of the
GNU Lesser General Public License (http://www.gnu.org/licenses/lgpl.html).
For more information about legal matters, see the file named LICENSE.TXT.

usage: gmt [options]
       gmt <module name> [<module-options>]

Session management:
  gmt clear history | conf | cache | all
                    Deletes gmt.history, gmt.conf, the user cache dir, or all of them

options:
  --help            List descriptions of available GMT modules.
  --show-bindir     Show directory with GMT executables.
  --show-cores      Print number of available cores.
  --show-datadir    Show directory/ies with user data.
  --show-modules    List all module names.
  --show-library    Show path of the shared GMT library.
  --show-plugindir  Show directory for plug-ins.
  --show-sharedir   Show directory for shared GMT resources.
  --version         Print GMT version number.

if <module-options> is '=' we call exit (0) if module exist and non-zero otherwise.

[cherkaue@scholar-fe02] $

Working Through the GMT Tutorial

The official GMT tutorial is very useful, but can be dense and does not always focus on what I think is most relevant for getting you started quickly. So here I have provided a roadmap to help you work through the tutorial more efficiently:

Best way to work through the tutorial is to start firefox from within thinlinc so that you can copy and paste the command directly from the tutorial into a command terminal (one that has GMT setup as described previously).
Section 1: Introduction
- Go ahead and read it, it won\'t take long.
Section 2: Session 1
- Some comments on Section 2.1 Tutorial Setup
  - The GMT man pages are not setup using my version of the code, however, type any GMT module at the command line and the manual page will be dumped to the screen (sorry, no way to scroll through it). A better idea is probably to have a second firefox tab opened to the on-line versions of the man pages, they are also more useful since the HTML version includes hyperlinks between the pages.
  - I suggest using "gv" to view your GMT built EPS files, see topics under the heading of Useful Information for more on what tools are available.
- Section 2.3 is a review of basic Bash shell commands, review as needed. Working with GMT will give you lots of Bash scripting practice.
- Section 2.4: You should be able to recreate all of the basic plots by copying and pasting the provided commands. The exercises in each subsection are designed to get you thinking about how the plots can be changed using the various flag options. Many of the flag options are common between GMT modules, so learning how to use them here will translate into a better understanding of how they work for the next tool. That said, you do not need to do every one of the exercises, you can pick and choose a few to try and move on. You can also get by with doing a couple of projections, and understanding that GMT can do many, many more (it has the same projection engine used by GDAL, QGIS, and even much of ArcGIS - ESRI has added a few specialty projections)
Sections 3 to 5
- The key to getting through the tutorial is to limit your exploration of the many exercises after each example plot.
- To get more information on what a flag option can do, start by looking at the man page for the command in question, for example type "gmt psxy" to get the manual page for the psxy module. Or find the psxy module man page on-line.
- If it is a common flag option, the description might not be complete in a more complex tool such as psxy. For things like projections "-J" and axes controls "-B", you actually need to check out the man page for psbasemap, e.g. "gmt psbasemap". The creators of GMT have tried to not overwhelm users by providing every detail of every option in every command, but that does mean that sometimes you have to look a little harder to find the answer. Many of these common options are also much easier to find in the most recent versions of the GMT on-line documentation, in fact they appear at the top of the Quick Reference page.
- Additional information can also be found int he GMT Cookbook. This is a collection of more complex plotting and analysis scripts that can provide insight into how to do more complex tasks with GMT. The python module matplotlib uses the same method to provide a resource for users to learn new tricks and get ideas about how to do more with the tool.

Useful Information for Working with GMT

This section includes a variety of tips and suggestions that have helped me work with GMT. The list will continue to grow as I get time to add more.

Helpful Hints for Developing Complex Images

Here are some of the tricks that I use to help me construct more complex images in GMT.

Start every GMT script with a simple psxy statement that draws a box of the exact size desired for your final image, for example

# This psxy command establishes the start of the figure (-K) and overwrites any previous version of the output file.
# It sets the map to rectangular (unprojected coordinates) with a 1:1 scale (-Jx1)
# It defines the size of the page using -R (this really just needs to be as big or bigger than the box being drawn)
# The origin of the plot is set in from the lower left corner by 1 inch (-X1 and -Y1) - assumes you are using inches not other measurement units
# It sets the pen used to a solid black (-Wblack)
# It turns the page to Portrait instead of using the default Landscape layout (-P)
# Finally the command accepts the next lines in the current script file as STDIN, until it reaches the string EOF ( "<< EOF" )
# The next 5 lines define the coordinates of the box corners - don't forget to clsoe the box.

psxy -Jx1 -R0/8.5/0/11 -X1 -Y1 -Wblack -K -P << EOF > OutFile.eps
0 0
4 0
4 2
0 2
0 0
EOF

Use a black line when setting up the image, then change the line color to white for the final version.
The box helps you line up figure components within the space you want to use.
Drawing the box first also establishes the boundaries for the EPS figure (the Bounding Box in postscript terminology). This improves clipping of the final image, and even the display of the clipped image in tools such as "gv" when the display is set to show the bounding box rather than a default page size (typically US letter or A4).

Before worrying about how to get the data into the script, layout all axis using the psbasemap command. This renders new figures very quickly, so you can tweak position and size of all axis and maps before starting to worry about other details.

Viewing Postscript Files on RCAC Clusters

Viewing postscript (PS) or encapsulated postscript (EPS) files can be a challenge depending on what tools are installed on the local computer. The standard postscript display tool is "gs", but it only allows for documents to be viewed as full sized pages (typically US letter), and always as portrait (so long side is vertical). That is not always useful. The following document viewers are now (as of Spring 2019) available on all of the RCAC maintained clusters (I have not tried them all):

gv - an oldie but one of my favorites
ghostscript-gtk
evince
xpdf
qpdfview
apvlv
zathura
zathura-pdf-poppler

Some of these may be better at displaying PDFs, but many will have the ability to do both.

Cleaning up GMT EPS Files for Use in Word and Other Applications

After using GMT to create a postscript figure file, I like to run the file through the Unix/Linux command line program "ps2epsi". This converts the postscript (PS or EPS) file to the Adobe encapsulated postscript interchange (EPSI) format. There are two benefits to the EPSI file that results from this conversion:

The figure bounding box is reset to include all aspects of the figure. Because GMT builds complex figures from sequences of simpler commands, it is possible that parts of a complex figure will fall outside the defined bounding box and therefore be cut out when trying to include the figure in another document. It is even more likely that GMT will simply extend the bounding box too far to make sure that incorporates all parts of your figure, resulting in a figure file that has too much white space. The process of converting the PS file to EPSI generally corrects both problems. See also my notes on setting up the bounding box for complex figures for my suggestions on how to avoid this problem in the first place.
The conversion to EPSI also adds a low-resolution preview image to the start of the file. For programs, such as word processors, that read EPSI files, the preview image will display on the page letting you get a sense of what the figure will look like when the document is completed. Note that when the document is printed or converted to a document format like PDF, the full EPSI will be rendered into a high-quality version of the figure. The preview image is just what you see when editing the document.

One thing to beware of, is that the preview image from the EPSI file will make it seem like the figure is displaying in presentation software such as Powerpoint. However, all you will be able to see is the low resolution preview image, as Powerpoint and similar software is not designed to interpret the actual EPSI file.

Converting Postscript to Raster Image File Formats

ImageMagick is a suite of image processing tools installed on many Unix/Linux systems, including all of the RCAC clusters. The "display" command will let you view image file formats (e.g., BMP, GIF, JPEG, PNG, TIFF, etc), while the "convert" command will allow you to convert image file formats, and do simple manipulations (e.g., change image resolution, convert backgrounds to transparent, crop images, etc.). To convert output from GMT to a raster image file format use the following commands:

Convert the original GMT output file to EPSI.
Determine the resolution desired for the final raster image in dots per inch (DPI), typically anywhere from 72 dpi for the web, 150-250 dpi for a good balance between space and clarity, and 600 dpi to 1200 dpi or greater for publication.
Use the convert command to convert from the EPSI file format into whatever raster format you want to use (you can use the flag option to set the output format, but convert defaults to the format defined by the extension on the output file (".png" for a PNG file, ".jpg" for a JPEG file, etc.)
```
# This statement converts the file inFile.epsi to a TIF file called outFile.tif with a resolution of 250 dpi
$ convert -density 250 inFile.epsi outFile.tif
```

Note

Add instructions for updating PATH in bash

things to learn
- layout with psbasemap
  - creating a simple line / scatter plot
  - adding time series axis
  - adding a legend
- spatial plots
  - adding coastlines and rivers
  - changing the projection
  - plotting points, filled points, contours and filled maps
  - adding a legend
- combining plots
  - layout with psbasemap
  - thinking in terms of final image size
  - filling plot with data
  - changing the plot data
    - discuss using a control file in bash?
- other stuff
  - setting a special color bar
  - time series axis (need to update that page)
  - setting fonts