Software : Tutorial: Using Linux to map routes and walks

Tutorial: Using Linux to map routes and walks
Apple Maps uses 80% less data than Google Maps
Apple avoids Motorola trouble, gets hit with Passbook patent suit

Tutorial: Using Linux to map routes and walks

Posted:

As a mountaineering and cycling enthusiast, I use my GPS to record my tracks and my digital camera to record the views.

To share my experiences, I wanted to be able to show my tracks on a map in a web browser. Like with Google Maps, I wanted to link my photos to a Point of Interest (POI) on the map and pop up the photo when the user clicks on the POI.

The whole process involves using a disparate collection of open source tools - first to extract tracks from the GPS, then to geo-tag the photos using the GPS data, and finally to display both the track and the photos on a map using the OpenLayers JavaScript library.

Download data

GPS Babel

The initial step is getting the data from your GPS. I have a Garmin eTrex, which has a serial port. However, many newer devices come with USB ports.

If your GPS unit has a serial port, you will probably need to get a USB to serial cable. These are widely available on eBay.

We shall use GPSBabel to download the tracks and waypoints from the GPS. GPSBabel is the Swiss Army knife of the GPS world. It converts waypoints, tracks and routes between many different formats.

If you're using a serial to USB converter, when you plug it in to your computer and run dmesg you should get output similar to:

[ 5753.489759] usb 5-1: pl2303 converter now attached to ttyUSB0
[ 5753.489786] usbcore: registered new interface driver pl2303
[ 5753.489789] pl2303: Prolific PL2303 USB to serial adaptor driver

If you're using a Garmin GPS, you will get the best results by downloading the raw tracklog. Saved tracklogs are stripped of much information, including time data. If you want to geotag your photos, you need the time data!

It's best to save your data in GPX format, as this is the most widely supported GPS data format. GPSBabel has a GUI front-end, which is written in Qt. The GUI builds a command line, which is passed to GPSBabel. The command is displayed in the GUI, so once you're familiar with the program you can use the command line to download tracks.

Many recent GPS devices have USB ports and will appear as a mass storage device, so you can download the tracklog using a file manager.

Specify a range

I clear the tracklog from my GPS only when it's getting full. However, I don't really want to download the whole log every time I have been for a walk or bike ride. GPSBabel lets you specify a whole range of filters, including time ranges, to specify what data is downloaded from your GPS.

The following command downloads the tracklog for the specified day and saves it as a GPX file:

gpsbabel -t -i garmin -f /dev/ttyUSB0 -x track,s tart=20120212000001,stop=20120212235959 -o gpx track.gpx

You are a geek, so your camera's clock agrees with your GPS clock? Err… I hate to admit this, but mine doesn't.

The clock in your GPS is synchronised with the GPS satellite, so should always be correct. Almost all digital cameras store information about when a photograph was created in EXIF tags. The relevant tags are 'DateTimeOriginal', 'CreateDate' and 'TimeCreated'. So what you need to do is update these tags using the difference between the camera's time and the GPS time.

We are going to use Phil Harvey's exiftool to correct the EXIF information. In my case, the camera's clock was 1hr 21 minutes ahead of the GPS clock, so I needed to subtract 1 hr 21 minutes from the time in EXIF tags. Using exiftool, the command is:

exiftool "-DateTimeOriginal-=0:00:00 01:21:0" \
"-CreateDate-=0:00:00 01:21:0" \
"-TimeCreated-=0:00:00 01:21:0" *.JPG

Note the - in front of the equals sign, which subtracts the time offset. Before updating your EXIF tags, you will probably want to create a backup copy of your photos in case you make a mistake. You can also create a copy of the EXIF data to a text file for all the photos in the current directory by using the following command:

exiftool -s *.JPG > exif_backup.txt

If it all goes pear-shaped, you can use this to restore the original data.

Geotagging

OpenStreetMap

In the next step, you will use the EXIF time stamps in the photos to correlate them with the GPS tracklog. We shall then write the GPS position data to the appropriate EXIF tags in the photos. This process is known as geotagging.

Now you are going to use GPS Prune to geotag the photos. Assuming that you've corrected any time differences between your GPS track and photos, the first step is to load the GPS track in GPSPrune. From the Photo menu, choose Add Photos and choose the directory containing your photos. From the Photo menu, choose Correlate Photos.

This dialog has various options which allow you to set Correlation Limits, as it's likely that the data in your GPS tracklog and photos match exactly. The two most useful options are:

Time Limit
Distance Limit

If the time or distance in the EXIF field and the tracklog are within the limits you have set, GPSPrune will consider the photo and tracklog point to be correlated. GPSPrune will then tick the checkbox for every photo where it can correlate the EXIF time with the GPS time.

Once you click OK, GPSPrune wil show points corresponding to each photo on your tracklog. Clicking on the point will display the associated photo. You can then write the latitude and longitude information to the relevant EXIF field in each photo. Select Save to EXIF from the Photo menu. This will display a list of photos to which the EXIF data will be written.

As a safety measure, you can choose not to overwrite the original photos, in which case they will be copied to a file with the extension_original. GPSPrune will now append new track points to the end of your GPS file, with the position of each photograph, time stamp and the photo's full path and filename. An example is shown below:

Creating thumbnails

You need to create a thumbnail for each image you want to display on the map. There are dozens of ways to do this, but you will use a simple shell script, which calls the imagemagick convert program to make thumbnails 75px wide. The thumbnails use the original filename prefixed with .thumb. Place the following script, in the directory containing the photos

FILES="$@"
for i in $FILES
do echo "Processing image $i ..." /usr/bin/convert-thumbnail 75 $i thumb.$i done

and run:

./thumbnail.sh *.JPG

which will create thumbnails from any files in the directory with the extension .JPG.

Point of Interest

In order to show the position where each photo was taken on your map, you will create a Point of Interest (POI) corresponding to each one. Luckily, OpenLayers will load the data to display the POI from a file, so you don't have to add each POI manually.

The various data fields are shown below. Note that fields must be tab separated and records separated by a line feed.

lat lon title description icon iconSize conOffset.

URLs can be placed in the text field. We shall use this to display a thumbnail of the photo, plus a link to the full-sized image and some descriptive text. You need to extract the track points, which contain the photo data, from the GPX file you created by geotagging the photos with GPSPrune.

GPSPrune helpfully appends this data to the end of your GPX tracklog, so we can easily extract it using a text editor. Copy the XML data, taking care to include all the necessary tags, and paste it into the template below. Note the template contains a single track point which shows what the correct syntax should be. Save the data to a file, for example photos.gpx.

<?xml version="1.0" encoding="UTF-8"?>
<gpx version="1.0" creator="GpsPrune v13 activityworkshop.net"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.topografix.com/ GPX/1/0" xsi:schemaLocation="http://www. topografix.com/GPX/1/0 http://www. topografix.com/GPX/1/0/gpx.xsd">
<name>Your Walk Name</name>
<desc>Export from GPSPrune</desc>
<trk><name>Your Walk Description</ name>
<number>1</number>
<trkseg>
<trkpt lat="53.13602777777778" lon="-3.9945555555555554"><ele>575</ ele>
<link href="./images/DSCF1869. JPG"><text>DSCF1869.JPG</text></link>
</ trkpt>
</trkseg>
</trk>
</gpx>

Now you need to convert the XML photo data from the photos.gpx file you created to tab separated data used to display the POI with OpenLayers. Since the data is just XML, you will use a Python script to do the conversion. The code can be found in full on our Archives.

Run the script from the Archives and redirect the output to a file:

./generate_poi.py > poi.txt.

Note that you must have a trailing blank line at the end of the file, or the last POI won't be displayed.

Place the generated file (poi.txt) in the same directory as the HTML page that generates the map.

Displaying on a map

GPS Prune

Now you have the geotagged photos, thumbnails and a text file containing the information to display the photos as POI on a map. The final stage is to create an HTML page that contains all the information required to load the map data, and to display the tracklog and photos. You can use almost any web server, since all the work is done in the browser.

To display the map and associated information, we will use the OpenLayers JavaScript library. OpenLayers is an open source JavaScript library for displaying map data. It works with most modern browsers and is completely client-based, requiring no special software on the server.

Using OpenLayers, you can display your data with the same JavaScript, using maps from all the most common mapping providers, including Google, Ordnance Survey and OpenStreetMap. However, if you want to use Google Maps or the Ordnance Survey, you will need to sign up to obtain an API key. You should read their terms and conditions carefully, as there are various restrictions on how you can use the data.

In contrast to this, OpenStreetmaps is completely unrestricted. Which service you use depends on your particular requirements. In the UK, I find that the Ordnance Survey is best for displaying walking data, but OpenStreetMap is best for cycling maps. However, using the OpenLayers LayerSwitcher widget, you can toggle your map between the different mapping systems, so you can use all three on the same web page if you wish.

The OpenLayers library is extensive and seems overwhelming at first sight. Luckily, there are plenty of examples to get you started. We are going to use the GPX file you created earlier to show the track on an OpenStreetMap. The map will also show the POI you created corresponding to the locations where you took photos.

Clicking on a Point of Interest will display a pop-up window showing a caption and a thumbnail of the photo. Clicking on the thumbnail will display the full-sized photo in a pop-up window.

LayerSwitcher

The first step is to generate a page that displays a map of the area you're interested in. You're going to use the OpenLayers LayerSwitcher, which will enable you to switch between various OpenStreetMap layers. You can also use it to switch between different maps, eg Google and OpenStreetMap.

The complete code for the web page is too long to show here, but is included over in the Archives.

Here are some tips and tricks:

Make sure that your page has a valid DOCTYPE, or weird things will happen.

You could host all the JavaScript files on your own server.

However, you will ensure that you get all the latest fixes (and new bugs, too) if you refer to the library on the remote site.

If you are using Google Maps or the Ordnance Survey, make sure that you are using a valid API key. The key is locked to your domain name and won't work on a different site.

If you are using OpenStreetMaps, the Cycle Map layer is also the best for displaying walking data, as it displays contour lines. The code displaying the map layer is in a single JavaScript block. First, you declare some variables that tell the map what latitude and longitude it should be centred on. Various other attributes, such as showing the LayerSwitcher and the Pan and Zoom bar, are also defined. Now, you need to add the layer that displays your GPX track on the map in its own layer.

'Carneddau' is the name of the track that is displayed in the LayerSwitcher. If you want to display more than one track, just copy this section and then change the values appropriately.

// Add the Layer with the GPX Track
var lgpx = new OpenLayers.Layer. GML("Carneddau", "carneddau.gpx", {
format: OpenLayers.Format.GPX,
style: {strokeColor: "green", strokeWidth: 5, strokeOpacity: 0.5},
projection: map.displayProjection });
map.addLayer(lgpx);

Next, you define the layer that will show your Points of Interest:

// Load our Points of Interest
var pois = new OpenLayers.Layer.Text( "POI", { location:"poi.txt",
projection: map.
displayProjection
});
map.addLayer(pois);

Finally, you can add a marker to show where you started and finished the walk.

// Add a marker to show the start point.
var size = new OpenLayers.Size(21, 25);
var offset = new OpenLayers.Pixel(-(size.w/2), -size.h);
var icon = new OpenLayers.Icon('http://www.openstreetmap.org/openlayers/img/marker.png',size,offset);
layerMarkers. addMarker(new OpenLayers. Marker(lonLat,icon));

You must define a div in the body section of the page where you want the map to appear. Note that the name of the div is the same as the layer used to define the map; being imaginative, I called it 'map':

When the page is loaded, the body.onloaded() function calls the Javascript init() function that loads all the map data. When you click on the thumbnail displayed in the POI, the full-sized photo is displayed in a pop-up window. The pop-up is created using JavaScript using the URL embedded in the text file containing the data for the POI.

Elevation charts

Elevation chart

I often want to visualise my GPX track as an elevation chart, which displays height against distance. If, as we've said, GPSBabel is the Swiss Army knife of GPS data, Gnuplot holds a similar status for graphs.

You can generate charts via GPSPrune (which uses GnuPlot), but it's much more flexible if you use GnuPlot directly. We're going to use the Python gpxplot script, written originally by Sergey Astanin, but forked by the author to provide more functionality. This fork is available on Github at https://github.com/geekinthesticks/GPXTools.

The script reads data from a GPX file and can output charts in several different formats. Run ./gpxplot.py --help for a summary of options. You will also need to install gnuplot.py, which is available for most distros.

If you don't have gnuplot.py installed, using the --gprint option will generate a GnuPlot script, which you can pass directly to GnuPlot from the command line.

To plot a chart of distance travelled against altitude to the screen:

./gpxplot.py --output-format gnuplot --y-axis elevation \
--x-axis distance --tzname 'Europe/
London' \
--imperial --file
arneddau.gpx

The default output is metric (height in metres, distance in kilometres), passing the --imperial option will show elevations in feet and distances in miles. If you want to output the plot to an image file, rather than the screen, use the --image option, followed by a filename.

The image format is determined from the filename extension, so --imagetrack.png would create a PNG file. Other supported output formats are GoogleChart, Gprint, table (columns separated by spaces) and orgtable (tables in Emacs org-mode format).

Times in GPX files are normally expressed in UTC format. If you want to show your data in local timezone format, use the --tzname option and pass the time zone in the following format – 'Europe/London'.

GnuPlot has many options to set colours, titles and a whole raft of other parameters. If you want to tweak the output, create a Gprint file and edit this to produce the formatting that you require. The edited Gprint file can then be used as input to GnuPlot to generate your final graph.

Conclusion

We have covered only the basics in this tutorial - there's lots more you can do. For example, you can add map layers for the Ordnance Survey and Google Maps, so your users are able to switch between different maps.

In addition to getting all the code for this article from the archives, you can get updates from the author's github repository at https://github.com/geekinthesticks/Plotting-gpx-Data-and-Showing-Geotagged-Photos-Using-OpenLayers.

Apple Maps uses 80% less data than Google Maps

Posted:

At last, here's something good to say about Apple Maps. According to analysis by data experts Onavo, Apple's in-house Maps app uses 80 per cent less data than Google Maps.

That's right, it's around five times more efficient.

That's because Apple Maps uses vector graphics, and so doesn't have to download map images again every time you adjust the view. The result? Less data consumed.

Of course Google Maps does have an offline mode as well, which isn't taken into account here. But still, it's some good news for anyone who bought an iPhone 5.

"On Google Maps, the average data loaded from the cellular network for each step was 1.3MB," Onavo says. "Apple Maps came in at 271KB - that's approximately 80 per cent less data!

"On some actions, such as zooming in to see a particular intersection, Apple Maps' efficiency edged close to 7x."

In Satellite View, Apple Maps uses around half as much data as Google Maps, Onavo found.

Mapology

CEO Tim Cook posted a letter to customers on the Apple website last week apologising for the failure of Apple Maps. He even recommended anyone using iOS 6 try alternative navigation aids like Nokia Maps, Microsoft's Bing, or Google Maps.

Apple Maps confused owners of iPhone 5s by putting entire towns in the wrong place, moving Heathrow Airport next to Hyde Park, and giving bridges and buildings a Dali-esque makeover.

As a result, just 4 per cent of iOS 6 users are sticking with it. In his open letter, Cook said Apple is working hard to improve Apple Maps.

Via BGR

Apple avoids Motorola trouble, gets hit with Passbook patent suit

Posted:

In August, Motorola filed a complaint with the U.S. International Trade Commission (ITC) against Apple, accusing the company of violating seven of its patents.

Nearly all Apple devices currently on the market figured into the complaint, including the iPod touch, several iPhones (including the iPhone 4S ), the iPad 2 and new iPad, as well as MacBook Pros and the MacBook Air.

The ITC was scheduled to look into the filings, but Tuesday Motorola withdrew the complaint, giving Apple somewhat of a reprieve from its recent litigious predicaments.

Had the ITC found Apple in violation of Motorola's patents, it could have imposed a U.S. sales ban on some or all of the products.

However, that brief moment of relief was soon lost when Apple was named in another new suit, this time from a developer alleging Passbook blatantly infringes on several of its patents.

Why Motorola's change of heart?

It's entirely possible Motorola and Apple could've come to some sort of an arrangement or settlement on the patent issue.

However, as Foss Patent pointed out, the motion to dismiss the charges made no mention of a deal between Apple or Motorola over the issue.

Foss Patent's Florian Mueller said the recent appointment of Judge Theodore Essex to the case may have influenced Motorola's decision.

As Mueller noted, Essex "doesn't make things easy for complainants," adding, "[m]aybe it's not just a coincidence that Motorola withdrew its complaint shortly after his appointment."

Earlier this year, HTC withdrew its patent infringement claim against Apple, though in that instance HTC's case was thought to have little merit.

TechRadar has reached out to Motorola, and will update this story if and when a request for comment is returned.

Passbook problems

Despite dodging the proverbial patent bullet from Motorola, Apple now faces a potential suit from yet another company.

Ameranth, a gaming and hospitality software developer, has filed a complaint with the ITC claiming Passbook violates four of its patents.

Introduced in iOS 6, Passbook stores information like boarding passes, coupons, and customer loyalty cards in one convenient location for ease of use.

The patents in question also deal with information storage and management, and to Ameranth, the way Passbook synchronizes that data is a bit too similar.

In the suit, Ameranth not only claimed Apple was aware of the patent violations, but also that one patent in particular (6,384,850) was "cited as a prior art reference in two Apple iPhone patents issued to named inventors Bas Ording and Steven P. Jobs."

Ameranth is looking to recover three times the amount of damages suffered as a result of Apple's infringements.

TechRadar has reached out to Ameranth and Apple, and will update this story if and when a request for comment is returned.

Yahuui Blog

Wednesday, October 3, 2012