One of the aims of Halcyon, the rendering engine behind Potlatch 2, is to provide a fully stylable map applet for web pages. Just drop in the SWF, give it a stylesheet (in MapCSS), and job done.

The SWF will talk to an OpenStreetMap database server. In theory you could point it at the main OSM API, but that’s primarily there for map editors, not as a free API for all-comers.

It would be friendlier to set up your own server. You can install the whole OSM ‘rails_port’ server stack for this, but it’s fairly complicated and involves a bunch of dependencies, so isn’t an option on your average shared host.

So I’ve written a little read-only API script for Halcyon use. You can get it from the OSM subversion repository in Potlatch 2/Halcyon’s resources directory.

It’s in Perl, comprises only one script, and has no dependencies (other than DBI for database access, obviously). It uses Flash’s AMF transport format, which is compact and fast. It needs only the current_ tables for nodes, ways, relations and tags, plus a nominal user 1 and changeset 1.

You might set it up as follows:

1. Download some OSM data (e.g. a planet extract)
2. Use Osmosis on your local machine to convert it to SQL and then upload the SQL dump to your remote host… or do it all on the remote host if you can
3. Upload tinyamf.cgi somewhere executable
4. Upload halcyon_viewer.swf, swfobject.js, a MapCSS stylesheet and any icons/fills necessary
5. Put the necessary embedding HTML in your page
6. That’s it!

I’ll post a real live example somewhere in due course.

Future versions of Halcyon will also be able to read directly from .osm files, so you won’t even need a database.

Today, oh happy day, Ordnance Survey released a bunch of map data for free. It’s an attribution-only licence, so you can do whatever you like with it as long as you say “data thanks to the nice guys at Ordnance Survey”. You can make maps, write clever location-sensitive apps, all that sort of thing.

Isn’t that great? Hoorah for the Ordnance Survey; for the guys at the Department of Communities & Local Government and Cabinet Office; and for Tim Berners-Lee and Nigel Shadbolt who pushed all this through.

The data includes postcode data and simple raster streetmaps. But perhaps the most interesting, for now, is a dataset called Meridian2 – real vector data that you can parse. Let’s take a look at how you can use it.

For this tutorial, you’ll need a computer with Perl installed and a fair bit of Perl knowledge.

What you’ll need

First of all, download the Meridian2 data from the Ordnance Survey’s OpenData site. (At the time of writing, the OS site is fairly bombed out and you might be better off getting it from the mirror site kindly provided by MySociety.)

You’ll see that these have bizarre file extensions: .shp, .shx, .dbf. These are shapefiles, Ye Olde Paleocentric File Format invented by ESRI back in 1807. The Wikipedia article about shapefiles has all the nitty gritty and is worth reading.

You’ll need a Perl module to parse shapefiles. The module in question is Geo::ShapeFile. There are many ways of installing Perl modules, but I use the CPAN shell:

sudo perl -MCPAN -e shell
install Geo::ShapeFile
(happy message ensues)
quit

Parsing the data

So let’s fire up a text editor and write a Perl script to use this data.

We’ll start by calling up the Perl modules that we’ve just installed:

#!/usr/bin/perl -w

use Geo::ShapeFile::Point comp_includes_m => 0, comp_includes_z => 0;
use Geo::ShapeFile;

$fn=shift @ARGV;
$shp=new Geo::ShapeFile($fn);

Next, let’s take the filename of the shapefile, which we’ll supply on the command line when we call this script – so Perl puts it in the @ARGV array – and then load that shapefile.

$fn=shift @ARGV;
$shp=new Geo::ShapeFile($fn);

Easy so far. You’ll have seen that each Meridian2 shapefile is actually three files: something.shp, something.shx and something.dbf. .shp and .shx contain the geometry (the line of the road), while .dbf contains the metadata (the road name and number, and so on). Fortunately, we just need to supply the ’something’ to Geo::ShapeFile, and it adds the three extensions as it needs them.

The shapefile contains a long list of shapes. We’ll parse them one by one.

for (1..$shp->shapes()) {
$shape=$shp->get_shp_record($_);
%db   =$shp->get_dbf_record($_);
$type =$shp->type($shape->shape_type());

So what are $shape, %db and $type?

$type tells you what sort of shape we’re dealing with, and is a string which’ll typically be ‘PolyLine’ (a line made up of a list of points) or ‘Region’ (the same thing, but denoting an area). There are various other complicated types but they’re not found in Meridian2. In fact, Meridian2 makes your life easy by not mixing different shape types in each file.

%db is a hash of the metadata for this shape – the road name, number and so on. For example, a typical hash from the A-roads file is:

_deleted => ”,
INDICATOR => ”,
ROAD_NAME => ‘ROTHAY ROAD’,
METRES => 61,
OSODR => O16AU69Q9R0TW,
CODE => 3001,
NUMBER => ‘A593′

Pretty simple: to find the road number of this shape, all we need to do is look at $db{’NUMBER’}.

Reading a shape

So all we need to do now is parse the shape itself. A shape has several ‘parts’, and a part has several points. So first we iterate through the parts:

for ($i=0..$shape->num_parts()) {
@part=$shape->get_part($i);

Then we iterate through the points in that part:

foreach $point (@part) {
$x=$point->X();
$y=$point->Y();
# do something with X and Y, for example…
print “$db{’NUMBER’} goes through $x,$y\n”;
}

And really, that’s it. We’ve extracted the points in the line. Where I’ve put a comment saying “do something”, that’s where you come in. You could draw a line, colouring it differently based on the road number. You could compare it to the user’s location, and work out which one is closest. Or whatever. That’s the fun bit.

Close your loops, and we’re done.

Finally, you can run your script like this (adjusting the filename and paths accordingly):

perl meridian2.pl meridian2/data/a_road_polyline

and you might want to CTRL-C it as the entire A-road network of Britain scrolls by.

What data does Meridian2 offer?

First of all, there’s the roads, grouped by classification. The shapefiles are motorway_polyline, a_road_polyline, b_road_polyline, and minor_rd_polyline. Warning: the minor roads are big.

You’ve also got rivers (polylines), lakes (regions) and woodlands (regions); railways and coastlines (both polylines); county and district boundaries (regions, and pretty big); and dlua, which means ‘urban areas’ (another big set of regions). There’s a set of points, which are easier still to parse, covering stations, settlements, and roundabouts. Finally, there’s some presentation-only stuff (text and cartotext) which you shouldn’t need to worry about for raw hacking.

I’d suggest you start with motorway_polyline, which is a small, simple dataset, then move onto something bigger.

Another warning: the data is very broken up. You’ll find plenty of roads split into a zillion polylines with five points or maybe fewer. I did a bit of hacking to join them back together in my Perl script; depending on your needs you may want to do similar.

A note on co-ordinates

All Meridian2 X and Y co-ordinates are OS eastings and northings: in other words, metres east and north of an arbitrary point somewhere off the Isles of Scilly.

Eastings and northings are much nicer to work with than the lats and longs which the rest of the world uses. You don’t have to worry about complex projections, you can just scale them down to your screen. You can find the distance between two points using Pythagoras without any Great Circle nonsense. And so on.

On the other hand, none of the rest of the world uses them, for fairly obvious reasons.

So unless your project is UK-only, you’ll need another Perl module to convert between eastings/northings and lat/long. I’ve previously used Geo::Coordinates::OSGB, but Matthew Somerville points out in the comments (below) that MySociety’s code is more accurate.

Finally, here’s one I made earlier: meridian2.pl is a hacked-together Perl script to generate Adobe Illustrator files from Meridian2 shapefiles, while attributes.pl just outputs the attributes (%db) from a Meridian2 shapefile.

Enjoy! Questions welcome in the comments.

For a long time, the most infamous bug in Potlatch wasn’t actually a bug in Potlatch: it was a bug in Linux Flash Player.

Linux FP doesn’t let you input international characters (i.e. anything apart from ASCII 32-127) into a textfield… and hasn’t done for a long time. So all those overseas mappers who want to type their street name – can’t. Epic fail is, clearly, epic.

Here’s the Adobe bug report for it. Feel the anguish in those comments.

And here’s how you fix it, I think. Many, many thanks to Linux-using OpenStreetMappers for their reports from my little debugging script, and especially to Ævar Arnfjörð Bjarmason who obviously has a vested interest in it working. I use a Mac so can only go on what others have said!

(more…)

CPAN has some really good e-mail modules – including one written by the BBC. How cool is that? But the documentation isn’t entirely transparent unless you’re a MIME wizard… which I’m certainly not.

Here’s what I’ve found works to send an HTML format e-mail with attachments. (For these purposes I was never sending more than five attachments, but it’d be trivial to change.) Working out that the outer part needed to be multipart/mixed was the bit that took me the time – simply attaching files to an Email::MIME::CreateHTML message puts them as extra alternative parts in a multipart/alternative message, so a mailer might just display one of the attachments as the message body.

use Email::MIME;
use Email::MIME::CreateHTML;
use MIME::Types;
use Email::Send;
use Email::Abstract;

# -----------------------------------------------------------
# You'll need to define:
#
# $html - the HTML for the e-mail
# $plain - plain-text version
# $sender_name - name of sender
# $sender_email - address of sender
# $name - name of recipient
# $address - address of recipient
# $subject - subject line
# @body - attachments (slurped into $body[1] etc.)
# @fn - filenames for attachments

# ----- Create HTML

my $htmlmail = Email::MIME->create_html(
    header => [],
    body => $html,
    body_attributes => {
        disposition => 'inline' },
    text_body => $plain);

# ----- Create base message

my $email = Email::MIME->create(
    header => [
        From => "$sender_name <$sender_email>",
        To => "$name <$address>",
        Subject => $subject ],
    attributes => { content_type => "multipart/mixed" },
    parts => [$htmlmail]
);

# ----- Add attachments

for ($i=1; $i<=5; $i++) {
    next unless $attach[$i];
    ($mimetype,$encoding) = MIME::Types::by_suffix($fn[$i]);
    $att = Email::MIME->create(
        attributes => {
            content_type => $mimetype,
            filename => $fn[$i],
            encoding => $encoding,
            name => $fn[$i],
            disposition => "attachment" },
        header => [ 'Content-ID' => "$year$mon${mday}_${contentid}_$i" ],
        body => $body[$i]);
    $att->header_set('MIME-Version');
    $att->header_set('Date');
    $email->parts_add([$att]);
}

# ----- Send mail

$Email::Send::Sendmail::SENDMAIL = '/usr/sbin/sendmail';
my $sender=Email::Send->new({mailer => 'Sendmail'});
my $response=$sender->send($email);
return $response;

Matt Amos (who should blog more) has done a really cool visualisation of the editors used by OpenStreetMappers. Blue is Potlatch, red is JOSM, green is Merkaartor, and grey is everything else.

Really big disclaimer: this is based on the created_by tag, and JOSM and Potlatch treat this differently. JOSM sets the tag whenever it adds a new object: Potlatch sets it whenever it adds a new one or modifies an existing one. So there’ll be an inevitable bias towards Potlatch. (Potlatch’s behaviour is arguably more sensible but I’ll leave that for another time!) But despite all that – isn’t the big mass of blue pleasing to the eye? Well, I thought so.

See the full image here.

Getting dotted lines in ActionScript seems to be one of those old chestnuts. The ever-resourceful Senocular has a neat ActionScript 2 solution, ported to ActionScript 3 here.

It’s pretty good, but wasn’t quite right for my needs. First of all, it spends quite a lot of time dealing with curves, which I don’t need – polylines are enough. Secondly, it just lets you define one ‘on’ length and one ‘off’ length, so you can do 6px on and 3px off, but not (say) 6 3 1 3. 

So here’s my 2am answer. Pretty compact, seems to work happily. I’ve not genericised it into a DashedLine class or such because that wouldn’t work for my project and I have no reason to think it will for yours, but feel free to do so. I think it’s a bit shorter than Senocular’s, too, though that might just be due to multi-statement lines…

dashedline.as