The SWF will talk to an OpenStreetMap database server. In theory you could point it at the main OSM API, but that’s primarily there for map editors, not as a free API for all-comers.

It would be friendlier to set up your own server. You can install the whole OSM ‘rails_port’ server stack for this, but it’s fairly complicated and involves a bunch of dependencies, so isn’t an option on your average shared host.

So I’ve written a little read-only API script for Halcyon use. You can get it from the OSM subversion repository in Potlatch 2/Halcyon’s resources directory.

It’s in Perl, comprises only one script, and has no dependencies (other than DBI for database access, obviously). It uses Flash’s AMF transport format, which is compact and fast. It needs only the current_ tables for nodes, ways, relations and tags, plus a nominal user 1 and changeset 1.

You might set it up as follows:

1. Download some OSM data (e.g. a planet extract)
2. Use Osmosis on your local machine to convert it to SQL and then upload the SQL dump to your remote host… or do it all on the remote host if you can
3. Upload tinyamf.cgi somewhere executable
4. Upload halcyon_viewer.swf, swfobject.js, a MapCSS stylesheet and any icons/fills necessary
5. Put the necessary embedding HTML in your page
6. That’s it!

I’ll post a real live example somewhere in due course.

Future versions of Halcyon will also be able to read directly from .osm files, so you won’t even need a database.

Gah. Give me a stash of Landrangers any day.

On Friday I succumbed and bought a 3G iPad. I’d just emerged from a stressful deadline and fancied some unaffordable retail therapy (sorry Mr HSBC. Please be nice to me), and I’ve always wanted an iPhone without the voice contract. And there was the small matter of having dropped my ghastly Huawei 3G modem in the canal which, frankly, is the least it deserved.

Having bought this, I couldn’t really resist Skobbler for the grand price of £1.50ish from the App Store.

Skobbler is a satnav app for the iPhone, and thus by extension, for the iPad in screen-doubling mode. It pulls down vector data and routes from the net, rather than having the geodata storedin memory. And the interesting bit, for me, is that it’s entirely based on OpenStreetMap data.

It’s very good. Rough edges abound, sure, but this is an amazingly nifty piece of code and design. I’d say it’s the fourth great product to spin off OSM. (For the record: OpenCycleMap, mkgmap, CycleStreets.)

The perspective display is the first “wow” moment: fairly standard in modern satnavs but a first for OSM. It pans and rotates beautifully, really making the most of the iPad’s CPU power. (I think the data is being served as vector tiles by CloudMade.)

The GPS accuracy is pretty good, and getting a fix is fast, too. Routes are pulled down quickly and revised when you go off-piste. Voice directions are clear, though the “Beware – watch your speed” gets a bit grating on the M42 with cameras at every gantry, and e A444 should be either the “four four four” or the “treble/triple four”.

The best bit: the OSM data, on the two journeys I tried (Charlbury-Hanborough and Charlbury-Burton), was more than up to the job. There was only one tiny missing street which caused some ambiguity – I’ll come to that later. But even as someone who looks at more OSM data than most, I was amazed how good it was: and the visibility that Skobbler brings to “getting routing right” can only make this better still. (Incidentally, their osmbugs interface is excellent and should be a good example for the main osm.org site to follow.)

What didn’t work?

The place selection UI is uncharacteristically clumsy, with various nested dialogues that didn’t seem too clear to me. It doesn’t appear to support postcodes, either, though no doubt the appearance of OS OpenData will change this.

The routing choices are a bit odd and seem to unnecessarily penalise roads below green-signed A roads. Skobbler tried to take me through Woodstock to Hanborough, which is a bit silly, and through Chipping Norton to Burton, which is odder still. In both cases I blithely sailed on the normal way and it recalculated quickly enough.

It fairly eats battery. Charlbury-Burton swallowed 25% of the charge, even when I put it to sleep on long streches of motorway. But then, most people will be powering it off a 12V adaptor.

Motorway junctions were often announced as “bear right” when they really meant “continue straight on”. Not really a problem. My biggest bugbear is how roundabouts are announced, and this seems to be common across all satnavs rather than being a fault peculiar to Skobbler:

“Take the third exit”

What? Is that this one over here, or was that the second, and does that little service road count, or… sorry, Mr HGV, didn’t mean to swerve across your lane… yes, beep to you too… bang.

“Continue straight on at the roundabout.” Please. It’s much easier. I think you can trust me to steer round the edge rather than ploughing straight through the middle.

Two reasons why this is particularly relevant to Skobbler. The “third exit” approach fails completely with a sometimes incomplete map database. At Wellesbourne it told me to take the second exit; actually, it should have been the third, but OSM was missing the real second exit (a service road to an industrial estate). “Turn right” would have been failsafe.

Secondly, that’s exaggerated still more when the database and the signs don’t agree, as here in Burton where OSM knows one road is the A5189 but the signs don’t tell you that. We should really tag that with ref:signed=no or somesuch, but “Take the second exit for the A5189″ is really, really confusing when there ain’t no sign of no 5189 (poetry).

But in the scheme of things that’s pretty minor. Skobbler is a seriously impressive achievement and one which more people should try… and not just because then I could tag a de facto 7.5T weight limit outside our house.

Incidentally, sorry for the lack of screenshot. I haven’t yet figured how to get WordPress to upload a pic from my iPad photo library. One thing at a time…

Inspired by this frankly hair-raising story of mini-Murdoch calling the editor of the Independent a fucking fuckwit (insightful commentary) and, of course, by Ridley Scott’s ‘1984′.

Isn’t that great? Hoorah for the Ordnance Survey; for the guys at the Department of Communities & Local Government and Cabinet Office; and for Tim Berners-Lee and Nigel Shadbolt who pushed all this through.

The data includes postcode data and simple raster streetmaps. But perhaps the most interesting, for now, is a dataset called Meridian2 – real vector data that you can parse. Let’s take a look at how you can use it.

For this tutorial, you’ll need a computer with Perl installed and a fair bit of Perl knowledge.

What you’ll need

First of all, download the Meridian2 data from the Ordnance Survey’s OpenData site. (At the time of writing, the OS site is fairly bombed out and you might be better off getting it from the mirror site kindly provided by MySociety.)

You’ll see that these have bizarre file extensions: .shp, .shx, .dbf. These are shapefiles, Ye Olde Paleocentric File Format invented by ESRI back in 1807. The Wikipedia article about shapefiles has all the nitty gritty and is worth reading.

You’ll need a Perl module to parse shapefiles. The module in question is Geo::ShapeFile. There are many ways of installing Perl modules, but I use the CPAN shell:

sudo perl -MCPAN -e shell
install Geo::ShapeFile
(happy message ensues)
quit

Parsing the data

So let’s fire up a text editor and write a Perl script to use this data.

We’ll start by calling up the Perl modules that we’ve just installed:

#!/usr/bin/perl -w

use Geo::ShapeFile::Point comp_includes_m => 0, comp_includes_z => 0;
use Geo::ShapeFile;

$fn=shift @ARGV;
$shp=new Geo::ShapeFile($fn);

Next, let’s take the filename of the shapefile, which we’ll supply on the command line when we call this script – so Perl puts it in the @ARGV array – and then load that shapefile.

$fn=shift @ARGV;
$shp=new Geo::ShapeFile($fn);

Easy so far. You’ll have seen that each Meridian2 shapefile is actually three files: something.shp, something.shx and something.dbf. .shp and .shx contain the geometry (the line of the road), while .dbf contains the metadata (the road name and number, and so on). Fortunately, we just need to supply the ’something’ to Geo::ShapeFile, and it adds the three extensions as it needs them.

The shapefile contains a long list of shapes. We’ll parse them one by one.

for (1..$shp->shapes()) {
$shape=$shp->get_shp_record($_);
%db   =$shp->get_dbf_record($_);
$type =$shp->type($shape->shape_type());

So what are $shape, %db and $type?

$type tells you what sort of shape we’re dealing with, and is a string which’ll typically be ‘PolyLine’ (a line made up of a list of points) or ‘Region’ (the same thing, but denoting an area). There are various other complicated types but they’re not found in Meridian2. In fact, Meridian2 makes your life easy by not mixing different shape types in each file.

%db is a hash of the metadata for this shape – the road name, number and so on. For example, a typical hash from the A-roads file is:

_deleted => ”,
INDICATOR => ”,
ROAD_NAME => ‘ROTHAY ROAD’,
METRES => 61,
OSODR => O16AU69Q9R0TW,
CODE => 3001,
NUMBER => ‘A593′

Pretty simple: to find the road number of this shape, all we need to do is look at $db{’NUMBER’}.

Reading a shape

So all we need to do now is parse the shape itself. A shape has several ‘parts’, and a part has several points. So first we iterate through the parts:

for ($i=0..$shape->num_parts()) {
@part=$shape->get_part($i);

Then we iterate through the points in that part:

foreach $point (@part) {
$x=$point->X();
$y=$point->Y();
# do something with X and Y, for example…
print “$db{’NUMBER’} goes through $x,$y\n”;
}

And really, that’s it. We’ve extracted the points in the line. Where I’ve put a comment saying “do something”, that’s where you come in. You could draw a line, colouring it differently based on the road number. You could compare it to the user’s location, and work out which one is closest. Or whatever. That’s the fun bit.

Close your loops, and we’re done.

Finally, you can run your script like this (adjusting the filename and paths accordingly):

perl meridian2.pl meridian2/data/a_road_polyline

and you might want to CTRL-C it as the entire A-road network of Britain scrolls by.

What data does Meridian2 offer?

First of all, there’s the roads, grouped by classification. The shapefiles are motorway_polyline, a_road_polyline, b_road_polyline, and minor_rd_polyline. Warning: the minor roads are big.

You’ve also got rivers (polylines), lakes (regions) and woodlands (regions); railways and coastlines (both polylines); county and district boundaries (regions, and pretty big); and dlua, which means ‘urban areas’ (another big set of regions). There’s a set of points, which are easier still to parse, covering stations, settlements, and roundabouts. Finally, there’s some presentation-only stuff (text and cartotext) which you shouldn’t need to worry about for raw hacking.

I’d suggest you start with motorway_polyline, which is a small, simple dataset, then move onto something bigger.

Another warning: the data is very broken up. You’ll find plenty of roads split into a zillion polylines with five points or maybe fewer. I did a bit of hacking to join them back together in my Perl script; depending on your needs you may want to do similar.

A note on co-ordinates

All Meridian2 X and Y co-ordinates are OS eastings and northings: in other words, metres east and north of an arbitrary point somewhere off the Isles of Scilly.

Eastings and northings are much nicer to work with than the lats and longs which the rest of the world uses. You don’t have to worry about complex projections, you can just scale them down to your screen. You can find the distance between two points using Pythagoras without any Great Circle nonsense. And so on.

On the other hand, none of the rest of the world uses them, for fairly obvious reasons.

So unless your project is UK-only, you’ll need another Perl module to convert between eastings/northings and lat/long. I’ve previously used Geo::Coordinates::OSGB, but Matthew Somerville points out in the comments (below) that MySociety’s code is more accurate.

Finally, here’s one I made earlier: meridian2.pl is a hacked-together Perl script to generate Adobe Illustrator files from Meridian2 shapefiles, while attributes.pl just outputs the attributes (%db) from a Meridian2 shapefile.

Enjoy! Questions welcome in the comments.

We could ask them. Actually, because we take an interest in these things – because we already go out and talk to people – we largely know. The roads are perceived as dangerous. Where there are safe routes, people don’t know about them. Helmet hair. Nowhere to park the bike. It rains.

What should we be doing?

Look! I fixed it for you! Everyone will start cycling to work now!

What?

So here, after much searching and head-scratching, is something that worksforme(tm). This post is here more as an aide-memoire than anything else, but someone else may find it useful.

Disclaimer: I’m still on Illustrator CS; all of this will no doubt be different in CS27 or whatever other overpriced piece of crap Adobe have come up with this week.

1. Download this PPD file and save it somewhere memorable. (I chose the Illustrator app folder.)
2. Go to File ≫ Print. Curse Adobe for not using the proper OS X print dialogue like everyone else does.
3. Change Printer to ‘Adobe PostScript® File’.
4. Change PPD to ‘Other…’, then select the PPD file from above.
5. Select your page size from the Media popup, e.g. A1. I guess if you were producing a funny-sized leaflet you could manually edit the PPD file to have your size in it – it’s just plain text.
6. Go to Setup and check that the Origin X and Origin Y are both 0mm.
7. Click ‘Save’ and call the file artwork.ps or something like that.
8. Open the resulting .ps file in Preview (context-click ≫ Open With ≫ Preview).
9. Breath a sigh of relief that you are now in a properly designed Apple app and don’t have to undergo the indignities of Adobe again.
10. Use File ≫ Save As… to save it as a PDF.

The tl;dr* version:

Generally good. Don’t release 1:25k and 1:50k rasters, that’s just gratuitous and we don’t need them. Provide an aerial photography API for rights-free tracing. Use PD or a database rights-aware licence, not CC-BY. kthxbye

* link almost certainly Not Suitable For Work, or indeed anywhere else

I do it like that because Old Charts do it like that too; because there’s a deliberately slightly retro feel to the WW maps (they’re essentially strip maps, like cruising guides used to have); and because I think it’s a great signifier of “we care about cartography” rather than “we have just thrown an OS map at our designer and told him to trace it, though not too obviously”.

Old Charts do it for a proper reason.

Roughly every two months someone complains that either a) the scale on OpenStreetMap is wrong or b) that OpenStreetMap doesn’t have a scale. (To be precise, a) happens, then the scale gets removed to stop the whingeing, then two months later b) happens.) You can read typical threads here and here but, in brief, the reason is because the scale isn’t consistent across latitudes, and that’s quite significant when you’re zoomed out a lot.

A bunch of us were talking about that this weekend, which is what just made me remember this.

On Old Charts, one bar represents one minute of latitude (or longitude). So the length of the latitude bars actually differs across the map.

The above is from a 3Mb university course presentation which blethers on about it for a while. 

It’s an excellent solution. I wouldn’t imagine there’s any likelihood of OpenLayers et al adding it any time soon. But doesn’t it look great?

I used to have a tablet computer. It was a beauty. It was A4 sized and had a full, moving-parts keyboard. The word-processor was speedy and yet powerful; but it was a proper computer, too. You could even program it. And it cost about £150.

Ah, Amstrad NC100 Notepad, how I miss thee.

It was such a great little machine. I’d tuck it under my arm then head off to the University Library in Cambridge to research some article or other I was writing for Keyboard Review. Then, at home, I’d connect up the serial-to-AppleTalk lead, run ZTerm, and fire the documents across. Bar a bit of search-and-replace to get rid of extraneous control characters, that was it. Done.

(The word-processing software was actually Protext, with which I was of course very familiar. Protext was the WP of choice on the Amstrad CPC, but also a ridiculously fast raw text editor. It came on a 16k sideways ROM; on another ROM I had Maxam 1.5, Arnor’s assembler. You’d write your source code in Protext then type ‘asm’, and Maxam would assemble it. To this day, the reason I indent code with hard tabs rather than spaces is Protext’s doing; a tab took up one byte; a space took up six. When you only had 38k for text, that was a big difference.

At one point I wrote 90% of the code, but only 30% of the UI, for a wizard hack that brought WYSIWYG editing and embedded graphics to Protext (CPC). It was going to be called Fidelity, after the utterly superb Durutti Column album. I never finished it but you should still buy the album.)

The NC100 used four real AA batteries, which is almost always better than any proprietary solution. (Maybe excepting Sony’s digital camera batteries.)

It had two failings. One is that Amstrad had always, from the CPC on, skimped on the keyboard decoding circuitry. If you pressed three keys together, a fourth character would also result. For the fast typist this is a real problem. On the NC100 there was one very common three-key combination (might have been I-O-N, I can’t remember) where the ghostly fourth key was cursor-up. So you’d be touch-typing at eight billion words a minute, and would briefly look down at the screen, only to see that three sentences ago you’d unwittingly ‘pressed’ cursor-up and all your subsequent text had been inserted into the previous line. This happened to me so many times.

The other failing is that Amstrad was enormously value-conscious. Which is generally a good thing, but I’d have preferred a £180 armour-plated NC100 to a £150 where I broke the power socket twice (which Rob Scott could fix) and the screen once (which he couldn’t).

If Apple were to build one of these, maybe with Mobile Safari and 3G networking, would I buy one? Hell yes.

Good news for:

• Google, Yahoo, Microsoft. Free maps, and unlike the US, good-quality free maps which they can start using right out the box.
• Ordnance Survey. I wrote here previously that OS’s best chance of surviving was to open up street name/geometries, boundaries, postcodes, peaks, rivers and PROWs, and to keep charging for the large-scale stuff. This seems to be pretty much what’s promised. I still believe that it’s absolutely the right decision for them. (Also, I am rather smug.)
• The Guardian. Launching a campaign is a risky business for any publication, especially a fairly obscure and, at times, seemingly fruitless campaign like ‘Free Our Data’. It has paid off – and of all the organisations campaigning for this, the Guardian is the only one that anyone has ever heard of.
• Apple et al. Insofar as Apple ever gives a shit about anything that happens outside the US, they no longer have to depend on anyone for UK iPhone maps. Not Google, not Tele Atlas. No-one. (Incidentally, if UK mobile carriers had any brains, they would now write their own mapping app and bundle it with their iPhone contracts. Fortunately they don’t.)
• Cartographers. Maps will now compete on cartography, not on data. This is an absolute shot in the arm for skilled cartographers and could go a long way to reviving the craft in the UK. With my Waterways World hat on, I’m delighted: our cruising guide maps can get better than they are now, yet anyone wanting to compete still has to learn how to produce lovely maps.
• Developers. Same applies. I am really looking forward to what people come up with. If I were an iPhone dev I would start writing that killer app now, ready to release when the data arrives.
• Wider Government. Full release instantly becomes the standard for public data. There is now absolutely no excuse for, say, the Environment Agency to withhold its fisheries data. That means more third-party sites that do funky things with public data. I suspect that will help in breaking the stranglehold of evil big outsourcers on Government IT projects.
• This blog because I can stop writing about boring map copyright law and start writing about fun things, like canals, organs and the new William Orbit album.

Possibly good news for:

• OpenStreetMap. I don’t think it’s a stretch to say this wouldn’t have happened without OSM. The inevitability that OSM would, in time, catch up with OS small-scale mapping absolutely vindicates the project. And, hey, complete data for the whole UK – what could be cooler?
      But on the other hand, everyone else has it, too. How do ongoing changes get integrated into the OSM database? Will the UK community survive a sudden change in tack from surveying the basemap to becoming a provider of ‘added value’? Will smaller public domain mapping projects create an informal, developer-led community without OSM’s harsh share-alike restriction? Will UK OSM developers (who lead the project) get bored of it now there’s not such a unique need? How many questions can I get in one paragraph?
      Oh, and there’s the licence. I dread to think what would happen if the chosen licence wasn’t compatible with OSM.

Bad news for:

• Tele Atlas and Navteq. See G-Y-M above. On the up side, their parent companies no longer have to bother collecting UK data for their satnavs/mobile phones. But that’s like saying Tesco giving free food away is good news for Sainsbury’s, which can now take it and resell it for 1p.