Système D

Free of comment redux (BBC edition)

Live tickers are quite the thing for election results, such as today's counts from the county council elections across England.

Unfortunately, the beloved BBC does insist on peppering its (otherwise excellent) live ticker with gobshites of the Speak You're Branes variety. One choice snippet from today's feed: "I would vote #UKIP at future elections as they stand for two key principles: Sovereignty and Democracy - unlike ConLibLab". Thanks, but MailOnline is that way ---->

So, as per making the Guardian readable, here's how to stop the bottom half of the Internet bubbling up into the live ticker. Find your user stylesheet, or install Stylish. Then simply add:

li.class-TWEET,li.class-EMAIL,li.class-SMS { display: none !important; }

Save, refresh, and enjoy an inanity-free BBC. Well, until Robert Peston comes on.

The great lost map scale

Walk into any map shop in Britain - even if it's just a humble branch of WH Smiths - and you'll find a shelf full of cheap road atlases, each at 1:250,000 or thereabouts. You'll find a selection of the classic OS Landranger at 1:50,000, the successor to the old "one-inch" map, and probably a slightly smaller collection of OS Explorer maps at 1:25,000.

But you're unlikely to find anything at 1:100,000. The scale has almost disappeared entirely from printed maps in Britain.

Its pre-metrication equivalent, the "half-inch" map (1:126,720), was once nearly as popular as the one-inch map. The Ordnance Survey formerly offered complete half-inch coverage of England and Wales, but the real masters of the art were Bartholomews. Their (newly metricated) series faded out in the '70s and '80s, but even today, it's such a commonplace in second-hand bookshops that you can usually pick up sheets for £1 each or less.

I have a real weakness for 1:100,000 maps. Partly it's because, for a cyclist, they're so much more practical. 1:25,000s are way too bulky for anything but the shortest ride, and let's not get into the Ordnance Survey's double-sided printing. 1:50,000s are ok for an afternoon ride if the sheetlines are favourable, but on any longer tour, you'll be filling your panniers up with them. (I think I gave up after trying to bundle seven Landrangers into a knackered old pannier for Lon Las Cymru.)

1:100,000s, however, give you the lie of the land - the woods, the contours, the rivers, the villages - in a fraction of the space. In a convincing (but sadly paywalled) article, Richard Oliver, historian of the 20th century Ordnance Survey, persuasively argues that 1:100,000-1:125,000 is the ideal scale for cycle touring. As a cyclist, I agree entirely.

And as a cartographer, I love the challenge. Any idiot, frankly, can draw a usable large-scale map. The real skill is in getting the information across in a small space... while still creating a beautiful map, one that communicates the lie of the land evocatively as only a good map can. The Landranger series, surely the world's greatest mass-market map, achieves wonders at 1:50,000 - but for a fairly ill-defined audience: walkers need 1:25,000, while motorists are happy with large-scale road atlases (or, increasingly, print-outs from the My First Cartography exercises of popular web mapping sites).

There's much more potential in designing something with a clear audience in mind. So my series of cruising guide maps for Waterways World (which has now grown to cover the entire canal network) has generally been at around 1:100,000. I certainly won't claim it's flawless - the monthly deadlines and the small budgets of a specialist magazine preclude that. But I hope it's clear and effective.

Sustrans' route maps, as drawn by Stirling Surveys, are also 1:100,000 maps of a linear route, prioritising the immediate surroundings of the route while spending less time on the further-flung corners. Rightly lauded, they clearly take a lot of time to draw, and I'm sad but not entirely surprised that Sustrans has been using less elaborate cartography from Four Point Mapping (ex-Cycle City Guides) of late.

So let's take a look at some older 1:100,000 maps, because we can.

This first example is from the Ordnance Survey Road Map of Oxford and Swindon (half-inch to the mile). The series was essentially completed in 1912-1915 though this edition is from 1927. Map dealer Forbes Robertson calls it "one of the most beautiful editions produced by the OS", and I can't argue with that.

Note not only the stipples for the great estates of Ditchley, Heythrop, Cornbury and Kiddington, but the elaborate handiwork for woodlands. Hill shading is discreet but highly effective: the Evenlode valley really stands out from the higher land around Chipping Norton. The colour palette is commendably restrained and something that many modern cartographers could learn from.

Moving on, this "Special District (RELIEF) Map" of Greater London dates from 1935 and was one of a very small series produced at the time (Birmingham, the Peak District, and the Cotswolds were the only other areas to be included).

The relief shading is rather peculiar, with a gentle orange wash bearing little relation to the contour lines, which almost look like they're out of register. The colours are more strident, but still chosen from a small palette. The typography is remarkably elegant - from the all-caps names such as Ampthill and Woburn, down to the tiny italics for railway station names (top left). It's a map where the whole is greater than the sum of its parts.

This map of Snowdonia is pretty much the last half-inch map the Ordnance Survey ever produced, a lone survivor which struggled on into the 1980s. (I'm trying to forget the unforgivable 1990s Tour series, noddy road-atlas cartography blown up to 1:100,000 in the colour photocopier, and a blemish on the reputation of the world's finest national mapping agency.)

The relief shading is really quite startling, even conceding that my scanner has picked out the colours more vividly than the original. Yet the design is a clear descendant of the 1920s series and retains its simplicity and immediacy; there's no long catalogue of colours and typefaces, just one, consistent design. I particularly like the "Mountain Rescue Post" labelling: why design an obscure symbol for a rarely-used feature when you can simply label it textually.

This Bartholomews half-inch map of the Peak District claims to have been published in 1946, but it doesn't really matter; the cartography didn't change that much over the 20th century. It's not dissimilar to the 1920s OS Road Map shown above, especially in its rather subdued style - something that must have seemed an anachronism in the 1970s as the gaudy road-atlas cartography of George Philip and the RAC was becoming the standard.

These maps were endorsed and (in an early precursor of OSM?) corrected by Cyclists' Touring Club members, and you can understand why: roads and contours are absolutely dominant here, with the odd spot-height, railway station, and helpful annotation of "Inn". What more does the cyclist need?

But the art of the 1:100,000 map is not completely dead. This is a scan from the contemporary AA Close-Up Atlas, one of two massive road atlases (the other being the Philip's Navigator) to provide national coverage at this scale.

Some people are rather sniffy about the AA's cartography, but I'm a fan. The typography and palette are well chosen. I very much like the way that residential roads are shown, but uncased. And there's a willingness not to simply copy what everyone else is doing. Look at the datasets incorporated into this example: pubs from the Good Pub Guide, petrol stations and speed cameras, names for country lanes, and best of all, the National Cycle Network. (Yep - that dotted green line is National Route 57.)

It's not quite suitable for cycling. The hill-shading is a barely discernible grey wash rather than actual contours that might tell you what lies in wait, and, well, the book is both rather heavy and too expensive for you to tear up happily. But it's close.

I don't hold out any great hopes for a revival in 1:100,000 printed mapping. But whenever I encounter a new mapping site, I take a while to look around zoom levels 12 and 13. One day, I hope, I'll encounter something as fine as these earlier examples.

Using console.log from Flash

Flash Debug Player has its own logging ability, but it requires setting up obscure config files, and then running a console viewer of some sort (even if it's just tail -f). Wouldn't it be much nicer if you could just use console.log as you do in JavaScript?

You can, and it's remarkably easy. Better still, it'll work in any Flash Player, not just the debugger. Just make sure you save this as console.as somewhere in your project, and import it into any code that needs it:

package {
    import flash.external.ExternalInterface;

    public class console {
        public static function log(...args):void {
            args.unshift("console.log");
            ExternalInterface.call.apply(null,args);
        }
    }
}

Then, to log to your browser console, just use console.log("Something to log", "Something else", 51237) or whatever you want.

One slight gotcha: Flash will attempt to convert anything you pass into JavaScript objects. That's generally good, but it will complain once you reach a certain depth limit. If that's causing it to fail, just cast to String instead: console.log(String(someObject)).

Site notes 1: Building from the ground up

I'm spending a lot of time at the moment working on a new web project. It's something I'm absolutely psyched about; it's one of the reasons I decided to make the leap from full-time salaried employment; it's got, I think, good potential; and, well, it's fun. But sitting all day in front of a computer with a (fairly) single-minded purpose can get a little wearing, and everyone needs the chance to vent sometimes.

I'm not ready to talk about the project itself yet, other than to say yes, it does have some maps, and no, it's nothing to do with canals. Instead, I'm intending to jot down a couple of notes here as the build progresses, noting what I'm enjoying working on, and owning up to the mistakes and rethinks along the way.

That's especially true right now, having just realised I made the wrong technology choice right at the beginning.

It's a content-heavy project, so it needed, I figured, a heavy Content Management System. My modus operandi is usually to build pretty much everything myself, but this site was going to be Serious Business. Enough time had elapsed since the Tridion nightmare back in 2005 for me to think about CMSs again. So, given that I wanted something open-source and highly hackable, Drupal was the obvious choice.

It was also the wrong one, but it took me six months to realise that.

That's not at all a slur on Drupal. Over those six months, I grew to like it a lot. Like everything, it has its quirks, its annoyances, but it's got plenty going for it. Modules for pretty much everything. A robust and well-designed admin interface. A friendly and clued-up community. Most of the stuff that baffled me at first (like the menu/router system), in time, I came to understand and occasionally respect - though I still couldn't get the hang of drush, admittedly. Anyway, I built a few little things in it, like the canal.travel linkblog, and it works well.

Then why switch?

Because I realised that I wasn't building the site I wanted. The technology was steering me to do things this way, that way. The good stuff I was getting from Drupal (admin interface, user management) was starting to be outweighed by the stuff it couldn't do with an easily downloadable module or some modest customisation, let alone out-of-the-box. In the end it was a forum issue which pushed me over the edge, but it could have been anything. And as time went on, I knew the balance would tilt even more.

The most successful thing I've ever built is the Charlbury website. That might not seem an obvious choice given that Potlatch 2 has over 50,000 users and Waterscape.com had, I guess, a decent six-figure number of visitors, whereas the Charlbury website has just under 5,000 unique visitors a month. That doesn't sound a lot... until you realise that Charlbury has a population of 3,000.

There are several reasons for that, but the main one, I think, is that the site has been able to iterate to meet the needs of the community. If someone comes up to me in the pub and says "you know, it'd be really useful if the website could do X", I don't have to tut and reply "sorry, don't think that'll be possible with our CMS". If it can be done, and I've got the branes to work out how - and the asker buys me a pint - then it gets done. Sure, the corollary is that I've had to write stuff that an off-the-shelf CMS would give me (like user management), but the Charlbury site is distinctive enough that there's not been too much of that.

So that's the model I'm returning to. It's not for everyone, but it suits how I work.

Of course, technology moves on. The Charlbury website recently celebrated its 10th anniversary, so it's no huge surprise that it's written in Ye Olde Perl as a bunch of not particularly elegant CGI scripts. There are better things in existence these days, so I've decided to build the new site in Rails... or almost. I'm actually using DataMapper with Rack, which when coupled with a router script is the same basic idea, but less likely to piss me off. Rails is "opinionated software" and I don't share all of DHH's opinions. (Did someone mention tabs?)

I think I've probably lost a fortnight to this, which is annoying, but better than realising the same thing six months after site launch. Encouragingly, I'm finding that I'm already working more efficiently in Ruby/DataMapper/Rack than I was in Drupal or than I do with the n thousand lines of Perl that make up the Charlbury site. A simple thing: every time I tick off a feature, I get to think "yay, I built that", rather than "whew, once again I managed to persuade the system to do something it didn't want to do". It's a much more enjoyable way of working.

It all starts with an editor – the blog edit

I gave a talk today at State of the Map US - officially the US local conference, but pretty much the second international one - about OpenStreetMap editors. Lots to say, and as ever, not enough time to say it in. So here’s my notes in slightly more prolix form. (Great word. Blame Catch-22 for that.)

The talk was called “It all starts with an editor” because that’s how OSM works: use editor to add data; it goes into a database; and a renderer makes a nice map out of the end of it. Incidentally, both the default editor and the default renderer come from the same tiny Oxfordshire town, Charlbury (population: 3,000). Go figure.

And my aim? To enthuse people to help make OSM’s editors better. It’s the single best thing you can do to help the OSM dataset grow, and I hope to explain why.

Part 1: why we need good editors

So let’s look at the editor of another crowdsourcing project: Wikipedia. Internet life without Wikipedia is now pretty much unthinkable, so this isn’t meant as excoriating criticism.

But still...

Wikipedia’s editor is a word-processor. We all know how to use a word-processor, so there’s a head-start right there. It has ‘bold’, ‘italic’ and ‘link’ buttons, just like your usual word-processor.

Except when you click ‘bold’, you get some text with asterisks either side. When you add a link, you get this vast amount of markup with ‘|cite ref’ and other obscure incantations. It’s like having bold and italic buttons in your word-processor, but what’s displayed on-screen is the raw RTF code. Seriously, I used word-processors on the Amstrad PCW in the mid-1980s that were more visual than this.

It’s not just the technology you have to get to grips with. Wikipedia has rules. Lots of them. I showed a slide of the names of Wikipedia’s policies - not the policies themselves, just their names - and it took up an entire four-column page of 10pt type. That’s a huge barrier entry.

Despite that… Wikipedia is world’s the sixth most popular website. So they must be doing something right.

If we were to take the same approach in OSM, we should ban Potlatch, mandate JOSM, and introduce lots more rules. Compulsory JOSM and lots of rules. Well, I guess the Germans would love it.

But it’s not enough. OSM is at its best when we harvest local knowledge. The principle for admittance to OSM should be “I know stuff that can go on this map”, not “I’m good at using an OSM editor”. The more local knowledge that can be added to OSM, the better the map is.

It’s already happening. I stumbled across a really good set of tutorials telling cyclists how to use Potlatch 2 to communicate their local knowledge to bikeplanner.org.

But it’s a moving problem. A typical OSM mailing list post in 2004: “Well, I got my patched Orinoco driver working, and I also got Kismet working and so it outputs what it finds by speaking to me with the Festival speech synthesis system”. We had a geek community for geeks, and the tools should be written to suit.

And in 2012: “Yes, I am new to osm. I want to add street names to the map in my town. The opening screen in Potlatch 2 shows on the left a list of points of interest: shopping, food and drink, etc. How do I add street names?”

The first guy clearly is going to be at home with JOSM (in fact, he’s Jono Bacon, now Ubuntu Community Manager). But the second guy. How do we help him?

The answer is that we write a more friendly editor. It sounds easy. It isn’t. Challenge 1: unlike Wikipedia and its word-processor, most people have never used a vector graphics app. Challenge 2: left unchecked, the tendency is for OSM’s barrier to entry to get higher, because people are inventing more and more complex tagging schemes. Challenge 3: our documentation sucks; the editor’s UI is the user’s sole guide. Challenge 4: browser-based and phone technologies are moving targets; five years ago you’d use Flash, today it’s all HTML 5.

And challenge 5 is that mapping is too much fun. Should I go out and survey a cycle route (yay!) or should I tackle one of those Hard Trac Tickets I’ve been putting off for months?

Traditionally there have been two schools of thought in writing editors. One is to minimise abstraction and give a raw-metal experience. Traditionally JOSM core tends to take that approach and any abstraction happens through plugins, whereas Potlatch has always abstracted - we even abstracted away a whole part of the data model (segments) in early versions of Potlatch 1.

Essentially, JOSM is a big powerful German-built 4x4, a raw, exhilarating driving experience, incredibly fast, very robust, and with all the bells and whistles you could ever want. Potlatch is a bicycle: simpler, not as speedy, essentially human-powered, but you can go anywhere with it and you don’t have to pass a test to use it.

I like bicycles.

Part 2: the Potlatch family

So, a brief history of Potlatch. Named after the newsletter of the Lettrist International, the predecessors of the Situationists. I wrote the first version because I wanted something that would let me edit OSM on my Mac (everything else was in Java, and OS X Java sucked), and because the whole nodes/segments/ways rigmarole was hard work in editors with no abstraction, and I wanted something as efficient as Illustrator’s line-drawing mode.

It wasn’t expressly conceived as a beginners’ editor. If there’s a design philosophy, it’s “a straightforward editor that doesn’t get in the way”. You should be able to do anything with it - i.e. it should give access to the full data model - but that’s not to say it has to be optimised for every single use case.

OSM needs a beginners’ editor and the Potlatch codebase is capable of providing that. But it wasn’t originally designed as such, and it’s not there yet.

Yet.

It started Potlatch 1 back in 200mumble. It grew like topsy, as did OSM. Perhaps its, and my, favourite hour was Tim Berners-Lee, doing his linked data spiel, demonstrating using P1 on-stage and saying “this is the future”. I shoud have just retired then; it doesn’t get better than that.

The UI is very much designed for the sort of mapping we were doing then. It’s more about creating, less about maintaining/refining. So, when you click the end-point of a way, it automatically assumes you want to extend it, rather than adding tags to the end-node. That was absolutely the right decision then. It’s probably not so appropriate now.

Potlatch 2 came along a few years later. Despite the name, it’s pretty much a complete rewrite; there’s virtually no shared code. Big changes include completely abstracting the tagging behind a friendly UI; a new MapCSS-based WYSIWYG renderer; and lots of tools to work with third-party data sources, because they’re a big part of mapping these days. (Lots of the credit for this goes to Andy Allan, who’s built an excellent SnapshotServer feature for this.)

Perhaps Potlatch 2’s greatest strength is that it has really good internal architecture, and that’s almost entirely the work of Dave Stubbs. Potlatch 1 was a bit of (well, a complete) lashup. (It was the first time I’d ever written any OO code. Go easy on me.) P2 is pretty much MVC, clean, and robust.

Another big thing with P2 was enabling custom deployments: customised versions for CycleStreets, HikeBikeMap, the Japanese community, and many other places. Even the USGS are using a version to update the National Map of America. That’s amazing and a real opening for the future.

So, chapter 3, and that’s iD.

iD is the P2 architecture ported to JavaScript. All the good stuff about the architecture is either ported or being ported. The one fundamental exception is the XML tagging preset architecture, which has proved to be a bit too complex for people, and which I’m replacing with a lighter, simpler, JSON-based system. But essentially, the underlying base is P2 in JavaScript.

Instead, I’d anticipate that the main change will be the UI.

I used to be an absolutely religious believer in modeless UI. I’m a long-term Mac user - my first one was a Mac IIsi in 1995 - and Apple’s Human Interface Guidelines, back then, extolled the virtues of modeless UIs. I love modeless UIs. No need to select a ‘draw tool’ or a ‘delete tool’ or a ‘move tool’ or any of that nonsense; instead, select item, do something with it.

It still works well, I think. Potlatch 2, for me, is very close to the way I like to work. But I know how P2 works (obviously I do, I wrote big chunks of it).

But put yourself in the shoes of a novice user. You create an OSM account, zoom in, and click ‘Edit’. Right… there’s a map.

No buttons.

What do I do now?

There’s two ways you can fix this. You can build a tutorial mode to hold the user’s hand; again, to take a Mac analogy, something like Apple Guide could be implemented and I’ve actually started work on that in P2.

The other thing you can do is (gasp) a modal UI. Modal UIs, if done well, can guide the user through what they want to do. You want to add a POI? Click the “Add POI” mode button, and follow the instructions.

I used to run scared from that because you end up creating wizards, and wizards are horrible; dialogue-based workflows that are fine for one-off config changes and first-time users, but really wasteful every time you want to draw a way or make a junction.

But my bolt-of-lightning realisation is that you can do both. If one of the modes (say, ‘Edit Object’) includes enough shortcuts to do P2-style modeless editing… then you’ve got the best of both worlds. The newbies get obvious modal editing, the experienced users get something seamless and fast.

iD is being written using the Dojo framework. Dojo is an MVC framework and toolkit with four main strengths. It’s very well suited for writing large applications, which an OSM editor certainly is. It has a great graphics library, and an OSM editor spends most of its time in graphics (and interactions with graphics) and not much in the traditional DOM. It has good UI widgets. And it’s an all-in-one solution: combining JQuery with Backbone with Require with JQuery UI is an alternative, but it’s nice that Dojo gives you the whole lot.

Funnily enough, those four strengths are exactly the strengths of the AS3/Flex framework which Potlatch 2 is written in. So porting the internals becomes very appealing.

(Incidentally, there are several reasons why it’s called iD. One is that doing anything in JavaScript involves getElementById. Another is that people find it extraordinarily, puzzlingly difficult to spell Potlatch. Another is that the 'i' is because I want something I can use on my iPad, and the 'D' is the D of Système D. And it’s also, continuing the French 1960s theme, a tribute to the Citroen iD.)

I keep going on about good architecture. Why is this important? Basically, because you really don’t want to reinvent this wheel every six months.

Take the ‘split way’ logic. This should be three lines of code; take original way, remove half the nodes, make a new way from those nodes. Except if you have a turn restriction, at which point you have to only keep that in one of the member ways. Or a route relation, in which case you have to preserve ordering, which is harder than it sounds. Or the way is P-shaped. Or whatever.

Andy wrote the 100 lines of code to do this and it took him a day. No-one else should have to waste that day. Reuse is good. The internals are very easy to get wrong… and they’re also incredibly boring to write.

Part 3: the future of OSM editing

Can we use this to build one single ultimate editor?

Google Map Maker is pretty exemplary. The UI is great for the beginner. It has some of the smartest programmers in the world behind it. But still, look at the GMM discussion groups, and you find comments like “I completely updated the island in OpenStreetMap in about 6 or 7 weeks. Changing roads in Google Map Maker is nearly impossible. Takes too long, the process is too unwieldy.” Or: “This editor is terrible. After working with the Potlatch editor in Open Street Maps I am very disappointed with GMM.”

I’m not damning GMM. It’s great at holding your hands into your first edit. There’s a whole lot we can learn from it. But you couldn’t build OSM with GMM alone. So we need more than one editor.

So, can we start to draw up a vision for future editor development?

Here’s a start. Three targets: desktop, online, mobile. Three levels: power-user, full-featured, targeted. And here’s an image of the grid.

You don’t have to fill every space. There’s not a lot of call for a version of JOSM that will run on the iPhone. Nor is there much point in writing single-purpose, targeted POI editors for desktop when you might as well fire up a browser and do it via the web.

I’m assuming that JOSM will continue, and long may it do so, with its precision German engineering. That fills the ‘power-user’ level perfectly, with a good second in Merkaartor.

And at the other end, there’s interesting developments in the ‘targeted editor’ space. Pushpin OSM is one. MapBox, with their Knight Foundation work, are also making really interesting noises in this space. It should be as easy for people to integrate an editor into their site as it is to integrate a slippy map, and having written that in the notes for my talk I was really pleased to hear that repeated later on.

But there’s still a space for a full-featured editor. As the bikeplanner.org tutorial showed, adding new cycle paths is a newbie task; and that requires a full geometry editor and relation support. Renaming a street - well, if only part of the street needs changing, you’ve got to do all that split way code.

This is the space that Potlatch, in its three incarnations, occupies. So the challenge for Potlatch is to remain as functional, but become easier to use.

We have to look at iD, rather than just incrementing P2, because Flash has two years left in the browser. We can do an AIR version for some platforms; iD is an answer to how we continue to supply it on the web.

But we shouldn’t just have two editors. To quote Chairman Mao, “let a thousand flowers bloom”. There should be more editors taking more new approaches. Let’s not reinvent the wheel by all targeting the same space, but rather, build different tools to cater to the many different potential audiences.

Part 4: Over to you

So, why should you get involved? Because, to return to the talk title, it all starts with an editor. I love mapping, but I’m aware that one day spent improving P2’s UI will have a better overall effect on the project than one day spent gathering housenumbers in my town.

We need code. But code doesn’t magically happen. It needs people to write it. And in terms of bang per buck, coding editors is the single most effective way to improve the quality of OSM.

Small patches are just as welcome as big stuff. I’m a big fan of “little things that mean a lot”. For example: last week, I changed the P2 toolbox palette to hide the advanced stuff at first, and run an extra sanity check before running a potentially destructive operation. It took half an hour or so, and the French mappers are really happy.

If you don’t code editors, there are other ways you can help. Stop overloading new concepts onto the existing data model; we don’t want the ‘split way’ code to become any more complicated. (Just as a rough guide, if your tagging solution involves uuencoding image data into tags, for example, you’re doing it wrong.) Yes, we do need a proper area datatype.

Write some docs. Film tutorial videos. Or write tagging presets. Editor coders are not the best qualified people to write tag config files, and it’s a distraction from their main task.

And be nice to those who put the effort in to writing editors. There are only two types of people who write editors right now: the bloody-minded and those who are funded. So if a newbie makes a mistake using an editor, don’t condemn the editor out-of-hand and demand it’s banned. Because, believe me, if you put a 12-year-old behind the wheel of a fast, powerful, German-engineered 4x4, they can do a lot more damage than if you put them on a bike.

(Fortunately the sysadmins are calm, unemotional people and have only ever once succumbed to the call to ban an editor from OSM. That was version 1722-1727 of JOSM, with the famous bug that transposed lats and longs.)

To hack on Potlatch 2, you need the Flash debug player (a free download from Adobe); the Flex command-line compiler (free download from Adobe or Apache); and a browser. The code is in git; clone mine at github.com/systemed/potlatch2, not OSM’s. There are readmes on the wiki and in the source. You don’t have to bother with ant if you don’t want to; just follow the instructions for debug builds.

And to hack on iD, you don’t really need anything special at all. Just check out the code from github.com/systemed/id, check out the Dojo docs, and enjoy.

Dive in. OSM will be better for it.

TomTom: from PND to FUD

(TomTom data is such high quality that people have to put up signs like this. Photographed in Slaithwaite, West Yorkshire, by Tim Brown, licensed CC-BY 2.0.)

You can tell you've got your competitors rattled when they start producing knocking copy about you. It's a testament to the success of OpenStreetMap that TomTom, the troubled, declining manufacturer of satnavs (PNDs) and geodata vendor, has published a rather cynical example of FUD about "open-source maps".

PNDs are Personal Navigation Devices. FUD is Fear, Uncertainty, and Doubt.

It's not really for me to say whether TomTom is good at manufacturing PNDs... but happily, it's really not very good at creating FUD.

Vandalism

One of TomTom's allegations is that "open-source maps" are vulnerable to vandalism:

Indeed the major benefit – the community aspect – has itself presented problems, leaving maps wide open to attack. A highly publicised case saw a leading provider suffer over 100,000 individual attacks, including reversals of the recorded directions on one-way streets.

This is more than FUD. This is plain wrong. TomTom is clearly referring to the case where OpenStreetMap caught Google contractors vandalising OSM. But let's look at what Steve Coast and Mikel Maron's original blogpost actually says:

Two OpenStreetMap accounts have been vandalizing OSM in London, New York and elsewhere from Google’s IP address, the same address in India reported by Mocality.

The most obvious vandalism started around last Thursday last week from these particular users however it may take us some time to do a full analysis. In fact over the last year we have had over 102 thousand hits on OSM using at least 17 accounts from this Google IP.

Can you spot the slight flaw in TomTom's argument? Not all of those "102 thousand hits" were "attacks". In fact, no more than a couple of dozen were - and those were swiftly spotted, and fixed, by the OSM community.

Of course, it's flattering that Google is so interested in OpenStreetMap that they consulted it over 100,000 times. Maybe they were dissatisfied with their current data provider?

Looks like they were. In fact, they've dumped their old data provider in both the US and Europe. Who was the data provider they dumped? That's right, TomTom. Turns out that their "expert map-makers" aren't quite so irreplaceable.

As a side-note (and yes, I know, "the plural of anecdote is not data"), it took the satnav/PND manufacturers several years to get the one-way direction right for the street 50 yards from my home. OpenStreetMap has always had the correct direction.

So I'm not sure that OSM should take too many lessons from TomTom about such "extremely dangerous mapping errors". I think it's pretty certain that the users of satnavs in these cases (1, 2, 3) are more likely to have been using TomTom data when they came to grief than OSM. TomTom would be better advised to put its own house in order before criticising others.

Data coverage and quality

The second thrust of TomTom's attack is that OSM data is of lower coverage:

In one particular instance, a leading open source map was compared against a professional TomTom map, and shown to have a third less residential road coverage and 16% less basic map attributes such as street names.

This criticism is fair enough. OpenStreetMap is a growing project. If you carefully choose the place to make your comparison, it's not difficult to find somewhere where OSM has just 84% of the streets of TomTom.

For an objective view, take ITOworld's OSM Analysis (requires login), which compares OSM against Ordnance Survey data in the UK. 283 areas have better than 84% coverage (the majority in the high 90%s); 125 have worse. So the average figure, in the UK at least, is considerably better than that cited by TomTom - but it's not hard to find areas that back up their contention.

This should be comforting to TomTom... were it not for the inexorable climb in OSM's coverage. 16% this month may be 10% later in the year, and so on.

However, TomTom's second criticism is wrong right now.

Worse still, it blended pedestrian and car map geometry, and included ‘a high number of fields and forest trails’ classified as roads.

This betrays a complete lack of understanding of OpenStreetMap data.

Pedestrian and car map geometry is intentionally blended in OpenStreetMap. OSM is one, seamless dataset. Pedestrians can walk along most roads: therefore roads are linked to paths to preserve pedestrian routing.

The attributes ('tags') for each object indicate whether it is accessible to cars, pedestrians, or both. It is absolutely standard practice when working with OSM data to filter on the basis of tags, so that you only get the data you need. So if you don't want to drive down forest trails, you simply filter out 'highway=track'.

This approach enables sites like MapQuest Open to offer planet-wide routing for pedestrians and cyclists, while sites using TomTom data only have pedestrian detail in certain urban areas. According to a recent study, even back in 2009,

within more densely populated areas, OSM data offer more overall data than the commercial provider [TomTom]

concluding:

for all cities analyzed in the US and Germany, OSM data resulted in shorter shortest paths [i.e. more complete and efficient routing] [than] pedestrian commercial data sets.

Compare the 2012 datasets, and OSM is richer still - and not just in densely populated areas.

TomTom doesn't cite the survey to which it refers, but since their article first came to light, Pascal Neis recognised it as his paper. And once again, TomTom has misinterpreted its conclusions - whether deliberately or through ignorance, who knows. The original context is this:

in the middle of 2010 OSM surpassed TomTom in the total number of streets recorded. However, a high number of field and forest trails caused this advantage for OSM.

Again, these are not "classified as roads", as TomTom claims. They are tagged as "highway=track". Pascal's study chose to consider these as streets (and, if you drive a 4x4, they are), but there is no suggestion that the data is misattributed in OSM. If your application wants to only focus on tarmaced roads, you simply filter out "highway=track".

It's a shame that TomTom didn't have the courtesy to attribute the study by Pascal (with Dennis Zielstra and Alexander Zipf) so that readers could make up their own minds, instead choosing to take an out-of-context quote. Perhaps they were a little afraid that people would read on to the conclusion:

According to our projection for the future, this discrepancy should disappear by the middle or end of 2012, and the OSM dataset for Germany should then feature a comparative route network for cars as provided by TomTom.

If you'd like to talk OpenStreetMap with someone who, unlike TomTom, actually understands it, I'm now undertaking consultancy work: find out more.

Flex losing KeyboardEvents

One of those "in case anyone else has this problem" postings...

Flex 4 has an exasperating bug whereby the keyboard focus can be lost, causing your KeyboardEvent listeners not to fire when a key is pressed. Generally this seems to happen when a UI element with focus, such as a dialogue box, is closed: the focus stays on the (now non-existent) dialogue box button, rather than returning to the stage. Consequently, any KeyboardEvent listeners attached to the stage won't trigger.

One commonly suggested solution is to add "stage.focus=stage" to the function you call when closing the dialogue box. But that's a bit nasty if you have 20 dialogues in your app - you really shouldn't have to add custom code to each one.

So, instead, I chose to add this to the Event.ENTER_FRAME function that I have running anyway (but I suspect there are probably other hooks you can add it to):

if (stage.focus && !stage.contains(stage.focus))
    { stage.focus=stage; }

This simply checks that the focused element is actually on the stage, and if not, sets the focus back to the stage.

Gerry and the Holograms lyrics

(Because some things deserve to be found when you search for them.)

Here we come / We’re Gerry and the Holograms / There are now 16 of me / And before I was one man

Gerry and the Hologram / Gerry and the Hologram

Lasers, lights and rods of glass / Appear to make me round / That is to say all 16 of me / ’Cause I’m Gerry and the Hologram / Gerry and the Hologram

Did I say 16 of me? / I’m sorry, we’re only one [two three four] / The others are just fragments / Called Gerry and the Hologram

Gerry and the Hologram / Gerry and the Hologram

Thoughts on Flash

Apparently the title has been used before.

The commentary about the fate of Flash seems never-ending; but it was amped up this week by Adobe's announcement that it's killing Flash-on-mobile. Some is thoughtful, much merely crowing. But there's one common thread: it concerns a mythical single product called "Flash".

There is no such product.

Let's set out the basics. There is a language, ActionScript 3: it's a hybrid of Java and JavaScript, and much less painful than JavaScript. You can use Adobe software (such as Flash Professional, a paid-for product intended for designers, and Flex SDK, a free command-line compiler) to write code in AS3. This code runs either under Flash Player (a browser plugin). It can also be compiled into more-or-less-standalone apps for mobile (iOS and Android) or desktop, using a technology called AIR.

Both AIR and Flash Player supply the same API to AS3, for graphics, networking and the like. There's also a free UI/forms framework, Flex, itself written in AS3.

Out of all this, there's only one area where Adobe makes money: the paid-for authoring tools. This is, after all, Adobe's core business. But they don't have to sink money into a player for Photoshop, for Illustrator, even for InDesign (unless you count Reader). Flash Player is a cost, and a significant one.

As HTML5 advances, it's becoming clear that Flash retains two strengths. One is hardcore pixel-mashing, particularly for games... particularly for 3D games. (In Adobe's words, "high definition entertainment experiences".) Most of the advances in Flash Player 10 and 11 are dedicated to this.

The other is Flex. Flex is an easy, if occasionally exasperating, way to build form-driven 'Rich Internet Applications'. It's full-featured, it speaks XML nicely, and it's internationalised. The Flex framework, too, is being actively developed.

So we have the curious situation where one runtime (Flash Player) is used for both performance-critical 3D games and enterprise-level data-driven webapps.

The two are likely to suffer very different fates.

Games are moving out of the browser. Mobile apps are the new gaming platform, and one where AIR is making serious inroads - not least as the best-placed platform for deploying the same code to iOS and Android apps.

Meanwhile RIAs, or webapps, or whatever you want to call them, no longer need Flash Player. Improved browser feature sets (loosely HTML5) are more than capable of doing what, five years ago, required Flash Player.

Adobe's latest announcement signals the eventual demise of Flash Player - not the death of Flash.

Adobe is too frustratingly erratic a company for any 'predictions' to be worthwhile. So let's call this a mere 'guess' instead:

In a few years, people will still be writing apps in ActionScript 3. Some will be compiled, via AIR, to iOS and Android apps: this is principally the games market. That's a dead cert. Less certain, but still likely in my view, is that other apps based on Flex will be compiled down to JavaScript and deployed in the browser. Meanwhile, Adobe continues to make the same money from authoring tools - but without the cost of a desktop player.

Free of comment

In print I prefer the Independent, but on the web, the Guardian is easily my favourite news site. Its breadth of coverage and speed of updating is matched only by... well, the witlessness of its commenters. It's a bit like this XKCD, only with added Europhobia and Dawkins.

Inspired by shutup.css, I'm now using a custom stylesheet to hide comments on all Guardian articles. Just one line of CSS makes the whole site bearable.

Installing it on Safari is trivial:

1. Download the stylesheet
2. Save it somewhere
3. In Safari, select it in Preferences > Advanced > Style sheet

And that's it. It's simple in Opera (my new favourite browser for non-speedy connections), too. In Firefox I think you could use Stylish.