UK Transport 2.0
It’s been a long time coming.
UK Transport is my app for multimodal public transport journey planning. When I worked for Agant Ltd., and developed UK Train Times, we often threw around ideas for an app that also included buses, coaches, ferries, London Underground, etc. Then, after I moved on, I realised I missed working on public transport apps. So I built my own, as a side project.
The first version of UK Transport shipped in early 2014. Part way through development, I made the decision to focus solely on schedule data, rather than realtime. This was partly in order to reduce the scope enough that I would actually ship something, and partly because the availability of realtime data back then was pretty poor. Good data was available for trains, but trains were already served by many other apps. So I shipped a multimodal planner using scheduled data. There were very few apps that did full door-to-door journey planning for the UK, so that was my unique selling point.
But I always intended to add realtime support. After shipping 1.0 I made a start, investigating sources of realtime data and building out the UI to support realtime departure boards and live progress. The first commits happened in February 2014.
The journey planner in the app is powered by SilverRail. My first port of call was to look what they provided in the way of realtime data. At the time, they offered realtime train predictions powered by either Darwin (the system belonging to National Rail Enquiries), or an experimental version powered by Network Rail Open Data. At the time, to use Darwin data you needed a paid-for license. I was (and still am) already paying per copy sold in order to use SilverRail’s journey planner, so an additional expensive API license was infeasible. But equally, the quality of SilverRail’s data based on the Network Rail APIs was poor. At first it didn’t support predictions, and when they were added they were naive and didn’t take into account the decisions made by the signallers.
I looked into using Realtime Trains to power the realtime information. While this may also have involved paying for another API, it would likely have been cheaper than Darwin, and the data and predictions were very good. In the end, I got quite far producing a version of UK Transport that used Realtime Trains to power the realtime departure boards and live progress.
But then, in May 2014, it was announced that Darwin would become free. This was a no-brainer to investigate. I could either adapt the app to query Darwin directly, or I could look into using SilverRail’s Darwin integration. While working on UK Train Times, I had experience of the direct Darwin API, and was aware of its limitations (notably a lack of a persistent ID representing a train service, meaning you could no longer refresh a train’s progress once it had left the station). So I looked into what SilverRail could offer.
SilverRail offered realtime data either through the SilverRail Journey Planner (SJP), which is what I already used, or through IPTIS, which is a rail-focussed API with additional features specific to rail data. I looked into the former first, since it would have required the fewest changes to UK Transport. But, at the time, there were some limitations: although the SJP offered realtime predictions, it didn’t distinguish between a predicted future arrival time compared with the actual time a train passed a station. I could only infer whether a train had passed a station by using the phone’s internal clock, which was not reliable. There were other issues too: problems with platform numbers, and some data reliability problems. (I believe that by 2016, most of these have been fixed. However, I went another route, as you’ll see if you read on!)
So I looked at IPTIS. The data was good quality, it was battle tested, and it seemed to meet my needs. It was yet another API to program into the app, and had its idiosyncrasies: IPTIS only lets you query information with three letter CRS codes (the codes that represent stations, such as LMS for Leamington Spa). It also only returns data using CRS codes: it doesn’t give the station names. You have to maintain your own list of station names inside the app in order to know what to present to the user. Also, I needed the mapping between the codes used by the SJP and by IPTIS, so I could know that the two APIs were talking about the same station or train.
But in spite of all these caveats, the data was good, so IPTIS was integrated into UK Transport. Next, what to do about buses…
The main UK realtime bus API is NextBuses, powered by Traveline. It’s been around for years, and is the same data that powers a lot of the orange dot-matrix signs on bus stops, or the SMS responder that lists the next buses at a stop. It’s supported in about 40 regions, including the West Midlands (where I live), but notably not London.
NextBuses is very simple, it only does one thing: you query a bus stop, and it tells you the next buses that will call at the stop. The results include both scheduled data and realtime data. However, it doesn’t give you any form of ID that refers to a certain bus or service, so there’s no way to track one particular bus as it moves stop to stop. You only get departure boards.
I wanted to be able to show the route for any given bus (as version one of UK Transport did). The SJP provided timetable bus data, so I decided to query both APIs, match the results up based on scheduled times, and use NextBuses to append realtime data to the SJP results. This works surprisingly well. There’s no live progress (you can’t tap on a row on the departure board and see where that bus is along its route), but (when you’re in a supported area) the departure boards show how many minutes late each one is.
I next looked at London buses. SilverRail had just started offering a TfL data feed, so I investigated that. Sadly it was not to prove very useful. You got the list of upcoming arrivals for a stop, along with their expected time. But unlike NextBuses, you didn’t also get a scheduled time. This is because London transport services (both bus and tube) are defined in terms of their frequency of arrival, so there isn’t really a timetable as we’d traditionally understand it. To make matters worse, the SilverRail TfL data at the time did not contain any form of vehicle ID, nor route code. So displaying the calling points for a service wouldn’t be possible, let alone a realtime location along its route.
In the summer of 2014, I had to take a number of months off work recovering from a hospital visit. I wasn’t able to concentrate well enough to work on an app, so development was put on hold. Then in the latter half of the year I started a new job, which consumed much of my time. It was December before I found much more of a chance to work on the app.
By this time it was the end of 2014. I’d also been improving version 1 along the way: adding support for larger screen sizes, building AGTable as a proper dynamic framework on iOS 8, various interface tweaks such as displaying the bus number on the journey planner. At this point I was also maintaining the Realtime Trains and IPTIS API support in parallel, since I didn’t yet know which one I’d end up shipping. It wasn’t until March 2015 that I decided to throw all my support behind IPTIS. London transport got left behind for a while.
I spent time adding support for the more obscure things that could come out of the train APIs: rail replacement buses, splitting and joining trains, tube trains that ran on the national rail tracks. By mid 2015 I was also playing with seeing what else I could pull out of NextBuses — I had a wonderful idea to fetch a bunch of queries for different stops along a bus route, and infer which item on each departure board was the same bus, thus being able to show a realtime dot. It worked, in a fashion, but it was way too slow on an actual device (since it needed loads of network requests)… and NextBuses has a pay-per-request business model. That was abandoned too.
My new job had turned me into a London commuter. Previously I had thought that if I couldn’t get realtime data for London working, it didn’t really matter — London was well served by other apps, whereas the rest of the country was typically left out in the cold. But now I was working in the capital, so I had a selfish need for that data. (It seemed like a cop-out to not use my own app for navigating!)
Not much work happened on the app between June 2015 and February 2016… I was too busy with my job, with the amateur opera society I was involved in, and with various other things. But the itch was still there. In February, I added a bunch of little things that I’d been wanting for ages, such as a “show current location” button on all the map screens. Now, it was finally time to tackle London transport.
The good thing was, by this time, TfL had launched their unified API, which had realtime information for bus, tube, DLR, tram and more. The data was a lot better quality than last time I looked into it. (Not perfect — some tube lines such as the Hammersmith and City are still full of errors, and Croydon tram data is abysmal, but better than it was.) Crucially, it did supply me with an ID representing each service, which meant that in theory displaying routes and live progress was possible. I started hooking the TfL API into UK Transport.
Did I run into problems? Of course! Based on the rest of this article, you shouldn’t be surprised! For instance, there’s no way to actually fetch the route of one specific tube train. You can fetch the route of a tube line… but it contains all the branches and joins. So how to display the route of one train? In the end, I settled with taking into account all the predictions for that train (which gave me a list of stations I knew it would call at), vetoing any branches that it couldn’t possibly go down, and then cutting the route off when the train got to its destination. In particular, this doesn’t care about stations the train has already passed, so I may be erroneously telling you a train came from Morden when it really started at Kennington. The data doesn’t exist for me to know either way. But it was good enough. Finally I had reasonable tube and London bus predictions!
I haven’t gone into the trouble with child vs parent stations (i.e. do we have a separate screen per platform of a tube station, or for the station as a whole). I haven’t mentioned platform number troubles, or the issues matching the IDs used by the journey planner with the IDs used by TfL. (Things like stringByReplacingOccurrencesOfString:@“9400” withString:@“940G”!) It’s not even going into the situation where one or the other of these APIs changes the unique identifier for a station. I haven’t discussed how, because I’m doing all of this client-side, any API changes necessitate a new app update, so I have to ensure resilience. I certainly haven’t gone into detail about all the regular expressions used to pull things like platform numbers out of station names (@“[^(]*\([^)]*\)[^(]*\(([^)]*)\).*”).
But enough is enough. Without running my own server and drawing my own inferences, I think I’ve done quite well incorporating realtime data from different sources into one multimodal app. Yes, the journey planner part only uses scheduled data (the realtime data appears on the departure boards and live progress views)… but again, the scope has to end somewhere in order to ship. My beta testers have been diligently using UK Transport 2.0 and reporting various issues for the past couple of months. It’s time to share it with the world.
I’m proud of what I’ve made so far. I use it all the time when navigating public transport. I hope you’ll like it too. And as before, this is not the end of the journey: there are plenty more things I could add in version 2.1, version 3.0, etc… in the future, when the stars align.