Bus Live Data Map

Bus Live Data Map  

  By: richardc on Dec. 14, 2017, 5:37 p.m.

Ref bus 'live' data map [http://smartcambridge.org/transport/map/]

Good concept, and quite hypnotic. Would like to understand more about the bus routes, but at the moment this web page looks rather too fragile to use.

1) Buses often appear to be in duplicate, or even triplicate, traversing the same route and starting and stopping together.

2) The symbols do not represent 'live' data - I can see Milton Road from my house, and buses a) do not appear when they should according to the map, but b) others which are not on the map appear without warning;

3) Occasionally buses take off across parks and through blocks of houses at great speed.

Could you indicate reasons and an idea of magnitude of the problem, please?

For example, appears that para 1 above is because some buses have two transponders, and 2b is because other buses do not have any, but an indication of roughly how many would be helpful. Thanks.

Re: Bus Live Data Map  

  By: ijl20 on Jan. 22, 2018, 2:49 p.m.

Apologies for a spectacularly delayed reply. If you'd like a feeble excuse it's that we're deeply buried in significant enhancements to the bus side of the platform...

For others reading this we're talking about the live 'real-time' bus map here: live bus map

1) Duplicate buses: Well spotted. This is was true with the 'data feed' (GTFS) we were receiving from the bus companies previously. A relatively small number of the buses were behaving as if they were carrying two separate position transmitters, transmitting slightly different locations on separate time schedules. At a guess, this is probably what was happening, so 10 points for a correct answer. By coincidence since your post we've moved over to an alternative data feed that doesn't seem to have that issue, but let us know if you spot it again. The analysis necessary to sort the data and work out where an apparent two buses were really two transponders on the same bus isn't something we can get to now as we are really busy with other stuff. Most importantly the new data feed has additional information such as the timetabled journey the bus is supposed to be following that radically improves our options in showing buses in a more meaningful way (see below).

2) The symbols aren't live, some buses appear without warning: The previous feed of position data was on a periodic 30-second batch transfer to us, so the position data was nominally 0..30 seconds old randomly for each bus. In practice there's an additional ~5-8 second delay in the bus companies sending us the data so you can add that to the delay. The new feed (SIRI) is transmitted on a more immediate basis (i.e. eliminating the stock 0..30 second delay) but there's still the delay of ~5-8 seconds before it comes to us. FWIW the data processing on our servers delays less than 0.01 seconds even when transmitting the same data to the five servers we use to process the data. Of course we are in dialogue with the data providers to ask they send the data when they get it without delay, and we're helping them improve their techniques. A bus appearing 'without warning' presumably means it's not sending it's position information. Not much we can do to detect that from our platform but gradually this will improve in part because of our work and the fact we're chivvying the industry to try a bit harder.

3) Buses take off across parks: We essentially move the bus in a straight line from point-to-point as we receive the position data, and sometimes this represents a jump as you've seen. In the prior version of the page (I think that you're referring to) we smoothly animated the bus over 30 seconds from point-to-point and with hindsight this introduced additional delay (due to the 30-second time of the animation) and made the 'park jumps' look worse. In the current version (see link above) we still move the bus point-to-point but do it in one second rather than thirty. Note at this stage the system itself has no idea of the map or the route the bus is supposed to be following, but see below.

We are working on a fundamental set of improvements in the use of the bus real-time data that have been enabled by the improved content of the 'SIRI' feed. The time/position data for the bus is pretty much unchanged, but now we can work out the timetabled journey each individual bus is on (i.e. it departed a particular stop at the start of its journey at a particular time). We have (already working) in development a high-speed 'timetable' server that given this "origin stop/time" can return the entire sequence of stops/times relevant for that particular bus (and can currently do this in 0.01 seconds). We are also working on advanced real-time analysis software that can accurately predict where on a bus journey a particular bus is, given its position and timetabled route (this is hardest part of what we're doing). This is a mix of Cambridge University research (where I work) combined with a genuine commitment to make information available to local travellers as that becomes possible.

 Last edited by: ijl20 on Jan. 22, 2018, 2:52 p.m., edited 1 time in total.

Re: Bus Live Data Map  

  By: richardc on Feb. 19, 2018, 10:35 a.m.

Thanks for explanation and for updates to displays. Icon is great – displaying the service number is what we need. I have done some sporadic monitoring over the last few weeks. Occasional pairs, but they seem genuine and ultimately split, so everything looks believable. You have also got rid of a couple of issues which were annoying – the screen tended to freeze after about a minute, and was also restoring to default zoom level with every update. Good to see they have been dealt with.

In terms of update rate, a modest delay is not a problem for those waiting at the stop. The critical part is that the system is believable until the bus comes into view. If your display has a lag of a few seconds, those at the stop will be pleasantly surprised when the bus arrives ‘early’.

However, the delay does seem a bit variable. General observation – nothing scientific – says it may be dependent upon the age of the vehicle. For older vehicles the delay may be up to about 25 seconds, but with modern buses, the delay is about 5-8 seconds. May only be indirectly related to the age of the vehicle – could be the sequence in which they are polled. If older equipment is polled first, with later vehicles (like the new PR fleet) being allocated a later position in the sequence, that would mean the PR fleet send information to the host a few seconds before the host updates you.

My only adverse comment would be that previously the movement was relatively smooth – albeit across the rooftops. Now the icons move forward and stop, apparently waiting for the next update. If you could get the industry to give you the information every 10 seconds, interpolation could be carried out in the following 10 seconds to produce a smoother display with maximum lag of 20 seconds.

Overall a huge improvement. Notice recent traffic on Twitter that apps are being compared favourably with the ‘real-time’ displays at the stops. I fail to understand how they can be so unreliable.

Re: Bus Live Data Map  

  By: jw35 on Feb. 23, 2018, 9:30 a.m.

I'm glad you like the improvements. The map is very much a 'work in prgress' both for showing the best we can currently achieve and for tryling different representations.

We are just consumers of the real-time data, we don't have any control over it and we have no particular insights into how it is created. Our analysis of the data we are recieveing shows that the frequency of reports by busses is predominantly every 20 seconds (95% of reports are within 21 sec of a previous report, 96% within 30 sec, 99% within 60 sec, and 99.9% within 240 seconds. The maximum observed delta is about 2.5 hours, but in all investigated cases deltas of over an hour are caused by otherwise bogus data.

What we don't currently have a good feel for is what reports we might be missing.

The distribution shows additional small peaks at 40, 60, 80, 100 sec, etc. presumably caused by failed or lost transmission. There are also peaks at 10 and 30 sec. (but not at 50 or above) suggesting some initial transitions on 10 second intervals. I would agree with your theory that perhaps some more modern vehicles may transmit more frequently. They could also reflect different monitoring equipment, perhaps in Stagecoach and Whippet busses.

We are currently going with the "move quickly to the last reported position" since we think it gives a better view of what we actually know. Future planned work on prediction will alow us to better show current predicted position. I'd argue that while a modest delay is not a problem for those waiting at the stop, it is a problem for anyone hurrying to a stop if the result is that they see the bus they wanted dissapearing into the distance - busses can cover quite a distance in 20 sec!


Jon Warbrick
Adaptive Cities Programme, University of Cambridge

Re: Bus Live Data Map  

  By: hught on June 5, 2018, 8:24 p.m.

The new interface is a definite improvement. However... the migration resulted in one key piece of functionality being "lost" (I don't doubt the data for it still exists, merely that it's no longer displayed to the user/potential passenger). Although the service number is now much more clearly and obviously displayed, what we now lack is any indication of the specific route that the particular bus operating that service is on. Clearly this doesn't matter on those few routes with absolutely no variation (such as the Citi 6). But for others it's crucial - for example, you need to know if a southbound Citi 2 is going to terminate at Sainsbury's or Addenbrooke's. Or which of the many out-of-town destinations a particular "B" journey is headed for.

And the same issue is relevant in those situations where traffic is at a standstill. You need to be able to tell whether a bus you're interested is heading inbound or outbound (unless it's at a terminus, obviously). Positioning on the map isn't precise enough for that to be clear. So all the user can do is wait for the bus to start moving again.

As I said above, I'm sure the data still exists and this is only a display issue.


Hugh Taylor

 Last edited by: hught on June 5, 2018, 8:25 p.m., edited 1 time in total.

Re: Bus Live Data Map  

  By: jw35 on June 7, 2018, 8:50 a.m.

Thank you, that's a good point. We haven't been activly working on the bus map recently, but a recent deployment has switched the default to one that uses a different internal communication method to access the real-time bus data. As you point out, this version does seem to display less information about the individual buses. I'll point this out to the people who are looking after the map.

As it happens the previous version of the map is still available at https://smartcambridge.org/transport/map_old/, though this may eventually go away.


Jon Warbrick
Adaptive Cities Programme, University of Cambridge

 Last edited by: jw35 on June 7, 2018, 9:01 a.m., edited 2 times in total.

Re: Bus Live Data Map  

  By: hught on June 7, 2018, 9:51 a.m.

And now for something completely different...

"Out of service" vehicles (sometimes?) show up on the map - but displaying "proper" service numbers. A moment ago a "26" was heading west along the A14 near Histon! OK, nobody's going to mistake that one for a real service. Here on the Huntingdon Rd I've seen the map showing various vehicles out of service, presumably heading back to the Fenstanton depot. At best this is unnecessary clutter; at worst, positively misleading. They don't show up on the County Council's RTPI data (or on any app I've seen). Perhaps the result of the different comms method you mentioned earlier. Is this data you can kill at your end, or does it need the operator to swtich off the GPS when the vehicle is out of service?

Re: Bus Live Data Map  

  By: jw35 on June 12, 2018, 9:19 a.m.

We have observed this too. There is nothing in the data we currently recieve that would alow us suppress such reports. Future work involves better matching of position reports to routes and the timetable which should alow us to detect buses that are not yet in service or which have finished their journey.

Your observation that other consumers of bus data don't have this problem is interesting...


Jon Warbrick
Adaptive Cities Programme, University of Cambridge

Re: Bus Live Data Map  

  By: hught on June 12, 2018, 11:51 a.m.

Ignore my final comment - it's unproven! The other apps and services I've encountered are all interested in "stop" data - not precise position of a vehicle (which is what you're doing). If the 26 is doing a circuit of the A14, then there are no stops to display any sort of information. And if a Busway "B" were heading up Huntingdon Rd, then the All Souls Lane stop wouldn't show anything, because we're not a stop along the "B" route. Perhaps if an "out of service" Citi 5 came up Huntingdon Rd (its normal route), then it would be interesting to see if the RTPI display said anything. But until then, ignore my comment.

Where this might become critical in the future is if something based on your platform were to offer me a "next bus at your stop", and simply picked up all vehicles heading in my direction, regardless of whether they're in service or not. But I think we're some way from that.

Re: Bus Live Data Map  

  By: richardc on June 27, 2018, 9 a.m.

Occasional buses on Milton Road also display the phenomenon mentioned by Hugh. It appears to be an operational issue. Buses travel along the road from the depot at Cowley Road to Drummer Street and vice-versa. Generally ‘Not in Service’, but odd ones have a rogue route code displayed, which implies the driver has failed to properly update.

The ANPR data also showed significant proportion entering system on Milton Road and leaving in Madingley direction (X5) or Saffron Walden (X13) for example, although of course the route code was not present on that data.

From your point of view it should be possible to detect buses not on correct route and ignore them, but then there is the contra-problem of buses on diversion. This does emphasise the importance of data verification, however. The original scheme proposed for adding extensive bus lanes to Milton Road indicated a huge number of buses as delayed, some by 30 minutes plus. Possibly similar data was not interrogated sufficiently rigorously in that case. Sure I do not need to labour the point here, but we need to be aware of how our information may be used by those who do not understand the limitations.

 Last edited by: richardc on June 27, 2018, 9:07 a.m., edited 1 time in total.

Re: Bus Live Data Map  

  By: ijl20 on June 28, 2018, 8:59 a.m.

In due course we'll get around to providing the next version of that bus map - for the past several months the main 'development' focus has been the SmartPanel with a combination of the layout framework, the multiple widget design, and also the detailed work on the analysis of bus timetables linked to the real-time bus position data (which is far more complex than it should be). We've already 'upgraded' the bus map to use the same source of real-time data as the widgets in the smartpanel (which was kind of two steps forwards, one backwards). In the next version of the bus map we'll try and take advantage of all the work we did on the smartpanel widgets.

So thanks for the feedback, and please keep it coming as it all helps. We (at the University) are not committing to any particular schedule as we're interleaving plenty of other work you don't see, but we are committed to continually enhancing the platform.