When is it (not) a good time to do maintenance?

With the global nature of the Internet and globalisation of businesses, there’s never really a good time to do maintenance. When it’s 1am in London, it’s 5pm in Silicon Valley and people are trying to wrap up their work day, and it’s first thing in the morning in Hong Kong, neither are going to be happy if they have a maintenance outage to deal with at such important times of the day.

So, you choose your disruptive maintenance windows carefully, to try and cause the smallest impact that you can.

However, if you know the users of the system are local, it’s much easier to choose your maintenance windows: usually when there are the least users on the system.

Try telling that to Transport for London.

This is the front end to TfL’s “Countdown” system. It tells you which buses are due at a given bus stop and an approximate time that they arrive. The countdown database is updated using location equipment on the buses, and drives LED displays at bus stops, and is accessible over the Internet, including a user interface designed for mobiles, and via SMS short code.

It’s especially useful when services on a route are infrequent, such as on a Sunday, where you may be looking at waiting up to 20 minutes for a bus if you managed to just miss the previous one. So, look back at the screen grab above, note the start time for the maintenance window.

Why do TfL think it’s a great idea to take the system down right at the time on a Sunday that people are heading out to visit family, maybe go out for Sunday lunch, or head to sporting events?

Wouldn’t a better time be the middle of the night on a Monday, when things are much quieter, with fewer users?

Paying techs extra to do system maintenance on a Sunday can’t be cheap either?

End of the line for buffers?

This could get confusing…The End of the Line by Alan Walker, licenced for reuse under the Creative Commons License

buffer (noun)

1. something or someone that helps protect from harm,

2. the protective metal parts at the front and back of a train or at the end of a track, that reduce damage if the train hits something,

3. a chemical that keeps a liquid from becoming more or less acidic

…or in this case, none of the above, because I’m referring to network buffers.

For a long time, larger and larger network buffers have been creeping into network equipment, with many equipment vendors telling us big buffers are necessary (and charging us handsomely for them), and various queue management strategies being developed over the years.

Now, with storage networks and clustering moving from specialised platforms such as fibre channel and infiniband to using ethernet and IP, and the transition to distributed clustering (aka “The cloud”), this wisdom is being challenged, not just by researchers, but operators and even equipment manufacturers.

The fact is, in lots of cases, it can often be better to let the higher level protocols in the end stations deal with network congestion rather than introduce variable congestion due to deep buffering and badly configured queueing in the network which attempts to mask the problem and confuses the congestion control behaviours built into the protocols.

So, it was interesting to find two articles in the same day with this as one of the themes.

Firstly, I was at the UKNOF meeting in London, where one of the talks was from a research working on the BufferBloat project, which is approaching the problem from the CPE end – looking at the affordable mass-market home router, or more specifically the software which runs on them and the buffer management therein.

Second thing I came across was a great blog post from technology blogger Greg Ferro’s Ethereal Mind – on his visit to innovative UK ethernet fabric switch startup Gnodal – who are doing some seriously cool stuff which removes buffer bloat from the network (as well as some other original ethernet fabric tech), which is really important for the data centre networking market with it’s latency and jitter sensitivity, which Gnodal are aiming at.

(Yes, I’m as excited as Greg about what the Gnodal guys are up to, as it’s really breaking the mould, and being developed in the UK, of course I’m likely to be a bit biased!)

Is the writing on the wall for super-deep buffers?

Regional Broadband – The Hidden Danger of Uber-projects

It was revealed this week that Digital Region, the centrally funded (to the tune of £90m – mostly public money) superfast broadband initiative in South Yorkshire is facing tough times, in particular a £9.2m loss on a revenue of only £167k (which only just pays the last CEO’s £100k salary – they are currently seeking a new CEO, one assumes to manage a turnaround).

The Yorkshire Post article goes on to explain another £4m of public funds have been ringfenced as a “security”, and that the four participating councils, already under budget pressure from Central Government austerity, may need to as much as £500k per year to secure the operations of Digital Region if the loan can’t be repaid. Is that throwing good money after bad, or is the situation redeemable?

This highlights my belief that these large centrally funded uber-projects contain a more significant risk of failure, and not of delivering the right product. The larger organisations that are able to bid and win such projects can come with higher overheads compared to the smaller community projects such as those serving areas with poor existing broadband service, who have a relatively captive and supportive market, and benefit from a tighter focus – for instance Rutland Telecom’s pioneering FTTC with unbundled sub-loop in Lyddington, which is using the same basic FTTC tech as DR is using, but on a smaller scale and in relative isolation.

The larger scale of the Digital Region deployment obviously needed a much bigger income to support the aggressive build and provisioning costs, along with what looks like a complex structure, and now that revenue hasn’t been realised. As can be seen on the DR website, and highlighted on the ThinkBroadband article, very few ISPs use the DR infrastructure to deliver service and is maybe one of the reasons they aren’t making their targets.

You have to ask yourself why this is? Continue reading “Regional Broadband – The Hidden Danger of Uber-projects”

What is it with the Virgin Brand?

Or “Why it’s easy to pick fault with Virgin Group companies“.

You may have noticed that I’ve recently been airing my opinion – or is it pent-up frustration – on the service that Virgin Trains provides on the UK’s West Coast Mainline out of Euston station in London. I long ago gave up trying to interact with their customer relations department about their failure to deliver either a promised element of their product, or sometimes what should be just their basic service – a comfortable journey from A to B.

It got me thinking about the wider point of why people seem easily dissatisfied with a service, and specifically Virgin Group as a whole, trains, planes, phones, tv, internet access, etc. A couple of quick searches, especially on social media, and it’s easy to find people going full tilt hating on Virgin Trains, dishing out brickbats to Virgin Media about busted broadband, or flying off the handle about the run-down (thankfully soon to be updated) Gatwick fleet on Virgin Atlantic.

The crux I’ve arrived at is that the Virgin brand tends to overpromise through it’s marketing and brand image and therefore sets itself up to underdeliver and disappoint.

Let’s look at the key connotations of the Virgin brand: Continue reading “What is it with the Virgin Brand?”

Now for the real “Up In The Air”

If you happened to be bored on a plane sometime in 2010, there’s a high likelihood you’ll have seen the film Up In The Air, and some of you may even relate to it. I remember that United Airlines even had the film doctored to remove much of the obvious product placement for competitor American Airlines from the film!

Air New Zealand 747-400, ZK-SUI by robertjamesstarling, on Flickr
Now, there's a nice plane. Still prefer it to the 777.

At the time, I was doing between 75k and 100k miles in flight each year, and while I wasn’t living the somewhat empty, itinerant existance of George Clooney’s character, I was almost certainly doing more travelling than most so-called “Traveller families” were travelling in the UK.

It meant that I could certainly relate to the film, the lead character’s pursuit of miles and elite status, and the benefits of choosing the correct airport security lane. I expect a lot of people reading this post (“Hi, Internet meeting circuit!”) can also relate to this.

I still never got down to living entirely out of a single roll-aboard case for more than a few days at a time though.

An independent film maker, Gabriel Leigh, has decided to make a feature-length documentary about the real frequent flyers, the people who really are “Up In The Air”, all the time, and often for no apparent reason. The film maker is appealing for backing on Kickstarter to raise the money to make the project.

Of course, the real irony would be if he can manage to fly to all the places he needs to when making the film using redeemed miles, rather than paying for a ticket!

Here’s a 20 minute taster of when he initially explored the phenomenon of the FlyerTalker and mileage runner.

One thing which really struck me about this short video was the chap in Tokyo, when he compared the airport to a city and a city to the airport. Everyone just going about their business, speaking their own language, doing their own stuff, in their own world, rarely interacting?

While it’s a true comparison for the mega-airports like Schiphol, DFW, Frankfurt and Heathrow, do we want our cities, our homes, our environments in which we live every day to become as impersonal as an airport? I don’t doubt for a moment that it is happening, but can’t help feeling I think that would be a sad state of affairs in the evolution of the city in the long term.

Please accept new prefixes XYZ behind ASfoo – make it stop!

Those of you who ran networks in the 1990s (possibly even in the early 2000s) will remember the excitement you had joining your first Internet exchange, plugging in that shiny new cable to your router interface, and setting up your first peerings.

Back then, you may also remember that in the rapidly growing Internet of the day, it was common courtesy to let your peers know that you’ve taken on a new customer, or acquired some new address space, so they could update their configs – particularly any filtering they were doing on the routes exchanged with you, which were often quite small and maintained manually, except for the largest providers.

Your message would go something like this:

Continue reading “Please accept new prefixes XYZ behind ASfoo – make it stop!”

I’m sorry. I didn’t quite understand that. Please state “Yes” or “No”

There’s a perfect tv programme, just up my street, airing next week…

Maybe it’s just the grumpy old man in me, but I don’t enjoy dealing with call-centre systems that have complex menu trees, repetitive announcements, auto-attendants, IVRs, and being told that my “business-is-important-thankyouforholding”.

Why can’t a person just pick up the phone and speak to me?

There is a cynical thing going on in some cases – a shady need for the company you’re phoning to keep you on the phone for a while – Call Revenue Sharing. This is where the destination of the phone number recieves a portion of the call termination revenue from the phone company. If the call isn’t long enough, the revenue share is negligible. The cynic in me, saying “yes” or pressing “1” for the umpteenth time, thinking this IVR or menu tree is designed to make the call sufficiently long enough to bring money into the organisation.

I’m therefore looking forward to Channel 4’s Richard Wilson On Hold which is broadcast at 8pm on Monday 16th January. I wonder if they will touch on Revenue Sharing, or just concentrate on the grumpy annoyance factor.