Conferences – Page 2 – Mike's Twitter Overflow

Beware the NTP “false-ticker” – or do the time warp again…

For the uninitiated, it’s easy to keep the clocks of computers on the Internet in synch using a protocol called NTP (Network Time Protocol).

Why might you want to do this? It’s very helpful in a large network to know that all your gear is synchronised to the same time, so that things such as transactional and logging information has the correct timestamps. It’s a must have for when you’re debugging and trying to get to the bottom of a problem.

There was an incident earlier this week where the two open NTP servers run by the US Naval Observatory (the “authority” for time within the US) both managed to give out incorrect time – there are reports of computers which synchronised against these (and more importantly, only these, or one or two other systems) had their clocks reset to 2000. The error then corrected, and clocks got put back.

Because the affected systems were chiming either only against the affected master clocks, or a limited number of others, the two incorrect times, but from a high stratum source, were taken as being correct and the affected systems had their local clocks reset.

There’s been discussion about the incident on the NANOG list…

Continue reading “Beware the NTP “false-ticker” – or do the time warp again…”

Recent IPv4 Depletion Events

Those of you who follow these things can’t have missed that the RIPE NCC had got down to it’s last /8 of unallocated IPv4 space last week.

They even made a cake to celebrate…

185-slash-8-cake — Photo (and cake?) by Rumy Spratley-Kanis

This means the RIPE NCC are down to their last 16 million IPv4 IP addresses, and they can’t get another big block allocated to them, because there aren’t any more to give out.

Continue reading “Recent IPv4 Depletion Events”

Networking equipment vs. Moore’s Law

Last week, I was at the NANOG conference in Vancouver.

The opening day’s agenda featured a thought provoking keynote talk from Silicon Valley entrepreneur and Sun Microsystems co-founder Andy Bechtolsheim, now co-founder and Chief Development Officer of Arista Networks, entitled “Moore’s Law and Networking“.

The basic gist of the talk is that while Moore’s Law continues to hold true for the general purpose computing chips, it has not applied for some time to development of networking technology. Continue reading “Networking equipment vs. Moore’s Law”

A table for 25? Not currying any favour with me…

Many of you will know that I’m involved in organising the UKNOF meetings.

Some of you will know that I don’t understand this obsession that many UKNOF attendees have with going en-masse for a curry (usually with someone’s employer picking up the tab) the evening beforehand.

What is the attraction, apart from maybe not having to pay for it yourself, of sitting at a big long table, when all it achieves is you having to yell at the person next to you in order to have a conversation while receiving iffy service of usually disappointing (sometimes downright poor) food?

It’s no good for mixing and networking, one of the attractions of going for dinner with industry colleagues, as you can only bellow your conversation at your immediate neighbours, either because everyone else is pissed and shouting, or just to make yourself heard over the loud sitar music.

Sitting in tables of 6-8 would help a lot with conversation, and probably improve service as well!

It’s also not a good dining experience. The most recent curry being a particular lowlight, when a) I hardly ate any of what I ordered because it was so unpleasant (and it wasn’t as though I’d ordered a phall!), and b) I was later unwell in the middle of the night. I should have seen the warning signs when they handed us each a sticky, laminated menu card, I guess.

While I don’t think of myself as entirely Grumpy Old Man as yet, I still don’t really see the attraction…

I also can’t talk about drunken behaviour in curry houses without a link to Rowan Atkinson’s Indian Restaurant sketch… It is a tricky bit of floor. Deceptively flat…

Successful 1st IXLeeds Open Meeting

I attended by all accounts a very successful first open meeting for the IXLeeds exchange point yesterday – with around 120 attendees, including many faces that are not regulars on the peering circuit making for brilliant networking opportunities and great talks from the likes of the Government super-fast broadband initiative, BDUK, and energy efficient processor giants ARM (behind the technology at the heart of most of the World’s smartphones), as well as more familiar faces such as RIPE NCC and LINX, among others.

Definitely impressed with the frank discussion that followed the talk by the DCMS’ Robert Ling on BDUK funding and framework, but still sceptical that it’s going to be any easier for smaller businesses to successfully get access to the public purse.

Andy Davidson, IXLeeds Director, was able to proudly announce that IXLeeds now provides support for jumbo frames via a seperate vlan overlaid on their switch, which is probably the only IXP in the UK which officially offers and promotes this service – at least for the time being. Of course, they are supporting a 9k frame size…

Well done to my friends and colleagues of IXLeeds for making it to this major milestone, and doing it in great style. It seems a long, long way from a discussion over some pizza in 2008.

The only thing I didn’t manage to do while in Leeds is take a look at the progress on the next phase of aql’s Salem Church data centre, but I’m sure I’ll just have to ask nicely and drop by aql at some point in the future. 🙂

End of the line for buffers?

This could get confusing…

buffer (noun)

1. something or someone that helps protect from harm,

2. the protective metal parts at the front and back of a train or at the end of a track, that reduce damage if the train hits something,

3. a chemical that keeps a liquid from becoming more or less acidic

…or in this case, none of the above, because I’m referring to network buffers.

For a long time, larger and larger network buffers have been creeping into network equipment, with many equipment vendors telling us big buffers are necessary (and charging us handsomely for them), and various queue management strategies being developed over the years.

Now, with storage networks and clustering moving from specialised platforms such as fibre channel and infiniband to using ethernet and IP, and the transition to distributed clustering (aka “The cloud”), this wisdom is being challenged, not just by researchers, but operators and even equipment manufacturers.

The fact is, in lots of cases, it can often be better to let the higher level protocols in the end stations deal with network congestion rather than introduce variable congestion due to deep buffering and badly configured queueing in the network which attempts to mask the problem and confuses the congestion control behaviours built into the protocols.

So, it was interesting to find two articles in the same day with this as one of the themes.

Firstly, I was at the UKNOF meeting in London, where one of the talks was from a research working on the BufferBloat project, which is approaching the problem from the CPE end – looking at the affordable mass-market home router, or more specifically the software which runs on them and the buffer management therein.

Second thing I came across was a great blog post from technology blogger Greg Ferro’s Ethereal Mind – on his visit to innovative UK ethernet fabric switch startup Gnodal – who are doing some seriously cool stuff which removes buffer bloat from the network (as well as some other original ethernet fabric tech), which is really important for the data centre networking market with it’s latency and jitter sensitivity, which Gnodal are aiming at.

(Yes, I’m as excited as Greg about what the Gnodal guys are up to, as it’s really breaking the mould, and being developed in the UK, of course I’m likely to be a bit biased!)

Is the writing on the wall for super-deep buffers?

Abstracting BGP from forwarding hardware with OpenFlow?

Interesting hallway discussion at RIPE 63 last week. Olaf Kolkman stopped me in the coffee break and asked if I could think of any way of intercepting BGP data in a switch/router, and somehow farming if out to an external control plane to process the BGP update, making the routing decisions external from the forwarding device and then updating the FIB in switch/router.

I had a think for about 15-20 seconds and asked “What about OpenFlow?”

In a classic switch/router type device, BGP packets are identified in the ingress packet processor and punted up to the control plane of the box. The routing process doesn’t run in the forwarding silicon, it’s running in software in the system CPU(s). The routing process evaluates the routing update, makes changes to the RIB in accordance, updates the system FIB, and then sends an internal message to program the forwarding silicon to do the right thing with the packet.

I’m assuming at this stage, that you understand the rationale behind wanting to move the routing decisions outside of the forwarding hardware? There are many reasons why you might want to do this: centralised routing decisions being one (hopefully quicker convergence?), the ability to apply routing policy based on higher level application needs (for example in cluster environments), to run routing decisions in powerful commodity hardware (rather than specialised, expensive, networking hardware), to run customised routing code to suit local requirements, or as Dave Meyer helpfully said, “Allows you do lots of abstractions”.

So, why not try and do this with Openflow?

It’s designed to make pattern matches on incoming packets in a switch and where necessary punt these packets to an OpenFlow controller for further processing. The OpenFlow controller can also signal back to the switch on what to do with further packets with the same properties, effectively programming the forwarding hardware in the switch. Not significantly different with how BGP updates are processed now, except it’s all happening inside the box.

It looks like OpenFlow could be the thing – except it’s probably a little young/incomplete to do what we’re asking of it at this stage – but it’s a rapidly developing protocol, and we’ve got folks who are well deployed in the SP arena that have already said they are roadmapping to build OpenFlow functionality into their SP product lines, such as Brocade.

It seems to me that there are plenty of open source routing daemons out there (Quagga, OpenBGPd, BIRD) which could serve as the outboard routing control plane.

So what seems to be needed is some sort of OpenFlow controller to routing daemon shim/abstraction layer, so that an existing BGP daemon can communicate with the OpenFlow controller, which seems to be what the QuagFlow project is doing.

Whither (UK) Regional Peering – Pt 3

Anyone still using C7513s? At the end of the last post, I vaguely threatened that at some point I’d go on to discuss IX Participant Connectivity.

The topic got a “bump” to the front of the queue last week, thanks to a presentation from Kurtis Lindqvist, CEO of Sweden’s Netnod exchange points, given at the RIPE 63 meeting in Vienna.

Netnod have been facing a dilemma. They provide a highly resilient peering service for Sweden, consisting of multiple discrete exchanges in various Swedish cities, with the biggest being in Stockholm – where they operate two physically seperate, redundant exchanges. They currently provide all their participants in Stockholm with the facility to connect to both fabrics, so they can benefit from the redundancy this provides. Sounds great doesn’t it? If one platform in Stockholm goes down, the other is up, traffic keeps flowing. Continue reading “Whither (UK) Regional Peering – Pt 3”

Just let IPv4 run out. It’s over. Just get on with it.

So, I’m currently at the RIPE 63 meeting in Vienna. Obviously, one of the ongoing hot topics here is IPv4 depletion, at times consisting of discussion on either a) the transition away from IPv4 to IPv6 via various transition mechanisms, and b) how to make the pitiful amount of IPv4 addressing that’s left last as long as possible.

One of the things that is often said about (b) is that it shouldn’t be done to death, IPv4 should just be allowed to run out, we get over it, and deploy IPv6. However (b) behaviour is to be expected when dealing with exhaustion of a finite resource.

There are similarities and parallels to be drawn between IPv4 runout and IPv6 adoption, fossil fuel depletion and movement to alternative energy techologies. The early adopters and the laggards. The hoarders and speculators. The evangelists and the naysayers.

So, for a minute don’t think about oil and gas resources being depleted, that’s way in the future. We’re facing one of the first examples of exhaustion of a finite resource on which businesses and economies depend.

If the IPv4 depletion and IPv6 (slow) adoption situation is a dry run of what might actually happen when something like oil runs out, then we should be worried, because we can’t just rely on carrier grade NAT to save us.

More specifics driving traffic to transit?

Interesting talk at RIPE 63 in Vienna today from Fredy Kuenzler of the Swiss network Init7 – How more specifics increase your transit bill (video transcript).

It proposes that although you may peer with directly with a network, any more specific prefixes or deaggregated routes which they announce to their upstream transit provider will eventually reach you, and circumvent the direct peering. If this forces traffic to your transit provider, it costs you money per meg, rather than it being covered in your (usually flat) cost of peering.

Of course, if it’s the one transit provider in the middle, they are getting to double-dip – being paid twice (once on each side) for the same traffic! Nice if you can get it!

So, the question is, how to find these more specific routes mark them as unattractive and not install them in your Forwarding Table, preferring the peered route, and saving you money.

Geoff Huston suggests he could provide a feed or a list of the duplicate more specific routes, crunching this sort of thing is something he’s been doing for ages with BGP routing data, such as the CIDR Report.

But the question remains how to take these routes and either a) keep them in the table, but deprefer the more specific which breaks a fundamental piece of decision making in BGP processing, or b) filter them out entirely, without affecting redundancy if the direct peering fails for any reason.

I started out being too simplistic, but hmm… having a think about this one…