There’s been some good efforts to fix the hiatus that’s been perceived to exist between the Internet operator community and the IETF recently. I hope I’m not giving them the kiss of death here… 🙂
A sense of frustration had been bubbling for a while that the IETF had become remote from the people who actually deploy the protocols, that IETF had become the preserve of hardware vendors who lack operational experience, and it’s no wonder they ship deficient protocols.
But, it can’t have always been that way right? Otherwise the Internet wouldn’t work as well as it does?
Well, when the Internet first got going, the people who actually ran the Internet participated in the IETF, because they designed protocols and they hacked at TCP stacks and routing code, as well as running operational networks. Protocols were written with operational considerations to the fore. However, I think people like this are getting fewer and fewer.
As time went by, the Internet moved on, a lot of these same folk stopped running networks day-in-day out, and got jobs with the vendors, but they stayed involved in the IETF, because they were part of that community, they were experienced in developing protocols, and brought operational experience to the working groups that do the development work.
The void in the Network Operations field was filled by the next generation of Network Engineers, and as time has gone by, fewer and fewer of them were interested in deveoping protocols, because they were busy running their rapidly growing networks. Effectively, there had been something of a paradigm shift in the sorts of people who were running networks, which differed from those who had been doing it in the past For the Internet to grow the way it did in such a short time, something had to change, and this was it.
At the same time, the operational engineers were finding more and more issues creeping into increasingly complex protocols. That’s bad for the Internet, right? How did things derail?
The operational experience within the IETF was suffering from two things – 1) it was becoming more and more stale the longer that key IETF participants didn’t have to run networks, and 2) the operator voice present at IETF was getting quieter and quieter, things suggested by operators had been largely rejected as impractical.
Randy Bush had started to refer to it as the IVTF – implying that Vendors had “taken over”.
There have been a few recent attempts to bridge that gap – “outreach” talks and workshops at operations meetings such as RIPE and NANOG sought to get operator input and feedback, however trying to express this without frustration hasn’t always been easy.
However, it looks like we’re getting somewhere…
Rob Shakir has currently got a good Internet Draft out aimed at building a bridge between the ops community who actually deploy the gear and the folks who write the protocol specs and develop the software and hardware.
This has been long overdue and needs to work. It looks good, and is finding support from both the Vendor and Ops communities.
It’s a “meta-problem” here is that one cannot exist without the other, it’s a symbiotic and mutually beneficial relationship that needs to work for a sustainable Internet.
I wonder if it’s actually important for people on the protocol design and vendor side to periodically work on production networks to ensure that they have current operational knowledge, and not relying on that from 10 years ago?
I think there are a couple of sides to this. My observation at IETF meetings is that we actually have several hundred operators that show up in v6ops (IPv6 Operations) and other operational working groups like opsawg and opsec. So the assertion that “operators don’t come” needs review. I can’t comment on IETF participant participation in RIPE/NANOG/etc, as I come to some but infrequently, but I similarly think that happens.
There is a fair amount of frustration expressed by people in each group when they are dismissed as “without clue” by people from the other. I suspect you can name people in the IETF that make you unhappy; I can name operators that have the same set of personal characteristics. I suspect that we need a certain amount of thickness of skin and détente.
As to “is operational experience important in protocol development”, yes it is. It’s not the only thing of importance – sometimes I would really like to take an operator aside and talk with them about the way they run their network and what could be improved if it was re-thought, as opposed to “fix the problem I had this morning now” or “could we please make IPv6 into IPv4 with more bits in the address”, which it is in some respects but isn’t in others. I would not agree that the original writers of the Internet architecture and its components all ran networks, but the protocol designers worked elbow-to-elbow with people who did. What I think we do need is more of that.
As mentioned in the opening paragraph, definitely an issue of perception, and you’re right, increasing the amount of co-operative, elbow-to-elbow, work is what’s needed, rather than the types of dismissive behaviour we can probably both think of…
Definitely take your point on the balance as well – operators can easily become focused on “dammit, just work” approaches because they have to deal with the immediate problems, and can benefit from being around those who can take a broader view.