Those of you who ran networks in the 1990s (possibly even in the early 2000s) will remember the excitement you had joining your first Internet exchange, plugging in that shiny new cable to your router interface, and setting up your first peerings.
Back then, you may also remember that in the rapidly growing Internet of the day, it was common courtesy to let your peers know that you’ve taken on a new customer, or acquired some new address space, so they could update their configs – particularly any filtering they were doing on the routes exchanged with you, which were often quite small and maintained manually, except for the largest providers.
Your message would go something like this:
Dear peers,
FoobarNET (AS64500) are going to announce the following new prefixes a.b.c.d/19
Please update your filters to accept this new prefix.Hugs,
The AS64500 NOC Bobs
Your NOC would duly recieve the mail, and go into the router config and update it so they could recieve your shiny new prefix.
The cynic in me also says that in the early days there may even have been a little bit of friendly bragging hidden between the lines in these messages – “Look at me! I’ve got another downstream multihomed customer, wheee!”, or “Look at my shiny new /16. Have you got one of those? No?”
Over the years, as the Internet expanded, and you went from having 30 or 40 peers on an IXP, to having upwards of 200 or 300 on the largest exchanges, it became necessary to automate things, or just to be less fussy, because you were rubbish at scripting or were in the boom time rat-hole of not having enough round-tuits to put the automation in place, and trusted your peers not to leak routes to you.
So you usually fell into one of these two camps:
- Automated prefix-list generation – You had a number of tools which looked what prefixes you should be receiving from your peer (e.g. from looking in a routing registry database), and building very specific filter/prefix lists from these and installing them in your router.
- Setting a maximum prefix limit – Rather than have to explicitly say which prefixes you should be receiving from your peer, you know you should receive no more than a predetermined limit (usually based on experience), so you set this maximum number in the config, and if any more prefixes are received valid or otherwise, the session will be torn down.
There are pros and cons to both methods. Option 1 specifically allows just the things you want to allow and ignores everything else, keeping the BGP session up. Option 2 is really there to protect your infrastructure from the suboptimal routing which will result if your peer messes up their configuration and sends you way more routes than they should – usually a full table – and in this case you probably really want to tear the session down as who knows what else they broke.
In either case, both of these common methods of ingress protection have made the “Dear Peers, please adjust your filters…” messages at best redundant, and at worst, an annoyance.
While CTO at LINX, I spent a long time trying to stamp out this outdated practice, however it seems like it’s largely been the equivalent of trying to hold back the sea, as sadly it looks like such retro stuff is still the new black (again), or maybe old habits just die hard, as these messages keep hitting IXP operations lists – ten years later!
They are also faced with the same exasperated reactions from the folks reading the list – that it doesn’t serve any purpose, it’s a distraction that they don’t need, and it’s a waste of both the sender and the recipient’s time.
All I can surmise is that there must be some bad habits which refuse to die out, or terribly outdated NOC operations manuals out there, for whatever reason, be it laziness, overwork, or some sort of procedural red tape, that still make the hapless NOC engineer send such messages. It would be interesting if a routing measurement project has the data and cares to analyse if such messages actually have any effect on the propagation of routing information.
As the Internet has grown, and many networks of different cultures and levels of experience join an exchange, such messages now get what I call a “cluefree” bit-wasting response, usually from a NOC on the mailing list which is used to dealing with downstream customer problems, and not used to dealing with peering relationships which are subtely but significantly different.
Common bozo responses are:
- What is your circuit ID?
- What is your customer reference number?
- We refuse these new routes as you aren’t a customer.
- What is our circuit ID?
…Er, shouldn’t they know that last one? There are various other evils too, but the crux of it is that these dumb responses take up time, and the whole thing is wasting time and making the more clueful folks angry.
This also opens up the Pandora’s box of making sure that contact from your IP peers is recieved at the appropriate place in your organisation.
What’s frightening is that it’s easy to draw the conclusion that these NOCs are poorly run, with inadequate staff training, and infrequently updated procedures manuals. It’s a bad advert for that company.
Anyway, maybe it’s time for another crusade amongst the operators to stamp out this quaint but frankly outdated practice.
Some ISPs become telecoms companies without noticing, particularly when they start selling VPN products (woo, circuit IDs!) . I’ve not seen much emphasis put on routing registries and automation of route filters, mostly because it’s a niche thing in most organisations that aren’t pure ISPs, at least in the context of the whole company. It probably doesn’t get the recognition it deserves considering how vital it is.
NTT/Verio in 2003 had their own routing registry. At FLAG 2003-2006 we had a tool which managed to munge RPSL data into Junos configs, but it seemed almost cutting-edge back in 2003, despite this being a problem that needed to be solved everywhere. Max-prefixes also got incremented by a small percentage based on number of prefixes as it grew.
So I think this might be a problem with lack of tools. The usual investment in tools, well, that goes into massive reporting behemoths and “network management systems” which are expected to solve every network administration problem you’ll ever come across, but essentially just ping and modify traps. “Oh, do you mean IP address management software?” is about as close to this as you’ll get.
To be honest, I’ve been out of the ISP routing world a bit too long to know current best practices, but routing registries and filters are certainly never discussed in the OSS (Operational Support Systems) and NMS world, which means something’s not getting through to the area where the money’s being spent.
I had a skim through the presentation below from NANOG which discusses the subject and some of the tools. Judging by some of the documentation of this software, it’s no wonder it’s got a lack of exposure.
Click to access NANOG51.Talk34.NANOG51%20IRR%20Tutorial.pdf
Also, thanks for the nostalgia trip 🙂 May all your resets be soft.